Small Group Discussions with Applicants and Other External Stakeholders Regarding Clarity, Understandability, and Usefulness of FDA's Benefit-Risk Framework (BRF)

Small Group Discussion Moderator Script: External Stakeholders and Applicants

benefit-risk framework implementation Evaluation

OMB Control No: 0910-0697 Expiration Date: 12/31/2023

Paperwork Reduction Act Statement: According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0910-0697. The time required to complete this information collection is estimated to average 90 minutes per response, including the time for reviewing instructions, participating in the discussion, and completing and reviewing any collection of information.

Send comments regarding this burden estimate or any other aspects of this collection of information, including suggestions for reducing burden to PRAStaff@fda.hhs.gov.

Your participation/nonparticipation is completely voluntary, and your responses will not have an effect on your eligibility for receipt of any FDA services. In instances where respondent identity is needed (e.g., for follow-up of non-respondents), this information collection fully complies with all aspects of the Privacy Act and data will be kept private to the fullest extent allowed by law.

The study we are conducting is on behalf of the U.S. Food and Drug Administration (FDA).

Introduction and Objectives

Thank you for taking the time to talk with us today. I am [name], and will be facilitating this discussion today. I am joined by my colleagues [name(s)], from Booz Allen Hamilton.

First, I want to provide some background on the purpose and objective of this discussion. Under the sixth authorization of the Prescription Drug User Fee Act, or PDUFA VI, FDA committed to continuing the implementation of the Benefit-Risk Framework into review documentation processes. The Benefit-Risk Framework is FDA’s framework to assess benefits and risks of new drugs and biologics and it was developed to improve the consistency and clarify of benefit-risk assessment during the pre-market review process. As a part of this commitment, FDA contracted Booz Allen to conduct an independent evaluation of its Benefit-Risk Framework (BRF). This includes assessing the Benefit-Risk Framework’s clarity, understandability, and utility as a communication tool for FDA’s external stakeholders. With that, the objectives are to:

1. Obtain your insights on the clarity and understandability of the Benefit-Risk Framework

2. Discuss how you would rate and assess the usefulness of the Benefit-Risk Framework

3. Identify how the Benefit-Risk Framework has been used and ways to which it could be adjusted to increase its usefulness

As part of our assessment of the Benefit-Risk Framework, we would like to ask you about your experiences with its use in reviews of [therapeutic area] drugs and biologics, such as [established name(s)].

This discussion should take about an hour to an hour and a half. I will ask questions, and [name(s)] will take notes. Booz Allen will keep your identifying information confidential. We will share only anonymized results outside our internal project team. Here are standard government statements about the voluntary and confidential nature of this information collection:

Do you have any questions before we start?

[Applicants only] Can you please state your title and what about FDA’s benefit-risk assessment is relevant to your role or otherwise of interest to you?

Discussion Questions

I am going to ask you about the understandability, clarity, and usefulness of the Benefit-Risk Framework prepared for [therapeutic area] drugs and biologics. Please feel free to ask me to clarify if anything is unclear.

1. Do you have any general comments about Benefit-Risk Framework(s) that you’ve seen that you would like to offer before I ask specific questions?

Understandability: Now, we are going to discuss how understandable you felt the logic and rationale were in the Benefit-Risk Framework. For the purposes of this discussion, we are defining “understandable” as whether you feel that FDA logically presented the key points in the Benefit-Risk Framework(s) and provided a clear rationale underlying the decision. Is this description clear to you?

2. Did you get a chance to read the Benefit-Risk Framework(s) in the review memo we sent prior to this discussion?

   1. What were the biggest “takeaways” for you?

   2. Did anything in the Benefit-Risk Framework(s) surprise you?

   3. After reading the Benefit-Risk Framework(s), did you have any further questions about how FDA came to its decisions?

3. In what ways is the content understandable?

4. In what ways can the understandability be improved?

5. On a scale of 1 to 4, how would you rate the understandability of the content presented in the Benefit-Risk Framework(s) you have seen, where 1 is not at all understandable and 4 is very understandable?

Clarity: Next, we are going to discuss how you clearly you felt the information and key points in the Benefit-Risk Framework(s) were stated. For the purposes of this discussion, we are defining “clear” as key points stated explicitly, parsed and sequenced appropriately, and grouped logically in a manner to which you can read the content easily.

6. Is the content presented clearly? If so in what ways?

7. In what ways can the clarity be improved?

For example:

• To what extent are key points stated explicitly?

• To what extent is content related to each key point grouped together?

• To what extent are key points presented in a logical sequence?

8. On a scale of 1 to 4, how would you rate the clarity of the Benefit-Risk Framework(s) you have seen, where 1 is not at all clear and 4 is very clear?

Usefulness: In this last portion of the discussion, I want to discuss to what extent the Benefit-Risk Framework(s) are useful to you.

9. How have you used or how do you plan to use Benefit-Risk Framework(s)?

   1. How useful is the Benefit-Risk Framework(s) for these purposes?

10. How can the usefulness of Benefit-Risk Framework(s) be improved?

11. How else would you like to use the Benefit-Risk Framework(s)?

    1. How would the Benefit-Risk Framework(s) need to be adjusted to accommodate these uses?

12. On a scale of 1 to 4, how would you rate the usefulness of Benefit-Risk Framework(s) for your purposes, where 1 is not at all useful and 4 is very useful?

13. Is there anything else that you would like to add about the Benefit-Risk Framework(s) that we haven’t covered today?

Closing the Discussion

Thank you very much for taking the time to talk with us. Your feedback is very helpful in giving us a sense of how the implementation of the Benefit-Risk Framework is working from a real-world perspective. Thanks again.