Scale Development

11

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Food and Drug Administration

[Docket No. FDA-2014-N-0373]

Agency Information Collection Activities; Submission for Office of Management and Budget Review; Comment Request; Risk and Benefit Perception Scale Development.

AGENCY: Food and Drug Administration, HHS.

ACTION: Notice.

SUMMARY: The Food and Drug Administration (FDA) is announcing that a proposed collection of information has been submitted to the Office of Management and Budget (OMB) for review and clearance under the Paperwork Reduction Act of 1995.

DATES: Fax written comments on the collection of information by [INSERT DATE 30 DAYS AFTER DATE OF PUBLICATION IN THE FEDERAL REGISTER].

ADDRESSES: To ensure that comments on the information collection are received, OMB recommends that written comments be faxed to the Office of Information and Regulatory Affairs, OMB, Attn: FDA Desk Officer, FAX: 202-395-7285, or emailed to oira_submission@omb.eop.gov. All comments should be identified with the OMB control number 0910-New and title, " Risk and Benefit Perception Scale Development." Also include the FDA docket number found in brackets in the heading of this document.

FOR FURTHER INFORMATION CONTACT:

Ila Mizrachi,

Office of Information Management,

Food and Drug Administration,

1350 Piccard Dr.,

PI50-400B,

Rockville, MD 20850,

301-796-5156,

Ila.Mizrachi@fda.hhs.gov.

SUPPLEMENTARY INFORMATION: In compliance with 44 U.S.C. 3507, FDA has submitted the following proposed collection of information to OMB for review and clearance.

Risk and Benefit Perception Scale Development (0910-New)

Section 1701(a)(4) of the Public Health Service Act (42 U.S.C. 300u(a)(4)) authorizes the FDA to conduct research relating to health information. Section 1003(d)(2)(c) of the Federal Food, Drug, and Cosmetic Act (the FD&C Act) (21 U.S.C. 393(b)(2)(c)) authorizes FDA to conduct research relating to drugs and other FDA regulated products in carrying out the provisions of the FD&C Act.

FDA requires that prescription drug advertisements be balanced in their presentation of risk and benefit information. Patients receive information on drugs not only from their doctors and pharmacies, through patient labeling and FDA-mandated Medication Guides, but also online, on social networks and via direct-to-consumer (DTC) television and print advertising. Moreover, research suggests that consumers struggle with the concepts of risk and efficacy and often overestimate drug efficacy . As a result, it is important for FDA to understand and accurately measure how consumers are making sense of this information and how it impacts decisions related to prescription drugs.

FDA’s Office of Prescription Drug Promotion (OPDP) has an active research program that investigates how direct-to-consumer advertising influences consumer knowledge, perceptions, and behavior. As OPDP’s research program has matured, the way in which we measure risk and benefit perception has evolved over time. This has resulted in perception measures that, while internally valid, tend to vary by study. Consequently, FDA needs a pool of reliable and valid measurement items for assessing consumers’ drug risk and benefit perceptions—as well as other elements of prescription drug decision making—consistently across studies. The purpose of this project is to create that measurement pool, thus increasing the rigor and efficiency of FDA’s research.

Design Overview

We will conduct pretesting prior to main data collection to assess the psychometric properties and identify any measurement challenges (e.g., misinterpretation, lack of variance) with candidate measurement items. We also will use the pretesting to examine factors that may affect future study results and analyses (e.g., response scale midpoints, moderating variables). We will conduct two sequential pretest waves (n=500 per wave; n=1,000 total) with the following target populations: (a) individuals diagnosed with chronic pain and (b) individuals diagnosed with hypertension.

Exhibit 1. Pretest Study Design

Wave	Medical Condition		TOTAL
	Chronic Pain	Hypertension
Wave 1	n = 250	n = 250	500
Wave 2	n = 250	n = 250	500
TOTAL	500	500	1,000

In the main study phase, we will conduct four sequential waves of iterative testing to fully assess the measurement properties of the candidate items and create the final pool of measurements. We will conduct the first two waves of the main study with members of the target populations (hypertension and chronic pain) to refine the measurement items for those groups and the second two waves with members of the general population who do not have the target health conditions to determine if measurement reliability and validity change when the advertised drug addresses a condition that study participants do not have (n=2,500 per wave; n=10,000).

Exhibit 2. Iterative Testing Design – Illness Population Sample

Wave 1
Chronic Pain Ad					Hypertension Ad
Ad Type	Drug Risk Level	Drug Benefit Level		Control	Ad Type	Drug Risk Level	Drug Benefit Level		Control
		High	Low				High	Low
Print	High	n=125	n=125	n=125	Print	High	n=125	n=125	n=125
	Low	n=125	n=125			Low	n=125	n=125
Television	High	n=125	n=125	n=125	Television	High	n=125	n=125	n=125
	Low	n=125	n=125			Low	n=125	n=125
Wave 2
Chronic Pain Ad					Hypertension Ad
Ad Type	Drug Risk Level	Drug Benefit Level		Control	Ad Type	Drug Risk Level	Drug Benefit Level		Control
		High	Low				High	Low
Print	High	n=125	n=125	n=125	Print	High	n=125	n=125	n=125
	Low	n=125	n=125			Low	n=125	n=125
Television	High	n=125	n=125	n=125	Television	High	n=125	n=125	n=125
	Low	n=125	n=125			Low	n=125	n=125

Exhibit 3. Iterative Testing Design – General Population Sample

Wave 3
Chronic Pain Ad					Hypertension Ad
Ad Type	Drug Risk Level	Drug Benefit Level		Control	Ad Type	Drug Risk Level	Drug Benefit Level		Control
		High	Low				High	Low
Print	High	n=125	n=125	n=125	Print	High	n=125	n=125	n=125
	Low	n=125	n=125			Low	n=125	n=125
Television	High	n=125	n=125	n=125	Television	High	n=125	n=125	n=125
	Low	n=125	n=125			Low	n=125	n=125
Wave 4
Chronic Pain Ad					Hypertension Ad
Ad Type	Drug Risk Level	Drug Benefit Level		Control	Ad Type	Drug Risk Level	Drug Benefit Level		Control
		High	Low				High	Low
Print	High	n=125	n=125	n=125	Print	High	n=125	n=125	n=125
	Low	n=125	n=125			Low	n=125	n=125
Television	High	n=125	n=125	n=125	Television	High	n=125	n=125	n=125
	Low	n=125	n=125			Low	n=125	n=125

Procedure

Pretests: Each participant will be randomly assigned to view either a print ad or a television ad for a fictitious prescription drug indicated to treat chronic pain or hypertension and will be asked to complete a brief online survey assessing their benefit/risk perceptions, intentions, and attitudes toward the drug. Based on the pretest findings, we will revise and remove candidate items prior to full-scale testing.

Main study: Each participant will be randomly assigned to view either a print or television ad for a fictitious prescription drug for hypertension or chronic pain and will be asked to complete a brief online survey assessing their benefit/risk perceptions, intentions, and attitudes toward the drug. In the first two main study waves, participants will view an ad that matches the sample’s medical condition (chronic pain or hypertension). In the final two main study waves, participants will be randomly assigned to view either the chronic pain stimuli or the high blood pressure stimuli.

The entire procedure is expected to last approximately 30 minutes. This will be a one-time (rather than annual) information collection. Note: The survey length has changed from 20 minutes to 30 minutes since the 60-day notice was published. This is due to the fact that cognitive interviews did not result in as much reduction in question numbers as originally expected. As this is a measurement validation study, it is important to include enough items on the questionnaire for sufficient comparison in order to identify those that perform the best. We have explained this change in survey length in responses to comments and have factored it in to the estimated burden.

In accordance with 5 CFR 1320.8(d), FDA published a 60 day notice for public comment in the FEDERAL REGISTER of April 21, 2014 (Volume 79, Number 76, pages 22143-22146). One comment was received from the company Eli Lilly, Inc. We respond to the points in Lilly’s comment below.

Comment: “Lilly seeks further clarity to better understand how FDA intends to apply the risk and benefit measurement items being developed through this study. FDA suggests in the Federal Register notice that the measurement items would be only used to enhance future FDA research initiatives; however, the precise nature and purpose of such planned research is unclear. Lilly suggests that any intended use of the measurement items to evaluate the effectiveness of drug advertising disseminated by industry would be inappropriate and beyond the jurisdiction and authorities granted to FDA.”

Response: Section 1701(a)(4) of the Public Health Service Act (42 CFR 300u(a)(4)) authorizes FDA to conduct research relating to health information. Section 903(d)(2)(C) of the Federal Food, Drug, and Cosmetic Act (the FD&C Act) (21 U.S.C. 393(d)(2)(C)) authorizes FDA to conduct research relating to drugs and other FDA regulated products in carrying out the provisions of the FD&C Act. We believe that these statutes provide a broad authority for FDA to conduct research related to prescription drug promotion as described in the information collection request. As already explained in the information collection request, the nature and purpose of this research is “to understand and accurately measure how consumers are making sense of this information and how it impacts decisions related to prescription drugs.” We believe that this research is crucial in ensuring that consumers are receiving prescription drug information that is truthful and non-misleading, and that prescription drugs are not being misbranded. FDA expects that any other purpose of this research will become clear only upon its completion, and FDA intends to make the research results and the final scale publicly-available.

Comment: “Although FDA intends to narrow the pool of survey questions during the pretesting stage of the research, we have concerns that the current questionnaire is extremely cumbersome and would likely exceed 20 minutes to complete. Further, based on the currently designed instrument, it is questionable whether in fact FDA would have success in respondents’ fully completing the survey.”

Response: Since the submission of the 60-day notice, the cognitive interviews have been completed (OMB control Number 0910-0695). We did not reduce the number of items as much as expected based on those interviews. Thus, we are recommending changing the questionnaire to 30-minutes in length, and burden estimates have been calculated accordingly. Even so, no respondent would ever answer the full list of questions provided in the 60-day notice; instead, the full questionnaire is the pool of items from which the questionnaire will be developed. We will test subsets of these candidate items using a form A/form B approach so that no respondent ever answers more than a 30-minute survey. In addition, some items may only be tested on one pretest and not the other or in one wave of a survey. No respondent would ever see all of these questions.

We take the survey length very seriously. We will be conducting two rounds of pretesting to refine the questionnaire and reduce the number of items, resulting in 30-minute (or shorter) questionnaires for the pretests and main study. We are sensitive to issues regarding respondent fatigue and its impact upon completion rates. We have employed similar online surveys on several previous studies, and we have obtained high completion rates, typically 90% or higher. For example, on a recent study (Experimental Study: Examination of Corrective Direct-to-Consumer Television Advertising [OMB control number 0910-0737])], we had a pool of 1,071 eligible respondents, and only 14 of those respondents failed to complete the survey. We anticipate that the completion rate for this study will be similar.

Comment: “In general, specific questions proposed in the draft questionnaire may be unanswerable by the respondent if not addressed specifically in the test stimulus. For example, Q23 “How long will Drug X/Drug Y’s negative side effects last once they begin?” If the duration of a drug’s side effects is not communicated in the stimulus, data captured would be purely speculative on the part of the consumer, especially without inclusion of a “don’t know or no opinion” option for the respondent.”

Response: Respondents will be exposed to information about the drug’s indication and side effects in the ad and will then be asked to provide their perceptions of the drug’s effectiveness and risk profiles. The questions are not intended to measure factual knowledge about the fictitious drug. By definition, one’s perception is a subjective assessment and, thus, does not need to be tied directly to a verbatim statement in the advertisement. Whether or not participants are forming perceptions about other attributes of the drug, such as how long side effects last, is an empirical question and the purpose of this study. Refining the questions, such as adding a “don’t know” option, will be further addressed by pretesting.

Comment: “In addition to the redundant and overlapping questions, several proposed questions appear to be unanswerable. The drafted questionnaire creates a high burden in complexity and time for the consumer and may cause significant respondent fatigue that could result in unreliable or incomplete data collection. Given these significant design issues related to the draft study questionnaire, Lilly suggests that FDA provide further details on how the questions in the draft questionnaire will be narrowed from the pretest stage to the iterative stage of the research and further evaluate the burden and likelihood to complete for the iterative testing stage.”

Response: The pool of questions will be narrowed and refined through two methods. The first method involved cognitive testing of draft measures (For a full discussion of the cognitive interviews, see OMB control Number 0910-0695. The goal of the cognitive interviews was to refine and narrow the measurement pool that will be subsequently pretested and then tested in an experimental study. The second method will involve iterative testing and analysis of draft measures to establish scale reliability and internal validity using survey methods. For a full discussion of the pretesting and experimental study see Design and Procedure sections above.

Comment: “Additionally, it is not clear why some batteries of questions, such as

those questions under the validity testing section (Q63-Q77) are included. These questions do not seem aligned with the research objective.”

Response: These items are included for the purpose of testing the convergent validity of the other items in our item pool (measures or risk and benefit perceptions). The items in Q63-Q77 come from the previously validated Beliefs about Medicines Questionnaire (BMQ) (Horne, Weinman & Hankins, 1999). As an example, if the benefit perception items perform as intended, they should be highly correlated with positive beliefs about medicines, as measured by the BMQ scale.

Comment: “Finally, questions 78-82 seem better placed in a battery of questions for the screening or consumer selection phase.”

Response: We believe that the constructs captured by questions 78-82 may moderate the relationship between ad content and respondents’ risk and benefit perceptions. We include them on the survey to keep the screener as short as possible, which reduces the burden on individuals who ultimately do not qualify for the study. They will not be used for screening as we do not plan to include or exclude any individuals based on their responses to these questions.

Comment: “Lilly suggests that the survey design be improved to better align with the research objectives, to avoid bias and to mitigate extreme respondent fatigue. Lilly recommends that FDA modify the data collection instrument to address the points noted above and seek additional public comment on the revised design.”

Response: Given our responses and points of clarification above, we believe that the current design is rigorous and meets FDA’s research objectives. The design allows us to test and validate measurement items for consumers’ risk and benefit perceptions. By randomizing respondents to the various ads with different benefit and risk information, we have controlled for underlying differences in respondent demographics and thereby have reduced the potential for selection bias (Kunz, Vist & Ochman, 2008) and enhanced study validity. As we have described above, we also have designed the study to minimize respondent fatigue by testing only the most promising candidate items and by ensuring a survey length of no more than 30 minutes.

Table 1. Estimated Burden¹

Table 1.--Estimated Annual Reporting Burden1
Activity	Number of respondents	Number of responses per respondent	Total annual responses	Hours per response2	Total hours
Pretest screener	2,000	1	2,000	.03	60
Main study screener	20,000	1	20,000	.03	600
Pretest	1,100	1	1,100	.5	550
Main Study	10,200	1	10,200	.5	5,100
Total	33,300	1	33,300	--	6,310

¹ There are no capital costs or operating and maintenance costs associated with this collection of information.

² Burden estimates of less than 1 hour are expressed as a fraction of an hour in decimal format.

³ With online surveys, several participants may be completing the survey at the time that the total target sample is reached. Those participants are allowed to complete the survey, which can result in the number of completes going slightly over the target number. Thus, if our target is 1,000, we have rounded up by an additional 100 to allow for some overage.

V. References

1. Lipkus, I. M. (2007). Numeric, verbal, and visual formats of conveying health risks: suggested best practices and future recommendations. Medical Decision Making, 27(5), 696-713.

2. Aikin, K. J., Swasy, J. L., & Braman, A. C. (2004). Patient and physician attitudes and behaviors associated with DTC promotion of prescription drugs–summary of FDA survey research results. Food and Drug Administration. Center for Drug Evaluation and Research.

3. Horne R., Weinman J., Hankins M. (1999). The beliefs about medicines questionnaire: The development and evaluation of a new method for assessing the cognitive representation of medication. Psychology and Health, 14, 1-24.

4. Kunz, R., Vist, G.E., & Ochman, A.D. 2008. Randomization to protect against selection bias in healthcare trials. The Cochrane Library, Issue 2.