0695 GEN IC CDER SSB 2017 Ext

U.S. Food and Drug Administration
Data To Support Drug Product Communications
OMB Control No. 0910-0695
SUPPORTING STATEMENT Part B: Statistical Methods
1. Respondent Universe and Sampling Methods
Testing includes various methods and approaches. The method(s) chosen for use depend on the
nature of the message or materials tested, as well as their intended audience. Recommended
methodologies and sample sizes are based on a review of the relevant literature, consultation
with experts in the field, and previous studies, regardless of source.
In general, testing relies on qualitative methods and is not intended to yield projectable results.
However, communication messages will be designed and marketed with specific audiences in
mind. In qualitative studies, quota sampling is often used to select a convenience sample of
individuals who meet certain qualifications that reflect characteristics typical of the target
audience. Response rate is not applicable to quota sampling because this type of sampling
results in a nonprobability sample which is not representative of the population. In qualitative
studies, respondents are usually initially contacted by telephone or through the mail; overrecruiting is done to compensate for not following up with non-respondents.
Where quantitative methods are used, information collection activities will target the particular
audiences with statistical sampling procedures employed to identify potential respondents. Mail,
telephone and Internet surveys typically will seek a convenience sample that nonetheless has a
reasonable diversity in key demographic characteristics such as age, gender, education, and
race/ethnicity. Telephone samples may be selected with random digit dialing (RDD) techniques,
lists, or with stratified sampling of telephone exchanges. For these samples, each sampling unit
(e.g., telephone household or respondent within a household) has a known non-zero probability
of selection.
2. Procedures for the Collection of Information
Questions in all testing methodologies include:
Standard measures of communications that are designed to assess to what degree the message
was successful in communicating information. These questions include measures of main idea
recall, comprehension, believability, personal relevance, and likes and dislikes.
Questions tailored for the communication message to address any special concerns the producer
of the message may have (e.g., the effect and /or appropriateness of graphic depictions of
negative health outcomes).

The methodologies planned for use in this submission will follow standard state-of-the-art
approaches adapted from marketing and communications research. In this context, the term
“testing” refers to testing messages, strategies, and communication materials, and should not be
confused with "pretesting of questionnaires" prior to their full-scale use.
The following methodologies will be used:
Individual In-depth Interviews: Individual in-depth interviews are used to elicit attitudes
and perceptions that offer insight into better understanding critical influences on people’s mental
models (i.e., belief structures), or for testing message concepts, draft materials, and
communication strategies. Individual in-depth interviews are ideal when the information in
question requires in-depth probing or when individual rather than group responses are considered
more appropriate. This methodology is appropriate for determining target audience attitudes,
beliefs, and feelings, particularly those addressing potentially sensitive or emotional topics. Indepth interviews are also cost-effective in eliciting comments on print materials. Individual indepth interviews can either be conducted on-line at a designated Internet location, conducted inperson, or conducted over the telephone. In some cases, respondents can be sent material in
advance, asked to read them, and told that someone will call to get their opinion. Respondents
for in-depth interviews are recruited from members of the target audience for the particular
message or print material being tested. They are also recruited based on meeting other
appropriate screening criteria, which may include their willingness to be interviewed. Specific
written instructions in the form of a screening questionnaire are used during the recruitment
process. The interviews themselves are conducted by skilled interviewers who follow a
prescribed discussion outline. A minimum of 20-25 respondents are queried in tests using this
method of data collection; in a standard test, 50 interviews are conducted. In-depth interviews
are generally 30 to 45 minutes in length.
Intercept Interviews: Intercept interviews involve positioning interviewers at a central point
or location commonly used by individuals who make up the desired target audience. In an
intercept interview, people are randomly asked to participate in a message or strategy survey
about health. This methodology is usually employed when reactions are desired on a nonsensitive topic over a fairly short period of time. Intercept interviews may be done in person or
by telephone:


Central Location Intercepts. In the case of central location intercepts, the point of
interception could be some place such as a shopping mall or health clinic. After several
initial screening questions, participants are asked a series of questions, often in relation to
draft material they have been shown. This is followed by an interview of up to 20
minutes in length, depending on the nature and complexity of the topic and material
presented.



Telephone Intercept Interviews. With telephone intercepts, the intercept point is usually
the terminus of a call-in number (e.g., a 1-800 number). The pertinent FDA Center will
have one or more 1-800 numbers to receive calls on drug-product-related topics. As an
2

add-on to some of these calls, several brief questions about communication messages can
be asked, providing valuable input on whether FDA messages are understood and
accepted. These telephone intercept interviews, conducted on a simple random sample
basis, may range in length from less than a minute to several minutes.
Focus Groups: Focus groups, or group interviews, are used to obtain insights into target
audience perceptions, beliefs, and attitudes in the early stages of the communication process (i.e.,
in concept, strategy, and materials development). Focus groups are usually composed of 8 - 10
people who have characteristics similar to the target audience or to subgroups of the target
audience. The groups are conducted by a professional moderator who keeps the session on track
while allowing respondents to talk openly and spontaneously. The moderator uses a loosely
structured discussion outline, which allows him/her to change direction as the discussion unfolds
and new topics emerge. Focus groups are valuable in exploring consumer reactions to message
concepts before additional resources are put into their development.
Self-Administered Surveys: Self-administered surveys can be used to validate belief
structures derived from mental models research results, or to test drafts of FDA concepts and
materials. Surveys can either be mailed to respondents along with the draft being tested,
accessed on-line at a designated Internet location, or distributed to respondents gathered at a
central location. Internet surveys can be administered to online panels and may use experimental
designs to test hypotheses about the relative efficacy of communication messages. Willing
participants recruited from the target audience are randomly assigned to treatment or control
conditions. After being exposed to the communication, participants are asked to provide
information about their recall, emotional and cognitive reactions, beliefs, and behavioral
intentions. When this method of pretesting is used, having at least 100 respondents per condition
is desirable. In some cases, a follow-up survey may be conducted within a reasonable timeframe
(e.g., one week) with those who complete the baseline survey.
When central location surveys are employed, people frequenting an expected location of the
target audience are randomly stopped and screened to determine whether they meet the predetermined selection criteria. When this method of pretesting is used, at least 50 respondents are
included; using approximately 100 respondents is desirable. When projects require in-depth
responses on a few questions, self-administered surveys may be enhanced through remote
oversight by an interviewer with the capability of adding pop-ups onscreen to probe answers by
respondents. FDA does not intend to generate nationally representative results or precise
estimates of population parameters using these surveys.
Gatekeeper Review: When public education materials are distributed to their intended
audiences through health professionals or intermediary organizations, the input of these groups
to the concepts and materials is critical to a program's success. As a result, these intermediaries,
or gatekeepers, are often queried through self-administered questionnaires as part of the testing
process. Twenty-five to 50 gatekeepers are generally interviewed as part of a materials test. The
questionnaire information is complementary to that requested of target audience members.

3

Omnibus Survey: An omnibus survey is a telephone or mail-back interview survey in
which different organizations add questions to a single questionnaire, thereby sharing the cost.
This technique uses random selection from an address list to speak to approximately 1,000
respondents with the intent of having a reasonable degree of diversity in key demographic
characteristics such as age, gender, education and race or ethnicity. FDA does not intend to
generate nationally representative results or precise population parameters using these surveys.
Because these surveys are conducted on a weekly or bi-weekly basis, they are an efficient way to
obtain test data from a larger number of consumers in a short period of time. To get such a quick
and cost-effective turnaround, however, the vendor can make up to only four callbacks, resulting
in a lower response rate than for custom surveys, where additional callbacks are made.
Generally, for the most cost-effective approach, computer-assisted telephone interviewing
(CATI) is used.
Because of the increase in the prevalence of adults who principally use wireless telephones,
random selection from an address-based list, which increases access to wireless users, shall be
used as appropriate to limit coverage biases that could potentially be introduced by landline-only
telephone surveys. Results from the July-December data collection of the 2009 National Health
Interview Survey suggest that one out of every six (16.3%) American adults live in mostly
wireless households.1 Moreover, ethnic differences have been diminishing. The percentage of
adults living in mostly-wireless households in late 2009 varied between 16.2% for non-Hispanic
black adults to 19.6% for non-Hispanic Asian adults.
FDA recognizes that, for the purposes of this generic clearance, omnibus surveys should only be
used for internal purposes (e.g., to better understand consumer perceptions before going public
with marketing campaigns). Omnibus survey results will not be used for program evaluation
purposes, or to make policy or regulatory decisions.
For all methodologies, professionally recognized procedures will be followed in each
information collection activity to ensure high quality data. Examples of these procedures include
the following:
A minimum of ten percent of telephone interviews will be monitored by supervisory
staff;
Data from mail or paper-and-pencil surveys will be computerized through scannable
forms or checked through double-key entry;
Observers will monitor focus groups, and focus group proceedings will be recorded; and
Data submitted through online surveys will be subjected to statistical validation
techniques (such as disallowing out-of-range values).

1

http://www.cdc.gov/nchs/data/nihs/earlyrelease/wireless201005.htm#wireless

4

All data collection and analysis will be performed in compliance with OMB, Privacy Act, and
Protection of Human Subjects requirements.
3. Methods to Maximize Response Rates and Deal with Non-response
In the case of data collection activities that involve interviews or telephone, mail, and in-person
surveys, several procedures proven effective in previous studies will be used to maximize
response rates:
Potential respondents will be informed about the importance of these studies and encouraged to
participate through a variety of methods, including letters of support from key individuals.
Experienced, highly-trained staff will moderate all focus groups and conduct all interviews and
surveys.
Interviewers will participate in thorough training sessions. Training topics will include study
objectives, question-by-question reviews of data collection instruments, strategies for engaging
respondents, role playing, and techniques for fostering respondent cooperation and survey
completion.
Well-defined conversion procedures will be established. If a respondent for a survey declines to
be interviewed, a member of the contractor's conversion staff will contact the respondent to
explain the importance of their participation. Conversion staff members are highly experienced
telephone interviewers whose style and persuasive abilities have demonstrated success in
eliciting cooperation. They receive a pay differential to acknowledge these skills, which also
serves as an incentive to the interviewer pool, whose completion rates are carefully monitored to
assess their qualifications to serve as conversion staff.
For telephone interviews, outgoing calls that result in a disposition of “no answer,” a busy signal,
or an answering machine will be automatically rescheduled for subsequent attempts. Up to 20
outgoing calls to a given number with dispositions of the sort listed will be made before
declaring it a non-response. Should a respondent interrupt an interview for any reason, such as
needing to attend to a personal matter, the interviewer will reschedule or, in the case of telephone
surveys, a predictive dialer will automatically reschedule the interview for completion at a later
time.
Fielding for telephone and mail surveys will occur over at least a six-week period. Based on past
experience, this time frame will allow the contractor to reach individuals who are on vacation,
out of the home during irregular periods, have a temporarily disconnected telephone, or who are
not answering the phone for some other reason. Interview staff will be able to provide
respondents with the name and telephone number of an official at FDA. This official will
confirm with respondents the importance of their participation.

5

A dedicated toll-free number will be established at FDA or a contractor’s office to allow
potential respondents to hear a pre-recorded message to confirm a study’s legitimacy. For mail
surveys, a number of techniques will augment response rates:


A self-addressed, stamped return envelope will be enclosed with each survey.



Surveys will be mailed to respondents using stamps instead of metered postage labels.



Creative and attractive graphics will be used to attract the attention of respondents (e.g.,
different colored paper for successive survey iterations).



Hand-signed cover letters will be sent with each survey.



Follow-up mail (up to 7 mailings) or phone contacts (up to 20 call-backs) will be made to
encourage participation; participant objectivity will be encouraged by reminding
participants about the importance of providing both negative and positive feedback.



Respondents will be allowed the option of faxing back completed surveys (and possibly
offered the option of completing the survey online).

4. Test of Procedures or Methods to be Undertaken
Before each information collection is implemented, a contractor will pilot test the instrument(s)
and method of data collection. Lessons from the pilot test will be identified, and changes as
necessary will be incorporated into the instrument and method. All pilot tests will involve no
more than nine individuals unless OMB clearance is sought for more than nine.
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing
Data
We held discussions on the specifications for FDA generic clearances for communications
testing of tobacco products and medical devices (OMB 0910-0674 and OMB 0910-0678,
respectively) that have specifications that are identical to those in this generic clearance.

6