Supporting Statement Part B Quantitative Testing for the Dev of FDA Communications 2019

part B. Statistical Methods

1. Respondent Universe and Sampling Methods

The participant universe for the quantitative testing may include a wide range of consumers and other FDA stakeholders such as producers and manufacturers who are regulated under FDA-regulated food and cosmetic products, dietary supplements, and animal food and feed. The samples will be selected from an appropriate target audience, which could be online panel members or samples obtained through in-person intercepts (e.g., in malls and schools). Sampled panel members will receive an email inviting them to participate in the study. Sampled intercept participants will be invited to participate in the study by trained recruiters. Participants who choose to participate will complete a screener and, if eligible, receive a questionnaire. Completed interviews will be monitored to ensure samples are diverse in terms of age, gender, education, and ethnicity/race. Other sources of samples may also be employed.

Studies under this collection will rely on quantitative methods and use convenience samples rather than probability samples. Therefore, the results are not intended to yield results that are statistically projectable, nationally representative or precise estimates of population parameters. When probability samples are employed (such as through an online panel), representative estimates to the national population will not be made.

2. Procedures for the Collection of Information

For most studies submitted under this generic clearance, FDA will use an online panel or market research vendor experienced in conducting in-person intercepts. When participants are recruited through online panels, the vendor will send email invitations to the target audiences using their market research panel. Each invitation will contain the OMB expiration date, the study title, the estimated amount of time for participation, and instructions for accessing the secure website. Once a participant enters the secure web site, a brief introduction will be presented informing the participant of the confidential and voluntary nature of the survey. Individuals who consent to participate in the survey will be able to access the survey by clicking on the link to the survey URL.

When participants are recruited using mall intercepts or intercept methods they will be invited to complete a screener to determine study eligibility. During the screening process, potential participants will be asked for personal information, including demographics and behaviors surrounding FDA-regulated food and cosmetic products, dietary supplements, and animal food and feed. This information will be used to determine eligibility. Similar to the process of using online panels, eligible participants will be told the title of the study, the estimated amount of time for participation, (if applicable) amount of incentive provided for successful completion of the survey and instructions for completing the study. Once a participant agrees and begins the study, a brief introduction will be presented informing the participant of the OMB expiration date and confidential and voluntary nature of the survey.

Questionnaires will be designed to measure responses to the study stimuli and collect demographic and other behavioral information from the participants. Participants may be asked to view study stimuli prior to responding to the questionnaire.

Summary of Protocol

• Survey screener — confirm eligibility.

• Random assignment to view stimuli, such as images, videos or messages (if appropriate).

Participants provide information on questionnaire in response to stimuli.

Potential Measures

Key Outcomes (measured post-exposure and/or at follow-up):

• Recall (aided and/or unaided)

• Reactions

• Beliefs

• Intentions

• Knowledge

Covariates and controls:

Age, gender, race, SES (income and education)

Analysis Plan (example for experimental studies)

Note: specific analysis plans will be developed according to the study protocol, and will be guided by the study purpose, design, research questions, and hypotheses tested (if appropriate). The following example is a typical experimental study design, one which would be tailored based on the specific study protocol.

Tests:

1. Tests of treatment effect: comparison of outcomes in treatment group(s) to control

2. Contrasts between treatment groups

Each quantitative test will be adequately powered to test the primary research hypothesis; key outcomes for the groups viewing a different version of the stimulus will be tested for statistically significant differences.

Unusual Problems Requiring Specialized Sampling Procedures

No specialized sampling procedures are expected, but if they are necessary, they will be described in the individual submissions.

Use of Periodic Data Collection Cycles to Reduce Burden

Each submission will be a one-time data collection effort.

3. Methods to Maximize Response Rates and Deal with Non-response

Experience with online studies suggests that about 15 percent of those who are sent survey invitations will complete a study. Despite, this low baseline value, FDA will implement several procedures to increase participation wherever possible. We will conduct cognitive interviews and pretests to help improve understandability of the questionnaire to reduce participant burden and to enhance interview administration. We will keep the questionnaire at a reasonable length to minimize break-offs. Tested recruitment and data collection procedures will be used to maximize cooperation and to achieve the desired response rates.

4. Tests of Procedures or Methods to be Undertaken

FDA may conduct cognitive interviews to evaluate and refine the draft questionnaire or study materials or stimuli. If the number of cognitive interview participants exceeds nine members of the public, the Agency will submit the cognitive interview protocol for review. Cognitive interviews help identify areas where the materials are ambiguous, burdensome, or confusing for participants. Study materials will be revised accordingly.

Additionally, FDA will conduct pretests to thoroughly test programmed or paper questionnaires and stimuli. Study materials will be revised based on the pretest findings.

5. Individuals Consulted on Aspects and Individuals Collecting and/or Analyzing Data

In general, FDA plans to use a contractor for recruiting, survey programming, and study administration. If needed, the contractor will also provide an analysis of the data and provide a summary report.

3