Experimental Study on Measuring Consumer Comprehension of Displays of Harmful and Potentially Harmful Constituents (HPHCs) in Tobacco Products and Tobacco Smoke
0910-NEW
Supporting Statement Part B
The respondent universe for this study is (1) youth aged 13 to 17 years old who are current cigarette smokers, susceptible to initiation of cigarette smoking, or current smokeless tobacco users; (2) young adults aged 18 to 24 years old who are current cigarette smokers, are smokeless tobacco users, or non-users of tobacco products; and (3) older adults aged 25 years or older who are current cigarette smokers, are smokeless tobacco users, or non-users of tobacco products.
Study participants will be recruited from a national online panel of adults managed by Lightspeed. Lightspeed will send panel members an email inviting them to participate in the study. Adolescent children of adult panel participants will be invited to complete the survey through an email invitation to their parents asking for their consent to solicit their child’s opinions. Panel members and children of panelists who choose to participate will complete the questionnaire. Completed interviews will be monitored to ensure samples are diverse in terms of age, gender, ethnicity/race, education, and socioeconomic status. We estimate 1,500 youth and 3,000 young adults/adults will complete the survey for a total of 4,500 respondents.
We do not intend to generate nationally representative results or precise estimates of population parameters from the experimental study; generating a representative sample of the size necessary for this study, using random digital dialing or another similar method, would be cost prohibitive. The study will use convenience samples rather than probability samples. Despite the attempt to match the study’s sample and the respondent universe in five demographic characteristics, matching is used solely to produce a sample with a reasonable degree of diversity in key demographic characteristics.
This section provides an overview of the study procedures, provides information on the degree of accuracy required for the study, and discusses the estimation procedures. There are no unusual problems requiring specialized sampling procedures.
For the information collection, Lightspeed will send email invitations to the target audiences using their market research panel. Adult (aged 18 or older) panel members will be sent an email directly inviting their participation in the study and instructions for accessing the secure web site for the survey. Youth will be invited to participate through an email invitation to an adult panel member who has indicated in their panel profile that they have a child in the eligible age range. Parents or guardians will be asked to provide permission before allowing their child to participate. Once a panel member or child of a panel member enters the secure web site, they will be presented with a brief introduction informing the participant of the confidential and voluntary nature of the study. Next, they will access the screener to determine eligibility based on the study inclusion and exclusion criteria. Those respondents who are determined to be eligible to participate and provide consent to participate will then be randomly assigned to one of six study conditions and complete the survey. The survey is estimated to take 20 minutes to complete.
If panelists decline, they will be categorized as non-respondents. Panelists not eligible to complete the survey will be categorized as ineligible.
For the purposes of estimating statistical power, we assumed a sample size of 4,500 subjects will be randomized in equal allocation (n = 750) to each of six arms corresponding to six different HPHC information formats. The primary outcomes of the study are domain score for Understanding, computed as the proportion of correct responses across the 10 items, and domain score for Misleadingness, computed as the proportion of misled responses across the 15 items. Secondary outcome is the domain score for Harm Perception, computed as the average of the Likert scores across the eight items.
Statistical power to detect significant changes (α = .05) in pre- and post-exposure domain scores for Understanding, Misleadingness, and Harm Perception was estimated via Monte Carlo simulation. Power to detect statistically significant effects is based on the assumption that items under each domain are independent and identically distributed and no correlation exists between pre- and post-exposure responses for each subject. Power to detect significant differences in domain scores between pre- and post-exposure are provided by HPHC display format in Table 1. Overall, power to detect changes for primary outcomes Understanding and Misleadingness scores are >99.9% for all display formats. Power to detect changes in Harm Perception scores in formats 1 through 6 are 40%, 86%, 90%, 97%, 75%, and 5%, respectively.
Table 1. Power to Detect Median Change in Pre- and Post-exposure Domain Scores at α = .05 by Format
HPHC Format |
Understanding |
Misleadingness |
Harm Perception |
Format 1 (OSU) |
>99.9% |
>99.9% |
40% |
Format 2 (UNC Spaghetti) |
>99.9% |
>99.9% |
86% |
Format 3 (UNC Bubbles) |
>99.9% |
>99.9% |
90% |
Format 4 (FDA) |
>99.9% |
>99.9% |
97% |
Format 5 (Infograph) |
>99.9% |
>99.9% |
75% |
Format 6 (Wildcard) |
>99.9% |
>99.9% |
5% |
Based on the results presented in Table 1 and given sample sizes of 750 subjects per display format, there is sufficient power at α = .05 to detect significant differences between pre- and post-exposure for primary outcomes Understanding and Misleadingness domain scores.
Statistical analyses will be conducted to address the study’s primary research questions. Following the recommendation of the U.S. Food and Drug Administration’s (FDA’s) Center for Tobacco Products’ (CTP’s) Office of Science (Yang, 2017), analytic procedures will be based on nonparametric tests of statistical significance. Participant Understanding of the information conveyed in the HPHC formats and Harm Perceptions will assessed with the Wilcoxson signed rank test. The Wilcoxon test is the nonparametric equivalent of the paired or dependent sample t test and the appropriate analysis to compare differences derived from the same population when the dependent variable is ordinal or continuous. It is used to assess differences from matched pair designs or repeated measures. Misleadingness of the HPHC formats will be assessed with the Mann-Whitney U test. The Mann-Whitney U test is the nonparametric equivalent to the independent t test and the appropriate analysis to compare differences that come from the same population when the dependent variable is ordinal or continuous.
Experience with online experimental studies suggests that about 10% of adults and 6% of adolescents who are sent survey invitations will complete a study. To maximize participation, we will incorporate best practices from similar online surveys into our data collection procedures. These include:
Conducting a pretest of nine participants to ensure the questionnaire is understandable and reduce participant burden.
Implementing a soft launch of the online survey to a small number of selected panel members to detect and resolve any technical difficulty.
Keeping the questionnaire at a reasonable length to minimize break-offs.
Including a brief introduction to the study that identifies FDA as the sponsor, states the purpose of the study, and provides toll-free telephones numbers for participants to call RTI with any questions about the study or their rights as a study participant.
Inviting panel members who appear to be eligible based on their member profile. As part of the registration process, panelists provide information about a range of sociodemographic characteristics, including age and smoking status that can be used to target particular groups. Lightspeed actively manages panelist profiles, requesting updated information on an ongoing basis to ensure that profile information is up to date.
Recruiting verified panelists. Lightspeed uses a double opt-in registration process whereby panelists are invited to participate and then must sign up through an opt-in confirmation email. This process protects against fraudulent account registrations and ensures that panelists are actively motivated to participate in surveys.
To minimize nonresponse, Lightspeed will conduct ongoing monitoring of response levels and drop-off rates. Non-respondents will receive an initial email invitation and up to two email reminders from Lightspeed requesting their participation in the survey. Lightspeed will work with RTI project staff to address any problems that arise throughout the course of the collection of information.
As with any study conducted using opt-in online panels, this study may be subject to nonresponse bias. However, the impact of such biases on study conclusions is mitigated by the experimental design of the study, which ensures random assignment of participants to condition. As in randomized clinical trials, study conclusions are based on comparisons between experimental and control conditions. We do not anticipate significant item nonresponse and thus have no plans to use imputation procedures. All analyses will be conducted using Stata 14.1 with specific estimators determined by the measurement of the outcome variable and model used.
RTI will conduct a small pretest to test the questionnaire and usability of the web-based study interface. This pretest will be in-person cognitive interviews with nine participants in the Washington, DC, metro area recruited through a focus group facility. At the end of the pretest, all strategies, algorithms, and programs for sampling, survey administration, and data compilation will be tested, validated, and readied for launch of the online data collection. The questionnaire and study protocol will be revised, if necessary, based on the pretest findings. Additionally, Lightspeed will begin data collection with a soft launch during which they will send invitations to a small subset of panel members to ensure the online survey is working properly.
The following individuals inside the agency have
been consulted on the design and
statistical
aspects of this information collection as well as plans for data
analysis:
Jennifer Bernat
Center for Tobacco Products
U.S. Food and
Drug Administration
Silver Spring, MD 20993-0002
Tel:
240-402-1078
Jennifer.Bernat@fda.hhs.gov
Katherine Margolis
Center for Tobacco Products
U.S. Food
and Drug Administration
Silver Spring, MD 20993-0002
Tel:
240-402-6766
Katherine.Margolis@fda.hhs.gov
The following individuals will conduct data collection:
Jessica Pepper
RTI International
3040 E. Cornwallis
Road
Research Triangle Park, NC
27709
Phone: 919-316-3180
jpepper@rti.org
Jennifer Alexander
RTI International
3040 E. Cornwallis
Road
Research Triangle Park, NC
27709
Phone: 301-770-8219
jalexander@rti.org
| File Type | application/msword |
| File Modified | 2019-07-02 |
| File Created | 2019-07-02 |