Supporting Statement: Part B
LGBT
Campaign: Wave 1B Online Quantitative Study Designed to
Prevent
Young Adult Tobacco Use
Statistical Methods
The one-time actual burden figures are listed in Exhibits 1 & 2, Part A.
Respondent Universe and Sampling Methods
The primary outcome of this study will be based on a non-random sample of 1,050 Lesbian, Gay, Bisexual and Transgender (LGBT) young adults who are 18-24 years old who smoked cigarettes on 1-29 days out of the past 30 days, but not every day. Participants will be randomly assigned to the control group, where they will not view any advertisements (ads), or to the ad-viewing group, where they will be asked to view one of three randomly assigned ad and provide quantitative and qualitative feedback about the ad. All enrolled participants will be asked to answer questions about their knowledge, attitudes, and beliefs about tobacco use as a check for potential unintended consequences of viewing the video ads.
The study is a cross-sectional design, and participants will be enrolled via targeted advertisements on social media, such as Facebook and Instagram. The screening criteria are based on age, number of days smoked cigarettes in past 30-day period, residence within the geographic target area, past participation in LGBT health-specific research, participation in LGBT culture, valid email address, and personal or close family or friends’ employment in the tobacco industry or with a market research company, an advertising agency, or a public relations firm.
As this study is considered part of formative research for campaign development and planning, these methods are not intended to generate nationally representative samples or precise estimates of population parameters. The sample drawn here is designed primarily to provide information on the perceived effectiveness of three video ads for FDA’s LGBT young adult campaign and to identify any potential unintended consequences of viewing the ads.
Sampling Methods
Advertising through social media platforms can help increase the diversity of the study sample and reach subgroups within the LGBT population that might be missed by other forms of recruitment (Chou et al., 2009). Data also suggest that social media engagement among LGBT 18-24 young adults is high. For example, a recent online survey found that 90% of LGBT respondents’ ages 18-24 indicated active use of Facebook and 62% indicated active use of Instagram (CMI, 2015). Social media ad targeting can be adjusted in real time, allowing the research team to react to shifting recruitment needs if a particular demographic is lacking in the overall sample. Social media advertisements may be deployed based on factors such as age, geographic location within or around a target campaign city, and interest in LGBT-related keywords. Respondents who click on any social-media sponsored ad will be redirected to the screener welcome page. The copy testing questionnaire will commence immediately after screener qualification and informed consent.
Qualified participants who complete the screener but do not complete the copy testing questionnaire soon after will be contacted via email with a reminder to complete the Copy testing questionnaire. This reminder will be in the form of an email message containing the link to the copy testing questionnaire, and up to two reminders will be sent to respondents within the study period.
The screener is designed to determine whether a respondent is qualified to complete the copy testing questionnaire. The screener will include the following major components.
Sexual orientation and gender identity: participants must self-identify as Lesbian, Gay, Bisexual, Transgender or Other non-straight/non-questioning gender or sexual minority, such as queer, pansexual or omnisexual to qualify.
Age: participants must be at least 18 years old and will not be invited for participation if they are older than 24 years old. Self-reported birth month and year will be asked, along with self-reported age, as an increased effort to mitigate potential fraud. The system-calculated age and self-reported age must match in order for the participant to qualify.
Past 30-day tobacco use: one item assessing non-daily smoking will be used as inclusion criteria, participants must be non-daily smokers, meaning they have smoked cigarettes on 1-29 days out of the past 30 days, but not every day.
LGBT cultural engagement items: participants must indicate engagement with the LGBT culture by responding in the affirmative to at least one of three LGBT cultural items.
Potential bias measures: participants must not have participated in LGBT health-specific research within the past 6 months. Participants must not work for nor have close friends or family who work for the tobacco industry, a market research company, an advertising agency, or a public relations firm.
Personal identifying information (email address and IP address): some select personally identifiable information will be necessary for incentive delivery, and to avoid fraudulent or duplicative entries. If the same email is used for two survey attempts, it will be automatically rejected. If two emails have the same IP address, the duplicate entry will be removed.
Demographics: Race/ethnicity and DMA will be collected to ensure a diverse participant sample. Zip code is being collected to ensure that participants are within determined geographic targets for the study (DMAs). Participants will be asked to provide their zip code at two different points within the Screener, and must match to qualify. Zip codes will be used to identify DMA of residence (e.g. San Diego). The de-identified dataset will include the DMA of residence and not the zip code. Post-hoc analysis may be conducted to understand potential variations between DMAs.
Incentive:
Participants who complete and submit the Copy testing questionnaire will receive a $15 online gift card via email within 72 hours of submission.
Sample Size
To obtain a final sample of 1,050 enrolled participants, we estimate that we will need to screen approximately 3,150 potential respondents. This estimate is based on previous research conducted within this target audience using similar methodologies.
Procedures for the Collection of Information
Participants will complete both the screener and copy testing questionnaire online using their own device. In addition, email addresses are collected at the conclusion of the screener, enabling researchers to contact participants with a reminder email if they are not able to complete the copy testing questionnaire at the time of screener completion. Participants will be sent up to two email reminders reminding them to complete the copy testing questionnaire.
Qualified participants will be randomly assigned to either an ad-viewing or non ad-viewing group. Participants in the ad-viewing group will view one of three rough-cut ads. Ad-viewing participants will then be prompted to complete a questionnaire designed to assess whether the advertisement provides an understandable and engaging message about the harms of tobacco use. Advertisements shown during copy testing will on average be approximately 30-45 seconds in length. Participants will have the option to replay an ad once it is complete.
All participants will answer a series of questions about tobacco use and exposure, and general questions about their attitudes and beliefs about the harms of tobacco use, which will be used to assess potential unintended consequences. Participants who do not view any advertisements are being included to measure for unintended consequences. Exhibit 5 indicates the variables to be assessed during the questionnaire and the participant groups that will be exposed to these survey items.
Exhibit 5. Structure of the Copy Testing Process and Questionnaire
Action or Variable |
Description |
Presented to Ad-Viewing Participants |
Presented to Control Participants |
Ad exposure |
Each of the ad-viewing participants will view one of three randomly assigned video ads. |
X |
|
Tobacco use and peer tobacco use |
Items on household tobacco use, peer cigarette use, and participant past 30-day tobacco use. |
X |
X |
Perceived ad effectiveness |
Items to assess ad effectiveness, presented immediately following the video ad. |
X |
|
Tobacco-related attitudes, beliefs and risk perceptions |
Items tailored to align with the tobacco facts chosen for inclusion in the video ads. Items assessing participants’ attitudes, beliefs, and risk perceptions related to tobacco use. |
X |
X |
Methods to Maximize Response Rates and Deal with Nonresponse
General Methods to Reduce Non-Response & Drop-Off
Several features of this study have been designed to maximize participant response rate and Questionnaire completion across recruitment methods.
Incentives: As participants often have competing demands for their time, incentives are used to encourage participation in research. Numerous empirical studies have shown that incentives can significantly increase response rates in cross-sectional surveys and reduce attrition in longitudinal surveys (e.g., Abreu & Winters, 1999; Castiglioni, Pforr, & Krieger, 2008; Shettle & Mooney, 1999; Singer, 2002). In this study, we will use an incentive in the form of a $15 online gift card per participant to equalize the burden placed on participants with respect to their time, and to provide enough motivation for them to participate in the study rather than another activity.
Reminders: A series of reminders will be utilized. Qualified participants who complete the screener but do not complete the copy testing questionnaire will be contacted via email with a reminder to complete the copy testing questionnaire. This reminder will be in the form of an email message containing the link to the copy testing questionnaire, and up to two reminders will be sent to respondents within the study period. These reminder email messages will include a unique link to the survey to enable participants to easily complete the questionnaire. These reminders are intended to decrease non-response by ensuring participants have convenient access to complete the questionnaire, and by encouraging participants who do not initially complete the questionnaire to complete it before the conclusion of data collection.
Mobile Phone Responsiveness: Both the screener and questionnaire will be optimized for performance on a mobile phone, in addition to other electronic devices such as tablets and laptops. This is especially important as research suggests that the LGBT population is known to be “early adopters” of technology such as smart phones and tablets (CMI, 2015). Based on this information and Rescue’s previous experience with online data collection, we expect that many LGBT young adults will attempt to complete the screener and questionnaire on a mobile phone. Ensuring that the surveys are optimized for mobile phone performance will reduce non-response and drop-off due to technical issues related to compatibility of the instruments with the mobile phone format.
Methods to Reduce Social Media Recruitment Non-Response & Drop-Off
Several methods specific to social media recruitment and data collection will be implemented to encourage completion.
Targeted Social Media Advertising: The social media advertisement campaign (Attachment E) will utilize specially developed ads and take advantage of social media platform targeting capabilities to deliver relatable advertisements to LGBT young adults who are most likely to qualify for the study. Social media ads will feature images and copy designed to appeal to LGBT young adults, in order to increase interest and the likelihood of completion. Additionally, these social media ads will be delivered to young adults most likely to be in the target audience via targeted advertisement delivery methods available in social media platforms, such as Facebook and Instagram. Using features of the social media platform such as demographic information from profiles and “likes” or hashtags, advertisements can be delivered to young adults who are most likely to be within the age range, living within the determined geographic targets, and those who identify with and engage in LGBT culture. This methodology will ensure that social media advertisements are delivered to LGBT young adults who are most likely to qualify, and that those participants feel a personal connection with the study, as a way to increase response rates.
Social Media Advertising Responsiveness: Adjustments in social media ads, including ad image and copy, will be utilized to maximize response. At any given time, low-performing ads will be removed from circulation. Ad performance will be assessed and compared overall, and segmented by device (i.e. mobile or desktop). Copy and images included in Attachment E will be introduced to the campaign and recombined in unique ways in response to ad performance. Additionally, advertising targeting may be revised to expand interest-based targeting to increase the sampling pool. These methods are benefits of the social media advertising recruitment approach, and allow researchers to react in real time to low response and qualification rates by adjusting the social media advertising and targeting to better reach the target audience and improve response rates.
Test of Procedures or Methods to be Undertaken
The campaign contractor Rescue SCG will conduct rigorous internal testing of the electronic survey instruments prior to their fielding. Trained researchers will review the screeners and questionnaire to verify that instrument skip patterns are functioning properly, delivery of campaign media materials is working properly, and that all survey questions are worded correctly and are in accordance with the instrument approved by OMB.
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
The following individuals inside the agency have been consulted on the design of the copy testing plan, survey development, or intra-agency coordination of information collection efforts:
Tesfa Alexander
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993-0002
Phone: 301-796-9335
E-mail: Tesfa.Alexander@fda.hhs.gov
Gem Benoza
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993
Phone: 240-402-0088
E-mail: Maria.Benoza@fda.hhs.gov
Leah Hoffman
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993
Phone: 240-402-7134
E-mail: Leah.Hoffman@fda.hhs.gov
Janine Delahanty
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993
Phone: 240-402-9705
E-mail: Janine.Delahanty@fda.hhs.gov
The following individuals outside of the agency have been consulted on questionnaire development.
Mayo Djakaria
Rescue Social Change Group
660 Pennsylvania Avenue SE, Suite 400
Washington, DC 20003
Phone: 619-231-7555 x 120
Email: mayo@rescueagency.com
Dana Wagner
Rescue Social Change Group
660 Pennsylvania Ave SE, Suite 400
Washington, DC 20003
Phone: 619-231-7555 x 331
Email: dana@rescueagency.com
Xiaoquan Zhao
Department of Communication
George Mason University
Robinson Hall A, Room 307B
4400 University Drive, 3D6
Fairfax, VA 22030
Phone: 703-993-4008
E-mail: xzhao3@gmu.edu
References
Abreu, D. A., & Winters, F. Using monetary incentives to reduce attrition in the survey of income and program participation. In: U.S. Census Bureau. Proceedings of the Survey Research Methods, 1999: 533-538.
Chou, W. S., Hunt, Y. M., Beckjord, E. B., Moser, R. P., and Hesse, B. W. Social Media Use in the United States: Implications for Health Communication. J Med Internet Res. 2009; 11(4), e48. doi:10.2196/jmir.1249
Community Marketing Inc., 9th Annual LGBT Community Survey. www.CommunityMarketinginc.com. Published in 2015. Accessed on Sept 7, 2016.
Castiglioni, L., Pforr, K., Krieger, U. The effect of incentives on response rates and panel attrition: Results of a controlled experiment. Surv Res Methods. 2008; 2(3):151-158.
Singer, E. The use of incentives to reduce nonresponse in household surveys. In: Wiley, ed. Survey nonresponse. 2002; 51:163-177.
Shettle, C., & Mooney, G. Monetary incentives in US government surveys. J Off Stat. 1999;15(2): 231.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-26 |