Generic Memo Part B

Supporting Statement: Part B

0910-0810

Multicultural Campaign: Wave 3 Online Quantitative Study of Reactions to Rough-Cut Advertising Designed to Prevent Multicultural Youth Tobacco Use

Statistical Methods

The one-time actual burden figures are listed in Exhibits 1 & 2, Part A.

1. Respondent Universe and Sampling Methods

The primary outcome of this study will be based on a non-random sample of 593 youth ages 13-17 years old who are either tobacco users or open non-users, and who are influenced by the Hip Hop peer crowd. Participants will be randomly assigned to the control group, where they will not view any advertisements (ads), or to the ad-viewing group, where they will be asked to view one of three randomly assigned ads and provide quantitative and qualitative feedback about the ad. All enrolled participants will be asked to answer questions about their knowledge, attitudes, and beliefs about tobacco use as a check for potential unintended consequences of viewing the video ads.

The study is a cross-sectional design, and participants will be enrolled via targeted advertisements on social media, such as Facebook and Instagram. The screening criteria are based on age, tobacco use status, Hip Hop peer crowd influence, residence within the geographic target area, valid email address, personal or close family or friends’ employment in the tobacco industry, and past participation in tobacco research.

As this study is considered part of formative research for campaign development and planning, these methods are not intended to generate nationally representative samples or precise estimates of population parameters. The sample drawn here is designed primarily to provide information on the perceived effectiveness of three video ads for the FDA’s Multicultural Campaign and to identify any potential unintended consequences of viewing the ads.

Sampling Methods

This study will utilize social media recruitment using targeted advertisements. Advertising through social media platforms, such as Facebook and Instagram, can help increase the diversity of the study sample and increase representation of traditionally underrepresented groups, including racial/ethnic minorities (Lane, Armin, & Gordon, 2015; Graham et al., 2008). Data also suggest that social media engagement among multicultural youth ages 13-17 is high. For example, a recent survey found that 66% of US youth ages 13-17 use Facebook, and 76% use Instagram, including 77% of Black and 80% of Hispanic youth (Lenhart et al., 2017).

For many social media platforms, ad targeting can be adjusted in real-time, allowing researchers to react to shifting recruitment needs if a particular demographic is lacking in the overall sample. Social media advertisements may be deployed based on factors such as age, geographic location, and interest in Hip Hop cultural pages or hashtags.

Respondents who click on any social media sponsored ad will be redirected to the screener welcome page. Upon screener completion, participants will be immediately notified if they qualify. All qualified participants will then be prompted to provide their parent/guardian’s email address, and their cell phone number if they prefer to receive the study link and reminders via text message (optional). Parent/guardian email address will be used to email the parental opt-out form to a potential participant’s parent or guardian. Qualified youth whose parents’ email does not bounce back and whose parents do not opt them out will be emailed and text messaged (if selected) a link to complete the questionnaire no less than 24 hours after the parental opt-out form is emailed to their parent/guardian, to allow sufficient time for parental-opt out to occur. Participants will provide informed assent/consent before they begin the questionnaire.

Qualified participants who complete the screener, whose parent email does not bounce back, and who are not opted-out by their parents/guardians will be contacted via email or text message (if selected) with a link to complete the questionnaire. Invited youth who do not complete the questionnaire within 24 hours of receiving the link will receive up to two reminders via email and text message (if selected) to complete the questionnaire.

The screener is designed to determine whether a respondent is qualified to complete the questionnaire. The screener is separated into two phases, with eligibility determined at the conclusion of Phase 1 in order to limit the amount of identifying information that will be collected from respondents who are not qualified to complete the questionnaire. The screener will include the following major components.

• Phase 1:

  • Demographic information: age (for verification of 13-17 age range), race/ethnicity, and sex;

  • Self-reported ever-use and number of cigarettes used in their lifetime;

  • Self-reported ever-use of other tobacco products (OTP);

  • Battery of questions to determine openness to cigarette and OTP use;

  • Questions related to previous tobacco research participation and tobacco industry affiliation;

  • Peer crowd influence;

  • Zip code (to ensure that participants are within determined geographic targets for the study);

  • Youth email address (to check against all current respondent data to avoid duplicates and reduce fraudulent activity); and

  • IP Address (collected automatically to reduce fraudulent activity and verify participant country of origin).

• Phase 2 (qualified youth only):

  • Parent email (to email the parental opt-out form to parents of eligible youth. Parental email address will not be used for any other purpose); and

  • Youth cell phone number (optional; to facilitate delivery of the questionnaire link by text message for youth who prefer this method. This functionality supports efficient data collection as 89% of US teens have or have access to a smartphone and 97% access the internet using mobile devices (Lenhart et al., 2017), making it likely that youth may desire to complete the questionnaire on a mobile device and therefore prefer to receive the link via text message).

Sample Size

To obtain a final sample of 593 enrolled participants, we estimate that we will need to screen up to 8,895 potential respondents. This estimate is based on previous research conducted within this target audience using similar methodologies.

2. Procedures for the Collection of Information

Participants will complete both the screener and questionnaire online using their own device. Email addresses and cell phone numbers (optional) are collected on the screener, enabling researchers to contact participants by email and text message (if selected) with the link to the questionnaire after the 24-hour parental opt-out waiting period, and with up to two reminders reminding them to complete the questionnaire.

Qualified participants will be randomly assigned to either an ad-viewing or non ad-viewing (control) group. Participants in the ad-viewing group will view one of three rough-cut ads. Ad-viewing participants will then be prompted to complete a series of questions designed to assess whether the advertisement provides an understandable and engaging message about the harms of tobacco use. Advertisements shown during copy testing will on average be 30-45 seconds in length. Participants will have the option to replay an ad once it is complete.

All participants will answer a series of questions about sensation-seeking, household tobacco use, peer cigarette use, and participant tobacco use, to allow comparison between ad-viewing and control groups to assess the effectiveness of the randomization process and ensure there are no confounding differences between groups. All participants will also answer questions assessing their knowledge, attitudes, and beliefs related to tobacco use, which will be used to assess potential unintended consequences by comparing responses between ad-viewing and control groups. Participants who do not view any advertisements are being included to measure for unintended consequences. Exhibit 5 indicates the variables to be assessed during the questionnaire and the participant groups that will be exposed to these survey items.

Exhibit 5. Structure of the Copy Testing Process and Questionnaire

Action or Variable	Description	Presented to Ad-Viewing Participants	Presented to Control Participants
Introductory questions	Household tobacco use, peer cigarette use, participant lifetime and past 30-day tobacco use, and participant sensation-seeking.	X	X
Ad exposure	Each of the ad-viewing participants will view one unique video ad.	X
Ad performance	Immediately following exposure to the video ad, ad-viewing participants will be presented with items to assess ad perceived effectiveness and emotional/attitudinal reactions to the ad.	X
Tobacco-related knowledge, attitudes, and beliefs; and attention check	Items assessing participants’ knowledge, attitudes, and beliefs related to tobacco use, along with two attention-check questions to assist in identifying participants not giving careful consideration to responses.	X	X

2. Methods to Maximize Response Rates and Deal with Nonresponse

Several features of this study have been designed to maximize participant response rate and questionnaire completion.

• Reminders: A series of reminders will be utilized to minimize drop off. Qualified participants who complete the screener but do not complete the questionnaire within 24 hours of receiving the link will receive up to two reminders via email and text message (if selected). These reminder emails and text messages will include a unique link to the survey to enable participants to easily complete the questionnaire. These reminders are intended to decrease non-response by ensuring participants have convenient access to complete the questionnaire, and by reminding participants who do not initially complete the questionnaire to complete it before the conclusion of data collection.

• Parental Opt-Out: A parental opt-out approach will be utilized. Due to the target population of this study, traditional active parental consent procedures would discourage participation among the very participants most appropriate for the aims of this study. Many youth who use tobacco or are at-risk for using tobacco are unlikely to seek out parental consent or have parents who provide active consent for their children’s participation in prevention programs (Levine, 1995; Pokorny et al., 2001; Unger et al., 2004; Severson and Ary, 1983). Demonstrating this point, there is consistent evidence of quantifiable differences in the characteristics of youth who participate in smoking cessation research when traditional written consent is required compared to waived parental consent, including participant demographics and smoking history (Kearney et al., 1983; Anderman et al., 1995; Severson & Ary, 1983). Utilizing a parental opt-out approach will remove a barrier that might discourage the target audience from completing the questionnaire, thereby reducing non-response.

• Mobile Phone Responsiveness: Both the screener and questionnaire will be optimized for performance on a mobile phone, in addition to other electronic devices such as tablets and laptops. This is especially important as 89% of US teens have or have access to a smartphone and 97% access the internet using mobile devices (Lenhart et al., 2017). Based on this information and Rescue’s previous experience with online data collection, we expect that many youth will attempt to complete the screener and questionnaire on a mobile phone. Ensuring that the surveys are optimized for mobile phone performance will reduce non-response and drop-off due to technical issues related to compatibility of the instruments with the mobile phone format.

• Token of Appreciation: In this study, we will use a token of appreciation in the form of redeemable gift credits in the equivalent of $10 total per participant to equalize the burden placed on participants with respect to their time, and to reduce overall burden by increasing questionnaire completion rates among youth who qualify on the screener. As participants often have competing demands for their time, a token of appreciation for participation in research is warranted. The use of a token of appreciation treats participants justly and with respect by recognizing and acknowledging the effort participants expend to participate. Numerous empirical studies have also shown that a token of appreciation can significantly increase response rates in cross-sectional studies and reduce attrition in longitudinal studies (e.g., Abreu & Winters, 1999; Castiglioni, Pforr, & Krieger, 2008; Shettle & Mooney, 1999; Singer, 2002). Additionally, evidence indicates that at-risk and multicultural populations may be particularly difficult to recruit and retain in health research (Hooven, Walsh, Willgerodt, & Salazar, 2011; Zand et al., 2006; Post, Gilljam, Bremberg, & Galanti, 2012; Patel, Doku, & Tennakoon, 2003; Siddiqui, Flay, & Hu, 1996; Giuliano et al., 2000; Murthy, Krumholz, & Gross, 2004), but that the use of a token of appreciation can be an effective means of recruiting and retaining participants from these populations (Martinson et al., 2000; Booker, Harding, & Benzeval, 2011; Caldwell, Hamilton, Tan, & Craig, 2010; Walter, Burke, & Davis, 2013).

4. Test of Procedures or Methods to be Undertaken

The campaign contractor Rescue has conducted rigorous internal review of the survey instruments. Trained researchers will review the screener and questionnaire to verify that instrument skip patterns are functioning properly, delivery of campaign media materials is working properly, and that all survey questions are worded correctly and are in accordance with the instrument approved by OMB.

4. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following individuals inside the agency have been consulted on the design of the copy testing plan, survey development, or intra-agency coordination of information collection efforts:

Gem Benoza

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20993

Phone: 240-402-0088

E-mail: Maria.Benoza@fda.hhs.gov

Mario Navarro

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20993

Phone: 240-402-4963

E-mail: Mario.Navarro@fda.hhs.gov

Matthew Walker

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20993

Phone: 240-402-3824

E-mail: Matthew.Walker@fda.hhs.gov

The following individuals outside of the agency have been consulted on questionnaire development.

Dana Wagner

Rescue Agency

2437 Morena Boulevard

San Diego, CA 92110

Phone: 619-231-7555 x 331

Email: dana@rescueagency.com

Carolyn Stalgaitis

Rescue Agency

660 Pennsylvania Avenue SE, Suite 400

Washington, DC 20003

Phone: 619-231-7555 x 313

Email: carolyn@rescueagency.com

Xiaoquan Zhao

Department of Communication

George Mason University

Robinson Hall A, Room 307B

4400 University Drive, 3D6

Fairfax, VA 22030

Phone: 703-993-4008

E-mail: xzhao3@gmu.edu

References

Abreu, D. A., & Winters, F. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. In: U.S. Census Bureau. Proceedings of the Survey Research Methods, 533-538.

Anderman, C., Cheadle, A., Curry, S., Diehr, P., Shultz, L., & Wagner, E. (1995). Selection bias related to parental consent in schoolbased survey research. Evaluation Review, 19, 663–674.

Booker, C.L., Harding, S., Benzeval, M. (2011). A systematic review of the effect of retention methods in population-based cohort studies. BMC Public Health, 11, 249.

Caldwell, P.H., Hamilton, S., Tan, A., & Craig, J.C. (2010). Strategies for increasing recruitment to randomised controlled trials: Systematic review. PLoS Medicine, 7(11), e1000368.

Castiglioni, L., Pforr, K., Krieger, U. (2008). The effect of incentives on response rates and panel attrition: Results of a controlled experiment. Survey Research Methods, 2(3), 151-158.

Graham, A. L., Milner, P., Saul, J. E., & Pfaff, L. (2008). Online advertising as a public health and recruitment tool: comparison of different media campaigns to increase demand for smoking cessation interventions. Journal of Medical Internet Research, 10(5), e50.

Giuliano, A.R., Mokuau, N., Hughes, C., Tortolero-Luna, G., Risendal, B., Ho, R.C.S., … McCaskill-Stevens, W.J. (2000). Participation of miniorities in cancer research. Annals of Epidemiology, 10(8 Suppl 1), S22-S34.

Hooven, C., Walsh, E., Willgerodt, M., & Salazar, A. (2011). Increasing participation in prevention research: Strategies for youth, parents, and schools. Journal of Child and Adolescent Psychiatric Nursing, 24(3), 137-149.

Kearney, K.A., Hopkins, R.H., Mauss, A.L., & Weisheit, R.A. (1983). Sample bias resulting from a requirement for written parental consent. Public Opinion Quarterly, 47, 96102.

Lane, T.S., Armin, J., & Gordon, J. S. (2015). Online recruitment methods for web-based and mobile health studies: A review of the literature. Journal of Medical Internet Research, 17(7), e183.

Lenhart, A., Kantor, L., Malato, D., Benz, J., Tompson, T., Zeng, W., & Swanson, E. (2017). Instagram and Snapchat are Most Popular Social Networks for Teens; Black Teens are Most Active on Social Media, Messaging Apps. Chicago, IL: The Associated Press-NORC Center for Public Affairs Research.

Levine, R.J. (1995) Adolescents as research subjects without permission of their parents or guardians: Ethical considerations. Journal of Adolescent Health, 17(5), 289-297.

Martinson, B.C., Lazovich, D., Lando, H.A., Perry, C.L., McGovern, P.G., & Boyle, R.G. (2000). Effectiveness of monetary incentives for recruiting adolescents to an intervention trial to reduce smoking. Preventive Medicine, 31(6), 706-713.

Murthy, V.H., Krumholz, H.M., & Gross, C.P. (2004). Participation in cancer clinical trials: Race-, sex-, and age-based disparities. Journal of the American Medical Association, 291(22), 2720-2726.

Patel, M.X., Doku, V., Tennakoon, L. (2003). Challenges in recruitment of research participants. Advances in Psychiatric Treathment, 9(3), 229-238.

Pokorny, S. B., Jason, L. A., Schoeny, M. E., Townsend, S. M., & Curie, C. J. (2001). Do participation rates change when active consent procedures replace passive consent. Evaluation Review, 25(5), 567-580.

Post, A., Gilljam, H., Bremberg, S., & Galanti, M.R. (2012). Psychosocial determinants of attrition in a longitudinal study of tobacco use in youth. Scientific World Journal, 2012, 654030.

Siddiqui, O., Flay, B.R., & Hu, F.B. (1996). Factors affecting attrition in a longitudinal smoking prevention study. Preventive Medicine, 25(5), 554-560.

Singer, E. (2002). The use of incentives to reduce nonresponse in household surveys. In: Wiley, ed. Survey nonresponse, 51, 163-177.

Shettle, C., & Mooney, G. (1999). Monetary incentives in US government surveys. Journal of Official Statistics, 15(2), 231.

Severson, H.H., & Ary, D.V. (1983). Sampling bias due to consent procedures with adolescents. Addictive Behaviors, 8(4), 433–437.

Unger, J.B., Gallaher, P., Palmer, P.H., Baezconde-Garbanati, L., Trinidad, T.R., Cen, S., & Johnson, C.A. (2004). No news is bad news: Characteristics of adolescents who provide neither parental consent nor refusal for participation in school-based survey research. Evaluation Review, 28(1), 52-63.

Walter, J.K., Burke, J.F., & Davis, M.M. (2013). Research participation by low-income and racial/ethnic minority groups: How payment may change the balance. Clinical and Translational Science, 6, 363-371.

Zand, D., Thomson, N.R., Budan, M., Braun, J.A., Holterman-Hommes, P., & Hunter, P.L. (2006). Predictors of retention in an alcohol, tobacco, and other drug prevention study. Evaluation Review, 30(2), 209-222.

7