0910-0788 SS OMB_Part A_FU2

Food and Drug Administration’s Evaluation of the Fresh Empire Campaign on Tobacco (EFECT)

0910-0788

SUPPORTING STATEMENT

A. Justification

1. Circumstances Making the Collection of Information Necessary

The 2009 Family Smoking Prevention and Tobacco Control Act (Tobacco Control Act) (Pub. L. 111–31) amended the Federal Food, Drug, and Cosmetic Act (the FD&C Act) to grant FDA authority to regulate the manufacture, marketing, and distribution of tobacco products to protect public health and to reduce tobacco use by minors. Section 1003(d)(2)(D) of the FD&C Act (21 U.S.C. 393(d)(2)(D)) supports the development and implementation of FDA public education campaigns related to tobacco use. Accordingly, FDA is currently developing and implementing a youth-targeted public education campaign to help prevent tobacco use among multicultural youth and thereby reduce the public health burden of tobacco. The campaign will feature events, advertisements on television and radio and in print, digital communications including videos and social media, and other forms of media. For the purpose of this OMB package, each of these campaign elements will be referred to as “advertisements” or “ads.”

The objective of the evaluation is to measure the effectiveness of CTP’s Fresh Empire campaign designed to reduce tobacco use among multicultural youth aged 12 to 17. FDA’s Fresh Empire youth tobacco prevention campaign will focus on reducing tobacco use among youth who affiliate with a Hip Hop peer crowd, and predominantly among African American, Hispanic, and Asian/Pacific Islander youth. The goal of the proposed information collection is to evaluate the effectiveness of these efforts in affecting specific cognitive outcomes related to tobacco use that are targeted by the campaign.

This study is designed to measure awareness of and exposure to FDA’s Fresh Empire youth tobacco prevention campaign and assess its impact on outcome variables of interest. The first data collection period was in mid to late 2015. The post-campaign data collection began approximately 6 months following the launch of the campaign. The data collection will end approximately 24 months after the launch of the campaign. This design, which includes cross-sectional data collection with an embedded longitudinal cohort, will facilitate analysis of relationships between individuals’ exposure to campaign activities and pre- to post-campaign changes in outcomes of interest between campaign and comparison cities. Research studies have demonstrated that receptivity to advertisements is causally antecedent to actual ad effectiveness (e.g., Davis et al., 2013; Davis, Uhrig, et al., 2011; Dillard, Shen, & Vail, 2007; Dillard, Weber, & Vail, 2007). We hypothesize that if the campaign is effective, the pre- to post-campaign changes in outcomes should be larger among individuals in campaign cities compared to individuals in comparison cities. Furthermore, the differences should be more pronounced for youth in campaign cities exposed to the campaign more frequently (i.e., dose-response effects).

The primary method to recruit youth for the pre-test survey was to send a brief mail screener to households in campaign and comparison cities. However, given that the target audience represents a relatively small proportion of youth, we complemented this approach by recruiting youth through social media. Those youth who are recruited through mail or social media will become members of the longitudinal panel. The pre-test survey includes measures of tobacco-related beliefs, attitudes, intentions, and behaviors. The outcome post-test survey includes measures of audience awareness of and exposure to the campaign advertisements as well as the aforementioned outcome variables of interest. The post-test questionnaire is presented in Attachment 1. The brief mail screener used to identify multicultural youth for the outcome pre-test survey is presented as Attachment 2. Attachment 3 contains the content of the web screener that will be used to identify eligible youth recruited using social media.

2. Purpose and Use of the Information Collection

The information obtained from the data collection activities is collected from individuals and will be used to inform FDA, policy makers in the United States, prevention practitioners, and researchers about the extent of multicultural youth’s exposure to the campaign’s activities and the extent to which exposure to these activities is associated with changes in targeted outcomes. While not exhaustive, the list below illustrates a range of purposes and uses for the proposed information collection:

• Provide critical data on the reach of the campaign among multicultural youth in targeted cities, particularly with estimates of the proportion of the population that was exposed to the campaign.

• Understand the influence of the campaign on targeted beliefs and attitudes.

• Inform FDA, policy makers, and other stakeholders on the impact of the campaign overall.

• Inform the public about the impact of the campaign.

• Inform future programs that may be designed for similar purposes.

To achieve these goals, data collection will consist of a pre-test survey and post-test surveys with youth in the target audience. The post-test surveys will be conducted among those youth who participated previously, with new cross-sectional participants being recruited to make up for attrition. Eligible youth will initially be 12 to 17 year old youth who affiliate with a Hip Hop peer crowd. Youth in the embedded longitudinal cohort may reach the age of 18 during the course of the evaluation. The sample will be predominantly African American, Hispanic, and Asian/Pacific Islander. The Fresh Empire campaign will target 44 cities. The data collection will occur in 15 campaign-targeted cities and 15 similar (“comparison”) cities. Collecting data in a subset of cities helps manage the costs of data collection, while not compromising statistical power (i.e., too much clustering reduces effective sample sizes). The embedded longitudinal cohort will also reduce cost, as well as respondent burden. The outcome study will rely primarily on a mail screener survey to identify eligible youth, followed by in-person data collection. We will supplement this approach by recruiting youth through social media. We will advertise in social media and invite youth 13 to 17 years old to complete the screening survey online. Consistent with the Federal Children’s Online Privacy Protection Act, we will not contact youth under 13 online. We will then ask eligible youth to provide contact information for their parents/guardians so that we can obtain consent for completing the outcome survey online. In post-campaign survey rounds, 15 to 17 year old youth recruited by social media will not require parental consent to participate in the survey. For youth 13 to 14 years old, we will continue to require parental consent.

To ensure that the youth who participate in the outcome evaluation are members of the target audience, i.e., multicultural youth who affiliate with a Hip Hop peer crowd, eligible youth will be identified using the same screening method used by the agency implementing the Fresh Empire campaign—Rescue. This is accomplished by presenting photos of males and females representing various peer crowds. The images will be displayed in two arrays stratified by gender. Respondents will be asked to rank order the three images depicting individuals who best represent their friend group and the three images that least represent it in each array. Survey participants will be categorized as members of the Hip Hop peer crowd based on this exercise. Eligible youth will be contacted and invited to complete the outcome survey.

The outcome survey will be self-administered on laptop computers provided by field interviewers or completed online. The pre-test survey will have a sample size of 2,100, with half of the sample (N=1,050) from 15 campaign-targeted cities and half (N=1,050) from comparison cities. The total sample for the post-test surveys will be approximately 6,300, with an equal number of surveys in campaign and comparison cities. We will estimate the proportion of baseline participants expected to complete successive post-test surveys and supplement that longitudinal sample with new cross-sectional participants to meet our target total sample size. This design permits an analysis of trends in outcomes between youth in targeted and comparison cities.

The original plan called for recruiting up to 500 participants for the pre-test surveys through social media platforms Twitter and Facebook. In actuality, there were fewer participants recruited through social media platforms as anticipated for the pre-test survey. Our social media recruitment were more successful during first follow up. Of the 6,300 post-test surveys, approximately 2,100 will be completed by youth recruited through social media, such as Twitter, Facebook and Instagram.

3. Use of Improved Information Technology and Burden Reduction

Use of an embedded longitudinal cohort will markedly reduce burden relative to a design consisting solely of cross-sectional surveys. In addition, this outcome study will rely on a mail-based screener and in-person computer-based outcome surveys for pre- and post-test data collection. The proposed approach of screening eligible youth by mail and recruiting eligible youth in-person provides a number of methodological advantages, including efficiency in identifying this hard-to-reach population, increased accuracy in measurement of key variables of interest, and reduced burden on study participants. Computerized administration permits the instrument designer to incorporate into the questionnaire routings that might be overly complex or not possible using a paper-based survey. The laptop computer and online survey that will be used to collect youth data can be programmed to implement complex skip patterns and fill specific wordings based on the respondent’s previous answers. Interviewer and respondent errors caused by faulty implementation of skip instructions are virtually eliminated. Second, computerized administration increases the consistency of the data. The computer can be programmed to identify inconsistent responses and attempt to resolve them through respondent prompts. This approach reduces the need for most manual and machine editing, thus saving time and money. In addition, it is likely that respondent-resolved inconsistencies will result in data that are more accurate than when inconsistencies are resolved using editing rules. FDA estimates that 18% of the respondents will use electronic means to fulfill the agency’s request.

The self-administered mail screener (see Attachment 2) will be programmed using a TeleForm —a machine-readable data form—so that the survey responses can be automatically captured using a TeleForm reader, which will obviate the need for manual data entry. Using this technology, the majority of surveys can be read electronically. Those that cannot be scanned will be coded by a data processor.

The computer-assisted self-interview technology for the outcome survey permits greater expediency with respect to data processing and analysis (e.g., a number of back-end processing steps, including coding and data entry, will be minimized). Data are transmitted electronically within 48 hours. These efficiencies save time due to the speed of data transmission, as well as receipt in a format suitable for analysis. Finally, this technology permits respondents to complete the interview in privacy. Providing the respondent with a methodology that improves privacy makes reporting of potentially embarrassing or stigmatizing behaviors (e.g., tobacco use) less threatening and enhances response validity and response rates.

The mail screener and in-person computerized survey sample will be supplemented by a sample of respondents who are recruited through social media. These respondents will be recruited through social media platforms, such as Twitter, Facebook and Instagram, and led to an online screener for the study (see Attachment 3). Respondents will be invited to complete the screener using a web survey programmed and hosted on RTI’s servers. This web screener will have the advantage of immediately notifying respondents if they are eligible for the full study. In addition, use of social media as a recruitment tool will cast a wider net to identify additional, eligible study respondents who are members of this hard-to-reach population.

Eligible respondents will be routed to the full web survey, and given a unique ID to use to enter the survey. Respondents will be able to quit the survey at any time and resume where they left off upon reentry. Respondents will also be emailed a link to resume the survey, contact information to ask questions, receive reminders to complete the survey, and receive a virtual gift card upon completion.

Administration of the survey using web methods will help to contain costs, allowing for a sample that is geographically diverse without driving up interviewer costs for travel during data collection.

4. Efforts to Identify Duplication and Use of Similar Information

FDA’s Evaluation of the Fresh Empire Campaign on Tobacco (EFECT) is new. To date, there has been no in-depth evaluation of this campaign in a real-world setting, and there are no existing data sources that contain measures on awareness of and exposure to the campaign. This proposed information collection therefore does not duplicate previous efforts. In designing the proposed data collection activities, we have taken several steps to ensure that this effort does not duplicate ongoing efforts and that no existing data sets would address the proposed study questions. We have carefully reviewed existing data sets to determine whether any of them are sufficiently similar or could be modified to address FDA’s need for information on the effectiveness of the campaign with respect to reducing youth tobacco-related outcomes. We investigated the possibility of using existing data to examine our research questions, such as data collected as part of ongoing national surveillance systems, evaluations of current or past state-level campaigns for youth, the National Youth Tobacco Survey, and the Youth Risk Behavior Surveillance System. Due to the timing of the campaign and the specificity of the target population, none of these existing data sources will be able to provide the necessary data collection needs of the campaign, none will include the necessary in-depth survey questions on awareness of individual ads and other campaign materials, and none contain all of the necessary outcome variables specific to the campaign’s messages.

5. Impact on Small Businesses or Other Small Entities

Respondents in this study will be members of the general public and specific subpopulations, not business entities. No impact on small businesses or other small entities is anticipated.

6. Consequences of Collecting the Information Less Frequently

Respondents to this collection of information will answer just one survey during each data collection period. While there are no legal obstacles to reduce burden, any lack of information needed to evaluate the Fresh Empire campaign may impede the federal government’s efforts to improve public health. Without the information collection requested for this evaluation study, it would be difficult to determine the value or impact of the campaign on the lives of the people they are intended to serve—multicultural youth. Failure to collect these data could reduce effective use of FDA’s program resources to benefit youth in the United States. Careful consideration has been given to how frequently the campaign’s intended audience should be surveyed for evaluation purposes. We believe that the proposed outcome study design will provide sufficient data to evaluate the campaign effectively.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances for this collection of information that require the data collection to be conducted in a manner inconsistent with 5 CRF 1320.5(d)(2). The message testing activities fully comply with the guidelines in 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In accordance with 5 CFR 1320.8(d), FDA published a 60-day notice for public comment in the Federal Register on January 5, 2015 (80 FR 230). FDA received two comments; however, only one was related to PRA. Neither required a response.

Comment: One comment stated that the media tracking survey and the outcome evaluation study proposed by FDA are critical to FDA’s efforts to develop and implement an effective multicultural youth tobacco prevention campaign.

Response: FDA agrees that this collection of information is necessary to the Agency’s efforts to promote and improve public health.

The following individuals inside the agency have been consulted on the design of the campaign evaluation plan, audience questionnaire development, or intra-agency coordination of information collection efforts:

April Brubach

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

9200 Corporate Boulevard

Rockville, MD 20850

Phone: 301-796-9214

E-mail: April.Brubach@fda.hhs.gov

Gem Benoza

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20903

Phone: 240-402-0088

E-mail: Maria.Benoza@fda.hhs.gov

David Portnoy

Office of Science

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20903

Phone: 301-796-9298

E-mail: David.Portnoy@fda.hhs.gov

Matthew Walker

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20903

Phone: 240-402-3824

E-mail: Matthew.Walker@fda.hhs.gov

Leah Hoffman

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20903

Phone: 240-743-1777

E-mail: Leah.Hoffman@fda.hhs.gov

Janine Delahanty

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20903

Phone: 240-402-9705

E-mail: Janine.Delahanty@fda.hhs.gov

The following individuals outside of the agency have been consulted on questionnaire development. Additionally, input has been solicited and received from FDA on the design of this study, including participation by FDA in meetings with OMB:

Matthew Farrelly

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-541-6852

E-mail: mcf@rti.org

Jennifer Duke

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-485-2269

E-mail: jduke@rti.org

Jane Allen

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-597-5115

E-mail: Janeallen@rti.org

Youn Lee

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-485-5536

E-mail: Younlee@rti.org

Amy Henes

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-541-7293

E-mail: ahenes@rti.org

Jamie Guillory

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-316-3725

E-mail: jguillory@rti.org

Patricia LeBaron

RTI International

230 W Monroe Suite 2100

Chicago, IL 60606

Phone: 312-777-5204

E-mail: plebaron@rti.org

Azucena Derecho

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-541-7231

E-mail: derecho@rti.org

Stephen King

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-541-8094

Email: stephenking@rti.org

Pamela Rao

Akira Technologies, Inc.

1747 Pennsylvania Ave NW Suite 600

Washington, DC 20002

Phone: (202) 517-7187

Email: prao@akira-tech.com

Xiaoquan Zhao

Department of Communication

George Mason University

Robinson Hall A, Room 307B

4400 University Drive, 3D6

Fairfax, VA 22030

Phone: 703-993-4008

E-mail: xzhao3@gmu.edu

Jeff Jordan

Rescue

3436 Ray Street

San Diego, CA 92104

Phone: 619-231-7555 x 150

Email: jeff@rescueagency.com

Mayo Djakaria

Rescue

660 Pennsylvania Avenue SE, Suite 400

Washington, DC 20003

Phone: 619-231-7555 x 120 

Email: mayo@rescueagency.com

Dana Wagner

Rescue

660 Pennsylvania Ave SE, Suite 400

Washington, DC 20003

Phone: 619-231-7555 x 331

Email: dana@rescueagency.com

9. Explanation of Any Payment or Gift to Respondents

Households that receive the mail screener will all receive a nominal incentive of a $2 bill to encourage completing and returning the screener before participating in this brief survey. The lead letter indicates that the $2 bill is intended for the potential youth participant, but that the adult recipient of the letter may keep the $2 if there are no eligible youth in the household. A meta-analysis of studies examining the use of incentives in mail surveys showed that pre-paid incentives and promised incentives increase participating in mail surveys by 19% and 8% respectively, compared to no incentives (Church, 1993). More recent studies confirm these findings (e.g., Montaquila et al., 2013; Brick et al., 2012; Beebe et al., 2005).

Youth invited to participate in the outcome evaluation surveys will receive incentives. Youth participants will be offered a $25 incentive for completion of the pre- and post-test surveys. We estimate that the pre-campaign survey will take 30 minutes to complete, and the post-test survey will take up to 45 minutes. The incentives are intended to recognize the time burden placed on participants, encourage their cooperation in subsequent post-test surveys, which will reduce both respondent burden and cost, and convey appreciation for contributing to this important study. Incentives are similar to those offered for most surveys of this type. Both surveys will take less than an hour to complete, and thus design protocols call for the same incentive amount. Numerous empirical studies have shown that incentives can significantly increase response rates in cross-sectional surveys and reduce attrition in longitudinal surveys (e.g., Abreu & Winters, 1999; Castiglioni, Pforr, & Krieger, 2008; Jäckle & Lynn, 2008; Shettle & Mooney, 1999; Singer, 2002; Singer and Ye, 2013). The decision to use incentives for this study is based on the need to promote continued participation by this hard-to-reach and specific population of multicultural youth who affiliate with a Hip Hop peer crowd (Beebe et al., 2005).

During the third follow-up data collection period, we plan to offer an additional $5 “early bird” incentive for longitudinal respondents who originally completed an in-person survey to encourage them to complete the survey online.

Several studies have shown that early bird incentives can improve response rates. In one study, individuals who received an early bird incentive were 1.8 times more likely to complete the survey within the first 7 days of data collection and were 1.69 times more likely to ever complete the survey (LeClere, Plummer, Vanicek, Amaya & Carris, 2012). Another study showed that an early bird incentive significantly increased the response rate in the first two weeks of data collection (29.7 vs. 20.1%) (Coopersmith, Vogel, Bruursema & Feeney, 2016). Biemer et al. (2017) used an experimental design to test the effectiveness of incentivizing mode choice – the researchers offered web and mail modes at the same time. Individuals in the experiment group were offered an additional $10 if they completed the survey via web.  The control group was offered the standard incentives. Incentivizing mode choice increased the overall response rate by 3.98 percentage points (42.78 vs. 38.80 for experiment and control group, respectively) and increased the proportion of respondents who completed via web (from 28% to 64%).

ExPECTT, the evaluation of the FDA’s public education campaign for tobacco use among youth (The Real Cost), promised an additional $5 incentive to participants who completed the survey online before the specified early bird date. Study staff found that this was an extremely effective method of facilitating timely data collection and promoting online completion of the surveys, which significantly reduced data collection costs. By eliminating the need for an interviewer to visit the household, this practice eliminated the cost of employing field staff and travel to the households. This method also reduced participant burden by not requiring them to complete the questionnaire on a specific day and time scheduled with an interviewer. In other words, online completion of the survey allows the participant greater flexibility. This method has also been used for the ExPECTT, RESPECT, and RuSTEC campaigns. For follow-up 1 of RuSTEC, 79% of completes occurred during the early bird period. Although still collecting data, 67% of the RuSTEC web responses for follow-up 2 occurred during the early bird period. For the ExPECTT campaign, 85% of follow-up 1 participants, 81% of the follow-up 2 participants, 79% of the follow-up 3 participants, and 80% of the follow-up 4 participants completed the survey during the early bird period.

Respondents who are recruited through social media (such as Twitter, Facebook and Instagram) and who complete the outcome survey online will receive a link to a virtual gift card via email, such as from Visa or Amazon, with a value of $25 upon completion of the survey.

A more detailed justification for the use of incentives is provided in Attachment 4. The use of modest incentives is expected to enhance survey response rates without biasing responses. A smaller incentive would not appear sufficiently attractive to participants. We also believe that the incentives will result in higher data validity as participants will become more engaged in the survey process. This will also enhance overall response to the pre-test and post-test surveys and reduce attrition at follow-up within the embedded longitudinal cohort. The use of incentives will help ensure that pre-test data collection is completed in a timely manner and potentially reduce the number of follow-up visits needed to contact non-respondents. Use of incentives within the embedded longitudinal cohort will reduce attrition which in turn will reduce respondent burden and the cost of post-test surveys. The specific amount of the proposed incentive is based on several previous projects conducted by RTI, including a survey used to evaluate FDA’s general market tobacco prevention education campaign and the National Survey of Child and Adolescent Well-Being, which found that use of similar incentives increased response rates among youth.

10. Assurance of Privacy Provided to Respondents

RTI’s Institutional Review Boards (IRB) will review and approve the consent and assent forms (Attachments 5a, 5b, 5c, 5d, 5e, 5f, 5g, 5i) for the outcome evaluation survey. These consent forms include language for parental consent and adolescent assent, or youth consent. The IRB’s primary concern is protecting respondents’ rights, one of which is maintaining the privacy of respondent information to the fullest extent of the law.

Concern for privacy and protection of respondents’ rights will play a central part in the implementation of the outcome evaluation study and will receive the utmost emphasis. Interviewers will be thoroughly educated in methods for maximizing a respondent’s understanding of the government’s commitment to privacy to the fullest extent of the law. Several procedures ensure that respondents’ rights are protected. First, the interviewer introduces himself or herself and the study to parents or guardians of eligible youth respondents using the Consent Scripts and the Study Description (Attachments 5a, 5b, and 6). As part of the process for obtaining informed consent, respondents are given a Study Description (Attachment 6), which includes information on their rights as study participants. Specifically, the Study Description states that respondents’ answers will be used only by authorized personnel for statistical purposes and cannot be used for any other purpose. Parental consent is obtained from the youth’s parent or guardian; subsequently, youth assent is requested. In post-campaign survey rounds, youth who have turned 18 do not require parental consent and provide their own consent. Signed consent and assent are waived in this study.

After obtaining informed consent, interviewers make every attempt to secure an interview setting in the respondent’s home that is as private as possible. In addition, the interview process, by design, includes techniques to afford privacy for the respondent. The self-administered portion of the interview maximizes privacy by giving control of the interview directly to the respondent. This allows the respondent to read the questions directly from the computer screen and then key his or her own responses into the computer via the keyboard.

Interviewers electronically transmit all completed data to RTI’s servers via secure encrypted data transmission within 48 hours of each day they work. On the data files, respondents are distinguished only by a unique identifier assigned to screenings and interviews. These identifiers will be used to link screener and interview data prior to analysis.

Security for respondents of the Web-based surveys will be assured in a number of ways: (1) we will obtain parental consent for all eligible youth screened online prior to completing the pretest outcome survey and for 13 to 14 year-old eligible youth prior to post-test outcome surveys, which is fully compliant with COPPA’s revised standards; each respondent will remain completely anonymous and will be known only by a unique alphanumeric variable; respondents will be asked to provide their email address to receive the incentive; (2) participants will log onto the secure server hosted by RTI using a link provided in the completed screener and a unique identifier and password, with the result that no information about the respondent’s identity will be connected to their survey responses; (3) respondents will be provided with information about the privacy of their data before they encounter the first survey item; (4) respondents will be required to provide their assent or consent to freely participate before they encounter the first survey item; and (5) respondents will have the option to decline to respond to any item in the survey for any reason. All those who handle or analyze data will be required to adhere to the standard data security policies of RTI.

To ensure data security, all RTI project staff are required to adhere to strict standards and to sign a nondisclosure agreement as a condition of employment on this project (Attachment 7). RTI maintains restricted access to all data preparation areas (i.e., receipt and coding). All data files on multi-user systems will be under the control of a database manager, with access limited to project staff on a “need-to-know” basis only. No respondent identifiers will be contained in reports to FDA, and results will only be presented in aggregate form.

Implementation of data security systems and processes will occur as part of the survey data collection. Data security provisions will involve the following:

• All data collection activities will be conducted in full compliance with FDA regulations to maintain the privacy of data obtained from respondents and to protect the rights and welfare of human research subjects as contained in their regulations. Respondents will receive information about privacy protections as part of the informed consent process.

• All data collectors will be trained on privacy procedures and be prepared to describe them in full detail, if necessary, or to answer any related questions raised by respondents. Training will include procedures for safeguarding sample member information in the field, including securing hardcopy case materials and laptops in the field, while traveling, and in respondent homes, and protecting the identity of sample members.

• All project employees will sign a privacy agreement that emphasizes the importance of respondent privacy and describes their obligations.

• All field staff laptop computers will be equipped with encryption software so that only the user or RTI administrators can access any data on the hard drive even if the hard drive is removed and linked to another computer.

• Laptops will use the Microsoft Windows operating system and require a valid login ID and password to access any applications or data.

• All data transferred to RTI servers from field staff laptops will be encrypted and transferred via a secure (SSL) broadband connection or optionally a secure telephone (land) line. Similarly, all data entered via the Web-based survey system will be encrypted as the responses will be on a Web site with an SSL certificate applied. Data will be passed through a firewall at RTI and then collected and stored on a protected network share on the RTI Network. Only authorized RTI project staff members will have access to the data on the secure network share.

• Respondents recruited through social media (such as Twitter, Facebook and Instagram) will also access the survey with a unique ID and password and will complete the survey on a secure server. Respondents who are part of the longitudinal cohort have the option to complete post-test outcome surveys online, following the same secure procedures. The result is that no information about the respondent’s identity will be connected with their survey responses.

All respondents will be assured that the information they provide will be maintained in a secure manner and will be used only for the purpose of this research. Respondents will be assured that their answers will not be shared with family members and that their names will not be reported with responses provided. Respondents will be told that the information obtained from all of the surveys will be combined into a summary report so that details of individual questionnaires cannot be linked to a specific participant.

Respondents will participate on a voluntary basis. The voluntary nature of the information collection is described in the introductory section of the Consent Process (Attachments 5a, 5b, 5c, 5d, 5e, 5f, 5g, 5i) and the lead letters (Attachments 8 and 8a).

11. Justification for Sensitive Questions

The majority of questions asked will not be of a sensitive nature. There will be no requests for a respondent’s Social Security Number (SSN). However, it will be necessary to ask some questions that may be considered to be of a sensitive nature in order to assess specific health behaviors, such as cigarette smoking. These questions are essential to the objectives of this information collection. Questions about messages concerning lifestyle (e.g., smoking, current smoking behavior, attempts to quit smoking) and some demographic information, such as race, ethnicity, and income, could be considered sensitive, but not highly sensitive. To address any concerns about inadvertent disclosure of sensitive information, respondents will be fully informed of the applicable privacy safeguards. The informed consent protocol (see Attachments 5a, 5b, 5c, 5d, 5e, 5f, 5g, 5i) will apprise respondents that these topics will be covered during the survey. This study includes a number of procedures and methodological characteristics that will minimize potential negative reactions to these types of questions, including the following:

• Respondents will be informed that they need not answer any question that makes them feel uncomfortable or that they simply do not wish to answer.

• Web surveys are entirely self-administered and maximize respondent privacy without the need to verbalize responses.

• Participants will be provided with a specific toll-free phone number (linking directly to the RTI IRB Office) to call in case they have a question or concern about the sensitive issue.

Finally, as with all information collected, these data will be presented with all identifiers removed.

12. Estimates of Annualized Burden Hours and Costs

12 a. Annualized Hour Burden Estimate

Information will be initially collected through interviews involving youth ages 12 to 17. Those youth will then be asked to participate in subsequent rounds. The sample will be predominantly African American, Hispanic, and Asian/Pacific Islander. Information will be collected prior to and following the campaign’s launch. To better understand youth’s awareness of and receptivity to campaign materials as the campaign evolves, we will collect data starting 6 months after the campaign launches and ending 24 months following the campaign’s launch. Statistical power estimates provide guidance on reasonable expectations for observing statistically significant change in outcomes of interest as detailed in Section B.1.

A mail-based screener was one of the methods used to identify eligible youth (Attachment 2). Parents or guardians will be asked to provide consent and their contact information on this form. For the pre-launch survey, the five-minute screener was completed by 13,816 households for a total of 1,151 burden hours for youth and an additional 229 hours for the parents or guardians. We will not use the mail-based screener for the post-test screening surveys because we are able to rely upon social media recruitment, described in further detail below. For this reason, estimated burden hours for youth and adults for the mail-based screener and consent process have been reallocated to the online screener as appropriate.

We will also recruit youth through social media (such as Twitter, Facebook and Instagram) as a secondary strategy to recruit youth 13 to 17. An online version of the screener described above will be used to identify eligible youth (included in Attachment 3). Eligible youth will be asked to provide their parents’ or guardians’ contact information. The pre-test survey required parental consent for all participants. For the post-campaign outcome surveys, newly recruited 15 to 17 year-old youth will not require parental consent and will therefore not be asked to provide their parents’ or guardians’ contact information. For newly recruited youth 13 to 14 years old, we will continue to require parental consent. The screener will take five minutes and will be completed by approximately 8,000 youth for the pre-test survey for a total of 666 burden hours. For the post-test survey, approximately 24,000 youth will complete the 5-minute screener, for 2,000 burden hours.

The process of parents and guardians providing consent for eligible youth will take approximately 1 minute. As noted, there were fewer participants recruited through social media platforms as anticipated for the pre-test survey; no more than 520 adults were contacted for consent for a total of 9 burden hours. For the post-test surveys, an estimated 6,000 adults will be contacted to provide consent for eligible youth for a total of 100 burden hours. This is a conservative estimate as not all eligible youth will require parental consent.

For the pre-test survey, 2,194 youth completed the questionnaire with an estimated burden of 30 minutes per respondent, for an annualized total of 1,097 hours. During the first post-test outcome survey, 2,404 youth (1,722 longitudinal and 682 cross-sectional) completed the survey (a larger sample size than anticipated due to successful social media recruitment efforts). During the second post-test outcome survey, a total of 2,255 youth completed the survey: 1,752 longitudinal cases and 503 cross-sectional cases (204 of which were removed from the analytic sample based on Hip Hop score). For the third post-test outcome survey, 2,100 youth (1,365 longitudinal and 735 cross-sectional) are expected to take the survey. For the post-test surveys, the estimated burden is 45 minutes per respondent, for a total of 5,070 burden hours (3 waves of longitudinal and 3 waves of cross-sectional). The number of respondents includes those originally recruited through the mail-based screener and those recruited through social media.

This data collection will take place in 2015, 2016 and 2017. Thus, the target number of completed campaign questionnaires for all respondents is 75,105, and the annualized response burden is estimated at 10,322. As a result of the revisions made to the participant numbers noted above, this revised estimate falls below our original approved 13,678 hours for this study. OMB approval is requested for 3 years. Exhibit 1 provides details about how this estimate was calculated. The Web self-administered surveys will be designed to maximize ease of response (at home on personal computers or mobile devices) and thus decrease respondent burden.

Exhibit 1. Estimated Annual Burden Hours^a

Type of Respondent	Activity	Number of Respondents	Number of Responses per Respondent	Total Annual Responses	Average Burden per Response	Total Hours
Youth aged 12 to 17 in the United States	Mail Screener and Consent Process-Pre-test outcome survey	13816	1	13816	0.0833 (5 min.)	1151
Adults 18 and older in the United States	Mail Screener and Consent Process- Pre-test outcome survey	13816	1	13816	0.0166 (1 min.)	229
	Online Screener and Consent Process- Pre-test outcome survey	520	1	520	0.0166 (1 min)	9
	Online Screener and Consent Process- Post-test outcome survey	6000	1	6000	0.0166 (1 min)	100
Multicultural Youth aged 12-17 in select media markets	Pre-test outcome evaluation survey	2194	1	2194	0.5 (30 min.)	1097
Longitudinal Cohort	First post-test evaluation survey	1722	1	1722	0.75 (45 min)	1292
	Second post-test evaluation survey	1752	1	1752	0.75 (45 min.)	1314
	Third post-test evaluation survey	1365	1	1365	0.75 (45 min.)	1024
Cross-Sectional Cohort	First post-test evaluation survey	682	1	682	0.75 (45 min.)	512
	Second post-test evaluation survey	503	1	503	0.75 (45 min)	377
	Third post-test evaluation survey	735	1	735	0.75 (45 min)	551
Multicultural youth aged 13-17 in the select media markets recruiting through social media and online panels	Pre-test online screener	8000	1	8000	0.0833 (5 min.)	666
	Post-test online screener	24000	1	24000	0.0833 (5 min)	2000
Revised Total		75,105				10,322

^a There are no capital costs or operating and maintenance costs associated with this collection of information.

12b. Annualized Cost Burden Estimate

Respondents participate on a purely voluntary basis and, therefore, are subject to no direct costs other than time to participate. There are also no start-up or maintenance costs. RTI has conducted many smoking-related surveys of similar length among youth. We have examined diagnostic data from each of these prior surveys and estimate that data collection for this study will take approximately 30 minutes per respondent for the pre-test outcome survey and 45 minutes for the post-test surveys. We estimate that the web surveys will also take 30 minutes per respondent for the pre-test outcome survey and 45 minutes for the post-test surveys. According to the U.S. Department of Labor (DOL) Bureau of Labor Statistics the average hourly wage in 2013 was $8.19 for ages 16 to 19. Thus, assuming an average hourly wage of $8.19 for youth respondents and an hourly wage for adults of $24.75, the estimated total cost to participants will be $90,135. This revised estimate falls below our original estimate of $119,809 for this study. The estimated value of respondents’ time for participating in the information collection is summarized in Exhibit 2.

Exhibit 2. Estimated Annual Cost

Type of Respondent	Activity	Annual Burden Hours	Hourly Wage Rate	Total Cost
Youth aged 12 to 17 in the United States	Mail Screener for pre-test survey	1,151	$8.19	$9,427
Adults 18 and older in the United States	Mail Screener for pre-test survey	229	$24.75	$5,668
	Online Screener and Consent process for pre- and post-test surveys	109	$24.75	$2,698
Multicultural youth aged 12 to 17 in the United States	Pre-test Survey	1,097	$8.19	$8,984
	Post-test Surveys	5,070	$8.19	$41,523
Multicultural youth aged 13 to 17 in the United States (social media	Online Screener for pre-test survey	666	$8.19	$5,455
	Online Screener for post-test surveys	2,000	$8.19	$16,380
Revised Total				$90,135

13. Estimates of Other Total Annual Costs to Respondents and/or Recordkeepers/Capital Costs

There are no capital, start-up, operating, or maintenance costs associated with this information collection.

14. Annualized Cost to the Federal Government

This information collection is funded through a contract with RTI. The total estimated costs attributable to this data collection are $8,028,118 (Exhibit 3). There are additional contract-funded activities occurring before and after this data collection that include project planning and data analysis. Other activities outside this data collection include coordination with FDA and its media contractor, evaluation plan development, instrument development, reporting, RTI IRB, and progress reporting and project management. This information collection will occur from 2015 through 2017.

Exhibit 3. Itemized Cost to the Federal Government

Government Personnel	Time Commitment	Average Annual Salary	Total
GS-13	25%	$73,846	$18,462
GS-14	15%	$87,263	$13,089
GS-15	5%	$102,646	$5,132
Total Salary Costs			$36,683
Contract Cost			$7,991,435
Total			$8,028,118

15. Explanation for Program Changes or Adjustments

This is a new data collection.

16. Plans for Tabulation and Publication and Project Time Schedule

Data from this information collection will be used to estimate awareness of and exposure to the campaign among multicultural youth. These estimates will take the form of self-reported ad recognition and recall that assess basic exposure as well as frequency of ad exposure. These estimates will also be calculated separately for each specific campaign advertisement.

Data from this information collection will also be used to examine statistical associations between exposure to the campaign and pre-post changes in specific outcomes of interest for campaign and comparison groups. We will conduct two primary types of analyses. The first will focus on aggregate changes in outcomes from the pre- to post-campaign periods between the campaign and comparison cities. The second analytic approach will focus on individual changes in outcomes as a function of campaign exposure, which will vary within and across campaign and comparison cities. The embedded longitudinal cohort may also permit some longitudinal analysis. The primary outcomes of interest among youth will be awareness of the campaign as well as tobacco-related beliefs, attitudes, intentions and behaviors. We hypothesize that there should be larger changes in outcomes among individuals with more frequent campaign exposure (i.e., dose-response effects).

In addition to relying on self-reported exposure, we will also utilize measures of market-level campaign intensity, which will be constructed with available data on campaign activities, including traditional and digital advertising and local campaign events. These data will be merged to the survey to provide an additional measure of campaign exposure among study participants. This will allow us to analyze the relationship between the market-level delivery of the campaigns and actual levels of awareness in each sample that is collected. This will also facilitate further analyses of the relationship between exogenous market-level measures of campaign dose and changes in the aforementioned outcome variables of interest.

The reporting and dissemination mechanism will consist of three primary components: (1) summary statistics (in the form of PowerPoint presentations and other briefings) on individual awareness of and reactions to the campaign, (2) a comprehensive evaluation report summarizing findings from this information collection, and (3) at least two peer-reviewed journal articles that document the relationships between campaign exposure and changes in the aforementioned outcomes of interest. The key events and reports to be prepared are listed in Exhibit 4.

Pre-test information collection must be completed before the launch of the campaign. OMB approval is requested as soon as possible.

Exhibit 4. Project Schedule

Project Activity	Date
Pre-test data collection	July-October 2015
Post-test data collection	April 2016-December 2017
Preparation of analytic data file	Approximately 4 weeks after completion of data collection
Data analysis	Approximately 5–12 weeks after completion of each analytic data file
Report writing and dissemination	Approximately 12-16 weeks after completion of each analytic data file

17. Reason(s) Display of OMB Expiration Date is Inappropriate

Not applicable. All data collection instruments will display the expiration date for OMB approval of the information collection.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

Not applicable. There are no exceptions to the certification statement.

References

Abreu, D. A., & Winters, F. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. Proceedings of the Survey Research Methods Section of the American Statistical Association.

Biemer, P. P., Murphy, J., Zimmer, S., Berry, C., Deng, G., & Lewis, K. (2017). Using Bonus Monetary Incentives to Encourage Web Response in Mixed-Mode Household Surveys. Journal of Survey Statistics and Methodology, 1–22. https://doi.org/10.1093/jssam/smx015.

Castiglioni, L., Pforr, K., & Krieger, U. (2008). The effect of incentives on response rates and panel attrition: Results of a controlled experiment. Survey Research Methods, 2(3), 151–158.

Centers for Disease Control and Prevention. (2012). Youth Risk Behavior Surveillance–United States, 2011. Morbidity and Mortality Weekly Report, 61(4), 1–162.

Coopersmith, J., Vogel, L. K., Bruursema, T., & Feeney, K. (2016). Effects of Incentive Amount and Type of Web Survey Response Rates. Survey Practice, 9(1).

Davis, K. C., Nonnemaker, J., Duke, J., & Farrelly, M. C. (2013). Perceived effectiveness of cessation advertisements: The importance of audience reactions and practical implications for media campaign planning. Health Communication, 28(5), 461–472. doi:10.1080/10410236.2012.696535

Davis, K. C., Uhrig, J., Bann, C., Rupert, D., & Fraze, J. (2011). Exploring African American women’s perceptions of a social marketing campaign to promote HIV testing. Social Marketing Quarterly, 17(3), 39–60.

Dillard, J. P., Shen, L., & Vail, R. G. (2007). Do perceived message effectiveness cause persuasion or vice versa? Seventeen consistent answers. Human Communication Research, 33, 467–488.

Dillard, J. P., Weber, K. M., & Vail, R. G. (2007). The relationship between the perceived and actual effectiveness of persuasive messages: A meta-analysis with implications for formative campaign research. Journal of Communication, 57, 613–631.

Farrelly, M. C., Davis, K. C., Haviland, M. L., Messeri, P., & Healton, C. G. (2005). Evidence of a dose-response relationship between “truth” antismoking ads and youth smoking prevalence. American Journal of Public Health, 95(3), 425–431. doi: 10.2105/AJPH.2004.049692

Jäckle, A., & Lynn, P. (2008). Respondent incentives in a multi-mode panel survey: Cumulative effects on nonresponse and bias. Survey Methodology, 34(1), 105–117.

Janega, J. B., Murray, D. M., Varnell, S. P., Blitstein, J. L., Birnbaum, A. S., & Lytle, L. A. (2004). Assessing the most powerful analysis method for schools intervention studies with alcohol, tobacco, and other drug outcomes. Addictive Behaviors, 29(3), 595–606.

LeClere, F., Plumme, S., Vanicek, J., Amaya, A., & Carris, K. (2012). Household early bird incentives: leveraging family influence to improve household response rates. American Statistical Association Joint Statistical Meetings, Section on Survey Research, 4156 - 4165

Murray, D. M., & Blitstein, J. L. (2003). Methods to reduce the impact of intraclass correlation in group-randomized trials. Evaluation Review, 27(1), 79–103.

Murray, D. M., & Short, B. J. (1997). Intraclass correlation among measures related to tobacco-smoking by adolescents: Estimates, correlates, and applications in intervention studies. Addictive Behaviors, 22(1), 1–12.

Shettle, C., & Mooney, G. (1999). Monetary incentives in U.S. government surveys. Journal of Official Statistics, 15, 231–250.

Singer, E. (2002). The use of incentives to reduce nonresponse in household surveys. In R. M. Groves, D. A. Dillman, J. L. Eltinge, & R. J. A. Little (Eds.), Survey Nonresponse (p. 163–177). New York, NY: Wiley.

Snyder, L. B., Hamilton, M. A., Mitchell, E. W., Kiwanuka-Tondo, J., Fleming-Milici, F., & Proctor, D. (2004). A meta-analysis of the effect of mediated health communication campaigns on behavior change in the United States. Journal of Health Communications, 9, 71–96.

Substance Abuse and Mental Health Services Administration (SAMHSA). (2012). Results from the 2011 National Survey on Drug Use and Health: Summary of national findings. NSDUH Series H-44, HHS Publication No. (SMA) 12-4713. Rockville, MD: Substance Abuse and Mental Health Services Administration.

U.S. Department of Health and Human Services (USDHHS). (2006). The health consequences of involuntary exposure to tobacco smoke: A report of the Surgeon General. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, Coordinating Center for Health Promotion, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health.

Wakefield, M. A., Spittal, M. J., Yong, H-H., Durkin, S. J., & Borland, R. (2011). Effects of mass media campaign exposure intensity and durability on quit attempts in a population-based cohort study. Health Education Research, 26(6), 988–997.

18