Part B - Quantitative Supporting Statement (POS Campaign)

Point-of-Sale Campaign: Online Quantitative Study of Reactions to Rough-Cut Advertising Designed to Encourage Adult Smokers to Quit Smoking

0910-0810

Supporting Statement: Part B

B. STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

The one-time actual burden figures are listed in Part A.

The primary outcome of this study will be based on a non-probability target sample of 2,844 adult smokers in the United States ages 25 to 54. The study is a post-test only randomized experiment with two general conditions: An execution exposure condition, in which a single execution as well as multiple ads together will be shown, and a control condition in which no executions will be shown. Participants will be recruited via an online panel. The screening criteria are based on age, smoking status, past quit attempts, cigarette purchase location, and intention to smoke in the future. The goal of the research is to determine whether advertisements developed to be placed at the Point-of-Sale provide an understandable and engaging message to encourage a future quit attempt without potential unintended adverse or counterproductive effects.

Sampling Methods

This study will recruit and screen potential participants online. Fors Marsh Group will handle all data collection and survey management. Fors Marsh Group will be working with Survey Sampling International (SSI)—a volunteer opt-in online panel provider, in order to provide the sample. SSI is a global sample provider with 62 proprietary panels as the core of their online sample. Panel members are recruited through a variety of methods including online communities, social networks, and websites of all types via banners, invitations and messaging of all types. All panel members go through rigorous quality controls before being included in any sample and have agreed to provide their opinions. SSI will invite panel members to participate in the study based on their likelihood to be within the target population (e.g., known smoker status). Eligible participants will be required to self-identify as being a current smoker or poly-user of tobacco (i.e., currently use one or more other types of tobacco products in addition to cigarettes) and having made a previous quit attempt via the screener form (Attachment B). Only participants aged 25–54 will be included in the study. Eligible participants must also report visiting a convenience store at least once a month and at least occasionally buying cigarettes at convenience stores. If they have participated in a tobacco research study in within the past 6 months, they will be excluded. Participants who indicate they are affiliated with a market research firm, ad agency, or tobacco company will be excluded from the study. Other demographic questions contained in the screener (e.g., gender, race/ethnicity, education) will not be used as inclusion/exclusion criteria, but these data will be included in the final data set for data analysis purposes. Participants will not be stratified by race/ethnicity or other demographic characteristics, but demographic questions will be used to ensure that enrollment includes a diverse population of adult smokers. Researchers will not inform ineligible individuals that they are being excluded as a result of anything related to their demographic profile or tobacco use behavior.

Table 8: Required Groups and Maximum Sample Sizes for Eight Executions (Single Exposure)
	Campaign Design 1						Campaign Design 2
	View 1a	View 1b		View 1c	View 1d			View 2a	View 2b	View 2c	View 2d
Exposure Group A	n = 316	---		---	---			---	---	---	---
Exposure Group B	---	n = 316		---	---			---	---	---	---
Exposure Group C	---	---		n = 316	---			---	---	---	---
Exposure Group D	---	---		---	n = 316			---	---	---	---
Exposure Group E	---	---		---	---			n = 316	---	---	---
Exposure Group F	---	---		---	---			---	n = 316	---	---
Exposure Group G	---	---		---	---			---	---	n = 316	---
Exposure Group H	---	---		---	---			---	---	---	n = 316
Exposures per Execution	n = 316	n = 316		n = 316	n = 316			n = 316	n = 316	n = 316	n = 316
Execution Exposures			n = 2,528
Participants in Exposure Groups			n = 2,528
Participants in Control Group			n = 316
Total Number of Participants			N = 2,844

Specific cigarette use status inclusion and exclusion criteria are:

1. Respondents who indicate in the Screener that they satisfy the criteria of a current smoker – that is, have smoked at least 100 cigarettes in their lifetime and smoke at least some days to everyday– will qualify for study participation.

2. Respondents who indicate in the Screener that they satisfy the criteria of having made a recent quit attempt – that is, they have stopped smoking for one day or longer during the last twelve months – will qualify for study participation.

3. Respondents who indicate that they have purchased cigarettes from convenience stores or gas stations in the last month will qualify for study participation.

4. Respondents who report that they have not smoked at least 100 cigarettes in their lifetime will be excluded from participation.

5. Respondents who report that they have not made a quit attempt in the last twelve months will be excluded from participation.

6. Respondents who report that they do not ever buy cigarettes from convenience stores or gas stations will be excluded from participation.

The screening, consent, randomization to groups, and survey will be one unified experience for the respondents. The invitation will include a link that will direct them first to the screener. If they are eligible to participate, they will move forward to the consent form, which they can review and provide consent. Those who agree to participate will then be randomized to either an experimental or control group and will proceed through their appropriate survey. After completed the survey, respondents will be directed back to SSI, where they will go to a thank you language landing page and be provided with their “e-reward”. Please refer to Attachment A for the informed consent form.

Sample Size

A power analysis was conducted to determine the appropriate sample size for this study in order to effectively detect differences between exposure and control groups. This sample size was determined to account for testing a total of eight ad executions. With eight advertisement groups and one control group in this study, there will be nine groups. Assuming a one-tailed test, a small effect size (Cohen’s d = 0.20) and an alpha of 0.05, the required sample size for both phases of this study to achieve a power of 0.80, which is generally considered adequate in social science research, is n = 2,790 (310 participants in each of the nine groups). Because study enrollment will be occurring simultaneously online, a buffer of approximately two percent (n=54) will be applied in order to account for over-enrollment in any of the groups for a maximum sample size of n=2,844 (a maximum of 316 participants in each of the nine groups). Based on previous experience recruiting participants for similar studies, it is anticipated that roughly three individuals will need to be screened for every one participant. Thus, it is estimated that 8,816 potential respondents will need to be screened for this study in order to reach the desired final sample size.

2. Procedures for the Collection of Information

Participants who are deemed eligible after taking the screener will be randomly assigned to one of the nine groups (there will be a total of eight advertisement exposure groups and one control group). All participants will first be asked questions on their current cigarette smoking behavior and their past quitting behavior in order to ensure participants are part of the target audience and to assess their prior motivation to quit smoking.

Participants in the exposure group will then be shown one out of the eight advertisements. Participants will answer open-ended and multiple choice scaled questions on the advertisement to measure their reactions and gauge the effectiveness of each ad. Participants will then be asked questions designed to measure their attitudes, knowledge, and beliefs about quitting smoking. Lastly, participants will be shown the ad they just saw along with three additional ads and answer questions on the campaign as a whole. The survey will take most participants in the exposure group up to 20 minutes to complete.

Participants in the control group will not be shown any ads and will only be asked demographic questions and questions related to their knowledge, attitudes and beliefs. The survey should take participants in the control group 3–5 minutes to complete. Comparisons will be made across control and exposure participants in order to identify any unintended consequences that result from the advertisements and to assess which specific ad executions resonated most with the target audience. The results of this survey will help to refine and determine which point-of-sale materials should be released into market.

All of the above-mentioned research activities will be conducted on a password-protected website. Table 9 indicates the variables to be assessed during the questionnaire and the participant groups that will be exposed to these survey items.

Table 9. Structure of the Copy Testing Process and Questionnaire

Table 2. Structure of the Copy Testing Process and Questionnaire
Action or Variable	Description	Presented to Ad-Viewing Participants	Presented to Control Participants
Tobacco use and smoking cessation behavior	Items on tobacco use, past quit attempts, and motivation to engage in smoking cessation.	X	X
Advertisement exposure	Each participant in an exposure group will be randomly assigned to view one of the eight print advertisements.	X
Perceived ad effectiveness	Participants in an advertisement exposure group will be presented with items to assess the ad effectiveness immediately following exposure to the print advertisement.	X
Tobacco-related attitudes, beliefs and risk perceptions	Items tailored to align with knowledge, attitudes, and beliefs related to tobacco use and cessation. Additionally, items will assess participants’ motivation to quit smoking and their perceived self-efficacy to quit smoking.	X	X
Full campaign exposure	Each participant in an exposure group will view the first ad along with three additional ads and answer evaluative questions on the entire campaign design.	X

Unusual

Problems Requiring Specialized Sampling Procedures

No specialized sampling procedures are involved.

Use of Periodic Data Collection Cycles to Reduce Burden

This is a one-time survey data collection effort.

3. Methods to Maximize Response Rates

General Methods to Reduce Non-Response & Drop-Off

Several features of this study have been designed to maximize participant response rate and Questionnaire completion.

• Incentives: As participants often have competing demands for their time, incentives are used to encourage participation in research. Numerous empirical studies have shown that incentives can significantly increase response rates in cross-sectional surveys and reduce attrition in longitudinal surveys (e.g., Abreu & Winters, 1999; Castiglioni, Pforr, & Krieger, 2008; Jäckle & Lynn, 2008; Shettle & Mooney, 1999; Singer, 2002). In the context of this research study, the target population is considered a harder-to-recruit population because of the screening criteria (e.g., current smokers who have made a quit attempt in the past year). This study will use “e-rewards” distributed through the online panel provider, equal to approximately $1.50 per participant to provide enough motivation for them to participate in the study rather than another activity. The incentive amount will be provided to participants for their entire burden time, which includes screening time and completing the survey.

• Online Completion: Participants will be emailed a link to complete the questionnaire online. Because participants can complete the questionnaire on their own time, study participation will be more convenient and they will be more apt to complete the survey. This will also decrease time and costs related to recruitment. This technology also permits participants to complete the instruments in privacy. Providing the participant with a methodology that improves privacy makes reporting of potentially embarrassing or stigmatizing behaviors (e.g., tobacco use) less threatening and enhances response validity and response rates.

4. Test of Procedures or Methods

The campaign contractor, FCB, and their subcontractor, Fors Marsh Group, will conduct rigorous internal testing of the electronic survey instruments prior to their fielding. Trained researchers will review the screeners and questionnaire to verify that instrument skip patterns are functioning properly, delivery of campaign media materials is working properly, and that all survey questions are worded correctly and are in accordance with the instrument approved by OMB.

5. Individuals Consulted in Statistical Consultation and Information Collection

The following individuals inside the agency have been consulted on the design of the copy testing plan, survey development, or intra-agency coordination of information collection efforts:

Chaunetta Jones

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20993

Phone: 240-402-0427

E-mail: Chaunetta.Jones@fda.hhs.gov

Tesfa Alexander

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20993-0002

Phone: 301-796-9335

E-mail: Tesfa.Alexander@fda.hhs.gov

Maria Roditis

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20993

Phone: 240-402-1928

E-mail: Maria.Roditis@fda.hhs.gov

Janine Delahanty

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20993

Phone: 240-402-9705

E-mail: Janine.Delahanty@fda.hhs.gov

The following individuals outside of the agency have been consulted on questionnaire development:

Sarah Evans

Director of Communication Research

Fors Marsh Group

1010 N. Glebe Road Suite 510

Arlington, VA 22201

Phone: 571-858-3752

Email: sevans@forsmarshgroup.com

Cameron Wimpy

Senior Researcher

Fors Marsh Group

1010 N. Glebe Road Suite 510

Arlington, VA 22201

Phone: 571-858-3763

Email: cwimpy@forsmarshgroup.com

Caitlin Krulikowski
Senior Researcher

Fors Marsh Group

1010 N. Glebe Road Suite 510

Arlington, VA 22201

Phone: 571-858-3771

Email: ckrulikowski@forsmarshgroup.com

Nikkita Sarna

Researcher

Fors Marsh Group

1010 N. Glebe Road Suite 510

Arlington, VA 22201

Phone: 571-858-3826

Email: nsarna@forsmarshgroup.com

Carol Issa

Account Supervisor

FCB

100 West 33rd Street

New York, NY 10001

Phone: 212- 716 7608

Email: Carol.Issa@fcb.com

Mark Hall

Senior Strategic Planner

FCB

100 West 33rd Street

New York, NY 10001

Phone: 212-885-3372

Email: Mark.W.Hall@fcb.com

References

Abreu, D. A., & Winters, F. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. In Proceedings of the Survey Research Methods Section (pp. 533-538).

Castiglioni, L., Pforr, K., & Krieger, U. (2008, December). The effect of incentives on response rates and panel attrition: Results of a controlled experiment. In Survey Research Methods (Vol. 2, No. 3, pp. 151-158).

Jäckle, A., & Lynn, P. (2008). Offre de primes d’encouragement aux répondants dans une enquête par panel multimodes: effets cumulatifs sur la non-réponse et le biais. Techniques d’enquête, 34(1), 115-130.

Singer, E. (2002). The use of incentives to reduce nonresponse in household surveys. Survey nonresponse, 51, 163-177.

Shettle, C., & Mooney, G. (1999). Monetary incentives in US government surveys. Journal of Official Statistics, 15(2), 231.

Page 12 of 12