Rationale for Outcome Evaluation Measures

Attachment 2a: Rationale for Outcome Evaluation Measures

Measurable population-level behavior change—such as a change in youth smoking prevalence—is the product of a series of changes in interrelated, individual-level beliefs, attitudes and perceptions about norms, and environmental-level factors such as smoke-free laws (Ajzen & Fishbein, 1980; Bandura, 1985; Hornik, 2002; Rosenstock, Strecher, & Becker, 1988; Trickett et al., 2011). Behavior change theory guides our understanding of how campaigns function (Ajzen, 1991; Bandura, 1985; Rosenstock et al., 1988; Trickett et al., 2011) and defines our expectations about the order in which campaign effects should occur: belief change, attitude and social norm change, intention, and finally behavior change (Fishbein, 1967). The Centers for Disease Control and Prevention’s (CDC’s) Best Practices for Comprehensive Tobacco Control Programs quantifies the timeline for these expectations, indicating that campaigns that deliver a sufficient amount of media will produce campaign awareness at 6 months, attitude change at 12 to 18 months, and behavior change at 18 to 24 months (CDC, 2007). A National Cancer Institute (NCI) study similarly concludes that campaigns “influence attitudes toward tobacco within a short period, followed by longer-term effects on smoking behavior” (NCI, 2008, p. 534). In practice, changes in beliefs, attitudes, and intention are often the first indicators of campaign effectiveness and, as a result, are among the first outcomes examined in the course of campaign evaluation (Cowell et al., 2009; Farrelly et al., 2005; Murray, Prokhorov, & Harty, 1994; Murukutla et al., 2012; Vallone et al., 2011a, b).

Evaluation Logic Model

Based on this evidence base and previous experience with tobacco prevention campaigns, we have mapped the expected relationships between specific campaign activities and downstream outcome indicators (Figure 1-1). This model further outlines key variables and other contextual influences on tobacco-related outcomes that may moderate the effects of the campaigns and therefore must be accounted for in our assessment of the campaigns’ impacts on key outcomes. Based on this model, we hypothesize that greater exposure to the campaigns will lead to greater changes in all key outcomes at all stages of time (short-term, intermediate, and long-term).

Figure 1-1. Evaluation Logic Model

Evaluation Questions

In this section, we present initial evaluation questions following the logic model described above and our current understanding of the creative direction of the general market and rural smokeless campaigns. The key evaluation questions we seek to answer fall under several broad domains, as outlined in Table 1‑1: implementation, campaign awareness and receptivity, campaign Web sites and social media, short-term cognitive outcomes, intermediate-term cognitive outcomes, and long-term behavioral outcomes. Although this is not an exhaustive list of all possible evaluation questions that this evaluation will address, the enumeration of questions in Table 1‑1 provides a detailed overview of the outcomes that are the focus of these campaigns and our evaluation. We present a combined set of evaluation questions for the general market and rural smokeless campaigns because of the significant overlap in the key questions. Table 1-2 provides the survey items enumerated by type and a description of how the various survey measures will be used in analyses.

Table ‎1-1. Campaign Key Evaluation Questions

Short-Term Cognitive Outcomes (illustrative)

Is cumulative exposure to the campaign associated with changes in tobacco-related knowledge, attitudes, and beliefs, including social normative beliefs, among youth overall and by sensation seeking and tobacco use status?

Is cumulative exposure to the campaign associated with an increase in awareness of the harmful ingredients in every cigarette?

Is cumulative exposure to the campaign associated with an increase in awareness of the risk of addiction from each cigarette?

Is cumulative exposure to the campaign associated with increases in the perceived risks of menthol cigarette use?

Is cumulative exposure to the campaign associated with an increase in the perceived risks of smokeless tobacco?

Intermediate-Term Cognitive Outcomes

Is cumulative exposure to the campaign associated with decreases in tobacco use susceptibility?

Is cumulative exposure to the campaign associated with increases in intentions to stop smoking and/or reduce daily consumption?

Long-Term Behavioral Outcomes

Is cumulative exposure to the campaign associated with a decrease in tobacco use initiation?

Is cumulative exposure to the campaign associated with a decreased prevalence of 30-day smoking and 30-day smokeless tobacco use?

Is cumulative exposure to the campaign associated with a decreased prevalence of established smoking and frequent smokeless tobacco use (20 or more days per month)?

Is cumulative exposure to the campaign associated with decreases in the number of days of tobacco use among current smokers and smokeless tobacco users?

Is cumulative exposure to the campaign associated with decreases in the average number of cigarettes smoked per day among current smokers?

Table 1-2. Survey Items by Type and Intended in Analysis, Outcome Evaluation Survey

Type of Item	Intended Use in Analysis of Outcome Evaluation Data
Youth
Demographics	control variable
Campaign Awareness and Receptivity	independent variable
Tobacco-related Attitudes, Beliefs, Risk Perceptions, and Social Norms	outcome variable
Intentions to Use Tobacco and Self-Efficacy	outcome variable
Cessation Intention, Behavior, and Motivation	outcome variable
Tobacco-related Behaviors	outcome variable
Other Tobacco Topics	control variablea
Media Use	control variable
Youth Environment	control variable
Parent
Demographics and Household Characteristics	control variable
Tobacco Use and Cessation	control variable
Home Media Environment	control variable
Parent Child Topics	control variable

^a Other tobacco products (e.g., cigars) and marijuana use are included to enable monitoring of potentially related outcomes not targeted by current media campaigns.

References

Ajzen I. The theory of planned behavior. OBHDP 1991;50:179-211.

Ajzen I, Fishbein M. Understanding attitudes and predicting social behavior. London, England: Pearson; 1980.

Albano JD, Ward E, Jemal A, et al. Cancer mortality in the United States by education level and race. J Natl Cancer Inst. Sep 19 2007;99(18):1384-1394.

Bandura A. Social foundations of thought and action: a social cognitive theory. London, England: Pearson; 1985.

Centers for Disease Control and Prevention. Best Practices for Comprehensive Tobacco Control Programs—2007. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; 2007.

Centers for Disease Control and Prevention (CDC). Framework for program evaluation in public health. (1999). MMWR Recomm Rep, 1999;48(RR-11), 1-40.

Cowell AJ, Farrelly MC, Chou R, Vallone DM. Assessing the impact of the national ‘truth’ antismoking campaign on beliefs, attitudes, and intent to smoke by race/ethnicity. Ethn Health. Feb 2009;14(1):75-91.

Farrelly MC, Davis KC, Haviland ML, Messeri P, Healton CG. Evidence of a dose-response relationship between “truth” antismoking ads and youth smoking prevalence. Am J Public Health. Mar 2005;95(3):425-431.

Fishbein M. A consideration of beliefs and their role in attitude measurement. In: Fishbein M, ed. Readings in attitude theory and measurement. New York, NY: Wiley; 1967:257–266.

Hornik R. Public health communication: Evidence for behavior change. London, NJ: Lawrence Erlbaum Associates; 2002.

Murray DM, Prokhorov AV, Harty KC. Effects of a statewide antismoking campaign on mass media messages and smoking beliefs. Prev Med. Jan 1994;23(1):54-60.

Murukutla N, Turk T, Prasad CV, et al. Results of a national mass media campaign in India to warn against the dangers of smokeless tobacco consumption. Tob Control. Jan 2012;21(1):12-17.

National Cancer Institute (NCI). The role of the media in promoting and reducing tobacco use. Tobacco Control Monograph No. 19. Bethesda, MD: U.S. Department of Health and Human Services, National Institutes of Health, National Cancer Institute 2008.

Rosenstock IM, Strecher VJ, Becker MH. Social learning theory and the Health Belief Model. Health Educ Q. Summer 1988;15(2):175-183.

Trickett EJ, Beehler S, Deutsch C, et al. Advancing the science of community-level interventions. Am J Public Health. Aug 2011;101(8):1410-1419.

Vallone DM, Duke JC, Cullen J, McCausland KL, Allen JA. Evaluation of EX: a national mass media smoking cessation campaign. Am J Public Health. Feb 2011;101(2):302-309.

Vallone DM, Niederdeppe J, Richardson AK, Patwardhan P, Niaura R, Cullen J. A national mass media smoking cessation campaign: effects by race/ethnicity and education. Am J Health Promot. May-Jun 2011;25(5 Suppl):S38-50.