Supporting Statement

Substance Abuse and Mental Health Services Administration

Center for Substance Abuse Prevention

Underage Drinking Prevention National Media Campaign

Materials Development and Testing Plan

Supporting Statement

1. JUSTIFICATION

1. Circumstances of Information Collection

The Substance Abuse and Mental Health Services Administration (SAMHSA) Center for Substance Abuse Prevention (CSAP) seeks a modification to its existing Office of Management and Budget (OMB) approval (OMB No. 0930-0196; Expiration Date: 09/30/2013) for its work to develop and launch the congressionally mandated Underage Drinking Prevention National Media Campaign (the Campaign). The purpose of this requested modification is to amend the existing OMB package to also include a series of quantitative surveys. These surveys will allow for the testing of Campaign materials, specifically during the pilot testing phase—results from which will inform the national Campaign and its launch, currently scheduled for the fall of 2012. Included in this submission are the information and surveys pertaining to this modification request only (the full, approved OMB package is available upon request).

Campaign History

On April 13, 2010, SAMHSA received OMB approval to conduct three phases of formative focus group testing (including approval of methodology and interview guides and screeners) for this important Campaign, as outlined below:

Phase 1: Formative focus group testing was conducted with parents and youth between the ages of 9 and 15 to understand their attitudes, beliefs, and feelings regarding underage drinking (focus groups, interviews, etc.).

Phases 2 and 3: Message testing with parents of youth ages 9 to 15 on a series of themes and concepts uncovered during Phase 1 to inform the final messages of the Campaign. Messages developed during Phases 2 and 3 were informed from the original focus group testing done with parents and youth.

Since the conclusion of these phases in the fall of 2011, and based on insights from this testing, SAMHSA developed a series of print, radio, and television public service announcements (PSAs)—the cornerstone of this Campaign—that it will test in five select pilot sites across the country and through others means. These pilot sites were chosen via a competitive request for proposal process and will be required to disseminate the PSAs in their local communities. The quantitative survey effort for which we seek OMB approval will allow SAMHSA to assess the effectiveness of the PSAs within these pilot sites and also provide insight for SAMHSA into what, if any, other changes it needs to make to these PSAs to increase their effectiveness when disseminated nationally during the launch. Without this effort, the Campaign will not have a measure by which to pre-test the materials intended for national distribution and therefore, leaves SAMHSA vulnerable to disseminating materials that have not been vetted with the intended audience, as strategic communication best practice advises.

Campaign Background

In 2003, Congress funded SAMHSA to address concerns about underage drinking after a study by the National Research Council drew attention to the alarming prevalence of alcohol consumption by our Nation’s youth. In 2006, Congress confirmed its commitment by passing the “Sober Truth on Preventing Underage Drinking Act,” or the “STOP Act.” The legislation called for a “multi-faceted effort” to include prevention, intervention, treatment, enforcement, and research. Today, through the STOP Act, SAMHSA supports national, State, and local efforts to prevent and reduce underage drinking.

The SAMHSA Underage Drinking Prevention National Media Campaign began with public service advertisements (PSAs) created by the Ad Council, first released in October 2005, with additional PSAs released in the spring of 2010. The Campaign has expanded to focus on broader marketing communication approaches, attempting to complement and build on all other SAMHSA underage drinking prevention efforts to create a unified, consistent voice.

Acknowledging that parents are the greatest influence on children, SAMHSA commissioned the Campaign to reach parents of children ages 9 to 15 to engage them more fully in preventing underage drinking. The Surgeon General’s Call to Action to Prevent and Reduce Underage Drinking (2007) states as one of its major goals “to engage parents,” and offers specific strategies for building parent skills and facilitating parental interaction. This Campaign will reach parents through traditional PSAs in TV, radio, and print venues, but also will incorporate other message dissemination tools such as social media, entertainment media, workplace communication, and work with intermediaries and partners. In order to test the meaningfulness and utility of the messages and materials prior to their launch in the fall of 2012, it is essential to gather information from pilot sites and others exposed to Campaign materials so that SAMHSA may better understand and gauge the impact the Campaign is having, and will continue to have, on parents and caregivers.

2. Purpose and Use of Information

This data collection effort will allow SAMHSA to test the Campaign during a pilot phase through process measures on Campaign activities, as well as outcome measures that will measure changes in parent knowledge, attitudes, and behaviors that result from being exposed to the Campaign. Given the budgetary and time restraints in reaching a large percentage of U.S. parents, SAMHSA has adopted a targeted strategy to effectively test the Campaign’s progress in accomplishing its goals of persuading parents with children ages 9 to 15 to start talking early with their children about the dangers of underage drinking. Specifically, SAMHSA will test the Campaign's effect on groups of parents who have been exposed to Campaign activities, materials, and messages. SAMHSA will do this through the pilot sites and the readily available Gallup Panel—A nationally representative Panel of households and individuals.

SAMHSA proposes using the Gallup Panel to assess awareness of the Campaign and effectiveness of Campaign messages and materials. The Gallup Panel is an existing mechanism easily accessed by Gallup and requires minimum effort for recruitment, incentives, and other typically resource heavy activity. The pilot sites also are an existing and easily accessed mechanism through which to conduct this work. The proposed materials and message testing will have both longitudinal and cross-sectional aspects. As part of the Campaign efforts, the following activities will be conducted to assess the effectiveness of the developed and still draft, PSAs and Campaign materials:

Quantitative Pre-Launch Pre-test

• Gallup will use a small sample of Panel members to provide feedback on Campaign materials prior to the national launch. Respondents will be asked to review specific Campaign materials and provide reactions to content, layout, and delivery.

Forced Exposure Longitudinal

• The testing will begin with an initial sample universe of Gallup Panel parents of children ages 9 to 15. This sample will be split in half to determine those who will receive the Campaign materials (“stimuli group”) and a representative “control group” who will not receive the Campaign materials.

• Gallup will administer a Web survey to establish a baseline of familiarity with the Campaign and its messages, and current knowledge, attitudes, and behaviors related to underage drinking.

• Within three weeks of the baseline survey of both the stimuli group and the control group, Gallup will send a set of Campaign materials to the parents in the stimuli group for use and review at home.

• Subsequent to the mailing of the materials to the select group of parents, Gallup will survey participants from both the stimuli and control groups without notice at the roughly three -month interval (or less) after initial exposure to assess how their knowledge, attitudes, and behaviors may have changed over time.

Natural Exposure/Event Evaluation

• In addition to the use of the Gallup Panel, SAMHSA proposes surveying a small sample of parents who attend specific Campaign events at each of the 5 pilot sites. Gallup will administer a pencil and paper survey of parents attending these pilot site (n=500) and partnership events. Respondents will be asked to assess the event and Campaign materials, and provide information on current knowledge, attitudes, and behaviors related to underage drinking. These surveys will be administered as participants enter these events.

• After the initial survey of event attendees, Gallup will conduct a roughly three -month follow-up (or less) with respondents via a Web survey to measure changes in their knowledge, attitudes, and behaviors as a result of seeing the Campaign materials at the pilot site, or other, event.

3. Use of Information Technology

The evaluative surveys will be conducted via Web. Gallup will program and maintain a Web-based platform to conduct the surveys.

4. Efforts To Identify Duplication

The information needed is specific to the Campaign’s messages and materials and is not collected anywhere else.

5. Involvement of Small Business Entities

As the information collection will go to individuals, no impact on small businesses, organizations, or government bodies is expected.

6. Consequences if the Information is Collected Less Frequently

This information will only be collected as outlined above, to test the success of the Campaign as well as to inform any future efforts regarding targeting parents for the purposes of reducing and preventing underage drinking. Any further information collected will be cleared in accordance with PRA requirements. There are no legal obstacles to reduce the burden.

7. Consistency with the Guidelines in 5 CFR 1320.5(D)(2)

The information collection effort will be in compliance with the guidelines set forth in 5 CFR 1320.5(D) (2).

8. Consultation Outside the Agency

There was no consultation outside the Agency for this phase of the work.

9. Decision to Provide Payment to Respondents

There are no incentives offered as part of these data collection efforts.

10. Assurances of Confidentiality

For the materials testing, individuals contacted will be assured that the survey is anonymous and that all sensitive information will be protected.   Below are four principals to guide the protection of privacy:

• Names and contact information are collected and stored separately from survey responses and are linked only by a unique code.  Names and contact information are only used for follow-up contact and only with permission of the participant.

• All data is kept on secured servers and access is limited to select personnel.  All personnel sign a confidentiality agreement every year and participate in mandatory data security training.

• Only aggregated data will be released.  No individually identifiable information will be used in any analysis, or released to any other party.

• Information is used only for the purpose it was attended and expressed in the survey document.  Respondent identities and information are never used for any other purpose.

Gallup is a member of CASRO and adheres to the code of standards and ethics for survey research.

11. Questions of a Sensitive Nature

On the surveys, parents will be asked about their concerns for their children. One or two topics may emerge that include activities that are illegal, such as underage use of alcohol. Other topics considered sensitive may be poor performance in school or how best to speak with children. These specific topics are not considered highly sensitive; a point confirmed in the results from previous data collection efforts where parents did not display angst or concern with answering these types of questions about their children.

12. Estimates of Annualized Hour Burden

For the testing, a total of five survey undertakings will be completed, the first being the quantitative pre-launch test with 250 respondents, the forced exposure baseline and follow-up with 2,500 respondents, and the natural exposure baseline and follow-up with 500 respondents.

Table 1: Respondent Burden Hour and Cost Estimates

Type of Respondent	Form Name	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours	Hourly Wage Rate1	Total Res. Costs
Gallup Panel Member	Quantitative Pre-launch Pre-test Survey	250	1	20/60	83	$16.27	$1,350.41
Gallup Panel Member	Forced Exposure Survey	2500	1	18/60	750	$16.27	$12,205.50
Gallup Panel Member	Forced Exposure Follow-up Survey	2500	1	18/60	750	$16.27	$12,205.50
Subtotal		5,250			1,583		$25,761.41
General Population	Natural Exposure Survey	500	1	18/60	150	$16.27	$2,440.50
General Population	Natural Exposure Follow-up Survey	500	1	18/60	150	$16.27	$2440.50
Subtotal		1,000			225		$4,881
TOTAL		6,250			1,808		$30,642.41

The annualized cost to respondents was estimated to be the burden hours estimate multiplied by the median hourly wage estimate (Source: Occupational Employment Statistics (OES), BLS). The median wage estimate ($16.27) used for the purpose of this calculation is for all occupations combined.

13. Estimates of Annualized Cost Burden to Participants

There are no total start-up costs, capital costs, or operation or maintenance costs.

14. Estimates of Annualized Cost to the Government

The estimated annualized cost to the government is $191,520. This cost reflects anticipated time spent by the project’s Contracting Officer’s Representative (COR) and Alternate COR to assist with the execution of this evaluation effort ($11,520); and the cost to SAMHSA for employing a contractor (Gallup) to administer the surveys ($180,000). Specific evaluation-related activities performed by the contractor include:

• Administering five total Gallup Panel and Pilot Site surveys;

• Traveling once to each of the five pilot sites; and

• Conducting analysis and report writing.

Cost estimates for project COR and Alternate COR activity are broken out by percentage of time and hours in Table 2: Estimated Annualized Cost to the Government.  Annual hours are based on a 40-hour work week for 48 weeks per year.

Table 2: Estimated Annualized Cost to the Government

Position	Percent FTE	Annual Hours	Rate*	Total Annual Cost
SAMHSA Alternate COR	10%	192	$40	$7,680
SAMHSA COR	5%	96	$40	$3,840
	Totals	2,976		$11,520

* Rate for SAMHSA staff includes fringe and overhead.

15. Changes in Burden

This is a new data collection.

16. Time Schedule, Publication, and Analysis Plans

Table 5: Timeline

Activity	Date
Quantitative Pre-launch Pre-test survey	May 2012
Forced Exposure Baseline survey	May 2012
Forced Exposure Follow-up survey	July 2012
Natural Exposure survey	May –July 2012
Natural Exposure Follow-up survey	July – October 2012

17. Display of Expiration Date

Expiration date will be displayed.

18. Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.

2. SURVEYS AND EVALUATIONS EMPLOYING STATISTICAL METHODS

B.1. Respondent Universe and Sampling Method

As mentioned in Part A of this supporting statement, SAMHSA will undertake testing of the Campaign messages and materials through process measures to assess Campaign activities as well as outcome measures to examine changes, if any, in parent knowledge, attitudes, and behaviors resulting from the Campaign. For this purpose, the following five surveys will be conducted: (i) Quantitative Pre-launch Pre-test Survey (ii) Forced Exposure Baseline Survey (iii) Forced Exposure Follow-up Survey (iv) Natural Exposure Baseline Survey and (v) Natural Exposure Follow-up Survey. The target population or the respondent universe for each of these five surveys is the population of parents in United States (50 States and DC) with children between the ages of 9 and 15. It is estimated that there are about 28 million children in the U.S. in this age-group (9 to 15) and about 17 percent of all households in U.S. contain at least one child from that age-group.

For the purpose of sampling eligible parents (i.e., parents of children in the age-group 9 to 15) for the first three surveys previously mentioned, the Gallup Panel will be used as the sampling frame. The following provides a description of the Gallup Panel: The recruitment for Gallup Panel began with an RDD (Random Digit Dialing) sample of telephone numbers, including both landline and cell phone samples. Following a list-assisted telephone sample design, a telephone sample representing the U.S. national population was obtained. Once the random sample was obtained, Gallup’s experienced interviewers called those numbers following an approved calling protocol and recruited members for this Panel. This process is repeated to ensure continuous recruitment for the Panel. Currently, the Panel includes approximately 47,514 households containing about 66,421 individuals. Respondents take a short survey about Presidential approval and other current event topics, and are asked if they would be interested in participating in additional surveys as a member of the Gallup Panel. Unlike opt-in Panels, the recruitment process for the Gallup Panel starts with a random sample of telephone numbers and, as a result, it is possible to derive the selection probability and hence the sampling weight for each respondent on the Panel. There is no time commitment to membership in the Gallup Panel. Rather, households and individuals are encouraged to remain members as long as they are willing and interested. As with any longitudinal design, Gallup’s Panel also is affected by attrition. Significant efforts are taken to retain Panelists for as long as possible. Gallup’s Panel also performs very well in terms of eliminating “professional” respondents who tend to volunteer for most of the opt-in Panels.

As previously mentioned, the Gallup Panel will be used as the sampling frame for sampling eligible parents. All Gallup Panel participants have been fully screened and a substantial amount of background data including “age of children” have already been collected, permitting rapid and efficient sub-sampling of subjects based on selection criterion. Further details of the sampling procedures to be used for each of the five surveys follow.

Quantitative Pre-launch Pre-test Survey: Gallup will use a small sample (approximately of size 250) of Panel members to provide feedback on finalized Campaign materials prior to the national launch. In order to ensure that the sample is nationally representative, the sampling frame of all eligible parents in the Gallup Panel will be stratified using appropriate stratification variables. For each respondent in the Gallup Panel, a significant amount of background information (including demographic variables) will be available for stratification. For example, geographic regions and relevant demographic variables may be chosen as stratification variables. A simple stratified sample design will be used to select this sample from the Gallup Panel. Given the relatively lower incidence of households with parents of children in the 9 to 15 year age-group, the use of the Gallup Panel will be an optimal way to effectively sample this population of parents.

Forced Exposure Longitudinal Survey (Baseline and Follow-up): An initial sample of size 2,500 will be drawn from the Gallup Panel for this survey using a simple stratified sample design. In order to ensure adequate representation from all subgroups, the stratification variables may be selected from demographic and other relevant variables including geographic regions. This sample will be randomly split in half to determine those who will receive the Campaign materials (“stimuli group”) and a representative “control group” who will not get the Campaign materials. A baseline Web survey will be conducted with both samples (of size 1250 each) to measure the familiarity with the Campaign and its messages, and their current knowledge, attitudes, and behaviors related to underage drinking. Within three weeks of the baseline survey of both the stimuli group and the control group, a set of Campaign materials will be sent to the parents in the stimuli group for use and review at home. Subsequent to the mailing of the materials to the stimuli group of parents, a follow-up Web survey of participants from both the stimuli and control groups will be conducted without notice at the 3-month interval after initial exposure to assess how their knowledge, attitudes, and behaviors changed over time.

Natural Exposure Survey (Baseline and Follow-up): SAMHSA proposes to survey a small sample of parents who attend specific Campaign events. Gallup will administer a pencil and paper survey of parents attending pilot sites and partnership events throughout the Campaign. The goal is to complete about 500 surveys with event attendees. Upon entering the event, respondents will be asked about their current knowledge, attitudes, and behaviors related to underage drinking. After the initial survey of event attendees, a follow-up survey will be conducted after three months (or less) with respondents via a Web survey to measure change in awareness, knowledge, attitude, and behavior, and to obtain feedback on the Campaign materials distributed at the event. For the initial survey, the sample will be selected from the event attendees. For the follow-up survey, the sample will consist of respondents of the initial survey.

B.2. Information Collection Procedures

The mode of data collection for the five surveys mentioned in section B.1 will be Web except for the Natural Exposure Baseline Survey where the survey of event participants will be administered using a paper and pencil survey.  Both methods take care to protect the privacy of the respondent by adhering to the strictest standards of data security allowed by law.   Paper surveys are separated from the paper containing respondent names and emails, and both sets of data are stored separately and securely.  Web data is entered using a unique single use code and the data is stored on a secure server accessible only by selected personnel who are trained in data security.

B.3. Methods to Maximize Response Rate

All necessary steps will be taken to achieve the maximum possible response rates. The survey questionnaires will be carefully designed keeping in mind the requirement of maximizing response rates. For the Web surveys, several email reminders will be sent periodically to boost co-operation and thereby increase the overall response rates. The expected response rate for the Quantitative Pre-launch Pre-test Survey and the Forced Exposure Longitudinal Surveys (Baseline and Follow-up) is expected to be around 65 percent. The Natural Exposure Surveys (Baseline and Follow-up) is assumed to be about 50 percent.

B.4. Tests of Procedures

The primary goal of data analysis will be to test the Campaign messages and materials and also to measure changes in parent knowledge, attitudes, and behaviors resulting from the Campaign, some of which were uncovered during the qualitative focus group testing phases of this project. The Quantitative Pre-launch Pre-test Survey data will be analyzed to finalize Campaign materials. Data from the Forced Exposure Longitudinal Surveys (Baseline and Follow-up) will be compared using statistical tests to detect changes, if any, in parent knowledge, attitudes and behaviors related to underage drinking as a result of the Campaign. Similar comparisons will be carried out using the Natural Exposure Survey data (Baseline and Follow-up) to assess the Campaign and also to review changes resulting from the use of Campaign resources.

B.5. Statistical Consultants

Name	Agency/Company/Organization	Number Telephone
Manas Chattopadhyay, PhD.	Gallup Organization	202.715.3179
Susan Conner, MSc.	Gallup Organization	202.715.3124
Camille Lloyd, MSc.	Gallup Organization	202.715.3188
Christina Zurla	Gallup Organization	202.715.3192
Robert Vincent	SAMHSA/CSAP/DSD	240.276.1582

List of Appendices

Appendix 1: Pre-launch Quantitative Survey

Appendix 2: Forced Exposure Baseline Survey

Appendix 3: Forced Exposure Follow-up Survey

Appendix 4: Natural Exposure Baseline Survey

Appendix 5: Natural Exposure Follow-up Survey

Appendix 6: Campaign Creative Materials

Appendix 7: Campaign Materials Testing Plan

1 http://www.bls.gov/news.release/ocwage.htm

11