OMB_ssa_2024 Clean

Programs to Reduce Underage Drinking

Supporting Statement

Check off which applies:

☐ New

☒ Revision

☐ Reinstatement with Change

☐ Reinstatement without Change

☐ Extension

☐ Emergency

☐ Existing

A. JUSTIFICATION

1. Circumstances of Information Collection

The Substance Abuse and Mental Health Services Administration (SAMHSA) is requesting an extension with revisions from the Office of Management and Budget (OMB) for the Survey of State Underage Drinking Prevention Policies, Programs, and Practices (the “State Survey”). This data collection is under OMB No. 0930-0316, which expires on October 31, 2024.

In recognition of the need for a coordinated approach to address the problem of underage drinking in the United States, the Sober Truth on Preventing Underage Drinking Act (the “STOP Act”) was passed by Congress in 2006, reauthorized in December 2016 as part of the 21st Century Cures Act (Pub. L. 114-255) and the Consolidated Appropriations Act, 2023 (Pub. L.117-328), and codified into law in 42 U.S.C. 290bb-25b: Programs to reduce underage drinking.

The STOP Act expresses the sense of Congress as follows:

A multi-faceted effort is needed to more successfully address the problem of underage drinking in the United States. A coordinated approach to prevention, intervention, treatment, enforcement, and research is key to making progress.

As the following graphic illustrates, this coordinated approach requires systemic engagement at all levels of society, from the individual and family to the community, state, and federal level. Each of the facets—prevention, intervention, treatment, enforcement, and research—are a necessary complement to the other facets. Consequently, the data collection methods described in this package demonstrate efforts to collect critical information at each systemic level, from prevention to enforcement.

Figure 1: Coordinated Approach

To facilitate the coordinated approach, the STOP Act established the Interagency Coordinating Committee on the Prevention of Underage Drinking (ICCPUD), composed of 16 federal officials representing agencies with underage drinking prevention programs or activities (Attachment 1).¹ While not enumerated in the (STOP) legislation, other agencies have chosen to participate. There are currently 22 federal departments and agencies participating in the ICCPUD.​ SAMHSA has been designated as the lead agency to work in concert with ICCPUD in fulfilling the Congressional mandates in the STOP Act.

The STOP Act contains four primary elements:

1. The award of community-based coalition enhancement grants for underage drinking prevention activities to eligible entities currently receiving funds under the Drug-Free Communities Act of 1997.

2. A national adult-oriented media public service campaign to prevent underage drinking, “Talk. They Hear You.” ® (TTHY), and an annual report to Congress evaluating the campaign.

3. An annual report to Congress summarizing federal prevention activities and the extent of progress in reducing underage drinking nationally, including data from national surveys conducted by federal agencies.

4. An annual report to Congress “on each State’s performance in enacting, enforcing, and creating laws, regulations, and programs to prevent or reduce underage drinking.” The State Survey that is the subject of this request gathers data used to develop the state-by-state report on prevention and enforcement activities related to underage drinking.

Driven by the legislation and coordinated by ICCPUD, each of these activities work together to prevent and reduce underage drinking. The data collection activities described in this package serve to assess the outputs and outcomes of efforts to align public health messaging with interventions: in addition to the annual State Survey and formative evaluation of ICCPUD’s Alcohol Policy Academy, data collection activities associated with two component resources from the TTHY national media campaign will be reviewed. (see Figure 2).

Figure 2: Aligning Public Health Messaging and Interventions through the STOP Act and “Talk. They Hear You.”

Underage Drinking and Its Consequences

Underage drinking not only impacts the health and well-being of those who consume alcohol but also extends its repercussions to their families, communities, and society at large. ² 

Alcohol is a factor in the deaths of approximately 3,900 youths in the United States (U.S.), shortening their lives by an average of 57 years.³ Data also reveal the wide-ranging health, social, and fiscal consequences associated with underage drinking.⁴

Alcohol is still the most widely consumed substance among America’s youth—used more often than marijuana or tobacco.⁸ Federal data indicate that 15.1 percent of 12–20-year-olds in the United States reported alcohol use in the past month; in comparison,11.8 percent of 12- to -20-year-olds had used marijuana; 13.4 percent had used tobacco products and/or nicotine by vaping; 2.1 percent had used illicit drugs other than marijuana; and 0.5 percent had used opioids.⁹ Further, underage youth who drink tend to binge drink and to consume more on a single drinking occasion than adults do. Approximately 54 percent of individuals ages 12 to 20 who reported drinking in the past month also reported binge drinking.¹⁰

Meaningful progress has been made in reducing underage drinking prevalence and related problems, such as traffic fatalities, over the past several years. However, it is unclear how recent changes in alcohol policies, enforcement strategies, primary prevention activities, training and technical assistance programs, and national campaign efforts may affect underage drinking. Concerning trends continue to emerge, including the ongoing development of new products that appeal to youth, in addition to observed changes in patterns of youth drinking behavior, such as combining alcohol with other drugs (e.g., marijuana, opioids, prescription drugs) Current (2022) NSDUH data indicate that 23.3 percent of individuals ages 12–20 who reported past month alcohol use also reported having used marijuana simultaneously or within 2 hours of the most recent use of alcohol. ¹¹

The Annual Report to Congress, State Reports, and the State Survey

In response to the health risks associated with underage drinking, the 50 states and the District of Columbia have adopted numerous policies, programs, and practices to alter the individual and environmental factors that contribute to underage drinking and its consequences. These efforts can potentially reduce underage drinking and its consequences and change norms that support underage drinking in American communities. Currently, there are no state or federally sponsored data sources, other than the State Survey, that have gathered information on state-level underage drinking policies, programs, and practices in a uniform and meaningful way.

To monitor progress toward effective responses to underage drinking, the STOP Act directs the U.S. Department of Health and Human Services (HHS) to develop the state-by-state report on prevention and enforcement activities related to underage drinking.⁵ More specifically, the STOP Act requires HHS to report on each state’s performance in “enacting, enforcing, and creating laws, regulations, and programs to prevent or reduce underage drinking.” The annual State Reports include data provided in each state’s Survey responses, which have been reviewed and approved by a Governor-designated representative.

The STOP Act lists nine categories of “best practices” that are required to be used in developing state performance measures. These have been collapsed into three categories for data collection purposes. Several of the items listed are publicly available and are collected independently to reduce the burden on the states. The three collapsed categories are:

Category #1: Enacting Laws and Regulations—The 50 states and the District of Columbia have enacted a significant number of laws and regulations that affect underage drinking. The STOP Act lists 16 specific underage drinking laws/regulations enacted at the state level (e.g., laws prohibiting sales to minors; laws related to minors in possession of alcohol), and 10 additional policies have been added pursuant to Congressional appropriations language or the Secretary’s authority under the STOP Act. SAMHSA uses existing sources of data to track the states’ performance in enacting these 27 legal policies through statute or regulation. Data regarding these policies are collected from the National Institute of Alcohol Abuse and Alcoholism’s (NIAAA’s) Alcohol Policy Information System (APIS), an authoritative compendium of state alcohol-related laws. The APIS data are augmented with original legal research.

Category #2: Enforcement and educational programs to promote compliance with these laws/regulations—The STOP Act states that information shall be included on whether each state “encourages and conducts comprehensive enforcement efforts to prevent underage access to alcohol at retail outlets, such as random compliance checks and shoulder tap programs, and the number of compliance checks within alcohol retail outlets measured against the number of total alcohol retail outlets in each State, and the result of such checks.” While the states’ laws and regulations are publicly available, data regarding the states’ performance in enforcing those laws and regulations, particularly the specific data described above, are not readily available.

Category #3: Programs targeted to youth, parents, and caregivers to deter underage drinking and the number of individuals served by these programs, and interagency and tribal collaborations—Each of the 50 states and the District of Columbia has created state-level programs aimed at preventing or reducing underage drinking. However, information about these programs is not available in a comprehensive way.

In summary, data from categories 2 and 3 do not currently exist in a complete or accessible form from publicly available sources. Some states collect some of the data, but not in a uniform fashion that allows meaningful cross-state comparisons.

SAMHSA has found that the most efficient and effective way to seek “input and collaboration” from the states, as the STOP Act requires, is through an annual survey that combines data from existing state databases and other data sources available to the states. Accordingly, data from Category #2 and Category #3 are collected by the State Survey, a survey tool administered electronically via an online platform, with approximately 200 possible questions that each state and the District of Columbia will complete (Attachment 2). As the Survey is specifically designed to only ask for data that has already been collected, nearly all states complete fewer than 200 questions. The State Survey is further described in Section 2.

In addition to understanding how States are engaged in efforts to prevent underage alcohol consumption, SAMHSA’s national youth substance use prevention campaign, “Talk. They Hear You.”®, helps parents and caregivers, educators, and community members get informed, be prepared, and take action.

Launched in 2013, the original goal of the campaign was to provide parents and caregivers with the resources they need to address the issue of alcohol with their children. Through strategic expansion, the campaign now supports SAMHSA’s efforts to prevent both underage drinking and other substance use by helping parents and caregivers, educators, and community members:

1. Get Informed: Increase their awareness of the prevalence and risks of underage drinking and other substance use;

2. Be Prepared: Equip them with the knowledge, skills, and confidence to prevent underage drinking and other substance use; and

3. Take Action: Encourage and increase their actions to prevent underage drinking and other substance use.

Campaign materials are continuously refined based on feedback from stakeholders and additional resources have been developed and tested since the campaign’s inception in 2012. While the campaign continues to refine and expand its resources to align with best practices for media campaign implementation as well as reflect up-to-date content, this package seeks to evaluate the Parents' Night Out (PNO) educational series. The PNO toolkit was developed for community implementation of interactive, facilitator-led, virtual or in-person, educational sessions designed to inform parents and caregivers about the realities of underage drinking and other drug use. In addition to increasing knowledge and awareness, PNO promotes early and frequent dialogue between adults and youth through the lens of motivational interviewing, an evidenced-based intervention supported by decades of research.

Parents’ Night Out also serves as an introduction to the campaign’s resource library consisting of public service announcements, infographics, discussion-starter guides, and a mobile application. These materials show parents and caregivers using everyday opportunities to talk with their children about alcohol and other drugs while providing age-appropriate materials to underscore the importance of starting the prevention conversation at an early age.

In addition to developing and disseminating the “Talk.  They Hear You.” campaign components, SAMHSA is also charged with evaluating the campaign. Message testing during the development of the media campaign materials coupled with case studies examining the influence of the campaign products on increasing awareness and knowledge about the risks of underage alcohol use as well as reported motivation to engage in conversation with youth was reported by parents. The current focus of the OMB application package focuses on extending the evaluation of the campaign to measure how the Parents’ Night Out educational sessions impact parent and caregiver knowledge, awareness, attitudes and/or behavior regarding talking to their children about underage drinking and other substances. The evaluation will employ a multiple-baseline component-oriented pre/post-test study design to assess the impact of this targeted campaign exposure.

The Parents’ Night Out series currently exists in multiple formats, including both an in-person and virtual delivery model as well as a shortened (1-hr duration) and extended (3-hr duration) versions. The 3-hour version includes exposure to a variety of TTHY campaign tools, including introduction to, and use of, the TTHY mobile application as well as opportunities to practice applying the motivational interviewing-based conversation tools. Use of the component-approach (e.g., adding, removing, or updating TTHY product/campaign materials based on feedback and evolution of the campaign in response to a changing landscape) allows for continuous improvement in response to ongoing feedback, enabling the campaign to utilize a community-engaged approach as well as remain current in an ever-evolving substance use landscape. The evaluation team will assess whether greater exposure to the materials results in increased knowledge, awareness, behavior change. Embedding evaluation into the larger campaign approach is also consistent with best practices in monitoring and evaluation research and SAMHSA’s larger strategic plan.⁶

2. Purpose and Use of Information

The data collection efforts included in this package will provide SAMHSA and the ICCPUD with a comprehensive understanding of state and national underage drinking prevention and reduction activities. The data collected through these mechanisms will be used to better inform and align public health messaging with intervention. The findings will be shared with Congress, states, and the broader public through reports, briefings, a national campaign, and school and health care settings.

Specifics for each data collection activity are described within each section of this submission: the STOP Act State Survey, the ICCPUD Alcohol Policy Academy formative evaluation, the Screen4Success Pilot implementation evaluation, and the evaluation of two TTHY campaign resources: the mobile application and Parents Night Out (PNO).

STOP Act State Survey

The State Survey has been distributed to all 50 states and the District of Columbia since 2011, achieving full participation (100 percent response rate) each year. This response rate over the entirety of the Survey’s existence is an indication of the states’ dedication to the task of preventing underage drinking, particularly given the challenges of gathering data from multiple state agencies, including behavioral health, enforcement, and other divisions of state government. It is also an indication that the data gathered and reported in the State Reports and State Performance and Best Practice Guide (SPBP) are useful to the states. In recent years, SAMHSA has hosted multiple virtual meetings to introduce the Survey to the state contacts responsible for completing it. These have been well attended, and attendees have expressed a strong interest in learning more about how they can utilize both their own state’s data, and the aggregate Survey data found in the SPBP to further their underage drinking prevention goals.

The purpose of the data collection through the State Survey is to create a compendium of the states’ best practices and performances in enacting, enforcing, and creating laws, regulations, and programs to prevent or reduce underage drinking. Congress mandated the collection of these data to provide policymakers and the public with otherwise unavailable but much-needed information regarding state underage drinking prevention policies and programs.

SAMHSA and other federal agencies that have underage drinking prevention as part of their mandate currently use the results of the State Survey to inform federal programmatic priorities and to track progress in the national effort to reduce underage drinking. The information gathered by the State Survey is an ongoing resource for state agencies and the general public that describes enforcement activities and funding priorities, assesses policies, programs, and practices in their own state, and familiarizes them with practices in other states. The survey results may also be used as a first step in research to develop states’ best practices guidelines for future Reports to Congress.

States are asked to complete the annual Survey that comprises the following four sections:

1. Enforcement of underage drinking laws including, but not limited to:

   1. The number of compliance checks (random and non-random) measured against the total number of alcohol retail outlets in each State.

   2. The result of these checks.

   3. Implementation of Shoulder Tap and Party Patrol operations.

   4. The number of sanctions (fines, suspensions, revocations) imposed on retailers for violations of underage drinking laws.

2. Underage drinking prevention programs targeted to youth, parents, and caregivers, including data on the approximate number of persons served by these programs.

3. State best practices’ standards and collaborations with tribal governments and state interagency collaborations used to implement the above programs; state participation in social marketing media campaigns intended to reduce underage drinking.

4. Workforce development activities, including strategies and funds expended on recruiting and retaining a behavioral health workforce.

Policy Academy Formative Evaluation

The Policy Academy strives to prevent and reduce underage and excessive drinking by increasing the capacity of community coalitions to modify the community context through the policy process. The Policy Academy includes 14 coalitions from across the U.S., with two individuals from each coalition serving as the participants (“coalition participants”). The Policy Academy formative evaluation is designed to measure changes in a coalition’s capacity to implement policy campaigns and progress through each of the ten policy steps in the Policy Academy.

As part of the Policy Academy, coalitions receive training and technical assistance from alcohol policy experts – they are referred to as “coaches” in the program. Each cohort will have a set of experienced coaches leading the work with the coalitions and will also include a set of mentees who will learn the policy process and gain experience training the coalition participants. As such, this evaluation will measure the increase in the policy training workforce through a coaches and mentee development pipeline.

This evaluation is designed to assess the implementation components of the Policy Academy – there is not an expectation to see policy change given the length of time it takes to pass a policy at the local or state level. Process measures evaluating changes in capacity and workforce development will be the focus of this nascent program.

The evaluation is comprised of seven surveys and one focus group. The focus group will collect qualitative data from the coalition participants on their experience and efforts to incorporate health equity into their policy campaign. The surveys and corresponding audiences are outlined below in Table 1.

Table 1: Policy Academy Data Collection Activities

“Talk. They Hear You.”

The “Talk They Hear You” campaign is comprised of a variety of tools and resources designed to decrease underage drinking by encouraging parents and caregivers, educators, and community members/organizations to proactively engage youth in conversations about alcohol another other drugs. Research has demonstrated that active and engaged adults can reduce underage drinking.⁷ One TTHY mechanism to engage adults is through PNO.

The Parents’ Night Out (PNO) Evaluation will assess changes in knowledge, skills, and confidence of parents and caregivers after receiving the training and materials for PNO and TTHY products. This evaluation will be delivered in partnership with community partners, who will be exposed to varying combinations of PNO and materials to determine change before and after exposure. The information gleaned in surveys of parents and caregivers will allow the evaluation team to assess whether PNO is being implemented as intended, and which products are most useful in increasing parents’ and caregivers’ capacity and intentions. The results will be shared with the implementation team for PNO curriculum modifications and for updating TTHY materials. Similarly, information collected through training and technical assistance via customer service feedback forms, webinar registrations, and newsletter sign-up will be used to continuously improve the materials to best serve the needs of the users.

The results of the PNO evaluation—as well as progress on the broader campaign evaluation —will be reported in an annual report to Congress. Section 2(d) of the STOP Act requires the Secretary of the U.S. Department of Health and Human Services to annually report on the production, broadcasting, and evaluation of the TTHY campaign. Since the STOP Act went into effect, SAMHSA has reported annually on the progress of the campaign, including related components such as PNO, and campaign evaluation in fulfillment of this act

3. Use of Information Technology

Information technology will ease the burden of those completing surveys and standardize and streamline data analysis and reporting. Qualtrics© Survey Suite will be used to develop and disseminate the electronic version of the surveys. The use of the Qualtrics© Survey Suite will enable SAMHSA to use branching logic in the electronic survey that provides respondents with different questions based on their responses to previous questions (and effectively enables respondents to skip over the questions that do not apply to them). Such branching logic will reduce the average time of the survey for respondents, thereby reducing overall burden. The use of Qualtrics© also provides SAMHSA with the ability to randomize response options in order to mitigate response option bias. Qualtrics© also includes built-in Section 508 compliance systems to accommodate respondents with vision impairments and built-in mobile accessibility to accommodate respondents using a range of electronic devices. SAMHSA owns and controls all data collected using Qualtrics©, and the data maintained in the Qualtrics© database will be kept confidential. Qualtrics© uses industry best practices to maintain data security and privacy and has devised proprietary methods to prevent disclosing data to the wrong requester due to programming errors.

STOP Act State Survey

As required by the STOP act, the unit of analysis for the State Survey is the state. Accordingly, there are 51 total respondents (50 states and the District of Columbia). However, data to complete the survey resides in various state agencies, and multiple staff may be called on to provide specific data elements.

To ensure that the State Survey obtains the necessary data while minimizing the burden on the states, SAMHSA conducted a lengthy and comprehensive planning process. SAMHSA sought advice from key stakeholders (as mandated by the STOP Act) by hosting multiple stakeholder meetings, conducted two field tests with state officials likely to be responsible for completing the State Survey, and investigated and tested various State Survey formats, online delivery systems, and data collection methodologies. As noted above, SAMHSA continues to gather feedback on the Survey through virtual meetings held to introduce the survey each year, and to provide fora for discussing the data collected and its usefulness to the states.

Based on these investigations, SAMHSA decided to collect the required data electronically, using an online survey data collection platform (Qualtrics). Links to the four sections of the Survey are distributed to states via email. The electronic format offers a key advantage since in most states, multiple agencies provide data and agency offices may be geographically dispersed. The electronic format allows agencies to distribute copies of relevant sections to the appropriate offices for completion. States have indicated the electronic format facilitates efficiency and coordination, and reduces burden.

Once a state completes the Survey in Qualtrics, their answers can be provided back to them for future years, only requiring them to respond to the question if there are changes to be made. This will reduce response burden and allow for more consistent tracking of data longitudinally.

Policy Academy Formative Evaluation

The Policy Academy Formative Evaluation surveys are distributed to coalition participants and coaches/mentees via email and administered through Qualtrics. Collecting data through Qualtrics improves response rates through its ease of use and ability to quickly provide feedback.

The Policy Academy Formative Evaluation surveys are distributed to participants via email and administered through Qualtrics. Collecting data through Qualtrics improves participant experience and allows participants to quickly provide feedback.

“Talk. They Hear You.” - Parents’ Night Out Evaluation

PNO data will be collected from participants through a survey delivered via email using Qualtrics. Completing the survey is not a requirement of the event, but an option to provide feedback to the campaign team. Collecting data through Qualtrics will improve the participant experience and allow them to quickly provide feedback.

4. Efforts to Identify Duplication

STOP Act State Survey

The STOP Act requires an assessment of the four categories of information discussed in Sections A.1 and A.2 for each of the 50 states plus the District of Columbia. SAMHSA is relying on existing data sources where they exist.

As noted on page 4, SAMHSA uses data on state underage drinking policies (Category #1 of the four categories included in the STOP Act) from the Alcohol Policy Information System (APIS). APIS data are augmented by SAMHSA with original legal research on state laws and policies addressing underage drinking to include all the STOP Act’s requested laws and regulations.

In contrast, state enforcement data (Category 2) are not publicly available. NIAAA comprehensively analyzed alcohol policy enforcement databases in 2005¹⁸ and found:

1. Data tend to be aggregated, making it difficult to differentiate between measures of enforcement that pertain to different alcohol policies and/or to different target populations, including those defined by factors such as age, which may be relevant to understanding the impact of enforcement on underage drinking.

2. Data collection may be limited to one or two years.

3. Sources used are not always consistent across years, raising issues of year-to-year comparability in longitudinal studies.

4. There are large gaps in the availability of data on significant measures. The available data are focused primarily on the actions of individual consumers (or violators of the law), whereas data on enforcement and compliance by alcohol merchants and retailers, institutions, or other corporate entities are much less available.

5. Data on enforcement resources (e.g., budgets, staffing levels, numbers of compliance checks conducted, etc.) are not readily available.

6. Databases often do not contain data from all 50 states and the District of Columbia, or data coverage varies from year to year.

In preparing this Supporting Statement, SAMHSA conducted extensive online research and expert consultation to determine whether any new sources of state underage alcohol enforcement data exist that could be used to prepare the state reports in lieu of conducting the survey. In addition, SAMHSA reviewed the databases described in the NIAAA 2005 document. SAMHSA determined that the conclusions drawn by NIAAA are still valid. No available sources exist for the enforcement data required by the STOP Act.

In preparing this Supporting Statement, SAMHSA searched for sources of data on state underage drinking prevention programs (Category 3). In addition, SAMHSA reviewed numerous state government websites to determine if data on the states’ underage drinking programs were available. Some data are available piecemeal, covering some topics for some states. Few of these data have been systematically collected, and they do not provide the longitudinal data required by the STOP Act. Most of the state websites did not provide data on all the prevention programs the states reported in the survey in the previous year. For those states that do compile some of the data elements that are requested, they can transcribe the data directly into the survey instrument.

In conclusion, no data sources were identified that approach meeting the requirements of the STOP Act. Thus, the State Survey is not duplicative of any current data collections.

Policy Academy

The Policy Academy is in its first year, and this evaluation is comprised of all new data collection. While some coalition participants may have completed other substance use prevention trainings and academies, none have been comprehensively evaluated and no public evaluation reports exist for those trainings. Thus, these data will be novel and unique to this academy and participants. The surveys were reviewed to ensure no unnecessary duplication between surveys. Additionally, the focus group questions will differ from those in the surveys.

“Talk. They Hear You.”

The data collection efforts outlined in this document are specific to the campaign evaluation and are not duplicated elsewhere. 

5. Involvement of Small Entities

STOP Act State Survey

This data collection will have no impact on small entities.

Policy Academy Formative Evaluation

The evaluation surveys that coalition participants and coaches/mentees Policy Academy participants complete will require approximately five minutes, twice per month (once after each training and once after each coaching call) – the time to complete these surveys is considered part of their Academy activities. Additionally, the focus group will take place during the in-person component of the Policy Academy and will take approximately one hour.

The coach evaluation survey will have no impact on small entities.

“Talk. They Hear You.” - Parents’ Night Out Evaluation

The TTHY campaign relies on partnerships with community organizations and will utilize their support in parts of the data collection.

The distribution of the PNO Evaluation survey will be facilitated by local organizations who host a PNO event. They will be provided with the link to the survey and will be asked to spend a few moments of the presentation to share the link. All data collection material creation and preparation, including programming and links, will be handled by the TTHY campaign team, minimizing the burden for small entities.

6. Consequences if Information is Collected Less Frequently

STOP Act State Survey

Each respondent must respond once, annually, in accordance with the STOP Act.

Policy Academy Formative Evaluation

The Policy Academy Formative Evaluation surveys and focus group are administered at different frequencies. The focus group will only occur once to provide qualitative insights that predominantly quantitative surveys alone, cannot. Surveys used to assess the coaching and training calls are distributed after each call. Without this information, the coaches and staff are not able to make regular and rapid improvements to the Policy Academy. Additionally, the baseline, follow-up, and six-month follow-up surveys are only administered to coalition participants, one time – these data are most useful for helping inform future Policy Academy activities. However, coalition participants and coaches/mentees are not required to complete any data collection activities and are not penalized if they do not.

“Talk. They Hear You.” - Parents’ Night Out Evaluation

The PNO Evaluation will utilize a multiple-baseline design, where three community sites will receive PNO materials and TTHY products. The evaluation surveys will be administered to participants before and after participation PNO and exposure to the campaign. Using multiple baselines allows evaluators to understand the impact of different “dosages” of materials and duration of activities. Collecting this in a single, static evaluation would prove more difficult for evaluators to draw meaningful conclusions about the potential influence of the campaign exposure in any perceived changes reported or observed.

7. Consistency With the Guidelines in 5 CFR 1320.5(d)(2)

All information proposed under this OMB package fully complies with 5 CFR 1320.5(d)(2).

8. Consultation Outside the Agency

a. Federal Register Notice

The notice required in 5 CFR 1320.8(d) was published in the Federal Register on 4/22/2024 (Vol. 89, page 29345). SAMHSA did not receive any comments.

b. Consultations Outside of the Agency

STOP Act State Survey

SAMHSA consulted with the ICCPUD and several experts in the revision of the State Survey. In addition, SAMHSA incorporated feedback obtained from respondents over the course of administering the survey under the current OMB approval. Based on these consultations, SAMHSA ensured that the data to be collected did not exist in another form, the survey instrument was clearly written, and the Survey was easy to complete.

Consultants

Cheryl McDonnell, Ph.D.

Senior Research Scientist

Synergy Enterprises, Inc.

Alicia Sparks, Ph.D., M.P.H.

Senior Principal

Synergy Enterprises, Inc.

Carol Cannon, M.A.

Research Scientist

Synergy Enterprises, Inc

Carrie Cristofes

Executive Director

National Liquor Law Enforcement Association

Kedar Dange, M.P.H.

Public Policy Analyst

National Alcohol Beverage Control Association

Policy Academy Formative Evaluation

The Policy Academy Formative Evaluation requires the expertise of a team of evaluators to ensure that the questions support a rapid cycle feedback model that will allow real-time improvements to the content and process of the Policy Academy. These consultants include experts in alcohol policy as well as evaluation methodology.

Consultants

Alicia Sparks, Ph.D., M.P.H.

Senior Principal

Synergy Enterprises, Inc.

Erin Burns, Ph.D.

Principal, Data and Evaluation Team Lead

Synergy Enterprises, Inc.

Rachel Edsall, M.P,H.

Associate, Data and Evaluation

Synergy Enterprises, Inc.

Michael Sparks, M.A.

President

SparksInitiatives

“Talk. They Hear You.” - Parents’ Night Out Evaluation

To complete the PNO evaluation, SAMHSA will partner with community organizations to deliver PNO and other TTHY materials. To train them in how to properly implement the materials, we will partner with an outside expert.

Consultants

Erin Burns, Ph.D.

Principal, Data and Evaluation Team Lead

Synergy Enterprises, Inc.

Rachel Edsall, M.P.H.

Associate, Data and Evaluation

Synergy Enterprises, Inc.

Debbie Berndt

Founder

Parent Movement 2.0

9. Payment to Respondents

No incentives are provided for completing any of the data collection activities described within.

10. Assurance of Confidentiality

Data will be kept private to the extent allowed by law.

STOP Act State Survey

As required by the STOP Act, all data will be gathered by state representatives designated by the Governors. The questionnaire requests the names of contact persons in five places with the following instructions:

Please provide the name and phone number or email of someone we can contact for additional clarification of the [type of data; e.g., enforcement, state expenditure, etc.] data reported in this section, if needed.

This person will NOT BE IDENTIFIED in any reports that result from this survey.

The sole purpose of requesting these names is to facilitate the process of seeking clarification when submitted data are ambiguous; no names will appear in the Report to Congress.

Survey data will be stored in password-protected, encrypted files. Access to these files will be limited to the data analyst and supervisor. The contact persons’ names are maintained in a confidential manner in compliance with contractual and regulatory requirements governing personally identifiable information (PII).

Policy Academy Formative Evaluation

For both the coalition and coach/mentee data collection activities, personally identifiable information including name, email, and professional affiliation will be collected from those who fill out the surveys and participate in the focus groups. This information will only be accessible by the Policy Academy staff through a secure SharePoint site. The Policy Academy staff may use this information to reach out to coalition participants or coaches/mentees to discuss the information they submitted. Any information shared with stakeholders will be deidentified.

“Talk. They Hear You.” - Parents’ Night Out Evaluation

The PNO Evaluation survey will ask participants for their name and email to match pre and post respondents and observe change over time. This information will be deidentified, and aggregated and shared with stakeholders.

11. Questions of a Sensitive Nature

STOP Act State Survey

No questions of a sensitive nature are included in the Survey.

Policy Academy Formative Evaluation

No questions of a sensitive nature are included in the Policy Academy Formative Evaluation surveys or focus groups.

“Talk. They Hear You.” -- Parents’ Night Out Evaluation Survey

The TTHY campaign discusses underage drinking, substance use, and mental health. Any information about these topics and the participant's personal experience will be delivered with resources for more support for that individual. The PNO Evaluation survey will ask about parent’s knowledge of child substance use, including which substances they know or suspect their child is using. The purpose of this information being collected is to better inform future targeted campaign products. PNO is comprised of resources to support parents and caregivers while having these conversations. Responses to these sensitive questions are voluntary.

12. Estimates of Annualized Hour Burden

STOP Act State Survey

Table 2 indicates the estimated total annual burden on each state for data collection will be 18.5 hours. This estimate includes time for reviewing instructions, searching existing data sources, gathering the necessary data, completing, and reviewing the collection of information, and entering the data into the form. The wage rate was obtained by taking an average of the wages of the types of employees who were responsible for filling out the survey in the pilot states.

The burden estimate in Table 2 is based on the lengthy and comprehensive planning process and pretesting conducted by SAMHSA. To design the State Survey, advice from key stakeholders (as mandated by the STOP Act) was sought by consulting with experts on specific sections of the survey, engaging the ICCPUD for review and feedback, conducting a pilot test with state officials likely to be responsible for completing the State Survey, and investigating and testing various survey formats, online delivery systems, and data collection methodologies. This draft went through an iterative process of review and revision with input by stakeholders and key informants and was expected to look as close to the final draft as possible. The state agencies responsible for filling out each section of the Survey were asked to estimate the amount of time it took to complete the Survey. These times were averaged and a burden of 18.5 hours per response was calculated.

Table 2: STOP ACT State Survey Estimated Burden for Respondents

Instrument	# of Respondents	Responses/ Respondent	Total Responses	Hrs. per Response	Total Hour Burden	Wage Rate	Total Hour Cost
State Survey	51	1	51	18.5	943.50	$28.07	$ 26,484.05

Policy Academy Formative Evaluation

Table 3 indicates the estimated total annual burden on the participants and coaches of the Policy Academy. The survey estimates include reading the instructions and questions and responding to each question. The focus group is scheduled for one hour, and includes introductions, instructions, posing of questions, and open discussion.

Table 3: Policy Academy Formative Evaluation Estimated Burden for Respondents

Instrument	# of Respondents	Responses/ Respondent	Total Responses	Hrs. per Response	Total Hour Burden	Wage Rate	Total Hour Cost
Focus Group	28	1	28	1	28	$27.10	$758.80
Participant Post-Coaching Call Survey	28	11	308	0.125	38.5	$27.10	1043.35
Participant Post-Training Call Survey	28	10	280	0.125	35	$27.10	948.5
Coach Post-Coaching Call Survey	3	77	231	0.17	39.27	$50	$1,963.50
Baseline	28	1	28	0.67	18.76	$27.10	$508.40
Follow-Up	28	1	28	1	28	$27.10	758.80
Six-Month Follow-Up	28	1	28	0.67	18.76	$27.10	$508.40

“Talk. They Hear You.” - Parents’ Night Out Evaluation

Table 4 indicates the estimated total annual burden on the participants of PNO. The survey estimates include reading the instructions and questions and responding to each question, and totals 7 minutes. The wage rate was determined based on the highest state minimum wage, as site locations have not yet been identified.

Table 4: Parents’ Night Out Evaluation Estimated Burden for Respondents

Instrument	# of Respondents	Responses/ Respondent	Total Responses	Hrs. per Response	Total Hour Burden	Wage Rate	Total Hour Cost
PNO Evaluation Survey	150	1	150	0.12	18	$16.28*	$293.04

* https://www.dol.gov/agencies/whd/minimum-wage/state

STOP Act and “Talk. They Hear You.”

Table 5 combines each of the total annual burden estimates for the State Survey, Policy Academy Formative Evaluation, and PNO Evaluation.

Table 5: Combined Estimated Burden for Respondents

Instrument	# of Respondents	Responses/ Respondent	Total Responses	Hrs. per Response	Total Hour Burden	Wage Rate	Total Hour Cost
State Survey	51	1	51	18.5	943.50	$28.07	$ 26,484.05
Focus Group	28	1	28	1	28	$27.10	$758.80
Participant Post-Coaching Call Survey	28	11	308	0.125	38.5	$27.10	$1043.35
Participant Post-Training Call Survey	28	10	280	0.125	35	$27.10	$948.50
Coach Post-Coaching Call Survey	3	77	231	0.17	39.27	$50	$1,963.50
Baseline	28	1	28	0.67	18.76	$27.10	$508.40
Follow-Up	28	1	28	1	28	$27.10	$758.80
Six-Month Follow-Up	28	1	28	0.67	18.76	$27.10	$508.40
PNO Evaluation Survey	150	1	150	0.12	18	$16.28*	$293.04

* https://www.dol.gov/agencies/whd/minimum-wage/state

13. Estimates of Annualized Cost Burden to Respondents

There will be no capital, startup, operation, or maintenance of services costs to respondents.

14. Estimates of Annualized Cost to the Government

STOP Act State Survey

The estimated cost to the government for the data collection is approximately $582,000. This includes approximately $578,300 for a 5-year contract for sampling, data collection, processing, reports, etc. and approximately $3,706 per year for SAMHSA costs to manage/administrate the survey for 2% of one employee (GS-15). The total annualized cost is approximately $116,401.

Policy Academy Formative Evaluation

The estimated cost to the government per 12-month Policy Academy plus a six month follow up is approximately $20,000 for data instrument development, analysis, and reporting. This includes approximately $1,853 for SAMHSA to manage and oversee the evaluation, a total of 1% of one employee (GS-15). The total annualized cost is approximately $21,853.

“Talk. They Hear You.” - Parents’ Night Out Evaluation

The estimated cost to the government is approximately $30,000 for site selection, data instrument development, analysis, and reporting. This includes approximately $1,853 for SAMHSA to manage and oversee the evaluation, a total of 1% of one employee (GS-15). The total annualized cost is approximately $31,853.

15. Changes in Burden

STOP Act State Survey

There is a .8-hour increase in the amount of time it takes to complete this survey. This draft went through an iterative process of review and revision with input by stakeholders and key informants and was expected to look as close to the final draft as possible. The state agencies responsible for filling out each section of the Survey were asked to estimate the amount of time it took to complete the Survey. These times were averaged and a burden of 18.5 hours per response was calculated.

Policy Academy Formative Evaluation

As the data collection tools are new, there is no change in burden.

“Talk. They Hear You.” - Parents’ Night Out Evaluation

As the tools are new, there is no change in burden.

The cost burden has been reduced as there have been significant efficiencies realized in this revised package. Namely, the survey fielding process has been streamlined through a drafted and tested communications plan, a staff who has experience fielding and managing this survey for multiple years, and a dedicated team specific to survey implementation. Additionally, the survey itself has been revised to reduce redundancies and make the review, quality control, and technical assistance more efficient. Taken together, these revisions have resulted in a lower cost to the government.”

16. Time Schedule, Publication, and Analysis Plans

STOP Act State Survey

Time Schedule—The State Survey is administered to the states in the spring of each year. Each state has 45 days from the receipt of the instructions to complete and submit the survey.

Analysis Plan—The analysis plan for the State Survey is designed to meet two goals:

1. Present each state’s data in a clear, concise, and easily assessable fashion.

2. Allow each state to speak for itself by including unedited text responses.

All data from the State Survey are descriptive, and each response will constitute a separate entry in the proposed data tables (see publication plan). No data reduction is required, and no comparisons across states are appropriate to the purposes of the Report to Congress.

As discussed earlier, the State Survey instrument requests contact persons for each section. These individuals will be contacted if data are missing or if potential problems with text entries are identified (e.g., ambiguities, grammatical problems). States will be invited to rewrite these entries. Consistent with the goal of allowing states to speak for themselves, however, the state respondents, including the Governor-designated survey representative, will have the final say concerning text entries.

Publication Plan—Analysis of data obtained through the State Survey will be part of the SPBP published annually. Individual state survey data will also be presented in tables within each State Report (SR) (Attachment 3), corresponding to the four major sections of the report. The attached tables present actual data collected during the 11^th Survey year.

Policy Academy Formative Evaluation

Time Schedule –Data collection will occur at the start of and throughout the Policy Academy, ending six months after the Academy completion. During this time, baseline, follow-up, and six-month follow-up surveys will be administered to each coalition participant, once. The focus groups will occur during the in-person training. Post-coaching and post-training call surveys for the coalition participants and coaches/mentees will be administered about monthly, after each policy step and meeting.

Analysis Plan – Process data from the post-coaching call and post-training call surveys will be reviewed monthly to provide deidentified feedback to the coaches and staff. This ensures the staff can make improvements during each cohort. Three summative reports will be created using qualitative and quantitative analysis methods. Quantitative data will be analyzed using descriptive statistics, including crosstabs and potentially chi-square tests. Qualitative data, including open-ended questions and focus group data, will include team coding and thematic analyses.

Publication Plan – The Policy Academy Formative Evaluation will produce a process evaluation report, an outcome evaluation report, and potentially a peer-reviewed manuscript.

“Talk. They Hear You.” - Parents’ Night Out Evaluation

Time Schedule – The evaluation survey for PNO will be administered following each PNO event held with one of the partner sites. Data will be pulled from this survey each month for analysis and reporting.

Analysis Plan – The PNO Evaluation survey will be for parent and caregiver responses to their child’s substance use to understand who is attending the sessions. Evaluators will export the responses into Excel, where they will run descriptive statistics, and qualitatively analyze response themes. This information, along with feedback on the structure and information delivered during the session, will guide any future iterations of PNO or other materials created.

Publication Plan - Data collection from the PNO Evaluation Survey will not be shared publicly. Reports developed will only be shared with the TTHY campaign team, and client, SAMHSA.

17. Display of Expiration Date

The expiration date will be displayed on all data collection tools distributed to participants.

18. Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.

1 Members of ICCPUD are listed on Attachment 1.

2 2020 RTC, page 18.

3 Centers for Disease Control, Alcohol-Related Disease Impact (CDC/ARDI), 2022.

4 2024 RTC, .

⁸ National Survey on Drug Use and Health (NSDUH), conducted by the Center for Behavioral Health Statistics and Quality (CBHSQ) of the Substance Abuse and Mental Health Services Administration (SAMHSA), 2022 data.

⁹ NSDUH, 2022.

¹⁰ NSDUH, 2022.

¹¹ 2024 RTC.

¹² 2024 RTC.

¹³ 2024 RTC.

5 As noted above, HHS has delegated this responsibility to SAMHSA.

6 SAMHSA 2023–2026 Data Strategy

7 Glenn, S.D., Turrisi, R., Mallett, K.A., Waldron, M.S., Lenker, L.K. (2024). Examination of Brief Parent-Based Interventions to Reduce Drinking Outcomes on a Nationally Representative Sample of Teenagers. Journal of Adolescent Health, 74(3) 449-457. https://doi.org/10.1016/j.jadohealth.2023.09.010

8 http://alcoholpolicy.niaaa.nih.gov/uploads/Enforcement_and_Compliance_Data_Sources_12_18_07.pdf

16