Ss-a Nitt_9-29-15

Now is the Time (NITT)–Project AWARE Evaluation—

Site Notification and Recruitment

SUPPORTING STATEMENT

Part A: Justification

The Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Mental Health Services (CMHS) is requesting approval from the Office of Management and Budget (OMB) for site recruitment letters and materials; the first of two data collections for the Now is the Time (NITT) evaluation. While NITT includes separate programs— Advancing Wellness and Resilience in Education -State Educational Agency (AWARE-SEA), Healthy Transitions, and two Minority Fellowship Programs (Youth and Addictions Counselors), this request is focused entirely on AWARE-SEA. SAMHSA’s initial request seeks approval for site recruitment materials and procedures for notification and recruitment of state grantees, grantee and nongrantee districts, and grantee and nongrantee schools. A subsequent package will request approval for the instruments to collect data across all four NITT programs (AWARE-SEA, Health Transitions, and two Minority Fellowship Programs).

AWARE-SEA aims to increase awareness of mental health issues among school-aged youth (grades K–12; hereafter referred to as “youth”); provide training for school personnel and other adults who interact with youth to detect and respond to mental illness in youth; connect youth and families/caregivers who may have behavioral health issues with appropriate services; and improve conditions for learning and behavioral health outcomes for all youth.

The AWARE-SEA evaluation will examine the process, outcomes, and impact of activities by SEA grantees and their local educational agency (LEA) and school partners. The study will evaluate the capacity of SEAs to address the aims of the program as described above.

Recruitment materials provided in appendices include (A) a state/grantee notification letter, (B) district notification and recruitment letters, (C) school recruitment letters, (D) text for an NITT brochure, and (E) AWARE-SEA “Frequently Asked Questions.”

A.1 Explanation of Circumstances That Make Collection of Data Necessary

AWARE-SEA represents a response to the third and fourth components of President Obama’s NITT Initiative: making schools safer and focusing on access to mental health services. AWARE-SEA is authorized under Section 520A of the Public Health Service Act, as amended, and addresses the Healthy People 2020 Mental Health and Mental Disorders Topic Area. AWARE-SEA grantees are required to provide mental health awareness training to adults who interact with youth, create partnerships to connect youth to mental health services, and create a school climate to reduce violence. The AWARE-SEA evaluation design will address each of these five priority areas: (1) collaboration and coordination; (2) mental health service system capacity and infrastructure; (3) early and accurate identification of mental health needs; (4) mental health service access for individuals in need; and (5) individual resilience and functioning, school/community safety.

The evaluation research questions, delineated below, will guide the evaluation in these priority areas:

AWARE-SEA Evaluation Questions

Consolidated Evaluation Priority Area: Collaboration and coordination

• To what extent do AWARE-SEA activities lead to enhanced collaboration and coordination across child and youth serving agencies?

• To what extent is information sharing across child-serving agencies enhanced as a result of AWARE-SEA?

• To what extent are families and youth involved in AWARE-SEA? Is level of impact correlated with the extent of family and youth involvement?

• What strategies effectively engage families in connecting with schools and communities?

• What are the barriers and facilitators to state and local-level collaboration, partnership development and shared decision-making? How were they addressed?

• What is the effect of collaboration within and between federal, state, and local agencies on AWARE-SEA grant activities, processes, and outcomes?

Consolidated Evaluation Priority Area: Mental health service system capacity and infrastructure

• How are state and local-level systems changing in response to AWARE-SEA?

• What evidenced-based programs and services are being provided as a result of the AWARE-SEA initiative (including modality type, intensity and duration)?

Consolidated Evaluation Priority Area: Early and accurate identification of mental health needs

• To what extent is there earlier identification of mental health problems (and co-occurring substance use) among youth?

• What is the effect of AWARE-SEA at the state and community levels on mental health literacy?

• How are the knowledge, skills and abilities of school officials and the range of actors in the public sphere that interact with youth to detect and respond to signs of mental illness in youth increased as a result of AWARE-SEA?

Consolidated Evaluation Priority Area: Mental health service access for individuals in need

• To what extent does AWARE-SEA increase access to mental health services for youth?

• What is the effect of AWARE-SEA on referrals to mental health services? Are youth referred and linked to mental health services that are community based, youth centered, family focused, and culturally and linguistically competent?

Consolidated Evaluation Priority Area: Individual resilience and functioning, school/community safety

• To what extent does AWARE-SEA contribute to making schools and communities safer?

• To what extent does AWARE-SEA lead to improved coping and resiliency among youth?

• Is school climate improved in grantee communities? If school climate is improved in grantee communities, can these improvements be linked to grant activities?

A.2 Purpose and Use of Information

According to the AWARE-SEA solicitation, AWARE-SEA grantees should increase the capacity of SEAs in three areas: (1) increase mental health awareness among youth; (2) train those who work with youth to identify and respond to mental health issues in youth; and (3) connect youth and families with mental health services. The intention is to encourage cross-system collaboration and make use of evidence-based strategies to address mental health needs. Specific AWARE-SEA objectives include increasing access to mental health services, linking SEAs with existing systems (e.g., juvenile justice, mental health), focusing on violence prevention through workshops for key stakeholders, increasing mental health awareness, implementing evidence-based behavioral health promotion strategies at multiple levels, early intervention, ensuring the provision of culturally appropriate services, using a multitiered behavioral framework (like Positive Behavioral Intervention and Supports), increasing mental health literacy via Mental Health First Aid (MHFA) training, and building capacity for sustainability.

Through the AWARE-SEA evaluation, SAMHSA will learn of effects on youth coping skills and resilience; school climate; engaging families in connecting with schools and communities; abilities of school officials and others to detect and respond to signs of mental illness in youth; and outcomes such as school violence, bullying, and perceptions of safety. SAMHSA will also learn about program implementation and the associations between implementation and outcomes.

Involvement of Grantees, Districts, and Schools

AWARE-SEA grants were made to 20 state education agencies, each of which will partner with three to five local education agencies (LEAs or school districts) in their state to plan and implement AWARE-SEA activities. AWARE-SEA activities may be implemented in all schools in the district or may be focused on a specific type (e.g., “high risk” schools or alternative schools) or all elementary, middle and high schools in an identified feeder pattern.

At the grantee, LEA, and school level, data will be collected from key staff in all identified partner organizations. School-level changes will be assessed through data collected from selected AWARE-SEA (i.e., “project site”) schools and comparison schools. Each grantee state will be asked to support the evaluation by encouraging the grantee districts to cooperate with the national evaluation contractor when contacted, enlist the participation of grantee schools, and provide access to data available through the district’s management information system (MIS). Each grantee district will also be asked to assist the study with identifying and encouraging the participation of comparison schools, where possible. One matched comparison school will be identified for each project site school. The comparison school is similar to the project site school in terms of demographic characteristics and rates of incidents of violence but is not implementing AWARE-SEA activities. Both project site and comparison schools will be asked to participate in the school administrator, teacher, and student surveys, and data abstraction from the schools’ MIS system.

The study may not be successful in recruiting a comparison school for each project site school because a suitable match cannot be identified or because the comparison school is unwilling to participate. In these cases, the study will attempt to recruit non-grantee districts and schools in a neighboring community where potential matched schools have been identified. The study will also compare grantee outcomes at the state and community levels to comparable measures from recurring surveys, including the Centers for Disease Control and Prevention’s National Health Interview Survey and the Youth Risk Behavior Surveillance System.

A.3 Use of Information Technology

There is no information technology involving respondents for site notification and recruitment activities. A web-based school contacting system will be developed to capture all contact information and communications with sites. It will mostly involve reading a letter about the evaluation study and accompanying materials, responding to a phone call in which the evaluation contractor will answer any questions grantees and their partners may have, and providing access to the next level of respondents (states provide access to districts, districts provide access to schools). Follow-up calls with district leads will be used to identify project site and comparison schools to invite for the study.

A.4 Efforts to Identify Duplication

Before requesting any site level information from grantees, a review will be completed of documents provided to SAMHSA as part of the award process and requirements, including grant proposals, award documentation, progress reports, and the comprehensive plan to gather site descriptive information that could inform the site recruitment efforts. The U.S. Department of Education’s National Center for Education Statistics (NCES) Common Core of Data will be used to identify potential comparison schools in non-grantee districts.

A.5 Involvement of Small Businesses or Other Entities

No small businesses will be contacted during site recruitment.

A.6 Consequences If Information is Collected Less Frequently

The site notification and recruitment efforts described in this submission are a one-time-only burden and are necessary for SAMHSA to conduct a national evaluation of AWARE-SEA. The AWARE-SEA grants were funded in late 2014, and planning has begun to implement activities as early as the 2015–2016 school year. Therefore, it is important that the evaluation obtain early implementation/baseline measures, ideally before the end of the 2015–2016 school year. To meet these data collection targets for the evaluation, recruitment of all identified districts and schools in the 2015–2016 school year is highly desirable. The 5-year study includes data collection at three time points: spring 2016, spring 2017, and spring 2018.

A.7 Consistency with the Guidelines in 5 CFR 1320.5(d)(2)

This information collection fully complies with the guidelines in 5 CFR 1320.5(d)(2).

A.8 Consultation Outside the Agency

The notice required by 5 CFR 1320.8(d) was published in the Federal Register on September 29, 2015 (80 FR 58496). No comments were received.

An Expert Panel with diverse expertise will advise the evaluation team on study design, instrumentation, and data collection. The panel members, listed below, include experts in school-based mental health programs, systems of care, school safety, school climate, youth violence prevention programs, and bullying prevention.

• Dr. Lynda Gargan, Interim Director, Federation of Families for Children’s Mental Health

• Dr. Susan Keys, Associate Professor and Senior Researcher, College of Public Health and Human Sciences, Oregon State University

• Dr. Benjamin Le Cook, Assistant Professor of Psychiatry, Harvard Medical School and Senior Scientist, Center for Multicultural Mental Health Research

• Dr. Michael Pullmann, Research Assistant Professor, Department of Psychiatry and Behavioral Sciences, University of Washington

• Dr. Pamela Orpinas, Professor, Department of Health Promotion and Behavior, University of Georgia

• Dr. Mark Salzer, Professor and Chair, Department of Rehabilitation Sciences, Temple University

• Dr. MaJose Carrasco, Director, Multicultural Action Center, National Alliance on Mental Illness

A.9 Payments or Gifts to Respondents

There will be no payments with regard to site notification and recruitment.

A.10 Assurance of Confidentiality

This collection is for recruitment purposes, and no data will be collected.

Other than the names and contact information for the state grantees, districts, and schools, which is information typically already available in the public domain and which was submitted as part of grant application and reporting requirements, no data collected during recruitment will contain personally identifiable information.

A.11 Questions of a Sensitive Nature

Questions of a sensitive nature will not be asked as part of site notification and recruitment.

A.12 Estimates of Respondent Burden

For site notification and recruitment, lead letters will be sent and a follow-up telephone call will be made to confirm participation and answer any questions. During subsequent communication by e-mail and telephone, state grantees will be asked to encourage district participation; district officials will be asked to encourage school participation. The following entities will be contacted:

• All 20 AWARE-SEA grantees at the state level

• An estimated 90 local education agency partners (3-5 districts per state, under the grant requirements)

• An estimated 396 schools in grantee districts that will be implementing AWARE-SEA activities (“project site schools”) (approximately 4-5 schools per grantee district are expected to participate in the evaluation). This estimate includes additional schools that may need to be contacted to replace grantee schools that are unable or unwilling to participate.

• An estimated 432 schools in grantee districts that are NOT currently implementing AWARE-SEA activities (“comparison schools”). This estimate includes additional schools that may need to be contacted to replace comparison schools that are unable or unwilling to participate. This estimate also assumes that the recruitment effort for comparison schools will be more difficult than that of project site schools and will require that a larger number of potential comparison schools be contacted compared to project site schools.

• Approximately 30 nongrantee districts will be identified and recruited as needed if no comparison school is available in a grantee district to form a matched pair with a project site school.

• Approximately 90 comparison schools in nongrantee districts will be identified and recruited as needed to form a matched pair for project site schools with no comparison school available. For each project site school without a comparison school, one best match and two alternates will be identified in each of the 30 districts.

Notification letters and recruitment materials are in appendices A through E.

Table A.1 summarizes the reporting burden associated with this notification and recruitment submission. It is estimated that it will take state, district, and school officials an average of 1 hour throughout the recruitment process. The total burden is 1,058 hours.

Table A.1. Estimates of Response Burden

Respondent	Number of Respondents	Responses per Respondent	Total number of Responses	Burden Hours per Response	Total Burden in Hours	Hourly Ratea	Estimated Monetary Cost of Burden
State grantee official	20	1	20	1	20	$47	$940
District official in grantee district	90	1	90	1	90	$47	$4,230
School official in grantee district – project site school	396	1	396	1	396	$47	$18,612
School official in grantee district – comparison school	432	1	432	1	432	$47	$20,304
District official in nongrantee district	30	1	30	1	30	$47	$1,410
School official in nongrantee district	90	1	90	1	90	$47	$4,230
Total Burden	1,058		1,058		1,058		$49,726

^aAssumes an hourly rate of $47 per hour (Bureau of Labor Statistics’ Occupational Employment Statistics for Education Administrators, Elementary and Secondary Schools, May 2014).

A.13 Estimates of the Cost Burden to Respondents

No cost to respondents is anticipated beyond the estimated burden cost described in Section A.12.

A.14 Estimates of the Annualized Cost to the Government

The estimated cost for site recruitment activities, which are limited to the AWARE-SEA evaluation, is no more than $403,617 for a period of approximately eight months and includes development of recruitment materials; development of a web-based school contacting system; mailing and postage; outreach to officials at the state, district, and school levels; and setting up logistics for data collection.

A.15 Changes in Hour Burden

This is a new project.

A.16 Time Schedule, Publication, and Analysis Plan

Site notification and recruitment of AWARE-SEA grantees and their school and district partners are being conducted for the purpose of enlisting sites for participation in the AWARE-SEA evaluation. Site notification and recruitment is planned during school year 2015–2016.

Contact information, status of site notification and recruitment activities, and initial data collection logistics information gathered during recruitment communications will be documented in a web-based School Contacting System.

No publications will emerge from this effort.

A.17 Display of Expiration Date

All recruitment letters and materials will display the expiration date for OMB approval.

A.18 Exceptions to Certification Statement

The certifications are included in this submission.

1