988 Evaluation Part A_03202025_Clean
988 Evaluation Part A_03202025_Clean.docx
SAMHSA 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation
OMB:
SAMHSA 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation
Supporting Statement A
Updated 7/10/24
Check off which applies:
☒ New
☐ Revision
☐ Reinstatement with Change
☐ Reinstatement without Change
☐ Extension
☐ Emergency
☐ Existing
Justification
The Substance Abuse and Mental Health Services Administration (SAMHSA) and its Behavioral Health Crisis Coordinating Office (BHCCO) request clearance for new data collection for the evaluation of the 988 Suicide & Crisis Lifeline and Crisis Services. BHCCO is the Federal lead overseeing the 988 Suicide & Crisis Lifeline, which is operated through a cooperative agreement with a single Lifeline Administrator.
The 988 Suicide & Crisis Lifeline and Behavioral Health Crisis Services Continuum (BHCSC) comprises three core structural elements outlined in the National Guidelines for Behavioral Health Crisis Care. These are crisis contact centers (“someone to talk to”), mobile crisis services (“someone to respond”), and crisis receiving and stabilization facilities (“a safe place for help”). The 988 Suicide & Crisis Lifeline and BHCSC provide access to behavioral health crisis care through a network of over 200 crisis contact centers.
Several sources provide funding for the BHSCS, including state general and local funds, SAMHSA’s Community Mental Health Services Block Grant (MHBG), SAMHSA’s Substance Use Prevention, Treatment, and Recovery Services Block Grant (SUPTRS BG), Medicaid, Medicare, Cell/Phone Telecom Fee, the Certified Community Behavioral Health Clinic (CCBHC) program, Mobile Crisis Response, Tribal Opioid Response (TOR) and State Opioid Response (SOR) grants. The National Association of State Mental Health Program Directors (NASMHPD) collaborated with SAMHSA to establish the Transformation Transfer Initiative in 2007 to fund and enhance behavioral health initiatives and core crisis services.
The 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation assesses the implementation and expansion of the 988 Suicide & Crisis Lifeline and BHCSC in the United States (U.S.). The evaluation will provide SAMHSA, grantees, and other interested parties with information to strengthen the BHCSC for all people in crisis. The evaluation aims to understand the system response to behavioral health crises and outcomes of interventions delivered, with a focus on understanding how improving access to quality crisis services and supports. The multi-method, multi-study evaluation combines primary and existing data to answer questions related to the implementation of, and outcomes from, 988 Suicide & Crisis Lifeline and BHCSC delivery, including:
The structure, composition, and collaboration patterns of the 988 Suicide & Crisis Lifeline and BHCSC at the national, state, territory, and Tribal levels.
How agencies associated with the 988 Suicide & Crisis Lifeline and BHCSC work together to provide behavioral health crisis services and the effect on broader use of crisis services like 911.
The effectiveness of the 988 Suicide & Crisis Lifeline and BHCSC in linking individuals to services and the relationship between the 988 Suicide & Crisis Lifeline and BHCSC and short- and long-term behavioral outcomes.
The effectiveness of the 988 Suicide & Crisis Lifeline and BHCSC on immediate reductions in suicidal ideation, homicidal ideation, and overdose risk.
The overall impact of the 988 Suicide & Crisis Lifeline and BHCSC on suicide and overdose mortality and morbidity by comparing long-term trends in public health outcomes before and after the implementation of the 988 Suicide & Crisis Lifeline and BHCSC.
The 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation is structured across three levels: System-Level, Client-Level, and Impact. There are five studies embedded within the three levels, with three of the five studies collecting data (The other two studies rely on existing data sources). Clearance is requested for data collection associated with nine instruments, which include Web-based surveys, inventory forms, and key informant interviews, as outlined in Exhibit 1.
Exhibit 1: Data Collection Instruments by Evaluation Level and Evaluation Study
System-Level Evaluation |
System Composition and Collaboration Study
|
System-Level Service Utilization Study – Uses extant data |
Client-Level Evaluation |
Client-Level Service Utilization and Outcome Study
|
Client-Level Risk Reduction Study
|
Impact Evaluation |
Impact Evaluation - Uses extant data |
Circumstances of Information Collection
Background
The United States is facing an unprecedented mental health and substance use crisis affecting people of all ages. In 2022, nearly one in five adults (59.3 million people) had a mental illness (AMI) (Substance Abuse and Mental Health Services Administration [SAMHSA], 2023b). The nation’s mental health issues go beyond those diagnosed by a professional. In 2021, two in five American adults reported experiencing symptoms of anxiety and depression (Vahratian et al., 2021) and 44% of high school students reported struggling with persistent feelings of sadness or hopelessness (Casey, 2022), exacerbated by the COVID-19 pandemic, social media, and gun violence. Additionally, in 2022, 70.3 million people aged 12 or older used illicit drugs, and 48.7 million people met diagnostic criteria for a substance use disorder (SUD) (SAMHSA, 2023b).
The pandemic has affected the public’s mental health and wellbeing in many ways, including through isolation and loneliness, job loss and financial instability, illness, and grief (Lopes et al., 2022). Behavioral health differentials, worsened by the pandemic, are evident in higher rates of mental health issues and SUDs among specific groups such as youth, young adults, Veterans, and racial and ethnic minorities (Centers for Disease Control and Prevention [CDC], 2022; McKnight-Eily 2021; SAMHSA, 2023a). For example, the pandemic's negative impacts have most heavily affected youth already at elevated risk of suicide, including those with disabilities, racial and ethnic minorities, and youth who are low-income, living in rural areas, immigrant households, involved with the child welfare or juvenile justice systems, or experiencing homelessness (Murthy, 2022).
Crisis care often becomes the primary access point for mental health treatment for individuals who struggle to find services, face a shortage of providers, or cannot afford care due to inadequate insurance coverage and high out-of-pocket costs. In 2021, there were 5,930 areas in the United States with a shortage of mental health professionals, affecting 129.6 million people who may not have access to adequate mental health services (Modi, Orgera, & Grover, 2022). Only 28% of the U.S. population live in areas with sufficient mental health professionals and psychiatrists to meet their needs. Most states have only 40% of the required mental health providers needed to serve their population (Modi et al, 2022).
Consequently, the burden of mental health care often falls on already overtaxed primary care providers and law enforcement professionals. This strain contributes to the fact that one in every ten visits to an emergency department is related to mental health issues (Santillanes et al., 2020). There has been a long-standing need in the U.S. to divert patients from overcrowded and under-resourced emergency departments and carceral settings into the mental health system (Vinson and Dennis, 2021). Given the national mental health needs, crisis care has become essential, with an increasing need for research to support and expand its infrastructure (Zabelski et al., 2023).
Challenges faced by those receiving crisis services include suicidality, self-harm, mental health concerns, substance use, crime, child protection, and interpersonal violence. A review of mental health emergency hotlines in the U.S. found that suicidality, depression, and interpersonal problems were the most common reasons for seeking help (Matthews et al., 2023). This study also showed that younger adult females were the most frequent hotline users, while older white males predominantly used Veteran crisis lines. Other studies have shown that females make up a higher percentage of crisis lifeline contacts, are more likely to repeat contacts, and often report being victims of violence, regardless of location or context (Barber et al., 2004; Krishnamurti et al., 2022).
Suicide is a major contributor to premature death in the United States and the second leading cause of death among people ages 10 to 34 (CDC, 2024). Millions of individuals report considering or attempting suicide each year. In 2022, an estimated 13.2 million adults had serious thoughts of suicide. Of these, 3.8 million made a suicide plan, and 1.6 million made a nonfatal suicide attempt (SAMHSA, 2023b).
Access to behavioral health treatment can reduce suicide risk factors, but early intervention is often unavailable. Although 18% of U.S. adults had a past-year SUD, only 5% received treatment within the past year (SAMHSA, 2023b). Additionally, 5.5 million adults with a mental illness are uninsured, and 58.9% of adults with a mental illness report that they have not sought treatment because it would cost too much (SAMHSA, 2023b). This lack of access can escalate mental health issues to crisis levels, making crisis care the only option for some. Barriers to treatment prevent early intervention and increase reliance on crisis systems when conditions become critical.
Risk factors for suicide include a range of individual, relationship, community, and societal factors. Substance use is a significant risk factor as it can exacerbate mental health issues, lead to impulsive behaviors, and increase feelings of hopelessness and isolation. Additionally, substance use disorders often co-occur with other risk factors such as depression, adverse childhood experiences, and social isolation, further elevating the risk of suicide (Health and Human Services [HHS], 2024)
According to the CDC, Non-Hispanic American Indian/Alaska Native (AI/AN) people and non-Hispanic White people have the highest suicide rates, while Veterans, those living in rural areas, and workers in industries like mining, construction, and agriculture also have higher-than-average suicide rates (CDC, 2023). Across race and ethnicity groups, men have higher suicide rates than women, making outreach to this group essential. For example, men aged 55–64 and 75+ have suicide rates three to five times higher than women in the same age groups (CDC, 2022). Given the effectiveness of crisis services (Gould et al., 2021), it is crucial to increase access to care for these groups.
Over the past two decades, addressing the mental health and overdose crises in the United States has become a priority for national leaders, experts, researchers, and health practitioners. The federal response has increased through policy and resource commitments, starting with the creation of the National Suicide Prevention Line (NSPL) in 2005 and its partnership with the Veterans Crisis Line in 2007. These resources have evolved into the current 988 Crisis Response.
For many years, SAMHSA’s NSPL served as an entry point to the behavioral health care system, offering immediate crisis care through calls, chats, and texts. In 2020, recognizing the need to expand and improve the national mental health crisis response, Congress designated a three-digit dialing code (988) to route clients to an expanded suicide and crisis lifeline. SAMHSA leads this effort in partnership with the Federal Communications Commission (FCC) and the Department of Veteran Affairs (VA). The 988 Suicide & Crisis Lifeline and BHCSC are designed for anyone in the nation experiencing any type of crisis, including those related to substance use, mental health, suicidal ideation, and emotional distress (SAMHSA, 2023a).
Recognizing the significant impact of the pandemic on the nation’s youth, a plan with initiatives to address the youth mental health crisis in schools and communities was developed. It also strengthens crisis care and suicide prevention infrastructure by expanding access to mobile crisis services in high-need communities and investing in behavioral health services for prevention, screening, treatment, and other support.
Recently, the federal government has invested nearly $1.5 billion to implement the 988 Suicide & Crisis Lifeline and BHCSC. This includes nearly $500 million in FY 2024 to the Lifeline Administrator and other grantees on state, territorial, Tribal and center levels to increase crisis contact center capacity and expand specialized services, including Spanish services, and videophone services using ASL for people who are deaf or hard of hearing. This investment underscores the Administration’s commitment to ensuring that anyone in crisis receives the necessary help and support.
Since the transition from the NSPL to 988 in July 2022, the 988 Suicide & Crisis Lifeline and BHCSC have answered over 9.6 million contacts (SAMHSA, 2024). Service improvement and expansion continue in all areas, enhancing crisis line features, crisis center supports, and funding. These efforts are part of a broader initiative to expand coverage for Americans and improve the 988 Suicide & Crisis Lifeline and BHCSC. Portions of the FY 2023 funding have been allocated to states and U.S. territories to improve local responses, coordinate across the crisis continuum, serve high-risk populations, and provide workforce training. Additional funding supports the 988 Suicide & Crisis Lifeline program follow-up services and expands access to crisis care for Tribal citizens.
The Need for Evaluation
The 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation aims to comprehensively examine the outcomes and impact of the 988 Suicide & Crisis Lifeline and BHCSC. Its objectives include evaluating the system’s effectiveness in engaging individuals in crisis, reducing mental health and substance use emergencies and suicide attempts; exploring consumer and provider experiences; and assessing overall outcomes related to crisis resolution and ongoing care. Consistent with SAMHSA’s commitment to evidence-based practices, this data collection effort can improve and advance the national response to mental health and substance use crises.
The 988 Suicide & Crisis Lifeline and BHCSC's current metrics provide essential insights into their reach and effectiveness, especially in rural areas. This data highlights the program's adaptability to the nation’s changing needs and supports evidence-based decision-making, reinforcing its crucial role in the national mental health support framework. However, these metrics are not exhaustive. Our proposed evaluation seeks to collect more detailed data to deepen our understanding and refine our approach.
This new evaluation will capture a broader spectrum of data beyond the initial metrics, addressing the unique demands of behavioral health crisis intervention. It will ensure the confidentiality and anonymity of individuals in crisis while collecting real-time interaction data, including the nature of the crisis, the urgency of the response, and the caller’s initial reaction to the intervention. Flexible and adaptable data collection methods will accommodate the unpredictable nature of mental health crises and various caller needs. This comprehensive approach will guide the evolution of crisis intervention strategies, continually assessing and improving the effectiveness and responsiveness of the 988 Suicide & Crisis Lifeline and BHCSC.
The data from the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation will play a significant role in informing policy and funding decisions, thereby improving the overall effectiveness of the U.S. mental health crisis response. The use of this data underscores the commitment to developing a more responsive and effective mental health support system at the national level. It supports key goals of the 2024 National Strategy for Suicide Prevention.
Clearance Request
SAMHSA is requesting OMB approval for a new data collection, the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation. We request approval for three years of data collection associated with the proposed design.
Overview of 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation and Crisis Care Continuum
The 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation is a comprehensive, multi-method study aligned with SAMHSA’s goal to evaluate the full spectrum of 988 Suicide & Crisis Lifeline services and related grant funding. This evaluation aims to understand system responses to behavioral health crises and the outcomes of interventions. The evaluation is structured across three levels: System-Level, Client-Level, and Impact.
Each level includes investigations to identify differentials in the use of the 988 Suicide & Crisis Lifeline and BHCSC services and their outcomes. Multiple studies within these levels will highlight key elements of crisis service activities and outcomes. Exhibit 2 provides an overview of the evaluation’s studies for the three-year data collection period with a five-year evaluation period.
Exhibit
2: Overview of 988 Suicide & Crisis
Lifeline and Crisis Services Program Evaluation Design
This comprehensive evaluation will enable SAMHSA and grantees of 988 crisis continuum services to make evidence-based decisions to improve program services and delivery. The evaluation will gather information on:
The characteristics, collaborations, and structures of the crisis services infrastructure within states, territories, and Tribal jurisdictions that support improved client outcomes.
Outcomes related to the reduction in the use of non-behavioral health crisis services (e.g., 911, law enforcement, emergency medical services).
The effectiveness of the 988 Suicide & Crisis Lifeline and BHCSC in linking individuals to referral services after contacting the crisis system and the relationship between engagement with crisis services and behavioral health outcomes.
Immediate reductions in risks of suicide, violence toward others, and overdose.
The impact of the 988 Suicide & Crisis Lifeline and BHCSC on suicide and overdose morbidity and mortality.
System-Level Studies
Two system-level studies examine the characteristics, collaborations, and structures of the crisis services infrastructure within states, territories, and Tribal jurisdictions that support improved client outcomes.
System Composition and Collaboration Study. The System Composition and Collaboration Study seeks to understand the organizational structures and collaborative dynamics of the 988 Suicide & Crisis Lifeline and BHCSC across various states, territories, and Tribal organizations. This study examines how different regions structure their crisis care systems, focusing on communication and integration among providers like crisis contact centers, mobile crisis services, and crisis stabilization units. It aims to understand how the 988 Suicide & Crisis Lifeline and BHCSC are implemented, highlighting inter-agency cooperation and resource management. To achieve these objectives, the study employs the System Implementation Survey, Crisis Continuum Provider Survey, along with Key Informant Interviews-Case Study Protocol for case studies and Key Informant Interview-Case Study Protocol - Case Sub-Study for data on the cost elements of implementing crisis case service.
System-Level Service Utilization Study. The System-Level Service Utilization Study aims to evaluate how the 988 Suicide & Crisis Lifeline and BHCSC changes crisis service utilization patterns, particularly observing shifts from traditional emergency services to behavioral health-focused crisis interventions. This study will assess whether the 988 Suicide & Crisis Lifeline and BHCSC implementation has resulted in a more behaviorally focused crisis response, thereby reducing use of non-behavioral health services like emergency departments or law enforcement. The investigation will leverage data from the System Composition and Collaboration Study and secondary data sources from emergency and behavioral health services. The study’s core purpose is to monitor if there is a shift in crisis response patterns and determine the 988 Suicide & Crisis Lifeline’s and BHCSC’s effectiveness in providing appropriate care.
Client-Level Studies
Two client-level studies will provide information about how the 988 Suicide & Crisis Lifeline and BHCSC fulfill their mission to connect those in crisis with the services and support needed to reduce crisis risk and improve overall behavioral health outcomes.
Client-Level Service Utilization and Outcome Study. The Client-Level Service Utilization and Outcome Study examines an individual’s journey through the crisis services system. It explores how effectively the 988 Suicide & Crisis Lifeline and BHCSC connect people to needed services and assesses the outcomes of these interventions. The study aims to capture the specific needs of those seeking help and evaluate how engagement with these services impacts their behavioral health outcomes. Using contact information from the Client Contact Disposition Form (CCDF), Client Contact Disposition Form: Parent or Caregiver Supplement (CCDF-PS), and a series of Client Experience Surveys (CES), the study will track individuals from their initial contact with the 988 Suicide & Crisis Lifeline or a BHCSC through to longer-term supportive care. This study will provide insights into how people engage with the 988 Suicide & Crisis Lifeline and BHCSC, the impact of these services on individuals, and common barriers to accessing services.
Client-Level Risk Reduction Study. The Client-Level Risk Reduction Study concentrates on the immediate outcomes of interactions with the 988 Suicide & Crisis Lifeline and BHCSC, specifically targeting risk reduction related to suicide, violence, and overdose. The study aims to assess the immediate impact of crisis contacts in reducing risks and provide a comprehensive understanding of client experiences with the 988 Suicide & Crisis Lifeline and BHCSC, evaluating the effectiveness of immediate crisis interventions in reducing risk factors and enhancing safety for individuals in crisis. This study will involve qualitative coding of contact recordings and interviews with a sample of clients (C-KII-DC) and involved third parties (C-KII-TPC).
Impact Evaluation Study. The Impact Evaluation Study will assess the broader effects of the 988 Suicide & Crisis Lifeline and BHCSC on key public health outcomes, such as suicide and overdose rates. It aims to determine their future impact on reducing suicidal thoughts, suicide attempts, non-fatal overdoses, and related deaths. Using secondary data sources like CDC mortality data and Medicaid claims, the study will analyze long-term trends in public health outcomes before and after the implementation of the 988 Suicide & Crisis Lifeline and BHCSC. This study will evaluate the effectiveness of these services in reducing mental health crises and provide insights for future policy and program improvements.
The key evaluation questions are presented below for each study in Exhibit 3.
Exhibit 3: Key Evaluation Questions for the Five Studies
Data Collection Instruments and Methods
Approval is being requested for nine data collection activities that compose the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation. Exhibit 4 provides a description of each instrument, the methods used, and respondents.
Exhibit 4: Data Collection Instruments
Exhibit 5 below lists instruments contained within the Client Experience Survey (CES).
Exhibit 5. Instruments within the Client Experience Survey
|
Use of the Information Collected
According to SAMHSA’s 2020 National Guidelines for Behavioral Health Crisis Care, comprehensive crisis care systems include core services like crisis contact centers, mobile crisis teams, and crisis receiving and stabilizing facilities. The information collected through the 988 Lifeline and Crisis Services Program Evaluation supports SAMHSA’s strategic priorities, which aim to improve behavioral health services in the U.S. These priorities focus on enhancing crisis care systems to promote mental health, prevent substance use and overdose, and support recovery, while ensuring equitable access and better outcomes.
Additionally, the evaluation aligns with the 2024 National Strategy for Suicide Prevention. This strategy, developed by over 20 agencies and offices across 10 federal departments, is guided by SAMHSA, the CDC, the National Institute of Mental Health (NIMH), and the Department of Health and Human Services’ Office of the Assistant Secretary for Planning and Evaluation (HHS/ASPE). The evaluation advances these strategic goals by providing data to improve public health and service delivery efforts related to mental health and suicide prevention.
Addressing SAMHSA’s Strategic Priorities
SAMHSA aims to increase access to a full continuum of care that provides timely and high-quality services to anyone who needs them. The 988 Lifeline and Crisis Services Program Evaluation aligns with SAMHSA’s priority “Enhancing Access to Suicide Prevention and Mental Health Services,” which aims to enhance access to suicide prevention and crisis care as crucial elements of the mental health continuum of care, so that people experiencing suicidal ideation and other behavioral health crises can receive the care they need and want in order to achieve well-being and thrive (SAMHSA Strategic Plan, 2023). Specifically, the evaluation addresses Priority 2: Goals 2 and 3, Objectives 2.1, 2.2, 3.1, and 3.2 (see below).
Objective 2.1 Improve access to suicide prevention services.
Objective 2.2. Improve the quality and effectiveness of suicide prevention services.
Objective 3.1. Improve the experience for people in crisis and for crisis care providers.
Objective 3.2. Improve allocation of resources across the crisis care ecosystem.
Addressing the 2024 National Strategy for Suicide Prevention
The 2024 National Strategy for Suicide Prevention addresses existing gaps and emerging issues to guide, motivate, and promote a more coordinated and comprehensive crisis care continuum. The 988 Lifeline and Crisis Services Program Evaluation aligns with the NSSP. Specifically, the evaluation addresses the following objectives within these strategic directions:
Objective 1.2. Create and enhance connections between state agencies, tribal nations, and local communities to increase the reach of comprehensive suicide prevention activities and to strengthen outcomes.
Objective 2.1. Assess community strengths and gaps to inform suicide prevention planning at the individual, relationship, community, and societal levels.
Objective 8.4 Promote effective continuity of engagement and care for patients with suicide risk when they transition between different health care settings and providers, especially crisis, emergency, and hospital settings, and between health care and the community.
Using Improvements in Information Technology
Every effort was made proactively to reduce the burden on individual respondents who participate in the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation. The web-based data collection system was designed to be intuitive and easy to navigate. Data collection via a computer or mobile web-enabled device allows grantees to conveniently submit data. In addition, system features help grantees track their data entry and responses.
Web-based Data Collection and Management System
The 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation uses the CSPDC, a web-based portal developed for this evaluation to efficiently gather data. This portal allows evaluation personnel to directly enter data and administer web surveys to respondents, ensuring timely and accurate data collection, which helps provide timely assistance and reduce stress for callers during crises.
The CSPDC is designed to be accessible and user-friendly, supporting a range of devices from desktops to mobile phones. The interface allows organizations to manage profiles, track progress, and respond to real-time data entry validations, enhancing data accuracy and reducing administrative burdens.
The CSPDC processes large volumes of data, including call patterns, response times, and resolution outcomes. Data is securely stored within the SAMHSA cloud environment, enabling comprehensive data analytics. This technology supports the goal of the 988 Suicide and Crisis Lifeline and Crisis Services Program Evaluation to continuously improve the 988 Lifeline and BHCSC through effective data analysis.
The CSPDC facilitates data collection, management, and dissemination, including communication between grantees and the evaluation team, secure data transmission and storage, and data quality monitoring. It serves as the portal for data collection forms, surveys, and data upload tools, supporting the evaluation by providing flexibility for new data collection protocols and linking existing data sources. The system offers tools for monitoring response rates in real-time, supporting data reporting and analysis needs, and downloading response data throughout the data collection cycle. Data sets are produced at least twice a year for delivery to grantees, with final data sets provided upon completion of data collection. Exhibit 6 details the core functionality of the CSPDC. The CSPDC will be secured at the NIST Moderate level and will only launch after receiving Authority to Operate (ATO) from SAMHSA.
Exhibit 6: CSPDC Core Functionality
4. Effort to Avoid Duplication
Before developing the data collection activities for the 988 Lifeline and Crisis Services Program Evaluation, literature was reviewed and a crosswalk completed to ensure that the data collection activities do not duplicate those already conducted by other components of the 988 Suicide & Crisis Lifeline and BHCSC, including metrics data and data collected by service entities such as Vibrant. Past and current national efforts, such as those from NRI Inc, the Center for Financing Reform & Innovation (CFRI), and others, were also reviewed to assess how their efforts align with each of the evaluation questions in the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation and to avoid duplication or similar information with existing mental health and crisis intervention data collection frameworks.
a. Existing Research
While previous studies have examined aspects of the 988 Lifeline and BHCSC, the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation is the first comprehensive evaluation funded to assess the 988 Suicide & Crisis Lifeline and BHCSC. Prior studies, such as the national survey of behavioral health program directors, revealed that many communities were unprepared for the 988 rollouts in terms of financing, staffing, and infrastructure (Matthew et al., 2023). Additionally, studies have highlighted significant increases in call volume following the transition to the 988 number, with some states experiencing a rise of over 30% in crisis calls (Purtle et al., 2023). Despite these findings, no evaluation has matched the scope, detail, and focus on intermediate- and long-term outcomes that this evaluation will provide, making it a unique and essential endeavor to understand the full impact and effectiveness of the 988 Suicide & Crisis Lifeline and BHCSC on a national level.
Currently, other federal and private entities are implementing data collection and analytic efforts related to the 988 Suicide & Crisis Lifeline and BHCSC. These Federal efforts, described below, informed the development of the 988 Suicide and Crisis Lifeline and Crisis Services Program Evaluation.
Utilizing Medicaid, Medicare, and Private Insurance Data to Understand BHCSC Service Trajectories, Clinical Outcomes, and Risk for Suicide or Other Violent Death
SAMHSA, through CFRI, seeks to understand financing methods of behavioral health care to identify opportunities, innovations, and challenges related to service delivery and accessibility. By employing Medicaid, Medicare, and private insurance claims data, researchers aim to implement a longitudinal study of individual outcomes throughout the crisis care continuum. These study efforts will continue over the fiscal year 2024.
NRI State Profile Reports
NRI, Inc., established in 1987 as a non-partisan, nonprofit organization, initially partnered with the National Association of State Mental Health Program Directors (NASMHPD). Since then, NRI has been collaborating with state agencies and the federal government to collect and analyze data on public behavioral health crisis systems and call centers, mobile crisis teams, short term crisis receiving and stabilization facilities, and crisis services financing. With a focus on performance measurement, NRI gathers data for annual reporting from state and some private psychiatric facilities, aiding in planning and evaluation at the national, state, and regional levels. NRI also assists states in meeting reporting requirements set by The Joint Commission.
Implementation of the Federal 988 Suicide and Mental Health Crisis Hotline Policy: Determinants and Effects of State Policy Implementation Financing Strategies
Researchers at New York University lead projects focusing on the financing strategies of state policies related to the implementation of 988. Their projects aim to characterize and explore the determinants of states’ 988 implementation financing strategies, assess the effects of user fee legislation on implementation outcomes, and investigate perceptions of financing determinants’ impact on policy implementation success. Employing legal mapping, national surveys, interviews, and a difference-in-differences analysis, the project team plans to generate knowledge to guide financing decisions for the early-stage implementation of 988, aligning with the National Institute of Mental Health’s (NIMH’s) Strategic Plan and suicide prevention goals over fiscal year 2024.
Vibrant Emotional Health Evaluation of a Network-wide Training Initiative
Vibrant Emotional Health, in collaboration with ICF, aims to evaluate how comfortable crisis counselors are with the 988 Suicide & Crisis Lifeline’s safety assessment and short-term intervention model. The project will improve training to enhance crisis center guidance. It will also measure differences in counselors’ skills with various groups and explore factors affecting their confidence, knowledge, and behavior. The evaluation will focus on the effectiveness of three mandatory core training courses designed to boost crisis counselors' confidence, knowledge, and skills in Lifeline Safety Assessment. Data collection will begin in summer 2024, with final analysis expected in early 2025.
988 Implementation Support
Over 2021-2022, the Education Development Center (EDC) worked on a project that successfully increased the capacity of 50 grantees to strengthen crisis services, reaching multiple stakeholders to prepare for implementation of the 988 Suicide & Crisis Lifeline and BHCSC. Landscape analysis reports provided insights into current Lifeline crisis centers, identifying areas for development, while strategic plans equipped state and territory grantees to handle the expected increase in 988 calls by ensuring quick, appropriate, and non-punitive responses to behavioral health crises.
5. Impact on Small Business or Other Small Entities
The 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation predominately focuses on individual crisis support, which reduces the need for extensive data collection from small businesses or entities. Unlike other initiatives that require significant data from businesses, this evaluation gathers data from individuals experiencing mental health crises, minimizing the burden on small businesses and crisis service providers.
When data collection from small entities like local crisis centers or small healthcare providers is necessary, the evaluation uses streamlined processes to gather only essential information, reducing administrative work. The evaluation employs user-friendly digital reporting tools that simplify data submission, designed to make the portal easy to navigate and efficient for small entities to participate.
The evaluation provides targeted training and technical assistance (T/TA) to support these entities. Targeted T/TA activities and resources will support evaluation participants in engaging in the evaluation by activity or survey type. Targeted T/TA will ensure evaluation participants understand required data collection and reporting requirements and receive ongoing support based on their level of participation. T/TA activities will include informational sessions and office hours and resources such as scripts and FAQ documents. SAMHSA maintains ongoing dialogue with providers to identify ways to further reduce the data collection burden, ensuring their participation remains sustainable and does not interfere with their core services. These measures help focus on individual respondents while collecting necessary data from organizational providers in the least intrusive way possible.
6. Consequences if Information Collected Less Frequently
Regular data collection for the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation is essential for accurately monitoring its implementation and detecting changes over time. Collecting data at the necessary frequency ensures that SAMHSA and other federal entities can efficiently evaluate service effectiveness, identify areas for improvement during the implementation of the 988 Suicide & Crisis Lifeline and BHCSC, and track progress.
Consistent and timely data collection informs federal policy development and decision-making. Insights from regular data on mental health trends, crisis intervention outcomes, and user demographics are crucial for shaping mental health policies, directing funding, and developing targeted initiatives. Without this data, policies risk becoming outdated or misaligned with the current mental health and substance use landscape.
The technological advancement of the 988 Suicide & Crisis Lifeline also depends on regular data updates. Continuous improvement of features like call-routing algorithms and digital support tools relies on ongoing data collection to meet the changing demands of mental health crisis intervention.
Organizations participating in the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation must provide effective crisis intervention services nationwide. Regular data collection is required to demonstrate compliance with federal mandates. Inadequate data collection can lead to non-compliance issues, posing legal challenges and concerns about the program’s adherence to federal requirements.
As the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation evolves, regular data collection ensures the system remains agile and responsive, quickly adapting to new trends, challenges, and best practices in mental health crisis intervention. Limiting data collection could hinder timely improvements, risking the program's relevance and effectiveness.
The instrument-by-instrument consequences can be found in Exhibit 7 below.
Exhibit 7. Data Collection Activities and Consequences If Information Collected Less Frequently
7. Consistency with the Guidelines of 5 CFR 1320.05(d)(2)
The data collection fully complies with the requirements of 5 CFR 1320.5(d) (2). Measures are also compliant with the race and ethnicity statistical standards laid out in 88 FR 5375, except for the CCPS (Attachment C). We are requesting exemption from using the race and ethnicity statistical standards laid out in 88 FR 5375 (2024 SPD 15) for this instrument. This instrument asks crisis agencies to provide information about the composition of their workforce in aggregate, which will require additional coordination and planning with these agencies. Based on the results of pilot testing efforts, we anticipate that many crisis service agencies will only collect required EEOC data. Thus, reducing the included race and ethnicity categories will mitigate missingness and enhance the reliability of data. We will comply with implementing the race and ethnicity categories in Figure 1 of 88 FR 5375 in our planned OMB renewal package in April 2028.
Consultation Outside the Agency
Federal Register Notice
SAMHSA published a notice in the Federal Register on May 29, 2025 (Volume 90 FR 22747, page 22747-22749), soliciting public comment on this study. No public comments were received.
Consultation outside the Agency
Consultation on the design, instrumentation, and statistical aspects of the evaluation has occurred with individuals outside of SAMHSA. An evaluation advisory Expert Advisory Panel (EAP), established in 2023 and convened in 2024, provided input and guidance on the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation design and instruments. The EAP members and positions are listed in Exhibit 8. A similar series of webinars in early 2024 with 988 Suicide & Crisis Lifeline and BHCSC implementors also provided feedback on the evaluation plan.
Exhibit 8. Expert Advisory Panel
Person |
Title |
Role |
Victor Armstrong, MSW |
Vice President |
American Foundation for Suicide Prevention |
Jennifer Battle, MSW |
Vice President, Community Access and Engagement |
The Harris Center for Mental Health and IDD |
Peter Britton, PhD |
Associate Professor, Department of Psychiatry
|
University of Rochester Medical Center Center for Integrated Healthcare, Syracuse VAMC |
Margie Belfour, MD, Ph.D. |
Chief of Quality & Clinical Innovation |
Parkland Hospital, Dallas, TX |
John Draper, Ph.D. |
President of Research and Development |
Behavioral Health Link, Inc. |
Dwight Holton, JD |
Chief Executive Officer |
Lines for Life |
David A. Jobes, Ph.D. |
Professor of Psychology |
The Catholic University of America |
Preston Looper, MS, LPC-S |
Founder & Principal |
Full Tilt Strategies LLC |
Jane Pirkis, Ph.D. |
Professor & Director |
Centre for Mental Health in the Melbourne School of Population and Global Health |
Jonathan Purtle, DrPH |
Associate Professor & Director of Policy Research |
NYU’s Global Center for Implementation Science |
Shelby Rowe, MBA |
Executive Director/Principal Investigator |
Suicide Prevention Resource Center |
Matthew Wintersteen, Ph.D. |
Associate Professor & Research Director Child & Adolescent Psychiatry Service |
Thomas Jefferson University |
For the System Composition and Collaboration Study, the Panel advised on the adoption of inclusive data collection methods. They recommended engaging with funding bodies like Medicaid or county administrations to include crisis service organizations not directly funded by SAMHSA. The Panel underscored the importance of enhancing coordination with healthcare systems to reduce emergency room pressures and better understand community roles in mental health emergencies. Suggestions included establishing formal channels for information exchange among partners and acknowledging state-level variations in the 988 Suicide & Crisis Lifeline and BHCSC’s implementation, which could influence outcomes.
In assessing the Client-Level Service Utilization and Outcome Study, the Panel highlighted the significance of recognizing informal support systems and incorporating strategies like safety plans or medication coping post-crisis contact to ensure sustained client wellbeing. Considering family dynamics, especially in youth evaluations, was also noted as important for capturing the broader context influencing recovery. The Panel advocated for research participation incentives to focus on altruistic benefits, encouraging engagement without financial compensation.
For the Client-Level Risk Reduction Study, the Panel suggested broadening the evaluation scope to include mental health and substance use treatments alongside suicide prevention efforts. They stressed that evaluating the effectiveness of the 988 Suicide & Crisis Lifeline should encompass a holistic approach, considering the intertwined nature of mental health, substance use, and suicide risk.
The Impact Study discussions covered different data aspects, such as officer-involved shootings linked to mental health crises highlighting the intersection of law enforcement and mental health services. The Panel suggested comparing international crisis care systems to gain a broader perspective on best practices and innovative approaches. Recognizing the variability in state-level data, they advocated for a flexible evaluation framework that accommodates different state approaches and legislative responses. Including racial data was highlighted to address differentials in mental health crisis responses and ensure equitable evaluation outcomes. The Panel also proposed exploring additional data sources like jail/prison records, telehealth usage, and third-party call data to capture a comprehensive picture of the 988 Suicide & Crisis Lifeline's impact. They emphasized the importance of integrating findings from qualitative interviews to enrich quantitative data, providing deeper insights into client experiences and system effectiveness.
The feedback from the advisory panel has been instrumental in enhancing the depth and comprehensiveness of the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation. These inputs underscore the necessity of employing wide-ranging and collaborative approaches in the evaluation and improvement of the U.S.’s crisis care systems.
Payment to Respondents
The 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation will use a research-based approach and will require participants from organizations, clinicians/client-facing staff, and client contacts. As a result, renumeration is suggested for respondents not directly affiliated with the organizations’ 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation at the time of their participation in appreciation of their time, potential inconvenience and burden of participation, and any related costs (e.g., mobile phone minutes or data, compensation for time). Thus, we recommend the following renumeration.
To offset any costs associated with their support of data collection (e.g., staff time), a selection of crisis contact centers, mobile crisis, and crisis stabilization providers will receive a stipend of $3,000 as an incentive for participation in the Client-Level Service Utilization and Outcome Study and the Client-Level Risk Reduction Study. Renumeration for respondents of the CES is a $20 incentive for their participation in each survey (enrollment, 3, 6, and 12-months) amounting to $80. Furthermore, $20 incentives will be distributed to up to 1,000 CCPS respondents, or 500 per survey administration, based on incentive lottery, C-KII-DCs and C-KII-TPCs will receive $50 after completion of the interview. Respondents to other data collection activities are primarily organizational staff or close affiliates. Therefore, no remuneration is planned for those activities.
10. Assurances of Confidentiality
Data will be kept confidential to the maximum extent allowed by law. Additionally, the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation will apply for a SAMHSA certificate of confidentiality (CC). The CC protects grantees and contractors from legal requests for names or other information that would personally identify participants in the evaluation of a grant, project, or contract. To assure the confidentiality of data collected during the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation and to ensure the protection of human subjects, evaluation data collection protocols and instruments will be reviewed and approved by ICF’s Institutional Review Board (IRB) prior to its collection. This review ensures compliance with the spirit and letter of U.S. Department of Health and Human Services (HHS) regulations governing projects that collect data from human subjects. Team Aptive IRB holds a Federal wide Assurance (FWA00002349 Exp. October 13, 2025) from the HHS Office for Human Research Protections (OHRP).
All protected data will be stored in the CSPDC, which will be supported by SAMHSA’s cloud environment and in the manner described in the Evaluation Plan, Section 10 Data Management and Storage Plan, submitted to, and approved by, SAMHSA for the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation on April 26, 2024. Furthermore, the CSPDC will facilitate data entry and management for the evaluation.
All data from the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation will be gathered electronically. Personal identifying information (PII) will only be gathered as necessary for survey administration or incentive distribution and will be stored separately from survey or interview data. Specifically, contact details needed for survey management and/or incentive distribution will be inputted into password-protected databases accessible solely to a select few, including data analysts and administrative personnel responsible for incentive distribution. These individuals have signed privacy, data access, and data use agreements. PII collected for survey administration will not be retained alongside survey responses, and all datasets will be stripped of any PII before analysis.
The confidentiality assurances provided by the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation are integral, complying with legal requirements and embodying the service’s commitment to offering safe, private, and compassionate support to individuals in mental health crises. Ultimately, the evaluation team developed processes, systems, and procedures to collect the minimum required PI and ensures data confidentiality, releasing it to the least number of people.
Specific procedures to protect the privacy of respondents for activities requiring PII for administration and/or incentive allocation are described in Exhibit 9.
Exhibit 9. Procedures to Protect Respondent Privacy
Activity |
Privacy Procedures |
SIS and CCPS |
Participating organization staff will enter information directly into the CSPDC to complete the SIS. To access the system, respondents will create individual usernames and passwords to protect their privacy. It is necessary to collect respondent contact information via the SIS to identify potential CCPS respondents and administer and distribute incentives for the CCPS. Respondent contact information will include agency affiliations, names, contact numbers, and email addresses and will be entered into a password-protected database, separate from survey responses. PII required for annual SIS and biennial CCPS administration will be maintained on Aptive’s secure servers. |
KII-CSP, KII-CSP-CSS, and Client KIIs |
KIIs for the participating organization case studies will include a select number of systems that provide services across more than one part of the BHCSC. Case study sites, selected at the state, territory, or Tribal nation level, will be chosen based on representation from the system configurations described in the SIS (e.g., hub and spoke, funding type). These identifiers, along with any necessary contact information, will be securely stored in a password-protected database, separate from the interview data. Potential interviewees will be invited to participate in the KIIs via notification through the CSPDC, and no PII will be requested during the interviews themselves. For the Cost Sub-Studies, limited PII will be collected from participating organizations to ensure effective communication and data collection. This information, including names and email addresses, will be entered into a separate, secure database distinct from the data collected on 988 implementation costs and costs incurred beyond the scope of SAMHSA grant activities or funding. The PII will be used for administrative purposes and will not be included in any public reports or datasets. The identities of individual participants or specific organizational details will not be disclosed in any reports or analysis, maintaining confidentiality. Client KII participants will include those that indicate interest in the CCDF. Prospective participants will be contacted via their preferred contact method to arrange interviews. Prior to interviews, participants will verbally consent to participate and no PII will be requested during the interviews themselves. In all KIIs, participants will be informed during the consent process about the specific use of their data and the measures in place for privacy protection. This will ensure that all participants are fully aware of how their information will be used and safeguarded, in compliance with the privacy protocols established by ICF’s IRB and relevant regulatory guidelines. |
CCDF |
Clients will be asked about their interest in participating at the contact’s close to ensure that counselors can establish rapport and effectively intervene in crisis events. The CCDF will be used to guide participant recruitment into the study and collects information about the best contact methods for survey invitations, client’s presenting concerns, contact disposition, and characteristics of contact (e.g., risk assessment, suicide attempt in progress). If a client agrees to participate, crisis staff will complete a CCDF for that client and submit it directly to the CSPDC. The CCDF includes a consent-to-contact form so clients may receive baseline and follow-up survey invitations. The consent-to-contact request will ask consumers to provide identifying information (e.g., name, email, phone number) necessary to distribute survey invitations at each timepoint. Contact information will only be used to distribute survey invitations and will be maintained in a secure, password-protected database. Contact data will be stored separately from de-identified CES and CCDF data. |
CCDF-PS |
Those under 18 will also be asked to provide contact information for their parent or guardian, who will receive the Client Contact Disposition Form: Parent Supplement. This supplement contains the CES consent form for youth and a small subset of CCDF-PS contact information questions to complete via email. Contact information will only be used to distribute survey invitations and will be maintained in a secure, password-protected database. Contact data will be stored separately from de-identified CES and CCDF-PS data. |
CES |
The CES will be administered to individuals who have expressed interest via CCDF submission and provide consent/assent for participation immediately prior to survey completion (see CCDF description for note about parent consent, where required). One week after their index contact with a participating 988 Suicide & Crisis Lifeline center and/or the BHCSC, clients will receive a survey invitation in alignment with contact preference indicated on the CCDF (text, email) to review a consent form for the CES and complete the baseline survey. Consenting participants will receive similar survey invitations 3, 6, and 12 months after completing their baseline CES. |
11. Questions of a Sensitive Nature
Special attention is given to the handling of sensitive data gathered from various instruments. The CES contains questions that are potentially sensitive. For example, certain questions request current mental health and health status, the types of behavioral health services received, and health outcomes. Furthermore, a subset of clients who are invited to participate in the CES will also be invited to participate in C-KII-DCs or C-KII-TPCs, where they will be asked to share additional details about their experience with the crisis system. These questions are central to SAMHSA’s goal of learning whether and how experiences with the 988 Suicide & Crisis Lifeline and BHCSC services influence behavioral health outcomes. Given the potentially sensitive nature of these questions, the following procedures are in place:
The CES web-based consent form explicitly advises potential respondents about the sensitive nature and content of the data collection protocol, as well as the voluntary nature of all data collection activities. The CES consent also contains contact information for the 988 Suicide & Crisis Lifeline.
Upon consent to participate, CES participants are reminded of their ability to skip questions, pause the survey, or stop the survey throughout survey administration. A second reminder is offered after respondents answer questions on their current mental health and wellbeing. If responses indicate an active crisis or moderate/high crisis risk, a dedicated screen will display a prompt with contact information for the 988 Suicide & Crisis Lifeline.
Any unanticipated or negative consequences will be immediately reported to the Evaluation Team’s IRB. In these situations, the principal investigator and project director will consult with appropriate clinical professionals to immediately determine if the participant presents a risk to themselves or others and make appropriate referrals.
During the C-KII-DCs and C-KII-TPCs, a trained interviewer will provide reminders that interview participants may choose to skip questions, pause the interview, or stop the interview throughout the interview administration. Interviewers will also assist the participant in connecting to crisis resources if active crisis risk is disclosed during the interview.
For all evaluation tools, organizations and staff will be asked to follow their organization’s crisis procedures if consumers express discomfort or active crisis risk during data collection.
12. Estimates of Annualized Burden Hours and Costs
This is a new data collection. Clearance is being requested for three years of data collection for the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation. Exhibit 10 below describes the burden and costs associated with 988 Suicide & Crisis Lifeline and BHCSC evaluation data collection activities. The cost was calculated based on hourly wage rates for appropriate categories presented in the Bureau of Labor Statistics (BLS) Occupational Employment Statistics (OES) May 2022 National Industry-Specific Occupation Employment and Wage Estimates. The participating organizations and number of respondents was calculated based on the reported number of crisis service agencies serving adults presented in the NRI (2023) State Mental Health Agency Profiles, including crisis contact centers (n = 544), mobile crisis teams (n = 1,287), and crisis stabilization facilities (n = 237). Estimates represent the number of these respondents who will be asked to participate based on the sampling criteria for each data collection activity (i.e., the use of a recruited sample of agencies for client-level components). Exhibit 10 shows an annualized summary of burden hours by respondent type.
Exhibit 10. Estimated Annualized Burden Hours and Costs (Across the 3-Year Clearance Period)
Type of Respondent |
Instrument |
Number of Respondents per year |
Responses per Respondent |
Total Number of Responses
|
Burden per Response (hours) |
Annual Burden (hours) |
Hourly Wage Rate ($) |
Total Annualized Cost ($) |
System Composition and Collaboration Study |
||||||||
Organizational Staff/Crisis System Administrator1 |
SIS |
73 |
1 |
73 |
0.75 |
54.75 |
$78.06 |
$4,273.79 |
Organizational Staff/Crisis Agency Manager2 |
CCPS |
1034 |
1 |
1034 |
1.50 |
1,551.00 |
$58.80 |
$91,198.80 |
Organizational Staff/Crisis Agency Staff3 |
KII-CSP |
35 |
1 |
35 |
1.00 |
35.00 |
$27.46 |
$961.10 |
Organizational Staff/Crisis Agency Staff3 |
KII-CSP-CSS |
13 |
1 |
13 |
0.50 |
6.50 |
$27.46 |
$178.49 |
Client-Level Service Utilization and Outcome Study |
||||||||
Organizational Staff/Crisis Agency Staff3 |
CCDF |
6,000 |
1 |
6,000 |
0.15 |
900.00 |
$27.46 |
$24,714.00 |
Parents/Caregivers4 |
CCDF Parent Supplement |
1,5605 |
1 |
1,560 |
0.10 |
156.00 |
$7.25 |
$1,131.00 |
Client4 |
CES - Baseline |
6,000 |
1 |
6,000 |
0.75 |
4,500.00 |
$7.25 |
$32,625.00 |
Client4 |
CES - 3 months |
1,500 |
1 |
1,500 |
0.65 |
975.00 |
$7.25 |
$7,068.75 |
Client4 |
CES - 6 months |
375 |
1 |
375 |
0.65 |
243.75 |
$7.25 |
$1,767.19 |
Client4 |
CES - 12 months |
94 |
1 |
94 |
0.65 |
61.10 |
$7.25 |
$442.98 |
Client-Level Risk Reduction Study |
||||||||
Client4 |
C-KII-DC |
30 |
1 |
30 |
1.00 |
30.00 |
$7.25 |
$217.50 |
Client4 |
C-KII-TPC |
10 |
1 |
10 |
1.00 |
10.00 |
$7.25 |
$72.50 |
Total |
16,724 |
|
|
|
8,006.10 |
|
$164,651.10 |
|
Exhibit 11. Annualized Summary Burden by Respondent Type
Respondents |
Number of Respondents |
Responses/ Respondent |
Total Responses |
Hours per Response (Total Hr/Total Responses) |
Total Annualized Hour Burden |
Organizational Staff/ Crisis System Administrator |
73 |
1 |
73 |
0.75 |
54.75 |
Organizational Staff/ Crisis Agency Manager |
1,034 |
1 |
1,034 |
1.50 |
1551 |
Organizational Staff/ Crisis Agency Staff |
6,048 |
3 |
18,144 |
0.55 |
9,979.2 |
Parents/Caregivers |
1,560 |
1 |
1,560 |
0.10 |
156 |
Client |
8,009 |
6 |
48,054 |
0.78 |
37,482 |
Total |
16,724 |
|
68,865 |
|
49,223 |
13. Estimates of Annualized Cost Burden to Respondents or Record Keepers
Programs are collecting most of the required data elements as part of their normal operations Program grant operations. Organizations will maintain this information for their own program planning, quality improvement, and reporting purposes. Therefore, there are no additional capital or start-up costs associated with this evaluation. There will be some additional burden on record keepers to provide potential respondent lists for data collection activities, as detailed in Exhibits 10 and 11.
14. Estimates of Annualized Cost to the Government
SAMHSA’s 988 and Behavioral Health Crisis Coordinating Office has planned and allocated resources for the management, processing, and use of the collected information in a manner that shall enhance its utility to agencies and the public. Including the Federal contribution to local evaluation efforts, the contract with Aptive Resources, and government staff to oversee the evaluation, the annualized cost to the Government is estimated at $3,414,907. These costs are described below.
A contract has been awarded to Aptive Resources for evaluation of the 988 Suicide & Crisis Lifeline and Crisis Services Program. The current evaluation contract with SAMHSA is funded to conduct the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation with various stakeholders including 988 crisis centers, mobile crisis response teams, crisis stabilization facilities, and other behavioral health crisis service continuum partners. The estimated average annual cost of the contract will be $2,788,202. This amount covers expenses related to developing and monitoring the national evaluation, including but not limited to, developing the evaluation design and instrumentation, developing TTA resources (e.g., manuals, training materials, etc.), conducting in-person and/or virtual TTA, monitoring of stakeholders, traveling to sites and relevant meetings, and analyzing and disseminating data. In addition, these funds will support the development of a Web-based data collection and management system for the 988 Suicide & Crisis Lifeline and Crisis Services Program and fund staff support for data collection.
There are four SAMHSA grant programs that have requirements for participation in the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation, and as such are being factored into the annualized cost to the government estimate. Each grantee program (988 state/territory, 988 Tribal nations, Community Crisis Response Program, and CCFU) is expected to fund an evaluator to conduct the self-evaluation, participate in the larger 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation and to satisfy the requirements of their respective program. It is estimated that 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation participation will require up to 0.05 full-time equivalent (FTE) to collect information, enter information into the Web-based data collection and management system, recruit participants, and conduct analyses at the local level. Assuming an annual evaluator salary of $66,440 based on the BLS May 2022 data for the Survey Researcher category, 5 percent effort for one grantee would be $3,322. Considering each grantee program’s number of grantees – 988 state/territory has 56, 988 Tribal nations has 100, Community Crisis Response has 12, and CCFU has 10, the total grantee cost would be $591,316.
It is estimated that the 988 and Behavioral Health Crisis Coordinating Office will allocate 0.30 of a full-time equivalent each year for Government oversight of the evaluation. Assuming an annual salary of $117,962 for a GS-13 step 1 pay scale, these Government costs will be $35,389 per year. This 0.30 FTE includes primary oversight from the COR and Technical Monitor.
15. Changes in Burden
This is a new data collection.
16. Time Schedule, Publication, and Analysis Plans
a. Time Schedule
The time schedule for implementing the cross-site evaluation is summarized in Exhibit 12. SAMHSA is requesting three-year clearance for this project.
Exhibit 12. Time Schedule
Evaluation Year 1 |
Evaluation Year 2 09/15/2025-9/14/26 |
Evaluation Year 3 09/15/2026-9/14/27 |
Evaluation Year 4 09/15/2027-9/14/28 (Planned) |
Estimated OMB approval date April 2025
|
|
|
|
b. Publication Plans
Team Aptive is committed to producing a series of comprehensive reports and scholarly articles. This initiative aligns with the overarching objective of informing both congressional decision-makers and the wider suicide prevention and mental health community about the impacts and effectiveness of the 988 Suicide & Crisis Lifeline and BHCSC.
Team Aptive will conduct a thorough analysis of the data collected during the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation and compile this information into detailed reports. These reports are essential for providing Congress with up-to-date information about the program, ensuring that lawmakers have the necessary insights to make informed decisions.
Furthermore, the significance of the 988 Suicide & Crisis Lifeline and BHCSC in the field of suicide prevention and crisis intervention warrants the dissemination of findings through professional journals. This approach aims to reach a broader audience, including researchers, policy makers, and program administrators, thus contributing to the collective knowledge in these fields. Team Aptive, in collaboration with SAMHSA will identify key findings from the evaluation for publication.
Each year, Team Aptive plans to develop and submit at least one peer-reviewed article. Topics for these articles will be chosen based on their relevance and potential impact, focusing on areas such as the effectiveness of the 988 Suicide & Crisis Lifeline and BHCSC, patterns in service utilization, and outcomes related to mental health support and suicide prevention. Before submission, all articles will be reviewed by the Contracting Office Representative (COR) for feedback on content and structure and for determining the most suitable journals for publication.
All drafts intended for publication will go through the standard SAMHSA review process required by SAMHSA, ensuring alignment with the goals of the 988 Suicide & Crisis Lifeline and BHCSC and adherence to high academic standards. The selection of journals for publication will be carefully considered, targeting those with a strong reputation in mental health and crisis intervention research. Some potential journals include:
|
|
Data collected through the 988 Suicide & Crisis Lifeline and Crisis Services Program Evaluation will be analyzed to address key evaluation questions and related sub-questions. Analysis plans for each study are described below.
System Composition and Collaboration Study
This study’s purpose is to understand the crisis care continuum structure in each state, territory, or Tribal Nation, and the extent to which crisis continuum agencies work together to support the 988 Suicide & Crisis Lifeline and BHCSC.
Through primary data collection and drawing upon the SIS, CCPS, and KIIs, the team will provide a comprehensive view of the current state of crisis care systems, highlighting areas of strength and identifying potential gaps or challenges in resource allocation, information sharing, and inter-agency collaboration. Specifically, to assess the structural frameworks and collaborative practices among crisis continuum agencies, we will analyze SIS data using descriptive statistics to provide a precise characterization of the related activities. This will include, for example: (1) detailed structures of the BHCSC, (2) implementation strategies of the 988 Suicide & Crisis Lifeline and BHCSC, and (3) communication patterns among BHCSC partners.
To assess system relationships and collaborations in each state and territory, the team will compute Social Network Analysis upon data gathered through the CCPS. Key metrics include: (1) the number and types of partnerships each organization has established, (2) the frequency and modes of communication between organizations, and (3) the flow of resources and information within the network. Analytic aims include understanding the density and centrality of the network; and identifying key nodes and connections to provide a clearer picture of the collaborative dynamics and resource distribution within the 988 Suicide & Crisis Lifeline and BHCSC.
Additionally, the team will employ mixed factorial ANOVA to compare crisis service structures, implementation, and collaborations across regions. Regression models will investigate differentials in 988 Suicide & Crisis Lifeline and BHCSC implementation and effectiveness, while t-tests and chi-square analyses will examine variations in collaborative practices and partnerships. Thematic analysis of qualitative data will be guided by coding rubrics developed for Consolidated Framework for Implementation Research (CFIR 2.0, Damschroder et al., 2022) and Consolidated Framework for Collaboration Research (CFCR, Calancie et al., 2021), to further identify factors affecting the 988 Suicide & Crisis Lifeline and BHCSC's implementation, number and interaction methods of partnerships, and integration of appropriate practices.
Team Aptive will leverage demographic and socioeconomic data from the American Community Survey (ACS), a comprehensive survey conducted by the U.S. Census Bureau, to assess staff representativeness and to inform selection of case study sites. The team will conduct comparative analyses as part of the case studies to study the implementation of processes and the development of formal relationships with Tribal nations and urban Indian Health Centers by different 988 Suicide & Crisis Lifeline and BHCSC centers. These case studies will offer insights into the nuances of crisis service delivery and collaboration, specifically emphasizing partnerships with Tribal communities and practices. For the cost case studies, the team will use a focused financial analysis of the 988 Suicide & Crisis Lifeline and BHCSC’s funding, reviewing budget documents and plans from states, territories, and Tribal entities. This will identify specific investments and strategies, highlighting the different types and scale of funding, its impact on operational efficiency and effectiveness, and guiding sustainability and financial optimization efforts. Through these efforts, the team will gain insights from all instruments and evaluation questions to understand statistical trends and the underlying narratives and contexts.
System-Level Services Utilization Study
This study examines whether the 988 Suicide & Crisis Lifeline and BHCSC is successful in creating a behavioral-health-system-first response to crisis events and the resulting reduction in use of non-behavioral health crisis services (e.g., 911, law enforcement, emergency medical services).
The team will leverage existing data (e.g., organizational reports, and secondary data sources) as well as findings from the Systems Composition and Collaboration Study to examine the impact of the 988 Suicide & Crisis Lifeline and BHCSC on the utilization of crisis services, systemic changes over time, and behavioral health differentials among service users.
Descriptive statistics will be employed to characterize annual utilization of crisis services (e.g., 988 Suicide & Crisis Lifeline and BHCSC, 911, and ED utilization) and crisis contact dispositions before and after the 2022 implementation of the 988 Suicide & Crisis Lifeline and BHCSC, establishing usage trends. If data allow difference-in-difference methods, the team will conduct interrupted time series (ITS) to model pre-988 Suicide & Crisis Lifeline and BHCSC trends over time in the usage of crisis services and compare to trends observed after the introduction of the 988 Suicide & Crisis Lifeline and BHCSC. Since there can be no control group, a single group ITS approach will be used where each state will be modeled with a pre- and post-988 Suicide & Crisis Lifeline and BHCSC implementation trend. Placebo analyses or sensitivity analyses will be conducted to further validate findings.
Organizations’ maturity in BHCSC implementation will be assessed through continuous scores based on implementation degree, coordination, and collaboration (e.g., partnerships between 988 Suicide & Crisis Lifeline and BHCSC crisis centers and 911 PSAPs, mobile crisis teams, law enforcement, etc.) of the BHCSC. Inferential analyses, such as linear regressions, may be employed to explore relationships between organizational maturity and usage trends. Additionally, the team will use descriptive statistics to summarize characteristics of the 988 Suicide & Crisis Lifeline and BHCSC workforce influence on contact resolution rates, referral volume, and linkages to BHCSC services. This analysis will include characterizing: (1) workforce-related strategies to reduce behavioral health differentials, (2) workforce supervision/monitoring/support practices, and (3) workforce education background characteristics. To assess whether these factors may be associated with 988 Suicide & Crisis Lifeline and BHCSC utilization, they may be included as moderation terms in the inferential steps of the previous assessment.
Client-Level Services Utilization and Outcome Study
This study draws upon data provided by individuals who have direct experience with the 988 Suicide & Crisis Lifeline and other crisis services. Through the CES and the CCDF (and the accompanying CCDF-PS), this study will provide SAMHSA with information on the service needs, outcomes, and experiences of individuals who receive care through a 988 Suicide & Crisis Lifeline or BHCSC organization.
The team will use descriptive statistics to regularly provide a precise characterization of enrolled client and index contact characteristics, along with patterns in referrals and service utilization drawing upon data from the CES. This analysis will include, for example: (1) the number and proportion of clients enrolled in the study based on the initial referring group (e.g., contact center, contact center follow-up program, mobile crisis, crisis stabilization); (2) demographics of enrolled clients; (3) the characteristics of index crisis contacts among enrolled clients, including presenting concerns, contact disposition, modality, referrals received, and other behavioral healthcare engagement at time of contact; (4) types of behavioral healthcare services received by participating clients; (5) crisis experiences within the 12-month follow-up period; and (6) general satisfaction with and results of services received within the 12-month follow-up period, including behavioral health outcomes and perceptions of care. The team will also provide summary statistics (e.g., mean, standard deviation, change scores between time points) to help further characterize the experiences of clients as they navigate the 988 Suicide & Crisis Lifeline and BHCSC.
To address questions about the clinical and functional behavioral health outcomes of clients, the team will use a mixed factorial ANOVA to examine changes over time among three client groups:
Those supported only by a 988 crisis contact center
Those receiving crisis stabilization services
All individuals who receive support through a participating crisis contact center, crisis stabilization center, and mobile crisis agency will be recruited to participate in the CES and its pre-requisite, the CCDF. Power calculations for the CES indicate a comparison of three groups (crisis contact support, mobile crisis service participants, and crisis stabilization services participants) over four waves of data collection (enrollment, 3, 6, and 12-months) using a mixed factorial analysis of variance assuming a medium effect size (Cohen’s f = 0.25), a non-sphericity correction of 0.75, alpha = 0.05, and power = 0.80 would require a total sample size of 356 (89 per group) after attrition. Our proposed sampling approach aims to ensure at least 356 responses at the 12-month follow-up to fully address evaluation questions, after accounting for up to 75% attrition between waves based on previous studies in similar populations (Gould et al., 2017; Witte, et al., 2010). Assuming 75% attrition per wave, that would require an enrolled sample baseline sample of 24,000, or 12,000 each from 988 Lifeline contact centers and community crisis response providers. Incentives will also be distributed to clients participating in individual-level data collection. Clients will be categorized based on the highest level of care reported at baseline (e.g., an individual using both 988 Suicide & Crisis Lifeline and mobile crisis services will be included in the ‘mobile crisis’ group). This approach will allow comparisons between these groups to explore the impact of the crisis care continuum and understand how different services interact to influence outcomes.
A similar mixed factorial ANOVA will compare clinical and functional outcomes based on whether the intervention during the crisis contact was voluntary or involuntary. The team will also explore the relationship between client-level and system-level factors through multi-level modeling. Although the multidirectional nature of the crisis care continuum and variations in service delivery may limit the variance necessary for this analysis, multi-level modeling can reveal important predictors of client outcomes and enhance longitudinal analysis. The suitability of this approach will be assessed using intraclass correlations to determine the extent to which units within the same group resemble each other. If appropriate, a series of multi-level models will be used as an alternative to the previously described regression analyses.
A series of regression models will be employed to examine potential differentials in service engagement following 988 Suicide & Crisis Lifeline and BHCSC and clinical outcomes based on demographic characteristics. Regression techniques will also be utilized to investigate differences in risk levels based on the number of follow-up contacts and referrals, while a combination of independent t-tests and chi-square analyses will be utilized to examine differences in referral volume, type, and service engagement concerning client and index contact characteristics.
A two-way mixed ANOVA will facilitate understanding of the relationship between care engagement status (e.g., whether an individual has existing behavioral health services) at index contact and overall crisis contact volume throughout the 12-month follow-up period. Overall, these methods will guide the team to delve deeper into clients’ journeys through crisis services, the types of services received, and behavioral health outcomes.
Client-Level Risk Reduction Study
This study will evaluate both the effectiveness of the 988 Suicide & Crisis Lifeline and BHCSC in reducing immediate suicidal, homicidal, and overdose risk, and is intended to provide a comprehensive understanding of client experiences with the service. Power analysis suggests that a sample of 75 participants is needed to detect an effect size of Cohen’s d = 0.5 with a 0.05 significance level and 0.8 power for the contact recordings that answer immediate outcome questions. We will oversample contacts to ensure representation of the presenting concern(s) (e.g., suicide, violence toward others, overdose) and to allow for those that lack sufficient codable data (e.g., hang ups, misdirected calls). Based on work completed by Team Aptive members Dr. Gould and Ms. Lake, we anticipate around 14% of contacts will be from repeat clients (for which only the first contact will be coded), 56% will be direct contacts from clients in crisis, and 14% will be third-party contacts. Approximately half of the direct crisis contacts (27% of all contacts) will be from suicidal clients. The prevalence of contacts from clients at risk of violence toward others or of non-suicidal overdose, and the distribution of presenting concerns on third-party contacts, are currently unknown; obtaining an estimate of the prevalence of these concerns is a goal of the current study. We aim to obtain and screen an initial sample of up to 300 randomly selected contacts from each center (approximately 100 per year per center from 2025 through 2027) to arrive at an eventual codable dataset of 75 contacts per center, with the expectation that approximately 27% will be from suicidal individuals and a smaller proportion will be from those experiencing homicidal thoughts or heightened overdose risk. If needed, additional contacts may be obtained and screened to ensure representation across presenting concerns.
Descriptive statistical methods will be employed to characterize all available recorded contact data, addressing inquiries related to typical outcomes of 988 Suicide & Crisis Lifeline and BHCSC contacts. This includes, for example: (1) the number of contacts obtained and coded, (2) the number and proportion of contacts where elements of suicide, violence/homicide, or overdose risk were observed, (3) the setting of these crises; (4) risk mitigation strategies and interventions employed by crisis counselors; (5) plans for after the contact; (6) whether emergency services are involved, either with or without the client’s consent; (7) behavioral changes from beginning to end of the crisis contact; and (8) client feedback communicated during the crisis contact.
The team will summarize these metrics by contact modality and presenting concern to help understand differences between groups. Additionally, nominal contact outcome data will be analyzed through a series of chi-square tests to explore relationships between presenting concern, risk acuity, risk mitigation strategies, and follow-up actions at the conclusion of contact.
Qualitative data analysis activities will also be conducted, including the development of coding rubrics and codebooks, utilizing qualitative data analysis software (e.g., MAXQDA) for data collected through the C-KII-DCs and C-KII-TPCs. The qualitative data collection team will implement composite scoring and categorization of qualitative data using a conventional inductive coding approach, adhering to established procedures. Mixed methods will be employed, and data from multiple sources will be triangulated to evaluate the C-KII-DCs, C-KII-TPCs, and data from the CES.
Impact Study
The Impact Evaluation Study will examine the impact of 988 Suicide & Crisis Lifeline and BHCSC on the outcomes of suicide and overdose deaths and other adverse crisis-related events. Because the 988 Suicide & Crisis Lifeline and BHCSC has been implemented nationally and a traditional randomized controlled trial is not possible in this context, the study will rely on a quasi-experimental ITS design using extant, secondary data sources (e.g., CDC mortality data, HCUP inpatient hospital discharge data, data from the NSDUH) gathered across multiple years prior to 988 Suicide & Crisis Lifeline and BHCSC implementation up through the most recent available reporting year, to establish longitudinal state-level trends in outcomes before and after 988 Suicide & Crisis Lifeline and BHCSC implementation.
To address the estimation challenge regarding forecasting multiple time-series data, the team will consider various methodologies that offer well-established and adaptable frameworks such as Autoregressive Integrated Moving Average (ARIMA) and its derivatives like Vector Autoregression (VAR), incorporating transformation such as logarithmic conversion when applied to count data (e.g., mortality rates). Additionally, specialized models like the “HHH4” model proposed by Paul & Held (2011) for epidemiological surveillance cater specifically to count data series, whereas machine learning algorithms like Bayesian Additive Regression Trees (BART) can be tailored to suit time-series analysis requirements.
The team will incorporate covariates into these models, examine trend data for seasonality, and adjust if significant, while accounting for changes in measurement instruments. We will use ensemble techniques, as proposed by Raftery et al. (2005), to combine results from individual models for a robust and comprehensive analysis. To ensure the robustness of our approach, we will conduct sensitivity analyses to identify alternative model specifications and perform placebo studies. For placebo checks, we will examine outcomes that should not be impacted by the 988 Suicide & Crisis Lifeline and BHCSC implementation, such as motor vehicle crash-related mortality or morbidity (excluding single-occupancy motor vehicle crash-related mortality).
Additionally, we will perform international comparisons using the synthetic control methodology (Abadie et al., 2015; Abadie & Gardeazabal, 2003; Ben-Michael et al., 2020). This method uses a weighted average of several countries from the pool of 37 OECD members to create a comparison group that maximizes the similarity of the comparison condition to the U.S. mortality outcome trajectory before 2022. This synthetic comparison provides a more informative assessment of the effect of the 988 Suicide & Crisis Lifeline and BHCSC changes compared to using a single country.
For state-level comparisons, we will use machine learning algorithms like regression trees (Breiman et al., 1984) to compute differences in estimated impact based on state-level BHCSC characteristics. These algorithms generate results that are easy to interpret and represent graphically. The information will be used to maximize predictive capabilities and potentially improve future suicide and overdose prevention policy decisions.
17. Display of Expiration Date
All data collection instruments will display the expiration date of OMB approval.
18. Exceptions to the Certification Statement
This collection of information involves no exception to the Certification for Paperwork Reduction Act Submissions.
1 BLS OES May 2022 National Industry-Specific Occupation Employment and Wage Estimates mean hourly salary for General and Operations Managers (code 11-1021), https://www.bls.gov/oes/current/oes111021.htm
2 BLS OES May 2022 National Industry-Specific Occupation Employment and Wage Estimates mean hourly salary for Social and Community Service Managers (code 11-9151), https://www.bls.gov/oes/current/oes119151.htm
3 BLS OES May 2022 National Industry-Specific Occupation Employment and Wage Estimates mean hourly salary for Counselors, Social Workers, and Other Community and Social Service Specialists (code 21-1000), https://www.bls.gov/oes/current/naics5_541720.htm#29-0000
5 This number represents an estimate based on the average distribution of monthly contacts by modality, cited in Lifeline Performance Metrics (SAMHSA, April 2024), and assumes that 40% of all individuals who contact 988 through chat or text (as cited in Gould et al., 2021 and Pisani et al., 2022) and 20% of those who contact 988 through phone call are below the age of 18.
SS
Part A 988
Evaluation
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Zanakos, Sophia |
| File Modified | 0000-00-00 |
| File Created | 2025-08-13 |
© 2025 OMB.report | Privacy Policy