SAMHSA Multiplier Survey
Supporting Statement
The Substance Abuse and Mental Health Services Administration (SAMHSA) of the U.S. Department of Health and Human Services has a long history of providing funds through grants and cooperative agreements to States and other entities to develop innovative programs to address a wide variety of needs. Through its three centers, Center for Mental Health Services (CMHS), Center for Substance Abuse Prevention (CSAP) and Center for Substance Abuse Treatment (CSAT), SAMHSA has sought to provide seed funding for programs that support its vision of a life in the community for everyone. Four programs from each of the three Centers will be selected to participate in the surveys (see B. 1 for sampling plan). SAMHSA is requesting the Office of Management and Budget’s approval for a survey for project directors to be used in this project (Attachment A).
Each SAMHSA grantee reports performance data to SAMHSA as part of their grant; however, little is known about the extent to which they are sustained after Federal funding ends. The Multiplier Survey is conceived as a pilot test of program management methodology and whether a retrospective design is the most effective way of understanding the development of program sustainability strategies after Federal funding ends.
The Multiplier Survey proposes to address this lack of information by identifying program factors, grantee characteristics and contextual factors that support sustainability and effective grant implementation by conducting one-time interviews with project directors from twelve programs selected from the three SAMHSA Centers. Section 501(d)(4) of the Public Health Service Act (42 USC 290aa) and Public Law 106-107 are the authorizing legislation for these surveys.
Kinds of Information to Be Collected
By understanding characteristics and factors that improve grant implementation and enhance sustainability at the end of Federal funding, SAMHSA will be better able to structure these programs to enhance the effectiveness of the “seed” money provided to states and other entities. Findings from this survey will inform the structure and implementation of future SAMHSA funding opportunities to address its specific goals and objectives. These include grant implementation, intended program outcomes at the conclusion of Federal funding, such as capacity improvement and system change, and program sustainability and influence on other programs or on systems at large. In this way, SAMHSA can provide funding opportunities for programs that support its vision of a life in the community for everyone. This survey may be the basis for a grant management and evaluation tool to be used by Federal project officers overseeing SAMHSA grant programs.
Project Director Interview (Attachment A): Project Directors from each of the twelve selected programs (see B. 1) will be asked to participate in a telephone interview about their project. The survey consists of 35 open-ended questions on topics including project intent and design, implementation, and sustainability. Using open-ended questions permits more in-depth responses and probing to identify important factors in sustainability.
Government Performance and Results Act (GPRA)
Project directors will be asked if the GPRA data they collected as part of their grant informed any modifications to the project and/or increased its sustainability or continued role in the community. Findings from these surveys may inform modifications to the types of performance measures collected and/or the method and timing of their distribution to SAMHSA grantees.
Technology will not be used for the Project Director interview, due to its qualitative nature, it was decided that an in-depth telephone interview with open-ended questions would permit a more complete understanding of pertinent factors of program influence and sustainability.
The information is collected only for the purposes of this project and is not available elsewhere.
While some of the individual grant projects may be small, not-for-profit entities, this one-time information collection effort will not have any significant impact on them. In addition, participation of project directors is voluntary. Project directors will have the opportunity to refuse to participate in the telephone interview. (see section A. 10).
Each respondent is asked to respond only once.
This information collection fully complies with 5 CFR 1320.5(d)(2).
The notice required in 5 CFR 1320.8(d) was published in the Federal Register on Friday, April 2, 2010, Volume 75, page 16812. No comments were received.
While not outside SAMHSA, a Workgroup, representing CMHS, CSAP and CSAT was created to assist in protocol development, identification of grant programs that met inclusion criteria, sampling plan and identification of appropriate project directors for in-depth interviews. It was felt that those within SAMHSA who had intimate knowledge of grant programs in the project would be better able to assist in the development of surveys and procedures to address the government’s concerns. A list of their names and contact information is in Attachment B.
In addition, contractor and subcontractor staff were consulted in the development of the protocols, including the following:
Howard Goldman and Garrett Moran, Senior Researchers: Westat, Rockville, MD;
Chanson Noether, Division Manager: Policy Research Associates, Delmar, NY.
Respondents will not receive any compensation for their participation in these surveys.
No personal identifying information is being collected. Names and contact information of project directors being interviewed will only be used to contact them. The contractor’s Human Subjects Coordinator and Institutional Review Board chair met and concluded that this project meets the criteria for exempt from the need for IRB review (45 CFR 46.1010(b)(5)(i)). A Certificate of Confidentiality is not required. Data will be stored in locked cabinets.
This information collection contains no questions of a sensitive nature, being focused on program implementation and sustainability.
The annualized hour burden is summarized in the table below:
Form Name |
No. of Res-pondents |
Responses per Res-pondent |
Total Responses |
Hours per Response |
Total Hour Burden |
Hourly Wage Cost |
Total Hour Cost ($) |
Project Director |
48 |
1 |
48 |
1.25 |
60 |
$70 |
$4200 |
TOTAL |
48 |
- |
48 |
- |
60 |
$70 |
$4200 |
The hour burden estimate was calculated, including the number of responses and hours per response, based on the contractor’s experience with this type of data collection and on informal testing with three Policy Research staff for each form. The hourly wage cost was calculated using a $70,000 estimate of project directors’ annual salary across the country. The loaded hourly wage for that salary is approximately $70.
There is no capital/startup or operation and maintenance cost involved in collecting the information.
The total estimated cost to the Federal Government for the Multiplier Surveys data collection activity is $188,455. This includes the cost of the survey development, implementation and analysis ($98,038) plus 5% of a SAMHSA project officer’s time at $100,000 annual salary ($5,000) for the year of the contract.
This is a new project.
Time Schedule
Once OMB clearance has been obtained, interviews will be scheduled with selected project directors from the grantee programs. (See B.1 for details on the sampling plan.) Project directors from all grantees will be invited via email to complete the web-based survey.
Activity |
Date |
Indentify project directors |
During OMB clearance period |
Submit final interview schedule to GPO |
Within 5 weeks of OMB approval |
Interviews completed |
Within 13 weeks of OMB approval |
Analyze data and distill policy recommendations |
Within 17weeks of OMB approval |
Submit draft final report |
Within 17 weeks of OMB approval |
Final report, incorporating GPO comments |
Within 22 weeks of OMB approval |
Analysis Plan
SAMHSA will use the survey data to obtain information on the following key issues, which, in turn, will inform its development and management of future programs within its three Centers:
Project Intent and Design
The impetus, intended population and champion for the project
Use of Evidence-based practices
Type of program, i.e. service enhancement or infrastructure or both
Project Implementation
Challenges encountered, modifications made in implementing program
Use of a planning period, if one was included
Use of technical assistance available and received
Systemic context of the project
Project Sustainability
Continuation after Federal funding ended, included how it was accomplished
Identification of significant contributing factors to the project’s continuation, if it did
Identification of significant contributing factors to the project’s discontinuation, if it did not continue
Type and use of any evaluation conducted and GPRA data collected
The in-depth survey is designed to yield qualitative data. Each will be analyzed by looking for recurring themes and factors across multiple programs.
Publication
Findings will be summarized in an internal report that will address three assessment areas:
Grant program factors associated with successful continuation and sustainability after Federal funding ends, including, but not limited to grant amount, type, and award length.
Grantee characteristics including, but not limited to baseline capacity, rate of spend down, carryover, and leadership
Contextual factors including state economic and demographic situation, infrastructure, level of oversight, involvement of multiple stakeholders
The internal final report will distill key elements and grantee characteristics that may be associated with long-term sustainability and relationship to other systems of service delivery within a community and/or State. It will also make recommendations on methodologies for effective grant management by Federal project officers and by project directors. Contract staff will make a presentation to SAMHSA staff of their findings at SAMHSA’s Rockville, MD, office at the conclusion of the project.
The expiration date will be displayed.
This submission describing data collection requests no exceptions to the Certificate for Paperwork Reduction Act (5 cfr 1320.9).
The Workgroup, Government Project Officer and Policy Research Associates, the contractor conducting this survey, chose the following criteria for SAMHSA grant programs to be included:
All funding must have been expended by September 30, 2008. This included any no-cost extensions.
Two main types of SAMHSA programs were chosen by the Workgroup as being most representative of the types of grants that SAMHSA funds. Infrastructure grants are designed to promote systems change as opposed to improving and delivering service. Service Grants traditionally focus on either improving the service delivery system and/or expanding the population(s) served. Workgroup representatives from each of he three centers (CMHS, CSAP, and CSAT) met with Center staff and identified one program from their Center that met each of the bulleted criteria below and the funding expenditure criterion described in the first bullet.
one infrastructure program with less than three years of funding
one infrastructure program with more than three years of funding
one services enhancement program with less than three years of funding
one services enhancement program with more than three years of funding
These twelve programs constitute the respondent universe for these surveysinclude
CMHS: Alternatives to Restraint & Seclusion State Incentive Grants; Child Mental Health Initiative; Older Adult Targeted Capacity Expansion; Targeted Capacity Expansion Grants for Jail Diversion
CSAP: Drug Free Community; Methamphetamine, Ecstasy, Inhalants and/or Other Club Drugs; Minority SA & HIV Prevention; State Incentive Grants
CSAT: Access to Recovery; Recovery Community Services Program; Screening, Brief Intervention, Referral & Treatment; Strengthening Treatment Access & Retention
Project Director Interview Sampling: All programs have multiple grantees. Four grantees from each program will be chosen for the in-depth telephone interview. For this pilot testing of this methodology, the sampling frame will be based on State’s funding per 100,000 persons, including SAMHSA block grant and discretionary funding, for mental health services for those programs from the Center for Mental Health Services. State’s total funding for substance use will be used to select grantees from the Center for Substance Abuse Prevention and Center for Substance Abuse Treatment. (See Attachment B for funding summaries for mental health and substance use SAMHSA and state funding. Data are from FY2006, the most recent available.) The funding summaries were divided into thirds. Using SAMHSA’s announcement of grantees for each included, if possible, two grantees in the top third of funding and two in the lower third will be selected to be interviewed. If no grantee fell into the top or bottom third, two closest to the top and bottom third will be selected to be interviewed In addition, where possible, the sampling will be purposeful to ensure geographic dispersion, and every effort will be made to select grantees from different states within the upper and lower thirds. It may not be possible to contact some project directors. In that case, another project director will be chosen from the sample. An introductory email will be sent to selected project directors inviting them to participate in the survey. The voluntary nature of their participation will be stressed. A ninety percent response rate is anticipated. (See Attachment A for the Project Director protocol and the introductory email to be sent to each project director.)
This is a one-time data collection to assist SAMHSA to better understand the extent to which their grant programs continue to have a role in their communities after Federal funding ends and factors that enhance project sustainability.
Project Director Survey: At least three attempts will be made to contact each project director selected for the study. If it is determined that the project director is no longer able or unwilling for whatever reason to participate in the study, the project director from another grantee will be selected. The SAMSHA Workgroup established to assist with this study will provide contact information for project directors of selected grantees. Because the numbers of project directors to be interviewed is relatively small, it is imperative that 48 of them be interviewed for the data to have the requisite validity and reliability.
The project director survey wasdeveloped to respond to the specific needs of this study. Workgroup members had extensive input in the development of the protocol and in the study design. Multiple iterations of the survey instrument was reviewed by the Workgroup.
Once a final draft was completed, it was reviewed by Policy Research staff, specifically Chanson Noether and Henry Steadman, and by Howard Goldman and Garret Moran of Westat.
Policy Research Associates has responsibility for instrument development and data collection for this project. They have been assisted by two consultants from Westat. All are listed below.
Susan Becker (Government Project Officer)
SAMHSA/OA
Phone: 240-276-2249
Contractors:
Henry J. Steadman (Project Co-Director)
Deborah Dennis (Project Co-Director)
Margaret Lassiter (Project Coordinator)
Policy Research Associates
Delmar, NY
Phone: 518-439-7415
Consultants:
Garrett E. Moran
Westat
Rockville, MD 20850
Phone: 301-294-3821
Howard Goldman
University of Maryland
Baltimore, MD
Phone: 301-983-1671
Attachment A:
Project Director survey
Introductory email
Attachment B:
Workgroup roster
Sampling Funding Summary
| File Type | application/msword |
| File Modified | 2010-11-29 |
| File Created | 2010-11-29 |