COMMON DATA PLATFORM
CLIENT/PARTICIPANT DATA COLLECTION
SUPPORTING STATEMENT
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
B1. Respondent Universe and Sampling Methods
On May 15, 2014, SAMHSA and two of OMB’s statisticians held a conference call on sampling vs. and census for SAMHSA’s data collections. Both Agencies agreed that SAMHSA’s data collections for discretionary grant programs which are for monitoring purposes will continue to use census as it has been the normal practice for years. The CDP is designed to collect measures to report on the performance of its discretionary services grant programs and thus will continue to use a census approach.
All SAMHSA grantees are required to collect and report certain data so the Agency can meet its obligations under their funding agreements.
One exception is the Screening, Brief Intervention, and Treatment (SBIRT program). For SBIRT, a 10% sample of clients was selected.
For Screening, Brief Intervention and Referral to Treatment (SBIRT) Grants Only:
Screening Only:
Baseline: For clients who are screened and who, based on the results of the screen, do not require any level of substance abuse intervention or treatment services, only the Record Management needs to be completed. No further information or re-assessment information is required
Screening and Brief Intervention (BI)
Baseline: For clients who are screened, and who, based on the results of the screen, should or do receive BI services, Section A2 (Planned Services) needs to be completed. It is important that all clients complete a tracking information sheet at baseline, in the event they are selected for follow-up.
Follow-Up: For a representative 10% sample of clients in this category, 6-month reassessment data should be submitted.
Follow-Up: For a representative 10% sample of clients in this category, 6-month reassessment data should be submitted. For individuals included in this sample, the appropriate sections should be submitted at follow-up. The instrument indicates the skip pattern. Please note that a 10 percent sample is required for each of the two levels of intervention (BT, RT).
With the exception of SBIRT, SAMHSA programs are often client level/participant interventions that are dramatically different from one group to another and may have insufficient population receiving a specific intervention to justify a sample. Populations from each group may not be similar and would not be appropriate to infer general findings about the successes or failures of a program because of the uniqueness of each group. Within populations, sample sizes may be too small to properly sample, leading to large sample variance and errors in findings about the programs. In these cases, where programs differ from group to group, it is important to gather data sufficient to draw statistically accurate conclusions about how the programs are performing and about what characteristics of the program may matter to the success of the program.
In addition to the GPRA, data collected by grantees will be used to demonstrate how SAMHSA’s grant programs are reducing disparities in access, service use, and achieving outcomes nationwide. To accomplish these purposes, SAMHSA expects grantees to utilize their data to (1) identify subpopulations (i.e., racial, ethnic, sexual/gender minority groups) vulnerable to health disparities and (2) implement strategies to decrease the differences in access, service use, and outcomes among those subpopulations. There will be subpopulations where sample size is too small to properly sample and where a census would be necessary obtain relevant and reliable outcome data.
All SAMHSA-funded discretionary grantees that provide direct services or engage in infrastructure development, prevention, or behavioral health promotion activities are required to participate in this data collection effort and will submit data based on their conducted activities.
Information collection procedures will vary by type of program. The client outcome measures for most providers will be extracted from previously established databases. Intake/baseline information is obtained by intake workers and/or counselors. For clients still in treatment 6 months later, the information will be obtained in the same way. In instances where clients are no longer in direct contact with the service provider, staff from the program will locate the clients and conduct the follow-up interviews. These interviews are to be conducted face-to-face.
Some programs collect their client information using paper and pencil methods. This project will not interfere with ongoing program operations. Programs will submit their data electronically via a web-based data entry process or upload process. The data for those clients with baseline, discharge, and follow-up data are matched using a unique encrypted client identifier.
Information data collection procedures will be the responsibility of individual grantees and may vary by type of program.
Some grantees have service providers conduct client-level baseline and follow-up assessments, while others have grant evaluators perform this function.
Some grantees may wish to collect client-level information using paper and pencil methods. SAMHSA will provide downloadable paper versions of the data collection instruments to facilitate this process. These grantees will then submit their data electronically via a web-based data entry process. The data for clients with both baseline and periodic reassessment data are matched using a unique encrypted client identifier developed by the grantee. Grantees will be clearly instructed not to use identifying information (i.e., social security number or initials) as the consumer identifier.
Required data collection points are:
BASELINE: For clients who have not previously been served by program, or who are returning to services following a discharge from the program, baseline data will be collected. For clients already enrolled in the program and continuing to receive services, administrative data should be submitted by the grantee within 30 days of initiating data collection. The timing of any subsequent data collection point(s) will be anchored to the baseline point the grantee indicates in this administrative record.
REASSESSMENT: SAMHSA requires client-level data collection every six months while the consumer is receiving SAMHSA-funded services. Ongoing periodic status review is viewed as consistent with good clinical practice.
DISCHARGE: Grantees must provide information on the type of discharge on all clients who are discharged. When the discharge is a planned event, the client will also be asked the questions on the client-level data collection instrument. The one exception to this requirement is when a client had responded to these same questions within the past 30 days as part of a Reassessment.
Each grantee also has its own plan for data collection, processing, data cleaning, control, and retention. Each plan describes how uniform data collection will be ensured, the time frame for conducting the assessments over the course of the project, and how participant protection will be assured. As mentioned above, these plans undergo peer review to ensure the adequacy and appropriateness of the study design and methods. The precise manner in which data will be collected and used depends on the specific grant program.
The Minority Aids Initiative (MAI) develops local capacity to provide substance abuse prevention services for individuals living with and affected by HIV/AIDS. These funds assist with outreach and training, addressing the special needs of racial and ethnic minorities, and studying the costs associated with delivering integrated care.
B3. Methods to Maximize Response Rates
Each grantee will establish its own client follow-up procedures as part of the original protocol. At the time of intake, information is typically obtained from clients to assist with locating them later. In addition, some providers are adept at using other community resources to assist with locating clients. Clients are typically cooperative with provider staff because of the relationship established during treatment. Since all participating grant programs propose a census at initial intake, considerable options also exist for non-respondent analysis and associated adjustments to the data such as weighting.
Follow-up has been a challenge to some grantees given the remote locations that they serve and the challenge of locating clients as far out as 6 months. For grantees that have not been aware of the strategies they can employ to begin the follow-up process at intake, how to maintain contact with clients, and the importance of good locator forms, several strategies have been implemented to assist the grantees with follow-up. First, follow-up training is offered which assists grantees in learning about and conducting follow-up at their sites. This program is offered to all grantees and after the grantees are trained through the grantee orientation process, monthly follow-up trainings are offered for those that need additional training or for new project staff. Individual grantee technical assistance is also available for sites that need additional follow-up instruction. These group and individual trainings are conducted by follow-up experts. Each grantee receives a follow-up tracking manual at these trainings that may be used as a future reference. A second strategy provides the grantees with data status reports on how close they are to meeting their follow-up goals. These reports are available from the web-based system to the grantees and Government Project Officers for the grants they are responsible. A third strategy is the automatic, system generated notice of when follow-up interviews are due for each client/participant. A fourth strategy provides technical assistance at national meetings. Experts, including grantees, have been identified and asked to make presentations at national grantee meetings on how to conduct follow-up. These sessions are well attended by grantees.
SAMHSA will maintain a contract to provide technical assistance for grantees and Government Project Officers (GPOs) on the collection, management, and utilization of data. Specifically, the objectives include:
Surveying grantees to determine current training requirements;
Maintaining current records of all SAMHSA data collection requirements;
Providing training to grantees related to the collection, management, analysis, and utilization of data; and
Providing technical assistance to grantees regarding data in manner that supports sustainable practices Researching and disseminating information related to data collection, management, and utilization.
This contract will support the Agency’s ability to ensure conformance to program specifications.
Issues related to response rates, as well as other data collection issues, are discussed at grantee meetings in order for GPOs to identify problems and provide technical assistance. In addition, GPOs monitor data collection efforts and provide technical assistance to individual grantees as necessary. Because collection of outcome measures is a stipulation of the grants, it is anticipated that all grantees will comply (as appropriate). The participants at each site to whom these measures will be administered are all voluntary respondents; therefore, grantees cannot guarantee full cooperation on the part of participants. As part of the terms and conditions of the grant award, sites are required to use the outcome data system. Historically, however, participant response rates across grantee sites have averaged 80%.
B4. Test of Procedures
Most of the data elements in the data sets have been taken from established data collection instruments that have a long history of use in the substance abuse field and have already been tested for validity and reliability, (i.e., ASI).
Feedback from the grantees also indicates that they routinely collect the same information requested of this data collection tool and some have integrated this tool into other tools that they routinely use to gather information. Some grantees report that they collect information in greater detail, (i.e., more response alternatives), but these are collapsed into standard categories.
Data elements include three client-level domains (Employment/Education, Crime and Criminal Justice, and Stability in Housing) and one system-level domain (Access/Capacity), which depends on common demographics collected on consumers. The content of these questions was appropriate for use, but additional value options were defined to reflect issues specific to the populations served by SAMHSA. The benefits of using these measures include a history of use in monitoring the performance of CSAT grantees, the ability to conduct cross-Center comparisons, and use of measures previously approved by OMB.
The infrastructure, prevention, and mental health promotion data elements are drawn from these grant’s existing performance indicators and modified to allow consistent reporting for SAMHSA. A pilot of nine grant Project Directors was conducted using the attached instrument; results indicated these data are already part of routine data collection for most of the pilot participants or are consistent with their funded activities.
B5. Statistical Consultants
The measures submitted here for OMB approval renewal are a result of lengthy consultation and discussion among SAMHSA personnel, based on previous discussions with grantees, administrators, and expert panels. Furthermore, these measures have been used by members of the grantee community, and at various meetings and conferences. The final selection of these measures was made by SAMHSA senior officials. The following SAMHSA staff were involved in the discussions:
Melanie Brown, M.P.H.
Public Health Advisor
SAMHSA/Center for Mental Health Services
1 Choke Cherry Rd. Room 6-1003
Rockville, MD 20850
(240) 276-1909
Christine J. Hager, Ph.D.
Social Science Analyst
SAMHSA/Center for Behavioral Health Statistics and Quality
1 Choke Cherry Rd. Room 2-1011
Rockville, MD 20850
(240) 276-1287
Christine.Hager@samhsa.hhs.gov
Mark Jacobsen. LCSW
Public Health Advisor
SAMHSA/Center for Mental Health Services
1 Choke Cherry Rd. Room 6-1065
Rockville, MD 20850
(240) 276-1826
Sarah Ndiangui, M.P.H.
Public Health Advisor
SAMHSA/Center for Substance Abuse Treatment
1 Choke Cherry Rd. Room 5-1062
Rockville, MD 20850
(240) 276-2918
Kathryn Wetherby
Public Health Advisor
SAMHSA/Center for Substance Abuse Treatment
1 Choke Cherry Rd., Room 5-1006
Rockville, MD 20850
Kathryn.Wetherby@samhsa.hhs.gov
LIST OF ATTACHMENTS
CMHS Client-Level Data Collection Instrument
CMHS Program-Specific Data Collection Instrument for PBHCI Program
CMHS Grantee-Level IPP Indicator Instrument for Select Programs
CSAP Data Collection Instrument for Youth
CSAP Data Collection Instrument for Adults
CSAT Client-Level Data Collection Instrument
CSAT Grantee-Level Aggregate Instrument for ATCC Program
Program-Specific Measures for the HIV CoC Program
Draft Reference Manual for CMHS/CSAT Grantees
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Department of Health and Human Services |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |