Crisis Counseling Assistance and Training Program Data Toolkit
SUPPORTING STATEMENT B
B. STATISTICAL METHODS
1. Sampling Methods
Ruggiero, K. J., Davidson, T. M., McCauley, J., Gros, K. S., Welsh, K., Price, M., . . . Amstadter, A. B. (2015). Bounce Back Now! Protocol of a population-based randomized controlled trial to examine the efficacy of a web-based intervention with disaster-affected families. Contemporary Clinical Trials, 40, 138–149.
Efforts used to inform this section of the OMB Supporting Statement include the following: (1) a retrospective evaluation of the CCP, (2) a cross-site evaluation of CCP grants funded as a result of the 2005 Gulf Coast hurricanes, and (3) analysis of the data from 2016 to 2021. The first two studies were performed by the National Center for PTSD as an interagency agreement with SAMHSA CMHS.
The retrospective evaluation involved an archival analysis of all available reports for crisis counseling grants implemented in the 50 United States that closed out over a 5-year interval. As such, all crisis counseling projects that closed out between October 1, 1996, and September 30, 2001, and were administered by states rather than territories, were eligible for analysis. Reports from 44 grants covering 28 disaster events were included in the quantitative analysis for the retrospective evaluation.
Given the catastrophic nature of the 2005 Gulf Coast hurricanes, a disaster-specific cross-site evaluation of funded CCPs was supported by SAMHSA CMHS. The cross-site evaluation of the 2005 Gulf Coast hurricane programs consisted of 22 grants in 17 states active between November 2005 and February 2007, thus 3 to 18 months after Hurricane Katrina. These grants utilized the 2005 OMB-approved CCP data collection forms.
The means (or averages) from these studies, combined with a review of the data trends from the past 5 years and consideration of the current CCP grants, were used to inform this section of the OMB Supporting Statement. The average number of respondents from 2016 to 2021 is 385,000 per year for the CCPs, distributed as follows:
Individual/Family Crisis Counseling Services Encounter Log = 284,700
Service providers (i.e., crisis counselors/program staff) will be required to complete this form for all service recipients who access individual or family crisis counseling services of 15 minutes or more.
Group Encounter Log = 24,500
Service providers will be required to complete this form for each group of service recipients who access group crisis counseling services and/or group public education services.
Weekly Tally Sheet = 63,000
Service providers will be required to complete this form for all contacts for whom data are not captured on either the Individual/Family Crisis Counseling Services Encounter Log or Group Encounter Log.
Assessment and Referral Tools = 5,400
These tools (Adult or Child/Youth) are intended for intense users of services, defined as all individuals receiving a third or fifth individual crisis counseling visit, or those experiencing serious reactions to the disaster. This tool will be administered by the service providers during encounters with all eligible service recipients beginning 3 months after the disaster.
Participant Feedback Form = 6,300
These forms will be collected from a sample of service recipients, not every recipient. The use of a time sampling approach is recommended, for example, soliciting participation from all counseling encounters 1 week per quarter.
Service Provider Feedback Form = 1,100
These forms will be administered to all service providers and data will be collected anonymously at least once at 6 months and/or 1 year after the disaster.
Most CCPs will collect encounter information using the mobile app for fast, timely, and reliable data entry into the online system and analysis. The CCP forms can also be collected on paper and then entered directly into the online database or forwarded to a central location for online data entry. The completion of forms is a part of the daily work requirements for service providers and does not interfere with ongoing program operations.
2. Information Collection Procedures
CCP crisis counselors and outreach workers will be responsible for completing the Individual/Family Crisis Counseling Services Encounter Log and the Group Encounter Log during the encounter or immediately after the service recipient(s) has/have completed the encounter session. Service providers will select appropriate response categories to items on the log based on their own perceptions of the service recipient or on information that recipients provide during the encounter. The Weekly Tally Sheet will be completed by the CCP crisis counselor or outreach worker at the end of the designated “week” period (for example, Sunday through Saturday as determined by the CCP and the SAMHSA CMHS project officer).
The Individual/Family Crisis Counseling Services Encounter Log will be completed by the service provider for all individuals who access this service. The Group Encounter Log will be completed by the service provider for all groups that meet for crisis counseling or for public education. The Weekly Tally Sheet will be completed by the service provider for all brief educational or supportive encounters not captured by any other form.
The Assessment and Referral Tools will be administered by CCP service providers and used as a checklist with the service recipient present. The Assessment and Referral Tools are intended to be completed by a trained service provider for all service recipients who access individual crisis counseling for the third or fifth time or express a strong need for intensive services outside of the scope of the CCP. These tools will be read aloud to the recipient by the crisis counselor and the recipient will be asked to respond to questions accordingly.
Both the Service Provider and Participant Feedback Forms will be completed in a location of the respondent’s choosing (for example, home or office) and anonymously. The Service Provider Feedback Form will be administered online and the Participant Feedback Form will be returned by mail or completed online. No individual identifying information will be collected on the forms to assure the anonymity of the respondent. The SAMHSA CMHS project officer along with FEMA staff will provide guidance to the state in determining the most appropriate method for the collection, processing, and sharing of findings from these forms.
3. Methods to Maximize Response Rates
The following logs/tools will be completed by a trained service provider as part of their job requirements:
Individual/Family Crisis Counseling Services Encounter Log
Group Encounter Log
Weekly Tally Sheet
Assessment and Referral Tools (Adult and Child/Youth)
To maximize response rates and compliance with the completion of these tools, all crisis counselors will be trained on the requirements of completing these forms as well as administration protocols. Procedures will be put in place for the oversight of the crisis counselors ensuring that they are monitored for compliance in completing all required forms. As such, our targeted response rates for the encounter logs (that is, Individual/Family Crisis Counseling Services Encounter Log, Group Encounter Log, and Weekly Tally Sheet) and the Assessment and Referral Tools is 100 percent. However, it is acknowledged that a 100 percent completion rate may not always be possible for various reasons (for example, the counselor may fail to complete a form as required, a form may not be available at the time of the encounter, or a form may be lost). Therefore, we estimate there may be a non-completion rate of between 2 to 5 percent yielding a completion rate of approximately 95 percent for these forms.
The Participant Feedback Form will be completed by adult service recipients (for whom individual or group crisis counseling services were provided). The form will be administered at 6 and 12 months after an event. At those times post-event, the program chooses a 1- or 2-week period during which selected sample members are asked to complete the Participant Feedback Form anonymously. The response rate will be calculated by comparing the total number of forms received and completed to the total number of forms that were distributed separately at 6 and 12 months. Methods to increase response rates include: (1) providing a stamped return envelope and (2) providing an accompanying letter from the program director encouraging participation and the need for the information to be returned to help improve the experiences of future service delivery. Furthermore, pre-testing of the form has helped to ensure that the form is easily comprehensible and brief enough to facilitate completion.
A review of previous data collected in the ODCES indicates that the typical response rate for the Participant Feedback Form is approximately 10 percent. We anticipate the same response rate for the current collection request. However, we may have a slight increase in reporting from previous years as we are requiring CCPs to administer this form if a longer ISP grant is requested and at least once during the RSP grant period.
Given this low response rate, it is likely that people who complete the Participant Feedback Form differ in program-important ways from people who do not. When findings are reported, the response rate and the likelihood of bias in the findings will be communicated clearly and aggregate level characteristics of responders as they relate to the program will be provided.
The Service Provider Feedback Form will be made available via an online link or paper distribution to all CCP service providers at 6 months and/or 1 year after the disaster. Response rates will be calculated separately for the 6 month and 1 year collection efforts. Methods to increase response rates among the service providers will include a letter to encourage completion of these forms.
4. Tests of Procedures
No new tests of procedures will be undertaken. Many instruments in the CCP Data Toolkit have been taken from established data collection tools that have already been tested for validity and reliability. In addition, SAMHSA CMHS staff members have had an opportunity to review the revised tools and all agree with the data items. The Assessment and Referral Tools, the Participant Feedback Form, and the Service Provider Feedback Form contain elements of the Short PTSD Rating Interview, or SPRINT, and the SPRINT-E, an expanded version of this form, both of which have been determined in research to be reliable and internally consistent. The Child/Youth Assessment and Referral Tool has items from the UCLA PTSD Reaction Index. Other items on these three forms, as well as on the Individual/Family Crisis Counseling Services Encounter Log and Group Encounter Log, evolved directly from previous studies (for example, retrospective and cross-site evaluation of 2005 Gulf Coast hurricanes), and through site visits, interviews, and focus groups with states, direct service providers, and federal staff. Demographics collected across all forms are considered standard items for collection in the research literature and speak directly to the goals of the CCP.
5. Statistical Consultants
The names and phone numbers of project officers and the consultant are as follows:
Federal Project Officers
Nikki D. Bellamy, Ph.D.
Center for Mental Health Services
Division of Prevention, Traumatic Stress, and Special Programs
Emergency Mental Health and Traumatic Stress Services Branch
5600 Fishers Lane, Room 14E05D
Rockville, MD 20857
Phone: 240–276–2418
Cell: 202–744–9042
Fax: 301–480–8966
Erik Hierholzer
Captain, United States Public Health Service
Substance Abuse and Mental Health Services Administration
Lead Public Health Advisor
Emergency Mental Health and Traumatic Stress Services Branch
Center for Mental Health Services
5600 Fishers Lane, Room 14E01D
Rockville, MD 20857
Phone: 240–276–0408
Cell: 240–499–5859
erik.hierholzer@samhsa.hhs.gov
Statistical Consultants
Abigail Woodroffe, Ph.D.
Principal Research Associate
IMPAQ International, LLC
201 Jones Road, Suite 1
Waltham, MA 02451
Phone: 443–259–5500
Everly Macario, Sc.D., M.S., Ed.M.
Senior Evaluation Research Director
SAMHSA Disaster Technical Assistance Center
c/o IQ Solutions, Inc.
11300 Rockville Pike, Suite 901
Rockville, MD 20852
Phone: 224–244–3965
Elizabeth Gall, MHS
Senior Research Analyst
SAMHSA Disaster Technical Assistance Center
c/o IQ Solutions, Inc.
11300 Rockville Pike, Suite 901
Rockville, MD 20852
Phone: 240–221–4297
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Graham, Carlos (SAMHSA/OA) |
| File Modified | 0000-00-00 |
| File Created | 2022-07-03 |