Crisis Counseling Assistance and Training Program Data Toolkit
SUPPORTING STATEMENT – Part B
B. STATISTICAL METHODS
1. Sampling Methods
Ruggiero, K.J., Davidson, T.M., McCauley, J., Gros, K.S., Welsh, K., Price, M., . . . Amstadter, A.B. (2015). Bounce Back Now! Protocol of a population-based randomized controlled trial to examine the efficacy of a web-based intervention with disaster-affected families. Contemporary Clinical Trials, 40,138–149.
Efforts used to inform this section of the OMB Supporting Statement include the following: (1) a retrospective evaluation of the CCP, 2) a cross-site evaluation of CCP grants funded as a result of the 2005 Gulf Coast hurricanes, and (3) analysis of the data from 2009 to 2011. The first two studies were performed by the National Center for PTSD as an interagency agreement with SAMHSA CMHS.
The retrospective evaluation involved an archival analysis of all available reports for crisis counseling grants implemented in the 50 United States that closed out over a 5-year interval. As such, all crisis counseling projects that closed out between October 1, 1996, and September 30, 2001, and were administered by states rather than territories, were eligible for analysis. Reports from 44 grants covering 28 disaster events were included in the quantitative analysis for the retrospective evaluation.
Given the catastrophic nature of the 2005 Gulf Coast hurricanes, a disaster-specific cross-site evaluation of funded CCPs was supported by SAMHSA CMHS. The cross-site evaluation of the 2005 Gulf Coast hurricane programs consisted of 22 grants in 17 states active between November 2005 and February 2007, thus 3 to 18 months after Hurricane Katrina. These grants utilized the 2005 OMB-approved CCP data collection forms.
The means (or averages) from these studies, combined with a review of the data trends from the past 2 years and consideration of the current CCP grants, were used to inform this section of the OMB Supporting Statement. The average number of respondents from 2016-2017 is 145,200 per year for the CCPs, distributed as follows:
Individual/Family Crisis Counseling Services Encounter Log = 117,200
Service providers (i.e., crisis counselors/program staff) will be required to complete this form for all service recipients who access individual or family crisis counseling services of 15 minutes or more.
Group Encounter Log = 15,000
Service providers will be required to complete this form for each group of service recipients who access group crisis counseling services and/or group public education services.
Weekly Tally Sheet = 11,100
Service providers will be required to complete this form for all contacts for whom data are not captured on either the Individual/Family Crisis Counseling Services or Group Encounter Logs.
Assessment and Referral Tools = 400
These tools (Adult or Child/Youth) are intended for intense users of services, defined as all individuals receiving a third or fifth individual crisis counseling visit, or those experiencing serious reactions to the disaster. This tool will be administered by the services providers during encounters with all eligible service recipients during beginning 3 months after the disaster.
Participant Feedback Form = 1,200
These forms will be collected from a sample of service recipients, not every recipient. The use of a time sampling approach is recommended, e.g., soliciting participation from all counseling encounters 1 week per quarter.
Service Provider Feedback Form = 300
These forms will be administered to all service providers, and data will be collected anonymously at least once at 6 months and/or 1 year after the disaster.
Most CCPs will collect client information using mobile app for fast, timely and reliable data entry into the online system and analysis, the CCP forms can also be collected on paper and then forwarded for central location for online data entry. The completion of forms is a part of the daily work requirements for service providers and does not interfere with ongoing program operations.
2. Information Collection Procedures
CCP crisis counselors and outreach workers will be responsible for completing the Individual/Family Crisis Counseling Services Encounter Log and the Group Encounter Log during the encounter or after the service recipient(s) has/have left the encounter location. Service providers will select appropriate response categories to items on the log based on their own perceptions of the service recipient or on information that recipients provide during the encounter. The Weekly Tally Sheet will be completed by the CCP crisis counselor or outreach worker at the end of the designated “week” period (for example, Sunday through Saturday as determined by the CCP and the SAMHSA CMHS project officer).
The Individual/Family Crisis Counseling Services Encounter Log will be completed by the service provider for all individuals who access this service. The Group Encounter Log will be completed by the service provider for all groups that meet for crisis counseling or for public education. The Weekly Tally Sheet will be completed by the service provider for all brief educational or supportive encounters not captured by any other form.
The Assessment and Referral Tools will be administered by CCP service providers and used as a checklist with the service recipient present. The Assessment and Referral Tools are intended to be completed by a trained service provider for all service recipients who access individual crisis counseling for the third or fifth time or express a strong need for intensive services. These tools will be read aloud to the recipient by the crisis counselor, and the recipient will be asked to respond to questions accordingly.
Both the Service Provider and Participant Feedback Forms will be completed in a location of the respondent’s choosing (for example, home or office) and anonymously. The Service Provider Feedback Form will be administered online, and the Participant Feedback Form will be returned by mail. No individual identifying information will be collected on the forms to assure the anonymity of the respondent. The SAMHSA CMHS project officer along with FEMA staff will provide guidance to the state in determining the most appropriate method for the collection, processing, and sharing of findings from these forms.
3. Methods to Maximize Response Rates
The following logs/tools will be completed by a trained service provider as part of their job requirements:
Individual/Family Crisis Counseling Services Encounter Log
Group Encounter Log
Weekly Tally Sheet
Assessment and Referral Tools (Adult and Child/Youth)
To maximize response rates and compliance with the completion of these tools, all crisis counselors will be trained on the requirements of completing these forms as well as administration protocols. Procedures will be put in place for the oversight of the crisis counselors ensuring that they are monitored for compliance in completing all required forms. As such, our targeted response rates for the encounter logs (that is, Individual/Family Crisis Counseling Services Encounter Log, Group Encounter Log, and Weekly Tally Sheet) and the Assessment and Referral Tools is 100 percent. However, it is acknowledged that a 100 percent completion rate may not always be possible for various reasons (for example, the counselor may fail to complete a form as required, a form may not be available at the time of the encounter, or a form may be lost. Therefore, we estimate there may be a non-completion rate of between 2 to 5 percent yielding a completion rate of approximately 95 percent these forms.
The Participant Feedback Form will be completed by of adult service recipients (for whom individual or group crisis counseling services were provided). The form will be administered at 6 and 12 months after an event. At 6 and 12 months after the event, the program chooses a 1- or 2-week period during which selected sample members are asked to complete the Participant Feedback Form anonymously. The response rate will be calculated by comparing the total number of forms received and completed to the total number of forms that were distributed separately at 6 and 12 months. Methods to increase response rates include: (1) providing a stamped return envelope; and, (2) providing an accompanying letter from the program director encouraging participation and the need for the information to be returned to help improve the experiences of future service delivery. Furthermore, pre-testing of the form has helped to ensure that form is easily comprehensible and brief enough to facilitate completion.
A review of previous data collected in the ODCES indicates that the typical response rate for the Participant Feedback Form is approximately 10 percent. We anticipate the same response rate for the current collection request. However, we may have a slight increase in reporting from previous years, as we are requiring CCPs to administer this form if a longer ISP grant is requested and at least once during the RSP grant period.
Given this low response rate, it is likely that people who complete the Participant Feedback Form differ in program-important ways from people who do not. When findings are reported, the response rate and the likelihood of bias in the findings will be communicated clearly, and aggregate level characteristics of responders as they relate to the program will be provided.
The Service Provider Feedback Form will be made available via an online link or paper distribution to all CCP service providers at 6 months and/or 1 year after the disaster. Response rates will be calculated separately for the 6 month and 1 year collection efforts. Methods to increase response rates among the service providers will include: (a) a letter to encourage completion of these forms.
4. Tests of Procedures
No new tests of procedures will be undertaken. Many instruments in the CCP Data Toolkit have been taken from established data collection tools that have already been tested for validity and reliability. In addition, SAMHSA CMHS staff members have had an opportunity to review the revised tools and all agree with the data items. The Assessment and Referral Tools, the Participant Feedback Form, and the Service Provider Feedback Form contain elements of the Short PTSD Rating Interview, or SPRINT, and the SPRINT-E, an expanded version of this form, both of which have been determined in research to be reliable and internally consistent. The Child/Youth Assessment and Referral Tool has items from the UCLA Posttraumatic Stress Disorder Reaction Index. Other items on these three forms, as well as on the Individual/Family Crisis Counseling Services Encounter Log and Group Encounter Log Forms, evolved directly from previous studies (for example, retrospective and cross-site evaluation of 2005 Gulf Coast hurricanes), and through site visits, interviews, and focus groups with states, direct service providers, and federal staff. Demographics collected across all forms are considered standard items for collection in the research literature and speak directly to the goals of the CCP.
5. Statistical Consultants
The names and phone numbers of project officers and the consultant are as follows:
Federal Project Officers
Nikki D. Bellamy, Ph.D., M.A.
Center for Mental Health Services
Division of Prevention, Traumatic Stress, and Special Programs
Emergency Mental Health and Traumatic Stress Services Branch
5600 Fishers Lane, Room 14E05D
Rockville, MD 20857
Phone: 240-276-2418
Cell: 202-744-9042
Fax: 301-480-8966
nikki.bellamy@samhsa.hhs.gov
Erik Hierholzer
Captain, United States Public Health Service
Substance Abuse and Mental Health Services Administration
Lead Public Health Advisor
Emergency Mental Health and Traumatic Stress Services Branch
Center for Mental Health Services
5600 Fishers Lane, Room 14E01D
Rockville, MD 20857
Phone: 240-276-0408
Cell: 240-499-5859
erik.hierholzer@samhsa.hhs.gov
Statistical Consultants
Lamyaa Yousif, Ph.D., M.Sc., M.B.Ch.B.
Lead Research Analyst
SAMHSA Disaster Technical Assistance Center
c/o IQ Solutions, Inc. |
11300 Rockville Pike, Suite 901
Rockville, MD 20852
240.221.4346 Voice
LYousif@IQSolutions.com
Everly Macario, Sc.D., M.S., Ed.M.
Senior Evaluation Research Director
SAMHSA Disaster Technical Assistance Center
c/o IQ Solutions, Inc. |
11300 Rockville Pike, Suite 901
Rockville, MD 20852
Phone: 224-244-3965
EMacario@IQSolutions.com
Mary Kay Dugan, M.A.
Managing Director and Principal Research Scientist
IMPAQ International, LLC
400 North 34th Street, Suite 320
Seattle, WA 98103
Phone: 206-939-4970
ATTACHMENTS
Individual/Family Crisis Counseling Services Encounter Log
Group Encounter Log
Weekly Tally Sheet
Assessment and Referral Tool (Adult)
Assessment and Referral Tool (Child/Youth)
Participant Feedback Form (REVISED FROM 2015 EXPIRATION FORM)
Service Provider Feedback Form (REVISED FROM 2015 EXPIRATION FORM)
Appendix A:
References
General CCP, Encounter Logs, and Feedback Surveys
Bellamy, N. D., Wang, M. Q., McGee, L. A., Liu, J. S., & Robinson, M. E. (2018). Crisis-counselor perceptions of job training, stress, and satisfaction during disaster recovery. Psychological Trauma: Theory, Research, Practice, and Policy. Advance online publication. http://dx.doi.org/10.1037/tra0000338
Brannen, D. E., Barcus, R., McDonnell, M. A., Price, A., Alsept, C., & Caudill, K. (2013). Mental health triage tools for medically cleared disaster survivors: An evaluation by MRC volunteers and public health workers. Disaster Medicine and Public Health Preparedness, 7, 20–28.
Brown, L. M., Framingham, J. L., Frahm, K. A., & Wolf, L. D. (2015). Crisis counselors’ perceptions and assessment of suicidal behavior among hurricane survivors receiving crisis counseling services. Disaster Medicine and Public Health Preparedness, 9(3), 291–300.
Cohen, J. A., Kelleher, K. J., & Mannarino, A. P. (2008). Identifying, treating, and referring traumatized children: The role of pediatric providers. Archives of Pediatrics and Adolescent Medicine, 162, 447–452. Retrieved from http://archpedi.jamanetwork.com/journal.aspx
Commers, M. J., Morival, M., & Devries, M. W. (2014). Toward best-practice post-disaster mental health promotion for children: Sri Lanka. Health Promotion International, 29, 165–170.
Connor, K., & Davidson, J. (2001). SPRINT: A brief global assessment of post-traumatic stress disorder. International Clinical Psychopharmacology, 16, 279–284.
Hamblen, J. L., Norris, F., Pietruszkiewicz, S., Gibson, L. E., Naturale, A., & Louis, C. (2009). Cognitive behavioral therapy for postdisaster distress: A community based treatment program for survivors of Hurricane Katrina. Administration and Policy in Mental Health and Mental Health Services Research, 36, 206–214.
Hamblen, J. L., Norris, F. H., Symon, K. A., & Bow, T. E. (2017). Cognitive behavioral therapy for postdisaster distress: A promising transdiagnostic approach to treating disaster survivors. Psychological Trauma: Theory, Research, Practice, and Policy, 9(S1), 130.
Jones, K., Allen, M., Norris, F., & Miller, C. (2009). Piloting a new model of crisis counseling: Specialized crisis counseling services in Mississippi after Hurricane Katrina. Administration and Policy in Mental Health and Mental Health Services Research, 36, 195–205.
Kronenberg, M. E., Hansel, T., Brennan, A. M. Osofsky, H. J., Osofsky, J. D., & Lawrason, B. (2010). Children of Katrina: Lessons learned about postdisaster symptoms and recovery patterns. Child Development, 81(4), 1241–1259. https://doi.org/10.1111/j.1467-8624.2010.01465.x
Navarro, J., Pulido, R., Berger, C., Arteaga, M., Osofsky, H. J., Martinez, M., . . . Hansel, T. C. (2014). Children’s disaster experiences and psychological symptoms: An international comparison between the Chilean earthquake and tsunami and Hurricane Katrina. International Social Work, 1, 14.
Norris, F.H., & Bellamy, N.D. (2009). Evaluation of a national effort to reach Hurricane Katrina survivors and evacuees: The crisis counseling assistance and training program. Administration in Mental Health & Mental Health Services Research 36(3), 165-175. doi:10.1007/s10488-009-0217-1
Norris, F., Donahue, S., Felton, C., Watson, P., Hamblen, J., & Marshall, R. (2006). A psychometric analysis of Project Liberty’s Adult Enhanced Services Referral Tool. Psychiatric Services, 57, 1328–1334.
Norris, F., Hamblen, J., Brown, L., & Schinka, J. (2008). Validation of the Short Post-Traumatic Stress Disorder Rating Interview (Expanded Version, Sprint-E) as a measure of postdisaster distress and treatment need. American Journal of Disaster Medicine, 3, 201–212.
Pekevski, J. (2013). First responders and Psychological First Aid. Journal of Emergency Management, 11, 39–48.
Riise, K. S., Hansel, T. C., Steinberg, A., Landis, R. W., Gilkey, S., Brymer, M. . . . . Speier, A. H. (2009). The Louisiana Specialized Crisis Counseling Services (SCCS): Final program evaluation. Unpublished manuscript.
Scheeringa, M., & Haslett, N. (2010). The reliability and criterion validity of the diagnostic infant and preschool assessment: A new diagnostic instrument for young children. Child Psychiatry and Human Development, 41, 299–312. https://doi.org/10.1007/s10578-009-0169-2
Steinberg, A. M., Brymer, M. J., Decker, K. B., & Pynoos, R. S. (2004). The University of California at Los Angeles Post-traumatic Stress Disorder Reaction Index: Child and adolescent disorders. Current Psychiatry Reports, 6, 96–100.
Uekawa, K., Higgins, B., Golenbock, S., Mack, A., and Bellamy, N. (2016). Psychometric properties of disaster event reaction items form the crisis counseling individual/family log. Disaster Medicine and Public Health Preparedness, Aug 12, 1-10.
Valenti, M., Fujii, S., Kato, H., Masedu, F., Tiberti, S., & Sconci, V. (2013). Validation of the Italian version of the Screening Questionnaire for Disaster Mental Health (SQD) in a post-earthquake urban environment. Annali dell’Istituto Superiore di Sanità, 49, 79–85.
Adult and Child/Youth Assessment and Referral Tools
Cohen, J. A., Kelleher, K. J., & Mannarino, A. P. (2008). Identifying, treating, and referring traumatized children: The role of pediatric providers. Archives of Pediatrics and Adolescent Medicine, 162, 447–452. Retrieved from http://archpedi.jamanetwork.com/journal.aspx
Fox, J. H., Burkle, F. M., Jr., Bass, J., Pia, F. A., Epstein, J. L., & Markenson, D. (2012). The effectiveness of Psychological First Aid as a disaster intervention tool: Research analysis of peer-reviewed literature from 1990 to 2010. Disaster Medicine and Public Health Preparedness, 6, 247–252.
James, L. E., & Noel, J. R. (2013). Lay mental health in the aftermath of disaster: Preliminary evaluation of an intervention for Haiti earthquake survivors. International Journal of Emergency Mental Health, 15, 165–178.
Krishnaswamy, S., Subramaniam, K., Indran, T., & Low, W. Y. (2012). The 2004 tsunami in Penang, Malaysia: Early mental health intervention. Asia-Pacific Journal of Public Health, 4, 710–718.
Kronenberg, M. E., Hansel, T., Brennan, A. M., Osofsky, H. J., Osofsky, J. D., & Lawrason, B. (2010). Children of Katrina: Lessons learned about postdisaster symptoms and recovery patterns. Child Development, 81, 1241–1259.
Navarro, J., Pulido, R., Berger, C., Arteaga, M., Osofsky, H. J., Martinez, M., . . . & Hansel, T. C. (2014). Children’s disaster experiences and psychological symptoms: An international comparison between the Chilean earthquake and tsunami and Hurricane Katrina. International Social Work, 1, 14.
Otsuka, K., Sakai, A., Nakamura, H., & Akahira, M. (2014). After the Great East Japan Earthquake: Suicide prevention and a gatekeeper program. Seishin Shinkeigaku Zasshi, 116, 196–202.
Riise, K. S., Hansel, T. C., Steinberg, A. M., Landis R. W., Gilkey S., Brymer, M. J., et al. (2009). The Louisiana Specialized Crisis Counseling Services (SCCS): Final program evaluation. Unpublished manuscript.
Rousseau, C., Measham, T., & Nadeau, L. (2013). Addressing trauma in collaborative mental health care for refugee children. Clinical Child Psychology and Psychiatry, 18, 121–136.
Scheeringa, M., & Haslett, N. (2010). The reliability and criterion validity of the diagnostic infant and preschool assessment: A new diagnostic instrument for young children. Child Psychiatry and Human Development, 41, 299–312.
Steinberg, A. M., Brymer, M. J., Decker, K. B., & Pynoos, R. S. (2004). The University of California at Los Angeles Post-traumatic Stress Disorder Reaction Index. Psychological Reports, 6, 96–100.
Steinberg, A. M., Brymer, M. J., Decker, K. B., & Pynoos, R. S. (2004). The University of California at Los Angeles Post-traumatic Stress Disorder Reaction Index. Current Psychiatry Reports, 6, 96–100. Retrieved from http://www.springer.com/medicine/psychiatry/journal/11920
Sampling Methods
Ruggiero, K. J., Davidson, T. M., McCauley, J., Gros, K. S., Welsh, K., Price, M., . . . Amstadter, A. B. (2015). Bounce Back Now! Protocol of a population-based randomized controlled trial to examine the efficacy of a web-based intervention with disaster-affected families. Contemporary Clinical Trials, 40, 138–149.
APPENDIX B:
Sample Data Tables
Primary Service |
Population Served |
Percentage |
Individual Crisis Counseling |
|
|
Group Counseling/Public Education |
|
|
Brief Educational/Supportive Contact |
|
|
Primary Service |
Population Served |
Percentage |
Individual Crisis Counseling |
|
|
Group Counseling/Public Education |
|
|
Brief Educational/Supportive Contact |
|
|
Age |
County |
Population Served |
preschool |
|
|
child |
|
|
adolescent |
|
|
adult (18–39) |
|
|
adult (40–64) |
|
|
adult (65+) |
|
|
County |
Handed to People |
Mailed to People |
Left in Public |
|
|
|
|
Year-Month |
County |
Total |
|
|
|
|
|
|
|
|
|
Year-Month |
County |
Total |
|
|
|
|
|
|
|
|
|
Year-Month |
County |
Total |
|
|
|
|
|
|
Year-Month |
Risk |
Total |
|
family missing/dead |
|
|
friend missing/dead |
|
|
pet missing/dead |
|
|
home damaged or destroyed |
|
|
vehicle or major property loss |
|
|
other financial loss |
|
|
disaster unemployed (self or household member) |
|
|
injured or physically harmed (self or household member) |
|
|
life was threatened (self or household member) |
|
|
witnessed death/injury (self or household member) |
|
|
assisted with rescue/recovery (self or household member) |
|
|
prolonged separation from family |
|
|
evacuated quickly with no time to prepare |
|
|
displaced from home 1 week or more |
|
|
sheltered in place or sought shelter due to immediate threat of danger |
|
|
past substance use/mental health problem |
|
|
preexisting physical disability |
|
|
past trauma |
|
|
N/A |
|
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | DHHS |
| File Modified | 0000-00-00 |
| File Created | 2021-01-15 |