Crisis Counseling Assistance and Training Program Data Toolkit
SUPPORTING STATEMENT
A. JUSTIFICATION
Circumstances of Information Collection
The Substance Abuse and Mental Health Services Administration (SAMHSA) Center for Mental Health Services (CMHS) is requesting approval for a revision from the Office of Management and Budget (OMB) for the Crisis Counseling Assistance and Training Program (CCP) Data Toolkit. The current forms in the toolkit (OMB No. 0930-0270) expire August 31, 2015. The CCP Data Toolkit contains seven continuing forms (i.e., Individual/Family Crisis Counseling Services Encounter Log, Group Encounter Log, Weekly Tally Sheet, Adult Assessment and Referral Tool, Child/Youth Assessment and Referral Tool, Participant Feedback Form, and Service Provider Feedback Form). The CCP (commonly referred to as the Crisis Counseling Program) is funded by the Federal Emergency Management Agency (FEMA) through the Robert T. Stafford Disaster Relief and Emergency Assistance Act (Public Law 93-288, as amended by Public Law 100-707).
Data collected through the use of this toolkit will be reported in the aggregate and will not be used for the purpose of reporting statistics for the general population. Rather, the data collected will be used to find statistics that will support the improvement of CCP service provision.
Services offered by the Crisis Counseling Program involve direct interventions to individuals and groups impacted by a major disaster or its aftermath. Educational activities and public information on disaster behavioral health issues are another component of the CCP. Additionally, disaster behavioral health consultation and training are also provided. In the event that FEMA declares a state eligible for the CCP, FEMA will look to the Director of the National Institute of Mental Health, as the delegate of the Secretary of the Department of Health and Human Services (HHS), to oversee the program (44 CFR 206.171 [f[a]). As such, SAMHSA CMHS (embedded within HHS) has become the designated representative for monitoring the Crisis Counseling Program, providing consultation, technical assistance, and guidance, and serving as point of contact to HHS for program matters.
Funded by FEMA and administered by SAMHSA’s CMHS, the CCP provides supplemental funding for individual and community crisis intervention services to U.S. states, territories, and other jurisdictions (hereafter referred to as “states”). States may apply for the Immediate Services Program (or ISP, which operates for the first 3 months after a disaster) and the Regular Services Program (or RSP, which operates for the next 9 months). The CCP has provided disaster behavioral health services to millions of disaster survivors since its inception and, as a result of 30 years of accumulated expertise, it has become an important model for federal response to a variety of catastrophic events. Recent CCPs include two programs in Colorado, one related to a wildfire and the second to a flood; programs in New Jersey and New York following Hurricane Sandy; programs in Washington and Alaska related to flooding and mudslides; and a program in Oklahoma in the aftermath of severe storms and tornadoes. These CCPs have primarily addressed the short-term behavioral health needs of communities through (a) outreach and public education, (b) individual and group counseling, and (c) referral. Federal disaster areas are eligible for a wide range of services, including the CCP.
Individual and group crisis counseling assists survivors in coping with current stress and symptoms in order to return to pre-disaster functioning. It relies largely on “active listening,” and crisis counselors also provide psycho-education (especially about the nature of responses to trauma) and help clients build coping skills. Outreach and public education serve primarily to normalize reactions and to engage people who might need further care. These roles are often, though not exclusively, performed by paraprofessionals who work throughout the community at sites including schools, churches, and workplaces. Although there are no formal limits to the number of sessions a person receives, crisis counseling typically involves no more than a few sessions. Since crisis counseling is time limited, referral is the third important function of CCPs. Counselors are expected to refer clients to formal treatment if the person has developed more serious psychiatric problems.
Regardless of their cause, disasters damage local infrastructures and strain the ability of local systems to meet the population’s basic needs. For the survivors, disasters may engender an array of stressors, including threat to one’s own life and physical integrity, exposure to the dead and dying, bereavement, profound loss, social and community disruption, and ongoing hardship. As a result of both the high prevalence and high stressfulness of disasters, the question of whether they impact behavioral health has been of interest for decades, and a substantial literature has developed that identifies and explains these effects. Based on a comprehensive literature review, the range of consequences experienced by disaster survivors is broad, including various psychological problems, such as depression, anxiety, and posttraumatic stress disorder (PTSD); physical health problems, such as sleep disruption, somatic complaints, and impaired immune function; chronic problems in living, such as troubled interpersonal relationships and financial stress; and resource loss, such as declines in perceived control and perceived social support. The data collected using the currently approved OMB CCP tools bear out these findings. According to crisis counselors completing the Individual/Family Crisis Counseling Services Encounter Log form, the most common event reactions among survivors were feeling sad or tearful (14 percent), anxious or fearful (11 percent), and irritable or angry (8 percent). Similarly, individuals who completed the Participant Feedback Form most frequently reported consequences of the disaster that included being bothered by bad memories, nightmares, or reminders of what happened; feeling down or depressed; and finding other stressful things harder to deal with because of what had happened.
CCPs have been required to collect data related to their program throughout the length of the program (44 CFR 206.171 [F][3]). However, until September 2005 there was no systematic mechanism for collecting the required data due to differences between disasters, programs, and states. In September 2005, OMB approved the CCP Data Toolkit (OMB No. 0930-0270) developed by SAMHSA’s CMHS with the assistance of the Department of Veterans Affairs’ National Center for PTSD. In August 2008, OMB approved the revised CCP Data Toolkit (OMB No. 0930-0270), to include minor revisions and the addition of a data collection form (Child/Youth Assessment and Referral Tool). In August 2012 we added a family/household component to the Individual Crisis Counseling Services Encounter Log to reduce the burden for crisis counselors who before that time needed to complete separate forms for interactions with various relatives or household members actively engaged during a single visit. At that time the form was renamed Individual/Family Crisis Counseling Services Encounter Log. Current data findings from November 2012 to January 2015 based on this revised log report that individual encounters make up 81.4% and family/household encounters 18.6%.
The current data collection forms will expire in August 2015. We are suggesting minor revisions to these forms with an August 2015 expiration date. Therefore, SAMHSA CMHS is requesting the approval of the revised CCP Data Toolkit for another 3-year period. The revisions include the addition of mobile app questions to the Service Provider Feedback Form (Attachment G) and minor revisions to the gender question on the Participant Feedback Form (Attachment F) and Service Provider Feedback Form (Attachment G). It is expected that State CCPs will use the forms in this revised toolkit for the purpose of continued collection of standardized information reported to SAMHSA CMHS for appropriate processing and analysis.
2. Purpose and Use of Information
CCPs by nature are delivered in a rapidly evolving environment in which decisions need to be made quickly on the basis of limited information. The prejudice is toward action, not deliberation. During the crisis, there may be little interest in collecting systematic information on how the program is working. This shortcoming makes it difficult to monitor program progress and provides few data with which to later determine program achievements. Without a systematic data collection process, programs have limited means of crystallizing what they have discovered from experience in a way that can be communicated to other people planning responses to future events.
The toolkit relies on standardized forms. Data will be collected throughout the program period about services delivered and users of services. At the program level, the data can be entered quickly and easily into a cumulative database that will be set up in advance to yield summary tables for both quarterly and final reports for the program. Because the data will be collected in a consistent way from all programs, data can be uploaded into an ongoing national database that likewise provides SAMHSA CMHS with a way of producing summary reports of services provided across all programs funded.
The data collection tools seek to gather information to better understand program reach, quality, and consistency. Program reach refers to the number of encounters crisis counseling staff have with disaster survivors. Program quality refers to whether the services were perceived as appropriate and beneficial by both service recipients and crisis counseling staff. Program consistency refers to the variability in service provision across geographical areas and whether this variability can be explained by differences in the areas and their populations. State CCPs will use the following components of the toolkit for data collection throughout the life of the program:
Encounter Logs. These forms document all services provided. Completion of these logs is required by the crisis counselors during both the ISP and RSP. There are three types of encounter logs:
Individual/Family Crisis Counseling Services Encounter Log (see Attachment A). Crisis counseling is defined as an interaction that lasts at least 15 minutes and involves participant disclosure. This form is completed by the crisis counselor for each service recipient or each family, defined as the person or persons who actively participated in the session (e.g., by verbally participating), not someone who is merely present. As mentioned above, in the past crisis counselors would complete a separate Individual Crisis Counseling Services Encounter Log for each participant in the family or household who participated. Information collected includes demographics, service characteristics, risk factors, event reactions, and referral data. The Individual/Family Crisis Counseling Services Encounter Log can be used for individual encounters or for family encounters (though an encounter of either type must be 15 minutes or longer for crisis counselors to use this form). Family data will be aggregated and analyzed separately from individual data. When the crisis counselor completes the form, he or she completes it for either an individual encounter or a family encounter, not both. Based upon the number of people reported in the encounter on the form, it will be clear if it is a family or an individual encounter. Recording data at the family rather than the individual level will not be as specific (or precise) as collecting information on all individuals. However, we have assessed the impact it may have and feel that this is an acceptable alteration, especially given the reduction in burden it permits (nearly 20 percent of encounters are family or household encounters). Since data are reported at the aggregate level, the data collected will still provide valuable information to the program. We examined entries in the “other” box in the “Focus of Encounter” part of this form and noted the following items: concern for others and/or family and friends; coping skills; AA/support groups/community groups; disaster planning and preparedness; exercise/nutrition/physical health tips; and prayer. However, there were not enough commonalities in the “other” box to suggest changes to this category in the Individual/Family Crisis Counseling Services Encounter Log.
Group Encounter Log (see Attachment B). This form is used to identify either a group crisis counseling encounter or a group public education encounter. The person completing the form uses a check mark at the top of the form to identify the class of activities (i.e., counseling or education). Information collected includes service characteristics, group identity and characteristics, and the focus or foci of the group’s activities.
Weekly Tally Sheet (see Attachment C). This form documents brief educational and supportive encounters not captured on any other form. Information collected includes service characteristics, daily tallies and weekly totals for brief educational or supportive contacts, and material distribution with minimal or no interaction, including social networking and mass media advertising efforts. The Weekly Tally Sheet will be used to measure reach, as it assesses the number of materials distributed and types of contacts. It will also address program consistency in that it will be used to capture data that in turn will be used to understand weekly trends and other phenomena within and across programs. Note that there is a category for social media (“social networking messages”) on the current Weekly Tally Sheet form. Instructions on this form for this category limit the entries to one for each post and prohibit counting likes or comments. We recommend not making changes in this social media category on this form at this time.
The following variables on the Weekly Tally Sheet are intended to capture the reach of the program through its less-than-15-minute interactions: brief educational contacts, telephone contacts, email contacts, material handed to people, material left at people’s homes, material left in public places, and community networking and coalition building.
Programs are encouraged to review the data using the “Weekly Trends” reporting features of the online system in order to ensure consistency of the program’s implementation with the intended service plan. Weekly Tally Sheets do not address the quality of the program.
Assessment and Referral Tools. Generally, these forms are used as an interview guide with adults or children and youth who have received individual crisis counseling on two or more occasions and who may need referral to further and more intensive services. However, these tools may be used at any time that a crisis counselor suspects that an individual is experiencing serious reactions to the disaster. Typically, these tools will be used beginning 3 months after the disaster (which may occur during the ISP or RSP) and will be completed by the crisis counselor or team leader.
Both the Adult Assessment and Referral Tool and the Child/Youth Assessment and Referral Tool are preexisting tools that were previously approved by OMB for use in the CCP. The question items for both of these tools have not changed for the present OMB approval request; included is administration instructions on the forms for the crisis counselor, as well as a privacy statement.
These Assessment and Referral Tools have been used by CCPs in their current forms and validated (see references below).
Brannen, D. E., Barcus, R., McDonnell, M. A., Price, A., Alsept, C., & Caudill, K. (2013). Mental health triage tools for medically cleared disaster survivors: An evaluation by MRC volunteers and public health workers. Disaster Medicine and Public Health Preparedness, 7, 20–28.
Commers, M. J., Morival, M., & Devries, M. W. (2014). Toward best-practice post-disaster mental health promotion for children: Sri Lanka. Health Promotion International, 29, 165–170.
Connor, K., & Davidson, J. (2001). SPRINT: A brief global assessment of post-traumatic stress disorder. International Clinical Psychopharmacology, 16, 279–284.
Hamblen, J. L., Norris, F., Pietruszkiewicz, S., Gibson, L. E., Naturale, A., & Louis, C. (2009). Cognitive behavioral therapy for postdisaster distress: A community based treatment program for survivors of Hurricane Katrina. Administration and Policy in Mental Health and Mental Health Services Research, 36, 206–214.
Jones, K., Allen, M., Norris, F., & Miller, C. (2009). Piloting a new model of crisis counseling: Specialized crisis counseling services in Mississippi after Hurricane Katrina. Administration and Policy in Mental Health and Mental Health Services Research, 36, 195–205.
Norris, F., Donahue, S., Felton, C., Watson, P., Hamblen, J., & Marshall, R. (2006). A psychometric analysis of Project Liberty’s Adult Enhanced Services Referral Tool. Psychiatric Services, 57, 1328–1334.
Norris, F., Hamblen, J., Brown, L., & Schinka, J. (2008). Validation of the Short Post-Traumatic Stress Disorder Rating Interview (Expanded Version, Sprint-E) as a measure of postdisaster distress and treatment need. American Journal of Disaster Medicine, 3, 201–212.
Pekevski, J. (2013). First responders and psychological first aid. Journal of Emergency Management, 11, 39–48.
Steinberg,
A. M., Brymer, M. J., Decker, K. B., & Pynoos, R. S. (2004). The
University of California at Los Angeles Post-traumatic Stress
Disorder Reaction Index: Child and adolescent disorders. Current
Psychiatry Reports, 6, 96–100.
Valenti, M., Fujii, S., Kato, H., Masedu, F., Tiberti, S., & Sconci, V. (2013). Validation of the Italian version of the Screening Questionnaire for Disaster Mental Health (SQD) in a post-earthquake urban environment. Annali dell’Istituto Superiore di Sanità, 49, 79–85.
Adult Assessment and Referral Tool (see Attachment D). This tool ensures the collection of information on characteristics of the encounter, risk categories, and demographics. The tool also includes the Short PTSD Rating Interview: Expanded Version, also known as the SPRINT-E, an 11-item measure of post-disaster distress including but not limited to symptoms of PTSD.
There have been no changes to the items on the Adult Assessment and Referral Tool. Script for instructions is provided for clarity. The script reads as follows:
“READ ALOUD: Occasionally, we find it helpful to ask survivors a few specific questions about how they were affected by the disaster and how they are feeling now. Your name or address is not being recorded on this form, and any information will be kept private to the fullest extent of the law. You may choose not to answer any question. May I ask you these questions? My first questions are about various experiences you have had in the disaster. Do any of the following apply to you?”
Child/Youth Assessment and Referral Tool (see Attachment E). This tool ensures collection of information on risk factors and demographics, and it includes items to assess post-disaster symptoms, as well as items for parents to rate their child’s feelings and behavior. When this form was developed, the symptom (or reaction) section of the tool was adapted from the University of California at Los Angeles (UCLA) Post-traumatic Stress Disorder Reaction Index with inclusion of additional items related to depression and functioning (Steinberg, Brymer, Decker, & Pynoos, 2004). Drs. Pynoos and Steinberg granted permission for this modification for use by the CCP Project Liberty after the terrorist attacks on September 11, 2001. This tool was then adapted by National Child Traumatic Stress Network in 2005 for use by the Louisiana Spirit Specialized CCP after Hurricanes Katrina and Rita.
There
have been no changes to the items on the Child/Youth Assessment
and Referral Tool. The following instructions are provided on the
form for clarity.
It is recommended that this form be
used with all children or youth who are intensive users of
services. Intensive users are people who are participating in
their third individual crisis counseling visit with any crisis
counselor from the program or who continue to suffer severe
distress that may be impacting their ability to perform routine
daily activities. This form should be used as an interview guide
(1) with children receiving individual crisis counseling on the
third and fifth occasions OR (2) with any child at any time if you
suspect the child may be experiencing serious reactions to the
disaster.
NOTE: Prior to administration of the Child/Youth Assessment and Referral Tool, make sure that consent has been obtained from a parent/caregiver for the child’s or youth’s participation. Children over the age of 7 may answer on their own behalf (with parental consent). For children 0–7, it is recommended that a parent/caregiver be interviewed with the child present. When there are concerns about the ability of a child over the age of 7 to understand and accurately answer the questions, it is advisable for the parent/caregiver to assist in answering the questions. Adolescents may not want to be interviewed in front of their parents. If a parent/caregiver is present, ask the adolescent if he or she wishes to be interviewed alone. See your program manager or CCP Evaluation Guidance and Administration document for further details.
This tool contains a script for verbal consent, which instructs the person administering the tool to: READ: “Occasionally, we find it helpful to ask children/adolescents or their parents/caregivers a few specific questions about how they were affected by the disaster and how they are feeling now. May I ask you these questions? My first questions are about various experiences you have had in the disaster.”
Crisis counselors, although paraprofessionals, receive training on how to use these forms in the required CCP trainings and have been administering them for years. According to the developers (Dr. Fran Norris and Dr. Melissa Brymer), these tools were developed for use by paraprofessionals. However, CCPs must have a protocol in place regarding what the crisis counselor should do if a referral is warranted and if a person being interviewed responds to the question “Is there any possibility that you might hurt or kill yourself?” with an answer of yes. The following guidance is provided to programs on the actual forms:
Many CCPs have team leaders or other staff with a mental health background to administer this tool to ensure that proper assessment and referral is carried out. All crisis counseling staff using this tool should have detailed training and guidance on use of the tool and when to make a referral for more intensive services. Prior to use of this tool, the CCP should have identified an organization or agency that is willing to accept referrals from the CCP for more intensive mental health or substance abuse intervention services.
The
following references provide support for the idea that an individual
other than a licensed professional can effectively administer these
tools:
Cohen, J. A., Kelleher, K. J., & Mannarino. (2008). Identifying, treating, and referring traumatized children: The role of pediatric providers. Archives of Pediatrics and Adolescent Medicine, 162, 447–452.
Fox, J. H., Burkle, F. M., Jr., Bass, J., Pia, F. A., Epstein, J. L., & Markenson, D. (2012). The effectiveness of psychological first aid as a disaster intervention tool: Research analysis of peer-reviewed literature from 1990–2010. Disaster Medicine and Public Health Preparedness, 6, 247–252.
James, L. E., & Noel, J. R. (2013). Lay mental health in the aftermath of disaster: Preliminary evaluation of an intervention for Haiti earthquake survivors. International Journal of Emergency Mental Health, 15, 165–178.
Krishnaswamy, S., Subramaniam, K., Indran, T., & Low, W. Y. (2012). The 2004 tsunami in Penang, Malaysia: Early mental health intervention. Asia-Pacific Journal of Public Health, 4, 710–718.
Kronenberg, M. E., Hansel T., Brennan, A. M., Osofsky, H. J., Osofsky, J. D., & Lawrason, B. (2010). Children of Katrina: Lessons learned about postdisaster symptoms and recovery patterns. Child Development, 81, 1241–1259.
Otsuka, K., Sakai, A., Nakamura, H., & Akahira, M. (2014). After the Great East Japan Earthquake: Suicide prevention and a gatekeeper program. Seishin Shinkeigaku Zasshi, 116, 196–202.
Riise, K. S., Hansel, T. C., Steinberg, A. M., Landis R. W., Gilkey S., Brymer, M. J., et al. (2009). The Louisiana Specialized Crisis Counseling Services (SCCS): Final program evaluation. Unpublished manuscript.
Rousseau, C., Measham, T., & Nadeau, L. (2013). Addressing trauma in collaborative mental health care for refugee children. Clinical Child Psychology and Psychiatry, 18, 121–136.
Scheeringa, M., & Haslett, N. (2010). The reliability and criterion validity of the diagnostic infant and preschool assessment: A new diagnostic instrument for young children. Child Psychiatry and Human Development, 41, 299–312.
Steinberg, A. M., Brymer, M. J., Decker, K. B., & Pynoos, R. S. (2004). The University of California at Los Angeles Post-traumatic Stress Disorder Reaction Index. Psychological Reports, 6, 96–100.
Additionally,
the follow guidance is provided to the programs in the CCP Data
Toolkit regarding referrals for children and youth:
For children over the age of 10 (or if the crisis counselor or parent/caregiver is concerned about a younger child), the [crisis] counselor may ask, ‘Have you had any thoughts or plans about either hurting or killing yourself?’ If the respondent answers ‘YES’ to this item, then the crisis counselor should immediately refer the child/youth for psychiatric or mental health professional intervention. The CCP should have protocols or procedures in place for how a crisis counselor should respond and who should be notified of this safety concern.
Many CCPs have team leaders or other staff with a mental health background to ensure that proper assessment and referral is carried out. All crisis counseling staff using this tool should have detailed training and guidance on use of the tool and when to make a referral for more intensive services. Prior to use of this tool, the CCP should have identified an organization or agency that is willing to accept referrals from the CCP for more immediate psychiatric intervention.
If the total number is four or higher, the counselor should discuss appropriate referral options for the child/youth and/or family. This includes being prepared to offer youth and parents the name at an organization that has agreed to accept CCP referrals and a contact at that organization.
As described above, the CCP must have a referral resource/center/agency for situations in which the crisis counselor has identified a child/youth as in need of immediate services. It is typically the case that the referral resource/center/agency has a licensed mental health professional on staff. Suggested licenses are clinical/counseling doctoral-level professionals (Ph.D., Psy.D.); social work (LCSW, LSW, D.S.W.); marriage and family therapists (LMFT); and so forth.
Participant Feedback Form (see Attachment F REVISED FROM 2015 EXPIRATION FORM). These forms are completed by and collected from a sample of adult service recipients, not every recipient. A time sampling approach (e.g., soliciting participation from all counseling encounters during a 1-week period 6 months and 1 year post-event) will be used. Information collected includes satisfaction with services, usefulness of the services provided, perceived improvements in one’s own functioning, types of exposure, and event reactions. In this application, we revised a question that read, “Sex,” with response options “Male” or “Female,” to now read, “How do you identify yourself?” Response options remained the same. We made this revision to capture data that account for more subjective gender identities, which go beyond biological sex.
The Participant Feedback Form is the only one used to collect information directly from adult service recipients, and the primary tool for collecting information on the quality of the program (the Service Provider Feedback Form is the other). It is used to inform program services at the local, state, and federal level. The questions about services relate directly to the goals of crisis counseling, such as reassurance and being helped to find ways to cope. There is a section on the ways in which the respondent was exposed to the disaster, and also on event reactions, such as posttraumatic stress, depression, impaired functioning, and perceived need for additional help. (This is the SPRINT-E, described earlier as part of the Adult Assessment and Referral Tool).
The SAMHSA Disaster Technical Assistance Center (DTAC) provides the following template to programs prior to adminstering the Participant Feedback Form, with instructions that it be customized to the program and provided as a cover sheet to the Participant Feedback Form:
[Date]
Dear friend:
[Name of or reference to disaster] caused many challenges and problems for many people. Our crisis counseling project, [Name of project], tries to help people cope with the stress of recovery. To do our jobs well, we need to know more about how people are doing now and if our crisis counselors are finding the best ways to be helpful.
Inside this packet is a very brief survey. It will take you only a few minutes to fill out but will be very helpful to us. Your answers will help us to help others.
The packet has a pen in it for you to use and keep. It also has a stamped envelope for you to mail in your survey after you are done. The survey does not ask for your name and is completely anonymous. No one will know which survey you filled in. Your answers will be used together with the answers of other people who are also kind enough to help us this week.
Please send your survey in by [Date]. Of course, it is fine if you choose not to participate. Also, you should feel free to leave any question blank that you don’t want to answer.
Your opinions are important to us. Thank you for taking the time to complete this survey.
Sincerely,
[Name of Project Director]
[Name of Project]
Service Provider Feedback Form (see Attachment G REVISED FROM 2015 EXPIRATION FORM). These forms are completed by and collected from the CCP service providers (i.e., crisis counselors) anonymously at approximately 6 months and 1 year post-event. The items on this form relate to the training, work environment, and level of job stress experienced by the crisis counselor. The form will be coded on several program- and worker-level variables to be shared with program management for review. The Service Provider Feedback Form provides additional information on the quality of the program. In this application, we revised a question that read, “Sex,” with response options “Male” or “Female,” to now read, “How do you identify yourself?” Response options remained the same. We made this revision to capture data that account for more subjective gender identities, which go beyond biological sex.
Lastly, the Service Provider Feedback Form added twenty-one (21) question items related to crisis counselor’s perceptions on use of paper versus mobile data entry processes for form completion of the following: Individual/Family Crisis Counseling Services Encounter Log, Group Encounter Log, and both Assessment and Referral Tools. Even though this has increased the response burden for the Service Provider Feedback Form, completion of the form is still estimated less than forty-five (45) minutes to complete.
Crisis counselors are the essential link between the program and the consumer. Crisis counselors and their supervisors are in a unique position to judge the quality of the services being provided and the extent to which they match the needs of the community. The Service Provider Feedback Form yields a standardized assessment of providers’ opinions and reactions to their work.
Ongoing program monitoring and information gathering will continue to increase the knowledge base established with the previous CCP Data Toolkit (approved in 2005, 2008, and 2012). This knowledge base persists to inform and guide the program at the federal level. From the systematic collection of data it is possible to interpret the factors responsible for differences in CCP implementation—that is, whether they derive from variations in setting (e.g., rural versus urban community) or program design variables that contribute to more successful outreach. By collecting data across future programs more completely and systematically, SAMHSA CMHS may be able to look at program data trends and make better judgments about program-level factors that influence service delivery. This goal requires a set of standardized tools that are useful for program monitoring and that feed into a cumulative national database. As of the 2012 submission of this OMB package, one article referencing the CCP was published:
Norris, F. H., & Bellamy, N. D. (2009). Evaluation of a national effort to reach Hurricane Katrina survivors and evacuees: The crisis counseling assistance and training program. Administration and Policy in Mental Health, 36, 165–175.
For each CCP grant that is awarded, two quarterly progress reports and one final report for the 9-month RSP grant are submitted to FEMA and CMHS project officers and a SAMHSA DTAC technical assistance specialist. Quarterly reports are due 30 days after the end of the 3-month reporting period. The final program report is due to the FEMA and CMHS project officers 90 days after the final day of program services. Program monitoring data are required in the quarterly reports and the final program report. Below, we provide a website link and login to our current CCP Online Data Collection and Evaluation System (ODCES) demonstration and training website so that OMB may view some of the reporting options available for data entered into the system that are accessible in real time at multiple levels (i.e., local service provider, state or territory, and federal). This website is a replica of the real CCP ODCES website, but it houses test data and is used for demonstration and training purposes. The login provided is for the state and territory level. After logging in, please view the reporting section (on the left-hand menu) to access the various reports offered to users. The website is maintained and managed by SAMHSA DTAC.
Access to the CCP Data Entry and Reporting demonstration and training:
http://www.ccpdata.org/CCP2Test
login: mitch.provider@umd.edu
password: provider
For a demonstration or questions regarding this website, please contact SAMHSA staff either Dr. Nikki Bellamy at 240-276-2418, nikki.bellamy@samhsa.hhs.gov or CAPT Erik Hierholzer, 240-276-0408, erik.hierholzer@samhsa.hhs.gov.
In addition, empirical knowledge about best practices is still very limited. Unsound counseling practices may be perpetuated and innovations and improvements not disseminated. By encouraging pilot testing of new innovations, such problems may be avoided in the future. This goal requires that programs have access to a set of optional tools that can be used for the testing of special initiatives.
In summary, whether the questions concern how to improve the reach of the service delivery system or how to improve the efficacy of the services themselves, systematic program monitoring provides a basis for the answers. Our proposed methodology for future CCP data collection processes, via the use of the CCP Resource/Data Toolkit, attempts to improve practice in a way that adheres to the goals and standards of program evaluation science while supporting the goals and standards of SAMHSA CMHS for delivering the highest possible caliber of disaster behavioral health program during a crisis.
3. Use of Information Technology
The forms, as well as the ability to submit the forms, will be available to all CCPs both electronically as well as in hardcopy. Crisis counselors will complete the Service Provider Feedback Form online, ensuring confidentiality. Following the completion of the data collection forms, data are entered into an online database, the CCP ODCES. This system, created in 2009, allows real-time data entry and reporting. All instruments will be available for download and printing from the SAMHSA DTAC website and the ODCES. Following download and printing, crisis counselors/program staff will complete paper-based versions of the Individual/Family Crisis Counseling Services Encounter Log, Group Encounter Log, Weekly Tally Sheet, Child/Youth Assessment and Referral Tool, and Adult Assessment and Referral Tool. Program participants (disaster survivors) will also complete a paper-based version of the Participant Feedback form. Data from these six forms are entered into an online secure database. Only data from the Service Provider Feedback Form are collected through electronic means.
Additionally, SAMHSA DTAC staff members are in the process of developing a mobile application that will offer CCP crisis counselors/program staff a more efficient alternative to the current paper forms they fill out to document their encounters in the field. From CCP Data Mobile Application focus groups conducted in late 2014 and early 2015, SAMHSA and SAMHSA DTAC staff members learned that CCP service providers and administrators anticipate that a mobile format for capturing and submitting CCP data will greatly enhance the existing system in terms of accuracy, efficiency, and overall user satisfaction. Providers are generally comfortable working with technology, and the providers interviewed described frequent use of tablets and smartphones during their CCP effort. The goal is to develop the following three forms by September 2015 for mobile device data entry: (1) Individual/Family Crisis Counseling Services Encounter Log; (2) Group Encounter Log; and (3) Weekly Tally Sheet. These forms will be tied directly into the existing desktop system’s back-end database and front-end functionality such as reporting, analysis, and quality control features.
4. Effort to Identify Duplication
These forms are specific to this program, and no other programs are collecting these data.
5. Involvement of Small Entities
The information requested will not have a significant impact on small entities.
6. Consequences If Information Collected Less Frequently
The Individual/Family Crisis Counseling Services Encounter Log will be completed by the crisis counselor for 100 percent of individuals or families who access crisis counseling services for 15 minutes or longer. The Group Encounter Log will be completed by the crisis counselor for 100 percent of groups that meet for crisis counseling or for public education. The Weekly Tally Sheet will be completed by the crisis counselor for 100 percent of other brief educational or supportive encounters not captured by any other form.
The Adult and Child/Youth Assessment and Referral Tools will be completed by a trained crisis counselor for 100 percent of service recipients who access the individual crisis counseling component multiple times (recommended on the third and fifth visit) or as deemed necessary. It is predicted that this will be less than 5 percent of all service users.
The Participant Feedback Form will be completed by service recipients. It will be made available at least twice during the CCP RSP grant to users of crisis counseling and education services and encounters. The sampling strategy will be determined by the state but will involve a target of at least two sampling occurrences during the 9-month program period. The Service Provider Feedback Form will be administered to all CCP service providers (i.e., crisis counselors and team leaders) at approximately 6 months and 1 year after a disaster.
The data being collected on the aforementioned forms are already required per 44 CFR 206.171 (f)(a). The introduction of these forms will provide a systematic method for data collection that will improve practice in a way that adheres to the goals and standards of the program and SAMHSA CMHS for delivering the highest caliber possible of behavioral health programs during a crisis. If CCPs do not collect the data at the aforementioned data points, this may lower the value of the data for SAMHSA CMHS use, in particular by losing measurement of intermediate and long-term effects.
7. Consistency with the Guidelines in 5 CFR 1320.5(d)(2)
This information collection fully complies with 5 CFR 1320.5(d)(2).
8. Consultation Outside the Agency
The notice required by 5 CFR 1320.8(d) was published in the Federal Register on May 18, 2015 (80 FR 28283). Key updates in this 2015 submission include the addition of new mobile app questions to the Service Provider Feedback Form (Attachment G) and minor revisions to the gender question on the Adult Assessment and Referral Tool (Attachment D), Child/Youth Assessment and Referral Tool (Attachment E), Participant Feedback Form (Attachment F), and Service Provider Feedback Form (Attachment G).
The following four experts reviewed the toolkit and found that it was written clearly and the language was concise and accurate. Further, the experts as well as previous users of the toolkit agreed that there should be a decrease in overall burden with the revised tools. More information on the rationale for the revised burden table is provided in Item 12 of this supporting statement. However, with the revised burden, people completing the forms should need no more than 8 minutes for the Individual/Family Crisis Counseling Services Encounter Log; 5 minutes for the Group Encounter Log; 12 minutes for the Weekly Tally Sheet; and 15 minutes each for the Adult Assessment and Referral Tool, Child/Youth Assessment and Referral Tool, 15 minutes for the Participant Feedback Survey, and 25 minutes for the Service Provider Feedback Form. The experts that were consulted included the following:
(1) Everly Macario, Sc.D., M.S., Ed.M.
Senior Evaluation Research Director | EMacario@IQSolutions.com
SAMHSA Disaster Technical Assistance Center
C/O IQ Solutions, Inc. | 224-244-3965 Voice | 800-308-3515 Toll-Free
11300 Rockville Pike, Suite 901, Rockville, MD 20852
http://www.samhsa.gov/dtac | DTAC@samhsa.hhs.gov
(2) Yukari Takata Schneider, Ph.D., M.P.H.
Market Research Analyst 2 | YSchneider@IQSolutions.com
SAMHSA Disaster Technical Assistance Center
C/O IQ Solutions, Inc. | 224-221-4254 Voice | 800-308-3515 Toll-Free
11300 Rockville Pike, Suite 901, Rockville, MD 20852
http://www.samhsa.gov/dtac | DTAC@samhsa.hhs.gov
(3) Melissa Brymer, Ph.D.
Director, Terrorism and Disaster Programs
National Center for Child Traumatic Stress
University of California, Los Angeles
11150 West Olympic Boulevard
Suite 650
Los Angeles, CA 90064
Phone: 310-235-2633, ext. 227
(4) Patricia Watson, Ph.D.
Senior Educational Specialist
National Center for Posttraumatic Stress Disorder
215 N. Main Street
White River Junction, VT 05009
Phone: 802-296-5132
Fax: 802-296-5135
9. Payment to Respondents
The crisis counselor respondents will not receive any payment, as completion of some of the forms in the toolkit is part of their regular work responsibilities within the CCP. These forms are the Individual/Family Crisis Counseling Services Encounter Log, Group Encounter Log, Weekly Tally Sheet, both the Adult and Child/Youth Assessment and Referral Tools, and, as needed, the Service Provider Feedback Form. The hourly cost associated with the completion of the Participant Feedback Form is the processing cost for these forms to be completed by people who have received crisis counseling services.
10. Assurance of Confidentiality
SAMHSA CMHS and its contractors or consultants will not receive identifiable client or participant records. Provider-level information will be aggregated to at least the program level.
Providers and all other potential respondents will be assured that protection is maintained throughout the data collection. All data will be closely safeguarded, and no individual identifiers will be used in reports, in which only aggregated data will be reported.
The following Paperwork Reduction Act Statement appears on all data collection forms that crisis counselors complete:
Paperwork
Reduction Act Statement
This information is being collected
to assist the Substance Abuse and Mental Health Services
Administration (SAMHSA) for the purpose of program monitoring of
FEMA’s Crisis Counseling Assistance and Training Program.
Crisis counselors are required to complete this form following the
delivery of crisis counseling services to disaster survivors.
Information collected through this form will be used at an aggregate
level to determine the reach, consistency, and quality of the Crisis
Counseling Assistance and Training Program. Under the Privacy Act of
1974 any personally identifying information obtained will be kept
private to the extent of the law. An agency may not conduct or
sponsor, and a person is not required to respond to, a collection of
information unless it displays a currently valid Office of Management
and Budget (OMB) control number. The OMB control number for this
project is 0930-0270. Public reporting burden for this collection of
information is estimated to average X minutes per encounter,
including the time for reviewing instructions, searching existing
data sources, gathering and maintaining the data needed, and
completing and reviewing the collection of information. Send comments
regarding this burden estimate or any other aspect of this collection
of information, including suggestions for reducing this burden, to
SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 2-1057,
Rockville, MD 20857.
For the Participant Feedback Form and the Service Provider Feedback Form, both of which are voluntary and anonymous, the statement will read as follows:
Paperwork
Reduction Act Statement
This information is being collected
to assist the Substance Abuse and Mental Health Services
Administration (SAMHSA) for the purpose of program monitoring of
FEMA’s Crisis Counseling Assistance and Training Program. This
voluntary information collected will be used at an aggregate level to
determine the reach, consistency, and quality of the Crisis
Counseling Assistance and Training Program. Under the Privacy Act of
1974 any personally identifying information obtained will be kept
private to the extent of the law. An agency may not conduct or
sponsor, and a person is not required to respond to, a collection of
information unless it displays a currently valid Office of Management
and Budget (OMB) control number. The OMB control number for this
project is 0930-0270. Public reporting burden for this collection of
information is estimated to average 15 to 25 minutes per encounter,
including the time for reviewing instructions, searching existing
data sources, gathering and maintaining the data needed, and
completing and reviewing the collection of information. Send comments
regarding this burden estimate or any other aspect of this collection
of information, including suggestions for reducing this burden, to
SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 2-1057,
Rockville, MD 20857.
11. Questions of a Sensitive Nature
The questions about mental health and behavioral health issues such as substance use could be considered sensitive, but they are either asked or discussed in the context of a disaster behavioral health program by trained personnel.
12. Estimates of Annualized Hour Burden
The revised adjusted figures are based upon a review of data trends among recent and currently active CCPs and the utilization of the Individual/Family Crisis Counseling Services Encounter Log, Group Encounter Log, and Adult and Child/Youth Assessment and Referral Tools, which are used at a lower rate than previously reported. Thus, the revised total amount of time that is estimated for completion of the CCP Data Toolkit, record management by provider staff, and entry into an online database by the CCPs is 7,638 hours. The annualized hourly costs to respondents are estimated to be $152,760.00. It is estimated from previous CCP reports that crisis counselors (i.e., outreach workers, paraprofessionals; estimated wage $20/hour) are expected to complete most data collection forms, and the hourly cost for the Participant Feedback Form is associated with processing costs. The revised burden estimates summarized in the following table (also refer to table footnotes) are based on the reported experience of SAMHSA CMHS CCP grantees and contractors in compiling, completing, and reporting on the previously approved CCP Data Resource Toolkit forms.
Data
|
Number of respondents |
Responses per Respondent |
Total Responses |
Hours per Response |
Total Hour Burden |
Hourly Rate |
Total Hour Cost |
Individual/Family Crisis Counseling Services Encounter Log |
2001 |
1962 |
39,200 |
0.13 |
5,096 |
20.00 |
$101,920.00 |
Group Encounter Log |
1003 |
333 |
3,300 |
0.07 |
231 |
20.00 |
$4,620.00 |
Weekly Tally Sheet |
2001 |
334 |
6,600 |
0.2 |
1,320 |
20.00 |
$26,400.00 |
Assessment and Referral Tools |
2001 |
14 |
2,8005 |
0.25 |
700 |
20.00 |
$14,000.00 |
Participant Feedback Form |
1,000 |
1 |
1,000 |
0.25 |
250 |
20.00 |
$5,000.00 |
Service Provider Feedback Form |
1006 |
1 |
100 |
0.41 |
41 |
20.00 |
|
|
|
|
|
|
|
|
|
Total |
1,800 |
|
53,000 |
|
7,638 |
|
$152,760.00 |
1 200 is based on typical average of 10 crisis counselors (or 10 full-time equivalent) per grant with an approximate average of 20 grants per year (i.e., 10 x 20 = 200).
2 On average, each crisis counselor will complete 196 forms over the course of the grant.
3 Average of one form per week for a pair of crisis counselors (i.e., 2 counselors completing 1 form = 100 crisis counselors) at 33 weeks that includes both ISP and RSP (1 x 33 = 33).
4 Average of 33 weeks for each grant that includes both ISP and RSP.
5 On average 5 percent of the individuals encountered will result in the use of this tool (i.e., 56,000 individuals x 5% = 2,800).
6 On average 50 percent of service providers/crisis counselors may complete or use this tool.
13. Estimates of Annualized Cost Burden to Respondents
There are neither capital or startup costs nor are there any operation and maintenance costs to respondents as these costs are assumed under the CCP grant funding to the states/territories or federally recognized tribes.
14. Estimates of Annualized Cost to Government
The cost to the government will include approximately 0.5 full-time equivalent (FTE) senior staff at a General Schedule 14, or GS-14, level for a total of approximately $59,619.00 annualized cost.
15. Changes in Burden
SAMHSA/CMHS is requesting a minimum increase in burden. The adjustment in burden will increase from previously approved 7,622 total burden hours to 7,638; 16 hours.
16. Time Schedule, Publication, and Analysis Plans
16.a. Time Schedule
No timetable can be given at this time due to the nature of this data collection effort. A crisis (i.e., natural or human-caused disaster such as a terrorist attack) must occur before a time schedule can be established. CCPs are initially funded to a state for 3 months (ISP), and then the state may receive funding for 9 months based on need (RSP). Collection of toolkit data will begin as soon as the CCP is established, and this information will be used to inform the program progress reports filed at 3, 6, and 9 months. A final report will be generated at the end of the program, typically 1 year after the initial application for the ISP grant.
The state CCPs will determine when they will collect the forms from crisis counselors for review and entry into the online database. The typical timeline is as follows:
Individual/Family Crisis Counseling Services Encounter Log and Group Encounter Log forms will be collected on an ongoing basis as service recipient contact is made. These logs will be submitted to the CCP staff member responsible for reviewing them on a regular basis (typically, at the end of each day, but depending on the CCP and the context of the event, this may occur once a week).
Weekly Tally Sheets will be completed at least once per week for each county visited and submitted to the CCP staff member responsible for their review.
Assessment and Referral Tools when completed will be collected on a daily or weekly basis and submitted to the CCP staff member responsible for their review.
Participant Feedback Forms will be collected twice, at 6 months and 1 year post-disaster.
Service Provider Feedback Forms will be collected twice, at 6 months and 1 year post-disaster.
16.b. Publication
Service recipient outcome data will be collected through the CCPs. Data will be used to monitor and provide feedback to the CCP while the program is active as well as to SAMHSA CMHS and FEMA project officers and program staff. Copies of quarterly and final reports for each CCP will be maintained by SAMHSA CMHS. In addition, presentations will be made at grantee or professional meetings and/or conferences, at which time aggregate data will be provided about the performance of the CCP that is hosting the meeting. Feedback regarding the CCP’s performance during that event will also be discussed in the context of other CCPs that bear comparison on some single variable or set of variables. Future uses of the data may include submission to present or publish aggregate-level findings to professional scientific organizations or journals in or related to disaster behavioral health to help improve service delivery through lessons learned. Any such presentation or submission for publication will adhere to the appropriate federal guidelines and policies.
16.c. Analysis Plan
Once a crisis occurs and a CCP is established, collected data will be used to monitor and provide feedback to the CCP while the program is active as well as to SAMHSA CMHS and FEMA project officers and federal staff. These data will be uploaded or entered into an online database that will be set up in advance to yield summary tables for both quarterly and final reports for the program. Quarterly reports are used to monitor delivery of services by each program throughout the life of the program, thus giving the project officers an opportunity to determine if service implementation is sufficient to meet the needs of the community and whether service recipients are appropriately identified and reached. This process helps to shape the response in vivo, or on an ongoing basis. The final reports will provide a comprehensive tracking mechanism to show how the CCPs were established and how they changed over time, lessons learned from the process of establishing and maintaining the CCP, numbers of service recipients reached, how and what services were used over time, and other program factors that will be used to inform the state as to how it can better respond to future disasters. Collected data will also become a part of an ongoing national database to produce summary reports of services across all funded CCPs. Because data at the program level will be collected systematically, it will be possible to perform analyses across system variables (e.g., variations in setting such as urban versus rural or variations in program design that lead to more effective outreach). This will enable SAMHSA CMHS to make better judgments about program-level factors that influence service delivery. The primary intent of the collection of data is to use the data internally for monitoring, evaluative, and training purposes.
The descriptive analysis primarily will utilize frequency distributions and counts from each of the toolkit forms in order to address such questions as the following:
How many service recipients were seen in this program?
What were the demographic characteristics of the service recipients seen in this program?
What were the demographic characteristics of the service providers in this program?
What were the levels of exposure to the event for service recipients?
What were the levels of stress associated with the event for service providers?
Where were services provided?
What services were provided?
The outcome analysis will primarily address the following questions:
Did the services meet the needs of the service recipients?
What were the reactions of service recipients to the disaster?
How adequately did the CCP serve the providers in the areas of training, workload, resource availability, supervision, support, and stress management?
Were there differences in reactions of service recipients to disasters based on geographic or demographic characteristics?
Each CCP grant has been required to collect data related to the program throughout the length of the program (44 CFR 206.171 [F][3]). However, until September 2005, there was no systematic mechanism for collecting the required data due to differences among disasters, programs, and states. In September 2005, OMB approved the CCP Data Toolkit (OMB No. 0930-0270), which was developed by SAMHSA CMHS with the assistance of the Department of Veterans Affairs’ National Center for PTSD. For the original 2005 OMB approval, the major objective proposed and achieved was to have consistent data collection processes, forms and tools for use, and administration across all awarded CCP grants in both the ISP and RSP timeframes. In 2008, OMB approved the second iteration of CCP data collection forms with the same OMB number and an expiration date of January 2012. This second iteration approved by OMB also had major objectives that were proposed and achieved, including the addition of event reactions on the Individual/Family Crisis Counseling Services Encounter Log, the provision for a Child/Youth Assessment and Referral Tool, and an online data entry and reporting system accessible 24 hours a day, 7 days a week, that could be utilized by all awarded CCP grants throughout ISP and RSP timeframes. The third iteration, again with the same OMB number and an expiration date of August 2015, added a family/household component to the Individual/Family Crisis Counseling Service Encounter Log. The current request for OMB approval represents the fourth iteration of these CCP data forms. The major objectives with this current fourth iteration are as follows:
Renew the forms with minimal changes.
Maintain continuity of data to promote efficient and quality analysis across programs and years.
Incorporate a mobile device collection process to enhance data quality and reduce paper burden.
As such, we expect that the ability will increase for federal partners and state CCPs to conduct extensive trend analysis with the Individual/Crisis Counseling Services Encounter Log and other forms in the CCP Data Toolkit from 2005 to present. Moreover, we believe that this increased ability will support the purpose of continued collection of standardized information for appropriate processing, analysis, and reporting. In these analyses, we will identify breaks in trend analyses by major revision of a given instrument, and provide separate trend lines for individual, youth, and family data.
17. Display of Expiration Date
The expiration date for OMB approval will be displayed on all data collection instruments for which approval is being sought.
18. Exceptions to Certification Statement
This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions. The certifications are included in this submission.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | DHHS |
| File Modified | 0000-00-00 |
| File Created | 2021-01-24 |