Recovery-SS-B-3.4.14

SAMHSA RECOVERY MEASUREMENT PILOT STUDY

SUPPORTING STATEMENT

B. Collections of Information Employing Statistical Methods

B.1. Respondent Universe and Sampling Methods

SAMHSA is proposing a pilot test of a Recovery Measures instrument among three grant

programs. The three programs that have agreed to participate are: Behavioral Health Treatment Court Collaborative (BHTCC); Cooperative Agreements to Benefit Homeless Individuals (CABHI); and the Primary and Behavioral Health Integration (PBHCI). The respondent universe will be clients of the grantees. Sampling procedures will not be used for selecting clients. It is expected that 50 to 150 clients in each of the 3 participating grant programs (300 clients overall) will agree to complete the instrument.

In consultation with the Recovery Measurement Expert Panel and SAMHSA statisticians, it was determined that a goal of 300 participants will provide sufficient power to examine the psychometric properties of the proposed instrument. The exact composition of the grantee population cannot be determined until the FY2014 funds are awarded. It is expected that the grantee populations will be heterogeneous in nature (e.g., grants focusing on homelessness, and/or primary care), in order to increase the generalizability of this data to individuals served by SAMHSA’s current and planned grant portfolios. The current estimate of 300 participants was determined to be sufficient to examine potential differences among grantee populations and be able to account for moderate attrition between baseline and the six-month follow-up time period. Participants are likely to vary as a function of the specific grantee sites that elect to participate; but participants are likely to include a large proportion of vulnerable individuals, including persons with mental health and/or substance use disorders. In order to qualify for participation, grantee sites will be required to:

• Describe the project to all individuals receiving a baseline GPRA interview during the study period;

• Collect informed consent forms from participants who agree to participate in the pilot;

• Agree to administer the Recovery Measurement instrument to clients during normally scheduled baseline GPRA interviews and during the normally scheduled six-month follow-up interview.

In order to qualify for participation, clients must meet the following inclusion and exclusion criteria:

Inclusion criteria include:

• The participant is 18 years of age or older;

• The participant is receiving SAMHSA-funded services at a participating grantee site;

• The participant is willing to complete the Recovery Measurement instrument during their baseline GPRA interview, and again at their normally-scheduled six-month GPRA follow-up interview;

• The participant has the legal and cognitive ability to provide informed consent to participate in the project.

Exclusion criteria include:

• The individual is younger than 18 years of age. For this pilot, SAMHSA has decided to focus on the adult population and to develop an adolescent-based tool at a later date. It is hoped that results from the current pilot study can inform the development of that tool.

• The individual does not have the legal or cognitive ability to provide informed consent to participate in this project. The grantee sites will be responsible for ensuring that participants have initialed the Voluntary Informed and are legally and cognitive able to provide consent.

• The individual is not willing to complete the instrument during both the baseline GPRA interview and the six-month follow-up interview.

• The individual is a non-English speaker. For this initial pilot, SAMHSA is collecting information on the English version of the proposed Recovery Measurement instrument. It is expected that SAMHSA will develop linguistically and culturally appropriate translations of the final version.

No weights will be calculated for analyzing the results. The final products, including internal reports for SAMHSA, will contain the descriptive results of the entries. The reports will provide findings using simple frequencies and other descriptive statistics.

B.2. Information Collection Procedures

Three SAMHSA grant programs have agreed to pilot test the Recovery Measures tool in order to gain an understanding of the usefulness and reception of the tool. No sampling will be conducted within these grant programs. Potential participants in the pilot test include individuals receiving SAMHSA-funded services at these grantee sites.

Recruitment of the grantee sites will be conducted through written letters that will detail the goals of the project and outline grantee requirements for participation. A sample copy of the letter is provided in Attachment 3.

Clients who agree to participate will be asked to complete the instrument at intake (baseline) and at 6-months post-intake. These are the points in time during which SAMHSA grantees routinely collect data on individuals participating in their programs. During each client’s normally scheduled GPRA interview, personnel at each grantee site will describe the purpose and goals of the Recovery Measurement project to each potential participant.

Potential participants will be provided a Voluntary Consent Form (Attachment 4) and informed that responding to the recovery questions will take approximately 10 minutes. Grantee staff obtaining informed consent will stress the voluntary nature of participation in this project. Each participant will be assured that participation will in no way affect their ability to receive SAMHSA-funded services or to participate in other aspects of the SAMHSA-required GPRA interview. Participants will also be assured that they will be able to end their participation at any time, with no adverse consequences.

Individuals who agree to participate in this project will be asked to initial and sign the Voluntary Informed Consent document (Attachment 4). Each participating grantee site will be asked to make two photocopies of each Voluntary Informed Consent form. Participating grantee sites will be asked to retain one copy of each form for their records. Each site will be asked to store the consent forms using established protocols for the security of sensitive information. Grantee sites will be instructed to securely destroy all copies of the consent forms at the end of the study period. Participants will be provided with the second photocopy of the consent form for their records. The original copies of the consent forms will be transferred to SAMHSA headquarters in Rockville, MD using established protocols designed to secure information and safeguard privacy and confidentiality. SAMHSA will store the original copies of each consent form for a period of seven years following the completion of this study. After this period has expired, consent forms will be destroyed in a secure manner.

Data collected as part of this project will include the Recovery Measurement instrument (Attachment 2). With few exceptions, SAMHSA requires that GPRA interviews be administered verbally to participating individuals. In order to assess the usability of this instrument as a potential addition to SAMHSAs required performance measurement set (GPRA) we are requiring that participating sites also administer the Recovery Measurement tool in the same format.

Participant responses will be recorded by project personnel on a hard copy of the Recovery Instrument. As seen in Attachment 2, the proposed data collection instrument contains no personally identifiable information; no demographic data will be collected as part of this pilot study.

Existing SAMHSA-required GPRA data collection forms that are routinely collected by grantee sites do include a unique identifier. SAMHSA plans to utilize the same unique identifier for both the required GPRA interview and the Recovery Measurement instrument. SAMHSA will then be able to match each participant’s responses to the Recovery Measurement questions to responses to all other GPRA questions. The plan to match this information to other de-identified GPRA data is directly addressed in our proposed Consent Form.

Each grantee site will be responsible for creating, maintaining and securing keys that match participants to the unique GPRA IDs. These keys will be stored and secured using existing site protocols for the protection of PHI, including the storage of the key in a separate location from the data. SAMHSA will not have access to the keys at any time.

All data will be collected at SAMHSA grantee sites. If SAMHSA is able to update its web-based based data entry and reporting system to include the proposed Recovery Measure items prior to the start of this project, participating grantees will be asked to enter data directly into the online tool as part of their normal GPRA reporting process.

SAMHSA’s existing data entry and reporting tools are HIPAA-compliant and utilize unique identifiers (GPRA ID) that will be used to identify participants in this project. If any other paper or computer-based records are created as part of this process, grantee sites will be instructed to store these records using existing protocols for the storage and protection of sensitive information.

If SAMHSA is unable to update its online reporting tool prior to the start of this project, participating sites will be required to collect the information using paper and pencil methods. The proposed data collection instrument does not include any personally identifying information, instead utilizing unique alphanumeric codes (GPRA IDs). If paper versions of the instrument are used, grantee sites will be instructed to store an original copy of the data using existing data storage protocols, and to mail a photocopy of the data collection instrument to SAMHSA. Once at SAMHSA, the data will be entered into a secure database on our internal servers. Hard copies of all data collection instruments will be stored in locked file cabinets to which only the PI has access. For reporting purposes, data will be aggregated and summarized so that no personally identifying information will be communicated.

B.3. Methods to Maximize Response Rates

In an effort to maximize response rates, as well as decrease rates of attrition, data collected as part of this project will be collected immediately following each participant’s normally scheduled GPRA interviews. SAMHSA requires all grantees to collect baseline GPRA data at intake, and follow-up data collection at six-months following the initiation of SAMHSA-funded services. This pilot project will follow established protocol.

SAMHSA will be pilot testing this tool among three grantee programs. It is expected that 50 to 150 clients within each grantee program will agree to participate and at least 75 to 80 percent will complete the tool at intake and 6 months post-intake.

B.4. Tests of Procedures

This study is a pilot test of the Recovery Measurement instrument. This measure is comprised of questions from the World Health Organization’s Quality of Life tool (WHO QOL 8) and SAMHSA’s existing set of Government Performance and Results Act (GPRA) measures. Data will be collected at two time points – at client intake and at six months post-intake. These are two points in time during which SAMHSA grantees routinely collect data on the individuals participating in their programs. SAMHSA has submitted an Application for Human Subjects Review for this Recovery Measurement Pilot Study to the Catholic University, Office of Sponsored Programs and Research Services, Committee for the Protection of Human Subjects.

The WHO QOL-8 instrument has been field tested in Europe, but has not been previously used in the United States. The 8-item index, also known as the EUROHIS-QOL 8, was developed as an adaptation of the WHOQOL-100 and the WHOQOLBRE. An article based on a study of the EUROHIS-QOL 8 item index was published in 2005 (Attachment 5 -- Silke Schmidt, Holger Mu¨hlan, Mick Power; European perspectives: The EUROHIS-QOL 8 Item Index: Psychometric results of a cross-cultural field study. European Journal of Public Health (2005), Vol. 16, No. 4, 420–428) doi:10.1093/eurpub/cki155) . The aim of the study was to test the psychometric properties of the EUROHIS-QOL 8-item index. In a survey of 4849 European adults, the EUROHIS-QOL 8-item index was assessed across 10 countries, with equal samples adjusted for selected socio-demographic data. Participants were also investigated with a chronic condition checklist, measures on general health perception, mental health, health-care utilization and social support. Findings indicated good internal consistencies across a range of countries, showing acceptable convergent validity with physical and mental health measures; the measure discriminates well between individuals that report having a longstanding condition and healthy individuals across all countries. Differential item functioning was less frequently observed in those countries that were geographically and culturally closer to the UK, but acceptable across all countries. The study concluded that the 8-item index showed good cross-cultural field study performance and a satisfactory convergent and discriminant validity, and can therefore be recommended for use in public health research.

The GPRA items used as part of this study are established, reliable and valid measures used by SAMHSA to report performance related to dimensions that the Agency delineated as supporting a life in recovery. The measures seek to capture information related to overcoming or managing one’s disease(s) or symptoms; a stable and safe place to live; meaningful daily activities; and relationships and social networks that support friendship, love, and hope.

B.5. Statistical Consultants

Co-Principal Investigator and Consultant: Joseph Shields, PhD

Associate Professor, Department of Social Work; Catholic University

Contractor, SAMHSA

Daytime Phone: 202-319-5474

Email: Shields@cua.edu

Co-Principal Investigator: Alyson Essex, PhD, MHS

Social Science Analyst, Center for Behavioral Health Statistics and Quality (CBHSQ), Substance Abuse and Mental Health Services Administration (SAMHSA)

Address: 1 Choke Cherry Road, Room 2-1002, Rockville, MD 20857

Daytime Phone: 240-276-0529

Email: Alyson.Essex@SAMHSA.HHS.Gov

ATTACHMENTS

Attachment 1: Certification of Institutional Board Review Approval from Catholic University

Attachment 2: Recovery Measurement Instrument

Attachment 3: Sample Grantee Invitation Letter

Attachment 4: Voluntary Informed Consent Document

Attachment 5: Published Article Documenting the Psychometric Properties of the EUROHIS-QOL 8

5