Co-occur-SS-Questions Final 1a3

MEASURES OF CO-OCCURRING INFRASTRUCTURE (MCI)

SUPPORTING STATEMENT

A. JUSTIFICATION

1. Circumstances of Information Collection

The Substance Abuse Mental Health Services Administration’s (SAMHSA) Center for Mental Health Services (CMHS) and the Center for Substance Abuse Treatment (CSAT) request approval from the Office of Management and Budget (OMB) for a revision of the data collection activities for the Measures of Co-occurring Infrastructure (MCI) for States receiving Co-Occurring State Infrastructure Grants (COSIG) (OMB No. 0930-0284), that expires on July 31, 2008. The only change is in burden hours due to an increase in the number of providers participating in the States. There are no changes to the two previously approved data collection tools: Screening, Assessment, and Treatment instrument and the Policies on Screening, Assessment, Referral, and Treatment instrument. Implementation of services for persons with co-occurring mental and substance use disorders is a SAMHSA priority, and a central part of SAMHSA’s overall performance measurement efforts is the development of a set of measures specific to this population.

The COSIG program is authorized under Sections 509 and 520 A of the Public Health Services act, as amended.

The Government Performance and Results Act (GPRA) of 1993 required Federal agencies to identify the goals of all funded programs and required reports on the program’s success in attaining those goals. Section 1115 of the Act, Performance Plans, states: “In carrying out the provisions of section 1105(a)(29), each agency is required to prepare an annual performance plan covering each program activity. Such a plan shall:

 “Establish performance goals to define the level of performance to be achieved by a program activity;

 “Express such goals in an objective, quantifiable, and measurable form unless authorized to be in an alternative form under subsection (b);

 “Briefly describe the operational processes, skills and technology, and the human, capital, information, or other resources required to meet the performance goals;

 “Establish performance indicators to be used in measuring or assessing the relevant outputs, service levels, and outcomes of each program activity;

 “Provide a basis for comparing actual program results with the established performance goals; and

 “Describe the means to be used to verify and validate measured values.”

Furthermore, in its reauthorization of the Substance Abuse and Mental Health Services Administration (SAMHSA) under the Children’s Health Act of 2000 (PL 106 310), Congress called on SAMHSA to collaborate with the States and other interested stakeholders to develop a plan “for creating more flexibility for States and accountability based on outcome and other performance measures.” Performance and outcome measures will reduce State and community reporting requirements while simultaneously presenting useful and reliable information to SAMHSA, Congress, and to other key stakeholders about the effectiveness of SAMHSA’s services and how the services are being applied across the country. Specifically, MCI will include performance measures to assess the extent to which SAMHSA’s COSIG grantees are providing screening, assessment, and treatment services to clients entering substance abuse and mental health facilities.

History of the Proposed Project

Section 3403 of the Public Law 106-310 – Children’s Health Act of 2000 (CHA) contained the requirements that SAMHSA: (1) change the Block Grants into performance-based systems, and (2) submit to Congress within two years a plan for what these performance based programs would look like and how they would operate. This plan would describe how the States would receive greater flexibility, what performance measures would be used to hold States accountable including the requirement that measures be developed for some specific populations including co-occurring populations, definitions for the data elements to be collected, the funds needed to implement this system, where these funds would come from, and needed legislative changes. In partnership with the National Association of State Alcohol and Drug Abuse Directors (NASADAD) and National Association of State Mental Health Program Directors (NASMHPD), SAMHSA began to develop an approach to measurement of co-occurring services infrastructure. In November 2002, a Technical Workgroup comprising representatives from NASMHPD, NASADAD, CSAT, and CMHS developed three performance concepts to support SAMHSA’s emergent activities related to co-occurring mental health and substance abuse disorders. The specific measures proposed for MCI derived from this workgroup and were developed with extensive input from the first cohort of seven COSIG States. The Technical Workgroup’s efforts to develop performance measures occurred in parallel with the development of COSIG, a SAMHSA discretionary program that awards grants to States to support activities to reduce barriers to care, and improve treatment services for persons suffering from co-occurring disorders. The COSIG program implemented principles established in SAMHSA’s 2002 Report to Congress on Co-Occurring Disorders, a report required by section 3406 of CHA. In that report, SAMHSA established the following performance measures for services to persons with co-occurring disorders.

 Increase the number of persons with co-occurring disorders served

 Increase the percentage of treatment programs that

a) Screen for co-occurring disorders;

b) Assess for co-occurring disorders; and

c) Treat co-occurring disorders through collaborative, consultative, and integrated models of care.

The COSIG Request for Applications (RFA), released in 2003, 2004, 2005, and 2006, emphasized these goals and SAMHSA’s focus on appropriate screening, assessment, and coordinated treatment for persons with co-occurring disorders. The measurement approach developed through the processes described will enable SAMHSA to assess the practices and policies of providers of mental health and substance abuse services for screening and assessing clients for co-occurring disorders, and the providers’ approach to service delivery for persons with co-occurring disorders.

2. Purpose and Use of Information

A major focus of SAMHSA’s COSIG program is increasing the number of substance abuse and mental health treatment programs that screen and assess for co-occurring disorders, and provide appropriate treatment for persons found to have such disorders. The proposed measures will enable SAMHSA to monitor program performance by COSIG grantees. To implement the performance measures, SAMHSA developed a set of instruments to collect data on providers’ practices and policy for screening, assessment, and treatment of co-occurring disorders. Data will be collected from participating treatment providers within 16 of the 19 current COSIG States (i.e., Alaska, Arizona, Arkansas, Connecticut, Delaware, the District of Columbia, Louisiana, Maine, Minnesota, Missouri, New México, Oklahoma, South Carolina, South Dakota, Texas, and Vermont. ), and in States receiving future COSIG awards. The additional COSIG states of Hawaii, Pennsylvania, and Virginia are not required to collect COSIG data.

The measures are not part of SAMHSA’s National Outcomes Measures (NOMs). As NOMs are adopted by SAMHSA, COSIG grantees will also report client-level outcome data for persons served in COSIG provider sites.

The instruments contain two components:

 Domain 1: Screening, Assessment, and Treatment: Assesses provider practices for screening, assessment, and treatment of clients with co-occurring substance abuse and mental disorders;

 Domain 2: Facility Policy on Screening, Assessment, Referral, and Treatment.

3. Use of Information Technology

Data for both Domains 1 and 2 will be collected at the level of the treatment facility. COSIG States are allowed to involve any number of treatment facilities within the State in their COSIG program. Only these facilities will collect and report data for the domains. Identified facilities will report to the State aggregate numbers of screening, assessment, and treatment practices for that facility during the reporting period, and the facilities policies on screening, assessment, and treatment. States will collect the information from the providers and report data to SAMHSA.

Domain 1 (number of persons screened and assessed, and types of treatment perceived by persons with co-occurring disorders) requires a treatment provider to track services for individual clients. The provider must record whether each client was screened and assessed during a defined period following admission, and the treatment disposition for those clients found to have co-occurring disorders. Some COSIG States are able to report all required data from existing automated State substance abuse and mental health database systems to extract the requested data. Other States are modifying their systems to simplify reporting and some are collecting data manually. Most providers in most States will need to implement tracking procedures to obtain information necessary to complete the MCI questionnaire.

Domain 2 requires only yes/no statements about provider policy and can readily be completed by providers.

The monitoring tools will be available in Word and Excel for respondents who wish to use one of these formats.

4. Efforts to Identify Duplication

The Measures of Co-Occurring Infrastructure are unique in that SAMHSA has no other system for gathering this information. Domain 1 counts the number of clients/patients screened, assessed, and treated for co-occurring disorders. No other SAMHSA data collection program is able to identify information about the performance of State and local facilities in screening, assessing, and treating co-occurring disorders or about the number of clients/patients with co-occurring disorders screened, assessed, and treated. Domain 2 obtains information from treatment facilities on their policies on screening, assessment, referral, and treatment.

5. Involvement of Small Entities

The data collection instrument has been designed by consensus among the COSIG States to minimize appropriate response burden. Small businesses are not significantly impacted by the requirements.

6. Consequences If Information Collected Less Frequently

The proposed data collection effort began in 2007 and is a current and ongoing effort that provides SAMHSA with data necessary to monitor performance of COSIG States in fulfilling SAMHSA’s goal of increasing the number of providers that screen and assess for co-occurring disorders, and that provide appropriate treatment for persons with co-occurring disorders.

Data collection will be ongoing at the provider level, with annual reporting from COSIG States to SAMHSA.

7. Consistency With the Guidelines in 5 CFR 1320.5(d)(2)

This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2).

8. Consultation Outside the Agency

In addition, the following individuals developed or reviewed and commented on the development of the measures:

Ron Kessler, Ph.D. Professor of Healthcare Policy 617 432-3587

Harvard Medical School, Boston, MA

Fred Osher, M.D . Center for Behavioral Health, Justice, and 410 646-3511

Public Policy, University of Maryland

Medical School, Baltimore, MD

Stanley Sacks, Ph.D. Director, Center for Integration of 212 845-4429

Research and Practice, National

Development & Research Institutes,

New York, NY

Sam Schildhaus, Ph.D. Formerly with National Opinion Research 202 296-1625

Center at the University of Chicago (NORC)

Yoku Shaw-Taylor, Ph.D. Research Scientist, NORC 202 429-1954

A Federal Register notice was published on January 14, 2008 (vol. 73 no. 9, p. 2267-2268) soliciting public comments on the information collection. No comments were received.

9. Payment to Respondents

Respondents will not receive any payment or gifts.

10. Assurance of Confidentiality

Data collection will be at the treatment facility level, and addresses provider-level policy and practice. Domain 2 applies only to provider policy. Domain one addresses provider services delivered to clients. No information will be collected from clients or about clients other than services received. Information will be aggregated at the provider level and forwarded to States with no client-identifying information, and no client-identifying information will be sent to SAMHSA.

Legal Protections:

The data collection will conform to all requirements of the Privacy Act of 1974, under the System of Records: Alcohol, Drug Abuse and Mental Health Epidemiologic and Biometric Research Data, HHS/ADAMHA/OA, #09-30-0036.

The 42 Code of Federal Regulations, Part 2, applies to all drug and alcohol treatment programs. It makes all records and data confidential. Such data cannot be released except:

• With patient consent

• To medical personnel in a medical emergency

• To a court in compliance with a court order

• For research or audit

It is pursuant to these regulations that interviewers become subject to fines up to $500 for the first violation of confidentiality and fines of up to $5,000 for each subsequent offence (2.1 Sec 408(f), 2.2 Sec 33(f)).

11. Questions of a Sensitive Nature

The collection of information about provider service practices is essential to the implementation of an effective monitoring system for COSIG grants. There are no questions about sensitive individual behavior, only questions about the process and structure of treatment. Procedures described above will ensure protection of all data and of individual rights.

12. Estimates of Annualized Hour Burden

SAMHSA will collect information from 16 providers of 19 participating in their States’ COSIG programs. Additional COSIG States and providers may be added in future years.

Data collection instruments are included in Attachment A.

The instruments were designed to collect the necessary information with as minimal burden to States and respondents as possible. An estimate of the annual collection burden on participants is presented in the table below. SAMHSA bases this estimate partially on information provided by potential respondents.

Estimate of Annual Cost Burden to Respondents

Data	Number of	Hours	Total	Hour	Total
Collection	Respondents	Per	Burden	Wage	Hour
		Response	Hours		Cost
Capacity to
Screen,	298	4.5 hours	1,341	$14.32	$19,203
Assess and
Treat
Measure 2:	298	3 minutes	15	$14.32	$213
Policy on Screening, Assessment, Referral, and Treatment
Total	298		1,356		$19,416

Total Annual Burden Hours: 1,356

13. Estimates of Annualized Cost Burden to Respondents

There are no capital and/or maintenance costs to respondents.

14. Estimates of Annualized Cost to the Government

The total cost to the government for activities directly related to this data collection is estimated to be $19,416.

15. Changes in Burden

Currently, there are 1,101 total burden hours. The Program is requesting 1,356. The increase of 255 hours is due to an overall increase in the number of providers participating in the Program. The number of participating States has decreased.

16. Time Schedule, Publication and Analysis Plans

This section contains plans for the study, including the time schedule, discussion of reports, and analysis planned

Time Schedule

The following schedule is expected:

TASKS	APPROXIMATE COMPLETION
OMB clearance anticipated	August 2008
Pre-collection meeting with COSIG representatives	One month after OMB approval
Start data collection	Two months after OMB approval
States submit data to SAMHSA	12 months after OMB approval
SAMHSA compiles State data	13 months after OMB approval
SAMHSA prepares report	14 months after OMB approval

Reports

Within 2 months of State data submission, SAMHSA will compile and analyze data submitted by States, and will produce an annual report addressing each of the key issues.

Analysis

SAMHSA will analyze the utility and completeness of the data collected and then will prepare a final report summarizing the results. SAMHSA will also analyze the data collected to develop a baseline to assess changes over time. The essential analysis will enable reporting on the following:

1. Percentage of clients screened for co-occurring disorders;

2. Percentage screened who were ‘positive;’

3. Percentage clients with positive screens who were assessed;

4. Percentage assessed determined to have co-occurring disorders; and

5. Treatment model for persons with co-occurring disorders.

These figures will be available at the facility level, at the State level, and at the COSIG program level. Because primary data will be submitted at the facility level, the data will enable SAMHSA to identify the number of treatment programs that screen, assess, and provide appropriate treatment for persons with co-occurring disorders. Because each provider will report the number of persons identified as having co-occurring disorders and their treatment dispositions, SAMHSA will be able to identify the number of persons with such disorders that receive treatment. And because identical data will be obtained annually, SAMHSA will be able to track changes in all of these measures over time.

Useful additional comparisons include mental health facilities, substance abuse facilities, and facilities that provide both mental health and substance abuse services.

One manner to test data validation is to compare results, as feasible, to other existing data sets. One comparison of the performance measures with other treatment data will be conducted. The National Survey of Substance Abuse Treatment Services (N-SSATS) does allow comparison of the results of the study with the percentage of facilities that report the provision of assessment services. The analysis will be supplemented by a test of the utility of the monitoring tools by speaking with appropriate State and Federal staff to ensure that they meet the policy needs of the States, CSAT, and SAMHSA. Finally, SAMHSA can select a sample of reporting facilities and conduct an onsite test of external validity through an audit of cases.

17. Display of Expiration Date

The expiration date will be displayed.

18. Exceptions to Certification Statement

These activities will comply with requirements in 5 CFR 1320.9. Certificates are included in this package.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

To minimize response burden, SAMHSA will allow COSIG states to employ sampling procedures for both providers within a state and clients within a provider.

The COSIG program consists of 19 States, 16 of which will collect COSIG data: Alaska, Arizona, Arkansas, Connecticut, Delaware, the District of Columbia, Louisiana, Maine, Minnesota, Missouri, New Mexico, Oklahoma, South Carolina, South Dakota, Texas, and Vermont. Three COSIG States, Hawaii, Pennsylvania, and Virginia are not required to report data. The participating States have included 298 providers in their COSIG program, and these providers serve, in the aggregate, over 253,946 clients annually. SAMHSA may add new COSIG States in future years, but this would not increase the size of provider and client universes since 5 of the 16 states grants currently collecting data end in September 2008. Table 1 presents information about facilities and clients within the 16 current COSIG data collecting states.

Table 1: Providers and Clients within the 16 COSIG States.

State	Number of COSIG-affiliated facilities	Number of facilities with more than 30 clients in year	Number of facilities with more than 20 clients in year	Number of facilities with more than 20 clients admitted in last 30 days	Total clients per year
Alaska1	74	49	52	N/A	8,764
Arizona	3	0	2	0	42
Arkansas2	30	30	30	29	167,105
Connecticut	7	6	7	3	2,463
Delaware	15	14	15	11	6,092
District of Columbia	22	21	22	1	2,157
Louisiana2	19	19	19	17	12,470
Maine3	20	10	20	10	4,478
Minnesota	20	17	19	10	13,114
Missouri1, 2	10	8	9	7	7,227
New Mexico2	3	3	3	0	195
Oklahoma1,2	15	15	15	12	16,542
South Carolina	3	3	3	0	325
South Dakota	32	22	22	6	4,665
Texas2	6	5	6	0	363
Vermont	19	19	19	13	7,944
TOTAL	298	246	268	119	253,946
1 These States did not provide data for the current estimates. Numbers shown represent 2006 estimates. 2 These 5 grants will not collect data after September, 2008. 3 Includes only those Maine providers continuing data collection beyond June, 2008.

The sampling strategy for both providers and for clients is the same. SAMHSA will randomly select a sample large enough to produce a 95% confidence interval with a margin of error of +/- 5%. Table 2 presents the sample size required for various size universes to achieve this criterion.

Table 2: Sample Size Table (Assuming a proportion=50% and desired 95% Confidence Interval of +/-5%)¹

Universe	Sample	Universe	Sample	Universe	Sample
10	10	125	94	1,250	294
20	19	150	108	1,500	306
25	23	200	132	2,000	322
30	28	250	151	3,000	340
40	36	300	168	4,000	350
50	44	400	196	5,000	357
60	52	500	217	7,500	365
70	59	600	234	10,000	370
80	66	800	259	50,000	381
100	79	1,000	277	100,000	383

For instance, for a universe of 100, a random sample of 79 is sufficient for a 95% confidence interval +/- 5%.

Sampling Facilities. Step one of sampling will be selection of facilities. As seen in Table 1, the current universe of COSIG-affiliated programs is 298 distributed among 16 States. Thirteen of the 16 States have fewer than 30 treatment facilities. As shown in Table 2, few facilities would be randomly deselected in states with 30 or fewer facilities, so in those states, all facilities will participate in data collection. In states with more than 30 facilities, SAMHSA will determine the exact number of facilities that must participate to reach the 95% confidence criterion, and randomly select from all facilities within the state those that must participate. This procedure will yield a total sample size of providers across the current 16 COSIG states, which is the number used in the burden table. States have the option of requiring all providers to participate. [None of the providers in the current COSIG data collection States serve fewer than 20 clients annually so SAMHSA will not need to exclude any small providers to avoid unstable counts at the facility level.]

Sampling Clients. Based on historical data, providers will estimate the number of clients to be served during the 12-month reporting period. Using Table 2, SAMHSA will provide to each provider the minimum number of clients that must be tracked for reporting purposes. Providers must collect appropriate data until the minimum number is reached. States may set higher requirements.

2. Information Collection Procedures

There are two domains:

• Domain 1: Screening, Assessment, and Treatment: Assesses provider practices for screening, assessment, and treatment of clients with co-occurring substance abuse and mental disorders;

• Domain 2: Facility Policy on Screening, Assessment, Referral, and Treatment.

Domain 1 requires a treatment provider to track services for individual clients. The provider must record whether each client was screened and assessed following admission, and the treatment disposition for those clients found to have co-occurring disorders. Some COSIG States are likely to use existing automated State substance abuse and mental health database systems to extract the requested data. Most providers in most States will need to implement tracking procedures to obtain information necessary to complete the MCI questionnaire. Please note that the monitoring tools attached to this document are not client tracking forms, but a format for reporting aggregate information about screening, assessment, and treatment practices within a provider during a defined time period.

Domain 2 requires only yes/no statements about provider policy and can be readily completed by providers.

Each participating State will be asked to compile specific information regarding the two co-occurring measurement domains. Of the 16 grants, collection is expected to continue for 11 grants and, for the two new grants, collection will begin in the first quarter of 2008. Collection will continue through the duration of the COSIG program. However, five participating States will not collect data beyond September, 2008 (Arkansas, Louisiana, Missouri, New Mexico, and Texas).

The data collection forms for this study have been designed via consensus by seven participating COSIG States, four other States, and SAMHSA.

3. Methods to Maximize Response Rates

The data collection instruments have been developed to minimize the response burden and to increase the likelihood of response by the treatment facilities. Sampling procedures will further reduce burden on respondent providers. The State governments are required to provide this information as a term of the COSIG grant program, and will be responsible for requiring their participating COSIG providers to engage in the collection and reporting of data.

4. Tests of Procedures

The original instrument was developed by a Task Force comprised comprising the Center for Mental Health Services (CMHS), the Center for Substance Abuse Treatment (CSAT), the National Association of State Alcohol and Drug Abuse Directors (NASADAD), and the National Association of State Mental Health Program Directors (NASMHPD). A workgroup with representatives from 11 States helped develop the data collection instruments and the procedures.

5. Statistical Consultants (Provided initial consultation)

The material has been reviewed by:

Sadeq Chowdhury, Ph.D. Senior Statistician 202 223-1637

National Opinion Research Center

at the University of Chicago (NORC)

James M. Herrell, Ph.D., M.P.H. Social Science Analyst 240 276-2789

Center for Substance Abuse Treatment (CSAT)

Susan Hayashi, Ph.D. Vice President 240 645-4588

JBS International

Charlene Le Fauve, Ph.D. Chief, Co-Occurring and Homeless 240 276-2787

Activities Branch, CSAT

Lawrence Rickards, Ph.D. Chief, Homeless Programs 240-276-1985

Branch, CMHS

Sam Schildhaus, Ph.D. Formerly with National Opinion Research 202 296-1625

Center at the University of Chicago (NORC)

Yoku Shaw-Taylor, Ph.D. Research Scientist, NORC 202 429-1954

LIST OF ATTACHMENTS

A. Instruments

1 The formula for determining sample can be presented as:

n*=z²*p(1-p)/d² and n = n*/(1+N/n*) with finite population correction, where:

n* = sample size when population size is large,

n = sample size adjusted for small population size,

N = population size,

p = target proportion to be estimated,

z = Normal distribution cutoff for a 95% confidence interval

d = desired half-width of 95% confidence interval

14