OMB Supporting Statement Part A Full Pkg 9 23 11_V4

Evaluation of the Substance Abuse and Mental Health Administration

Primary Care Behavioral Health Integration Grant Program

OMB Supporting Statement

A. Justification

1. Need for Information

This submission is a revision to currently approved collection #0990-0371, an Emergency Clearance request covering the first six months of data collection starting February 15, 2011 and ending August 14, 2011. This submission will cover data collection for the period starting August 15, 2011 and ending October 1, 2013.

This evaluation is being funded by the Assistant Secretary for Planning and Evaluation (ASPE) and the Substance Abuse and Mental Health Services Administration of the U.S. Department of Health and Human Services and conducted for the Substance Abuse and Mental Health Services Administration/Center for Mental Health Services’ (SAMHSA/CMHS) Primary Care Behavioral Health Integration (PBHCI) grant program. The PBHCI grants are covered under the requirements of P.L. 103-62, the Government Performance and Results Act (GPRA) of 1993; Title 38, section 527, Evaluation and Data Collection; and 38 CFR section 1.15, Standards for Program Evaluation. The GPRA requires Federal government agencies to evaluate their performance on a regular basis, and the analyses will include items reported for GPRA.

Four-year PBHCI grants were awarded to thirteen grantees on October 1, 2009. A second group of nine grants and a third group of 34 additional grants were awarded October 1, 2010. On September 23, 2009, ASPE awarded task order no. OS 11025 to RAND Corporation to design an evaluation of the PBHCI program and on October 5, 2010 ASPE awarded task order no OS 42345 to RAND Corporation to conduct the evaluation. The total evaluation will take place over thirty-six months, beginning October 1, 2010 and ending October 1, 2013.

2. Use of Information

The purpose of the PBHCI grants is to improve the overall wellness and physical health status of people with serious mental illnesses (SMI), including individuals with co-occurring substance use disorders, by supporting communities to coordinate and integrate primary care services into publicly-funded community mental health and other community-based behavioral health settings. The information collected through the evaluation will assist SAMHSA in assessing whether integrated primary care services produce improvements in the physical and mental health of the SMI population receiving services from community-based behavioral health agencies.

To achieve this program evaluation goal, ASPE has contracted with RAND to conduct an evaluation that will answer the following three research questions (RQs):

• RQ1: Is it possible to integrate the services provided by primary care providers and community-based behavioral health agencies? In answering this question, we will address several process evaluation issues related to integrated care. Specifically, using program-level data, we will assess program penetration of services, evolution of implementation plans, and reported and actual progress toward implementation goals, indications of quality of care at the program level, and sustainability of the processes that are implemented. Using client-level data, we will assess the extent to which programs delivered specific services, as well as the quality of services delivered to the target population.

• RQ2: Does the integration of primary and behavioral health care lead to improvements in the mental and physical health of the population with SMI and/or substance use disorders served by these integrated models?

• RQ3: Which models (and respective model features) of integrated primary and behavioral health care lead to better mental and physical health outcomes for the population served?

The project team has conceptualized the evaluation in terms of the Donabedian (1980) quality of care model, which includes the components of structure, process, and outcomes of care, and has aligned the research questions with these three components of this model. The team has also developed an optimally efficient, cost-effective data collection methodology for each component of the model, and each related research question, that utilizes data currently collected by the PBHCI grantees to the fullest extent possible and augments the existing data with additional collection only as needed.

For purposes of the independent evaluation, there will be two cohorts of sites, those funded in 2009 (13) and those funded in 2010 (43). Data for both groups will be collected for a year, after the first full year of each grantee’s project implementation, however, a subset of the data will be submitted to SAMHSA throughout the life of the grant for ongoing performance measurement and monitoring.

In order to answer the research questions, ASPE is seeking Office of Management and Budget (OMB) approval for data collection utilizing six instruments:

1. Site Visit Interviews for leadership, care coordinators, and primary care and mental health providers at PBHCI sites and key staff at control sites

2. Web-based Survey for key staff at each grantee site

3. Client Physical Exam and Survey

4. Individual Service Utilization Data

5. Quarterly Reports from grantees

6. Physical Health Indicators to be reported through TRAC

Below, we describe each data collection instrument, as well as how it will be used for this evaluation, in detail. Table 1 provides additional detail about how the content areas in each data collection instrument will be used to answer the evaluation’s key research questions.

Site Visit Interviews

The site visit interviews at the PBHCI grantee sites will enable the evaluators to gain an in-depth understanding of how a select number of programs have implemented the primary care-behavioral health integration program, both globally and with respect to specific required features of the program, such as screening/referral, registry/tracking, care management and wellness/prevention. It will provide an opportunity to learn about barriers and facilitators to implementation of the program. Site visits will also enable the evaluators to directly observe how the sites are implementing various features of the program. The site visits to the matched control sites will provide the evaluators with information about their services and the extent to which they have program features similar to the intervention sites such as registries, care management and wellness services.

The program leadership interview will be administered to 2-4 administrators, which may include program managers, medical directors, CFOs, key administrators, and evaluators/data managers for PBHCI, at each grantee site. The care coordinator interview will be administered to 1-2 care coordinators at each site. The primary care provider interview will be administered to 1-2 physicians and 1-2 nurse practitioners or physician assistants (2-4 primary care providers total) at each site and may include 1-2 wellness educators where available. The mental health provider interview will be administered to 2-4 psychiatrists, psychologists, social workers, case managers and/or peer specialists at each site. The control site interview will be administered to 4-6 key staff at the control sites, which may include administrators, care coordinators, and/or care providers.

The site visits will be performed during year 3 of the evaluation.

Web-Based Survey

The web-based survey of PBHCI staff will enable the evaluators to gain a better understanding of how the various grantees have implemented the PBHCI program, both globally and with respect to specific required features of the program, such as screening/referral, registry/tracking, care management and wellness/prevention. The web-based survey is a key component of the PBHCI evaluation because it allows the evaluators to obtain important information about program implementation in an efficient, cost-effective way from all participating sites. Site visits will yield more thorough and detailed information, however site visits to all grantee sites would be time and cost-prohibitive. Web-based surveys are an efficient method for obtaining program implementation information from informants at all grantee sites.

At each grantee site, the survey will be administered to 2-4 administrators, which may include program managers, medical directors, CFOs, key administrators, and evaluators/data managers for PBHCI; 1-3 care coordinators; 1-2 physicians and 1-2 nurse practitioners or physician assistants (2-4 providers total); 1-2 wellness educators where available; and 2-4 psychiatrists, psychologists, social workers, case managers and/or peer specialists.

The web-based surveys will be administered during year 3 of the evaluation.

Client Physical Exam and Survey

We propose a client-level physical exam and survey so that the evaluator can assess the impact of participating in the PBHCI program on physical health outcomes (BMI, HgBA1c or blood glucose, blood pressure, triglycerides, cholesterol, waist circumference, and breath carbon monoxide) and other indicators of subjective well-being (daily functioning, substance use, social support, housing, employment, perception of care, service utilization, diet, perception of care, medications and side effects) All of these data elements will be primarily used to answer RQ2. The client-level physical exam and survey also includes demographic information so that the evaluator can assess disparities in physical health and well-being among different groups of PBHCI clientele.

A total of n=5500 unique patients from 10 PBHCI grantee intervention sites and 10 non-PBHCI matched-control sites will complete the client-level physical exam and survey. Patients from the control sites will complete the physical exam and survey on 2 occasions: at baseline and again at one year follow-up. There is no other source of physical health indicators information from the control sites. Patients from the intervention sites will only complete the physical exam and survey at one year follow-up. Baseline data for patients at intervention sites will come from physical exams and surveys completed as part of the PBHCI program. Importantly, physical exams that are part of the PBHCI program will include all of the same data elements as those administered at one year follow-up. Similarly, surveys completed as part of the PBHCI program have several identical data elements in areas of focus for the evaluation (e.g., demographics, daily functioning, substance use, housing, education, employment, perception of care, social connectedness, service utilization, etc.) While baseline data for the participants at the intervention sites can be extracted from clinical registries and medical records, patients from the intervention sites will complete the contractor-administered physical exam and survey at 1 year follow-up to ensure acceptable longitudinal retention of the baseline sample, and timely, standardized, unbiased collection of physical health and well-being data.

The evaluator will work with the control sites and the contractor to enroll n=3000 control patients (300 at each of 10 sites) to complete the baseline administration of the client physical exam and survey, expecting that n=2500 patients (accounting for 15% attrition) will complete the exam and survey at 1 year follow-up.

Individual Service Utilization Data

The purpose of collecting individual service utilization data from grantee’s clinical registries or other similar data systems is to quantify the type and amount of clinical services received by each client so that the evaluator can link individual service utilization to indicators of physical health and well-being. Data will be extracted from clinical registries and/or medical records (depending on each sites’ data management system). Sites with electronic medical records and/or registries should be able automate the process of report generation, which will substantially reduce the burden associated with this task. Grantees receive technical assistance with their clinical registries from SAMHSA’s Center for Integrated Health Services (CIHS).

Grantees will generate reports for all active PBHCI clients during the evaluation period. Individual service utilization data will be due at the same time as quarterly reports, as the two reports are intended to be complementary; the individual service utilization data includes a quantitative summary of each site’s clinical activities and the quarterly report includes a qualitative summary of a site’s clinical and managerial activities. Individual service utilization data (and quarterly reports) will be submitted once every three months during the 2^nd year of each grantee’s PBHCI clinical activities for the independent evaluation but will be submitted to SAMHSA throughout the life of the grant for ongoing performance measurement and monitoring purposes.

Quarterly Reports

Quarterly reports will allow the evaluator to assess how all the sites are implementing primary care-behavioral health integration on an ongoing basis. In particular, it will allow the investigators to assess key accomplishments and barriers, staffing changes, infrastructure activities, and implementation of specific program components. It will help the evaluators understand which patients are deemed eligible for the program and how funding is being used to support the program.

Quarterly reports will be completed by all grantees and will be submitted to the evaluator during year 2 of program implementation for the independent evaluation but will be submitted to SAMHSA throughout the life of the grant for ongoing performance measurement and monitoring purposes.

Physical Health Indicators for TRAC

We propose that 6 required and 2 optional physical health indicators be added to the TRansformation ACcountability (TRAC) system for use by the PBHCI grantees. TRAC is the web-based system through which all grants funded by the SAMHSA Center for Mental Health Services (CMHS) report performance measurement data (OMB Approval No. 0930-0285). Through TRAC, each grantee is required to collect and report data on behavioral health outcomes for each person receiving services as a result of the grant, using a standard protocol. These data are collected by grantee staff members who interview each service participant at initial entry into the program, every six months while enrolled in services, and when discharged from the program. In order to accommodate program-specific performance monitoring needs, the system allows individual grant programs to add a small number of OMB-approved data elements that are critical for assessing core outcomes for the program. In addition to client-level data, grantees also report grant-level infrastructure changes through TRAC.

The required indicators to be added to TRAC for PBHCI include height, weight, HgBA1c or blood glucose, blood pressure, triglycerides, and cholesterol, which are biomarkers for obesity, diabetes, hypertension, hyperlipidemia, and hypercholesterolemia, respectively. The optional indicators include waist circumference and breath carbon monoxide which are indicators of metabolic syndrome and smoking status, respectively. We also propose the addition of an optional question to determine whether an 8 hour fast occurred prior to the blood tests.

The evaluator will use the physical health indicators entered into TRAC to determine whether physical health care can be integrated into behavioral health care (RQ1) and to test for improvement in physical health from before to after enrollment in PBHCI services (RQ2). The evaluator will also use these data to test which models (and respective model features) of the PBHCI programs lead to better physical health outcomes for the population served (RQ3).

Reporting of physical health indicators through TRAC will facilitate standardized reporting and consolidation of the physical health data from all 56 grantees. Physical health data will be extracted from grantees’ clinical registries and/or electronic medical records and entered into the TRAC system every 6 months. This interval coincides with the TRAC-required bi-annual client-level interviews so that all TRAC data can be entered simultaneously and grantees’ data management burden can be minimized. The independent evaluation will utilize data submitted during year two of each grantee’s implementation. However, SAMHSA will continue to collect the TRAC data throughout the life of the grant as part of its ongoing performance measurement and monitoring.

Table 1: PBHCI Evaluation Data Sources

Data Source	Level of Observation	Research Question	Content Areas	Analysis
Grantee Quarterly Reports	Program	RQ1, RQ3	Program accomplishments, staffing changes, consumer involvement, barriers, infrastructure activities, wellness programming, data collection, grant-funded programming, sustainability, eligible patients, contact w/ SAMHSA, alternate funding sources	Use coded data to identify challenges/barriers to integrating services, strategies to overcome barriers, and to categorize sites according to models and features of integrated care
Individual Service Utilization Data	Program and Consumer	RQ1, RQ3	Physical health services, mental health services, substance abuse services, wellness services, provider contacts	Descriptive statistics for the number of individuals using categories of PBHCI services at each site and process of care indicators. Regression analyses to examine relationships between program-level structure/model features and client-level process of care indicators in order to identify model features associated with rates of appropriate care.
CMHS TRAC	Program and Consumer	RQ1, RQ2	Exam (primary and secondary indicators); Client interview (demographics, functioning, stability in housing, education and employment, crime, perception of care, social connectedness, services received) Program (policy development, workforce development, finances, organizational change, partnership/collaborations, accountability, types/targets of practice, awareness, training, knowledge/attitudes/beliefs, screening)	Descriptive statistics for individuals served by each program. Intervention sites will use this data for baseline measures of physical health indicators.
Physical Health Assessment (exam and questionnaire)	Program and Consumer	RQ2, RQ3	Exam (primary and secondary indicators) Questionnaire (demographics, daily functioning, substance use, housing, education/employment/crime, social connectedness, service utilization, diet/nutrition, physical activity and fitness, physical health and healthcare, medications and side effects	Inferential statistics (ANCOVA or propensity score analysis) to compare individuals at intervention sites and matched control sites on changes in physical health indicators over time. Extend the individual-level difference-in-difference analysis of program outcome effects to include process of care indicators as predictors of outcomes.
Site Visit Interviews	Program	RQ1, RQ2, RQ3	Collaboration across MH/PC, program structure, screening and referral, registry/tracking, performance monitoring, care management, EBPs, wellness/prevention/ early intervention, self-management support, consumer involvement, electronic capabilities, cultural competency, implementation	Qualitative analyses and coding will identify themes in each site visit domain. Based on the themes identified, conceptual maps will be generated to characterize the way sites in general (as well as types of sites or individual sites) describe issues regarding the integration of care. Data will also be used to inform interpretation of quantitative analyses for RQ1.
Web-Based Survey	Program	RQ1, RQ3	Role and caseload, collaboration between MH/PC, structure, care management, wellness/prevention/ early intervention, other activities, screening/referral, registry/tracking, performance monitoring, EBPs, self-management support, consumer involvement, cultural competency, implementation	Quantitative data will be used for descriptive statistics to characterize sites, identify challenges/barriers to integrating services, strategies to overcome barriers, and to categorize sites according to models and features of integrated care.

3. Involvement of Information Technology

The addition of the physical health indicators to TRAC for the PBHCI grantees will enable SAMHSA and the evaluator to capture a standardized set of performance indicators using a uniform reporting method.

Quarterly reports and individual service utilization data will be submitted electronically to a secure SharePoint site in the form of Word and Excel (or other database) documents.

The project will use Web conferencing technology when appropriate to provide any necessary training on data collection. Support for the use of the TRAC system is provided by the host of the system. Extensive technical assistance and training for PBHCI grantees will also be provided by the National Council for Community Behavioral Healthcare which holds an award for this purpose.

4. Efforts to Identify Duplication

The site visit interviews, web-based survey, client physical exam and survey, individual service utilization data, quarterly reports and the additional TRAC physical health indicators to be used for the PBHCI evaluation are unique to this initiative, and the frequency of data collection has been reduced to a minimum. The information from these instruments is needed to determine the success of planning and implementation and the impact of the PBHCI activities on client outcomes.

In formulating the evaluation plan, we have carefully considered how to minimize burden and have included the following approaches to do so:

• Quarterly Reports/Site Visits/Web-based Survey: To the extent possible, information regarding grantee implementation issues will be gathered through review of the quarterly reports that grantees have been preparing and submitting to their SAMHSA Project Officer in the normal course of the grant program. The web-based survey will provide comparable data for analysis across all 56 sites. Site visit questions for the ten intervention sites will further supplement the information from the quarterly reports and surveys for a small subset of grantees and have been tailored to different stakeholders to minimize the time that grantee staff must spend in interviews. Site visits to the control sites will be the only source of information about their services and programs.

• Client Physical Exam/Survey: Given that the client physical exam and survey will be administered only once in the intervention sites and twice in the control sites, there is no feasible way to reduce the client burden by collecting data less frequently.

• Individual Service Utilization Data: This information should be readily available through the patient registries or electronic health records used for the PBHCI program and does not represent any duplication of effort.

• Physical Health Indicators reported through TRAC: While entering this information into TRAC may represent some duplication of effort since results will likely be entered into an electronic medical record, chart or registry, it is critical to collect this information in a standardized way that is also connected with the SAMHSA-required client-level TRAC data. The data will be entered concurrently with the entry of the client’s other TRAC data and is only expected to add up to 5 minutes of data entry time.

5. Impact on Small Business

Grantees vary in size from small entities through larger provider organizations. Every effort has been made to reduce the number of data items collected from grantees to the least number required in order to effectively evaluate the PBHCI program. Further, the use of an external contractor for the client physical exam and survey is intended to reduce the burden on the control and intervention sites. Finally, a number of the grantees have contracted with external evaluators who are assisting with data collection and reporting.

6. Consequences of Not Collecting the Information

If the data are not collected, the evaluator will not have adequate information to answer the three research questions. Inclusion of all planned data sources to yield information about structure, process, and outcomes is necessary to achieve a complete representation of quality of care. If this information were not collected, the evaluator would be unable to answer RQ2. Site visits will take place only once during the evaluation. If they were not completed, the evaluator would have inadequate information to assess the structure and processes of care in place to answer RQ1 and RQ3. The web-based survey will provide information about the structure and processes of care across all grantees but not with the level of detail afforded by site visits. Quarterly reports will provide important information regarding what is required to integrate services and the barriers faced by the grantees, information critical to addressing RQ1 and RQ3. The collection of Physical Health Indicators through TRAC will allow the evaluator to test which models (and respective model features) of the PBHCI programs lead to better physical health outcomes for the population served. The individual service utilization data will provide valuable information on individual service use that will be important in addressing the relationship between client service use and outcomes.

7. Special Circumstances

This project involves none of the special circumstances listed in the documentation.

8. Adherence to 5 CFR 1320.8(d) and Outside Consultations

The notice of proposed information collection activity was published in the Federal Register on March 28, 2011, pages 17129-17130.

The evaluation plan was developed in consultation with a six-member Technical Advisory Group (TAG) made up of experts in the area of evaluation design and implementation, mental health services research, the integration of community-based behavioral health services and primary care services, and a mental health services consumer with evaluation experience. The TAG met once in person to review the summary of grantee activities and to provide input on specific questions related to the design of the evaluation. A second TAG meeting was held by phone with the purpose of soliciting comments and feedback on the draft evaluation design and the proposed data collection instruments. The following individuals were TAG members:

Daniel Ford, MD, MPH

Professor of Medicine & Vice Dean for Clinical Investigation

Johns Hopkins School of Medicine

dford@jhmi.edu

Neil Korsen, MD

Medical Director

MaineHealth

korsen@mainehealth.org

Jon Morgenstern, PhD

Professor & Director, Substance Abuse Services Department of Psychiatry

Columbia University Medical Center

jm977@columbia.edu

Margaret Park, MDiv

Recovery Specialist, Office of BH

Allegheny County Department of Human Services

margaret.park@alleghenycounty.us  

Joe Parks, MD

Medical Director

Missouri Department of Mental Health

joe.parks@dmh.mo.gov

Linda Rosenberg, MSW

President & CEO

National Council for Community Behavioral Healthcare

lindar@thenationalcouncil.org

Additionally, two experts were retained for consultation on the design of the evaluation:

Jurgen Unutzer, MD, MA, MPH

Professor and Vice-Chair of Psychiatry

University of Washington

Adjunct Associate Professor

Department of Psychiatry and Biobehavioral Sciences

UCLA

unutzer@u.washington.edu

Benjamin Druss, MD, MPH

Rosalynn Carter Chair in Mental Health

Emory University

bdruss@emory.edu

9. Provision of Payments or Gifts to Respondents

RAND will enter into contractual agreements with mental health agencies participating in the outcomes evaluation. Up to 10 grantee intervention sites will be paid $10,000 for their work as independent data collection agents. Payment will be provided upon successful completion of the following tasks:

1. Mental health agencies will develop a list of PBHCI clients who agree to be contacted to learn about the research project. Agencies will share the list of names and current contact information with RAND so that clients can be contacted and invited to participate in the evaluation.

2. Mental health agencies will host staff from OnSite Health Diagnostics [a RAND subcontractor and national on-site biometric and diagnostic health screening company, fully HIPAA compliant] for up to 4 visits lasting 2-4 days each while they interview and conduct physical health screenings with clients who have consented to participate in the research project.

Up to 10 control sites will be paid $25,000 similarly for their work as independent data collection agents. They will be paid in two separate installments following the completion of the tasks detailed below:

1. Mental health agencies will develop a list of PBHCI clients who agree to be contacted to learn about the research project. Agencies will share this list of names and current contact information with RAND so that clients can be contacted and invited to participate in the evaluation. 

2. Mental health agencies will host staff from OnSite Health Diagnostics [a RAND subcontractor and national on-site biometric and diagnostic health screening company, fully HIPAA compliant] for up to 4 visits lasting 2-4 days each while they interview and conduct physical health screenings with the sites clients who have consented to participate in the research project. The first installment of $10,000 will be paid upon successful completion of this first screening session.

3. Mental health agencies will host staff from OnSite Health Diagnostics a second time, approximately one year after the first session, for up to an additional 4 visits lasting 2-4 days each while OnSite Health Diagnostics staff interview and conduct physical health screenings with clients who participated in the first round of interviews and physical health screens. The second installment of $15,000 will be paid upon successful completion of this second screening session.

All clients will be remunerated for their participation in the exam and survey in the form of a gift card to a local chain store (e.g., Target or WalMart). The payment will be $20.00 if the individual completes the informed consent, the exam and the survey and $10.00 if he/she completes only the consent and the exam or survey. Clients at the control sites will be administered the physical exam and survey at two points and will therefore receive up to $40.00 for their participation over the two sessions. The evaluation team consulted with staff at several grantee sites and with internal experts to determine that this payment would be sufficient to motivate (but not coerce) individuals to take part in the study.

10. Assurance of Privacy

Client physical exam and survey: The contractor will assure the respondent of the privacy of information collected in basic language in an advance letter (Attachment 13) which will be mailed to each potential respondent about two weeks before they are contacted. The language used in the letter will be close to a 6^th grade reading level. In the introduction to the physical exam and survey, the respondents will be reminded about the voluntary nature of their participation and that information collected will meet all requirements of the Health Insurance Portability and Accountability Act (HIPAA) and the Privacy Act.

Site visit interviews: All stakeholders invited to take part in the interviews will be provided with an informed consent form (Attachments 11, 12) to read and review with the research staff prior to the interview. The consent form states that the information gained will be used for research purposes only and will not be attributable to any individual.

Web-based survey: Participants will provide informed consent for the survey using an electronic signature, and all survey information will be maintained on a secure system. Respondents will be identified by unique numeric IDs and not names or other identifying information.

The evaluation has been approved by RAND’s Human Subjects Protection Committee. The application is included as Attachment 18.

11. Justification for Sensitive Questions

As previously stated, the client survey will assess, for example, daily functioning, employment and social connectedness. None of the items will assess sexual behavior or attitudes, or religious beliefs. Consumers will, however, be asked to report on drug and/or alcohol use. These topics may be considered sensitive, but this information is necessary to include in the survey to answer the study’s research questions. Assessment and treatment of co-occurring substance use disorders are key components of the PBHCI program.

12A. Estimate of Annualized Hour Burden

Table 1 provides estimates of the average annual burden for collection of the proposed information. The estimates provided in the text below cover the burden for the 3-year life of the evaluation. Annual burden and cost are provided in the Tables 2 and 3 below.

• Client Physical Exam and Survey: We estimate that it will require an average of 1 hour to complete the exam and survey with the consumers at the 10 control and 10 intervention sites, including time for the introduction, completion of the informed consent form, providing re-assurance of confidentiality, and responding to questions. Clients at control sites will complete the physical exam and survey at two points in the evaluation period, while clients at intervention sites will complete the contractor-administered exam and survey only once, as baseline data will be collected in the normal course of treatment and extracted later. (300 clients/site * 10 control sites * 1 hour at baseline) + (250 clients/site * 20 sites * 1 hour at follow-up).

• Site Visits: We estimate that the intervention sites’ leadership interviews will require 1.5-2 hours per respondent (10 * 4 * 2h), the care coordinator interviews will require 1-1.5 hours per respondent (10 * 2 * 1.5h), the PC provider interviews will take 1-1.5 hours per respondent (10 * 4 * 1.5h), and the MH provider interviews will take 45 minutes-1 hour per respondent (10 * 4 * 1h), at each of the 10 sites to be visited. We estimate that the key staff interviews at the control sites will involve 4-6 respondents and will take approximately 1.5-2 hours (10 * 5 * 2h).

• Web-Based Survey: We estimate that completing the web-based survey will require an average of 1-1.5 hours for leadership and 1 hour for PC providers, MH providers, care coordinators and wellness educators. We estimate that up to 10 individuals at each site will complete the survey, assuming a 70% response rate (56*10*1.5h).

• Individual Service Utilization Data: We estimate that the collection of data to complete the report could require up to 8 hours if done manually. It will be completed up to 11 times by all 56 PBHCI funded sites (56 * 11 * 8h). (It will not be collected for the first quarter, as the 2^nd cohort will not generally be providing the PBHCI-funded services to their clients yet.)

• Quarterly Reports: We estimate that completion of the report will require on average 2 hours. It will be completed up to12 times by all 56 PBHCI-funded sites. (56 * 12 * 2h)

• TRAC indicators: We estimate that entry of the 6-8 physical health indicators will require approximately 5 additional minutes for the individual entering data into TRAC at all 56 PBHCI sites. Information will be entered up to 6 times for an estimated 500 clients receiving PBHCI services during the evaluation for a total of 3,000 data entry sessions per site.

Table 2: Estimated Annualized Burden Hours

Type of Respondent	Instrument Name	No. of Respondents	No. Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Grantee Data Staff	Individual Service Utilization Data	56	4	8	1,792
Grantee Data Staff	TRAC Indicators	56	1,000	5/60	4,667
Grantee Project Directors	Quarterly Reports	56	4	2	448
SMI Clients	Client Exam and Survey-Baseline	1,000	1	45/60	750
SMI Clients	Client Exam and Survey-Follow-up	1,667	1	45/60	1,250
Grantee Leadership	Site Visit Interview	40	1	2	80
Grantee MH Providers	Site Visit Interview	40	1	1	40
Grantee PH Providers	Site Visit Interview	40	1	1.5	60
Grantee Care Coordinators	Site Visit Interview	20	1	1.5	30
Control Site Leadership	Site Visit Interview	50	1	2	100
Grantee Key Staff	Web Survey	560	1	1.5	840
Total					10,057

12B. Estimated Annualized Burden Costs

Table 3: Estimated Annualized Burden Costs

Type of Respondent	Instrument Name	Total Burden Hours	Hourly Wate Rate*	Total Respondent Costs
Grantee Data Staff	Individual Service Utilization Data	1,792	$15.00	$26,880.00
Grantee Data Staff	TRAC Indicators	4,667	$15.00	$70,005.00
Grantee Project Directors	Quarterly Reports	448	$34.00	$15,232.00
SMI Clients	Client Exam and Survey-Baseline	750	$15.00	$11,250.00
SMI Clients	Client Exam and Survey-Follow-up	1,250	$15.00	$18,750.00
Grantee Leadership	Site Visit Interview	80	$40.00	$3,200.00
Grantee MH Providers	Site Visit Interview	40	$40.00	$1,600.00
Grantee PH Providers	Site Visit Interview	60	$50.00	$3,000.00
Grantee Care Coordinators	Site Visit Interview	30	$28.00	$840.00
Control Site Leadership	Site Visit Interview	100	$40.00	$4,000.00
Grantee Key Staff	Web Survey	840	$40.00	$33,600.00
Total				$188,357.00

* Hourly wage estimates are based on salary information provided in 10 PBHCI grant proposals representing mostly urban locations across the country and represent an average across responders of each type.

13. Capital Costs

There will be no capital, start-up, operation, maintenance, nor purchase costs incurred by the sites participating in data collection for the evaluation.

14. Estimate of Annualized Costs to the Federal Government

The annual cost to the government of the proposed data collection for the evaluation consists of 10 percent of the Government Project Officer’s salary (grade 14, step 1 at $105,211 for 2010 in the Washington DC metropolitan area) and 100 percent of the contract awarded for the conduct of the PBHCI cross-site evaluation by the U.S. Department of Health and Human Services, Assistant Secretary for Planning and Evaluation ($1,417,599 for 3 years.. The estimated total cost of these expenses is $1,449,162 for three years, and the average annual cost is $ 483,054. .

15. Changes in Burden

There has been one change in the evaluation plan since the submission of the Emergency 6 Month Package. This change is the inclusion of the web-based survey in year 3 of the evaluation. For this reason, and the fact that this submission represents the average annual burden over 3 years, the annual burden hours and costs are greater in this submission than the previous one. Years two and three have greater burden and cost compared to year one of the study due to the inclusion of more client exams/surveys, site visits, and the web survey. An additional change is the reduction in the total number of grantees from an estimate of 60 in the earlier submission to the actual number of 56 grantees.

16. Plans for Tabulation and Publication

The evaluation contract for the PBHCI grant program anticipates that aggregate results from the national evaluation will be incorporated in text and charts in the following publications:

• A Baseline Evaluation report due October 4, 2011

• A Follow-up Evaluation report due October 2, 2012

• An Executive Summary of the evaluation of the PBHCI grant program and a Final Evaluation Report describing the data collection, analysis, and findings on what approaches were found to be successful in integrating primary and behavioral healthcare in community mental health settings due October 1, 2013.

ASPE and SAMHSA may also choose to incorporate the aggregate results from the cross-site evaluation in journal articles, scholarly presentations, and congressional testimony on outcomes resulting from the PBHCI grant program.

Table 2. Schedule for Use of PBHCI Cross-Site Evaluation Instruments

Activity	Date
Evaluation contract begins	October 1, 2010
OMB approval	By February 1, 2011
Data collection begins	By February 1, 2011
Data collection ends	By September 31,2013
Data analysis	By September 31, 2013
Completion of Report	October 1, 2013

17. Expiration Date

The expiration date of the OMB approval will be displayed on advance letters to prospective respondents to the client survey.

18. Exceptions to the Certification Statement

There are no exceptions identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-I.

15