OMB Supp Statement-A-06.5.13 Final

Site Visits with Grantees Integrating HIV Primary Care, Substance Abuse, and Behavioral Health Services

Supporting Statement

A. Justification

A.1. Circumstances of Information Collection

The Substance Abuse and Mental Health Services Administration (SAMSHA) is requesting approval from the Office of Management and Budget (OMB) to conduct in-person Site Visits with Minority AIDS Initiative – Targeted Capacity Expansion (MAI-TCE) Grantees Integrating HIV Primary Care, Substance Abuse, and Behavioral Health Services. This is a new project request targeting the collection of programmatic level data (e.g., services provision, program administration, consumer involvement, evaluation planning, organizational capacity) through one-on-one and group interviews with grantee personnel. Data collected through interviews will occur in project years one and two and support the cross-site evaluation. For the purposes of this request, the MAI-TCE Evaluation Site Visit Protocol and the Grantee Self-Assessment Form (Attachments A and B) have been included for review.

The goals of the MAI-TCE project are to facilitate the development and expansion of culturally competent and effective integrated behavioral health and primary care, which include HIV services and medical treatment within 11 of the 12 Metropolitan Statistical Areas (MSAs) and Metropolitan Divisions (MDs) most heavily impacted by HIV/AIDS. The program also supports the integration of behavioral health services (i.e., prevention, treatment, and substance abuse) into the CDC’s Enhanced Comprehensive HIV Prevention Plans (ECHPP). Interviews conducted with MAI-TCE grantees during site visits are an integral part of efforts to evaluate: (1) the effectiveness of program implementation across the grantee sites; (2) grantee efforts to integrate behavioral health, substance abuse and HIV care; (3) the variety of program models in use across the grantee sites; and, (4) grantee efforts to engage and successfully reach their target populations.

SAMHSA will conduct a total of two in-person site visits with each of the 11 MAI-TCE program grantees. Currently, MAI-TCE grantees are in the initial year of implementing SAMHSA’s vision for integrated care models in their communities. These site visits are intended to provide additional qualitative context to MAI-TCE grantee programming efforts, and will be used in conjunction with client level data collected through the TRAC and Rapid HIV Testing (RHT) data systems. The analysis of information provided by grantees during site visits will only be used in a descriptive manner to help explain grantee programming and efforts to provide services to their MAI-TCE target populations.

SAMHSA will conduct one-on-one and group interviews with grantee staff who will provide information on their program’s integration of primary care and behavioral health services. While participating in the evaluation is a condition of the grantees’ funding, participating in the interview process is voluntary. The Protocol was designed to collect information about: specific program components; HIV testing integration challenges, successes, and lessons learned; HIV care and evidence-based behavioral health services for their specific populations of focus; and engaging consumers in the Behavioral Health and Primary Care Network Committee and other aspects of the project, including how cultural competence is operationalized.

A mixed interview approach will be used with MAI-TCE grantees for conducting site visits. Each site visit will focus on one grantee and involve staff with in-depth knowledge of that grantee’s MAI-TCE Program (e.g., Program Director, Clinical Director) or knowledge of a specific element of that grantee’s continuum of services (e.g., clinical staff, case managers, outreach staff). Experience in conducting site visits with SAMSHA grantees has shown us that this approach to conducting site visits minimizes service disruptions, reduces burden for all programming staff, and produces in-depth qualitative information needed to conduct a comprehensive evaluation.

To determine the interview mix for each site visit, all grantees will be contacted prior to a site visit and included in all aspects of planning. We will work with grantees to determine (1) who at the grantee site is best suited for addressing questions in specific sections of the interview guide; and, (2) if the topic area (e.g., program administration, programmatic and clinical services) would best be addressed in an individual or group interview setting. At the conclusion of planning, an agenda will be drafted outlining the entire site visit, who will be interviewed, the time allocated for each section, and what will be covered in either a group or individual setting.

The site visits will provide context to the information obtained through TRAC and RHT and provide data that are not captured through these mechanisms. Direct contact with project staff will provide critical information on site processes, client flow, and the impressions of grantee site staff on how these systems are working. This information will enhance our understanding of the RHT and TRAC NOMS data that will be submitted by grantee sites. Most importantly, gathering contextual information about the quantitative data that will be submitted will help clarify site-specific data reporting issues that need to be addressed. Specifically, the methodology allows us to better conceptualize questions to be asked in a quantitative fashion from a larger sample at a later time

This program is authorized under Section 501(d)(4) of the Public Health Service Act (42 USC 290aa).

A.2. Purpose and Use of Information

The MAI-TCE Program is part of the Congressional Minority AIDS Initiative, which was developed to improve HIV-related health outcomes for racial and ethnic minority communities disproportionately affected by HIV/AIDS and to reduce HIV-related health disparities. The program also supports the goals of the National HIV/AIDS Strategy, the Department of Health and Human Services (HHS) 12 Cities Project, and the Centers for Disease Control and Prevention’s (CDC) current efforts through the ECHPP to reduce HIV risk and incidence in the areas most affected by the HIV epidemic. The program is a part of SAMHSA’s Health Reform Strategic Initiative. Our evaluation proposal was drafted in consultation with the SAMHSA Center’s Team (CMHS, CSAP and CSAT) and meets the requirements outlined in RFTOP #270-11-0376.

The goals of the SAMHSA MAI-TCE program aim to:

• Support the development and expansion of culturally competent and effective integrated behavioral health and primary care networks in racial and ethnic minority communities;

• Facilitate the integration of HIV prevention, rapid HIV testing, and treatment activities into existing mental health and substance abuse networks;

• Increase access to and use of integrated services; and

• Create behavioral change related to mental health, substance abuse, and HIV risk in targeted populations.

The following questions provide a sample of the program-level questions that SAMHSA plans to ask during the site visits:

• Please describe how clients flow through your system of services, from client outreach to intake to reception of behavioral health/primary care services.

• How does your agency promote effective communication and information sharing across diverse staff?

• Share a few of the barriers the project has faced and the plans to overcome these barriers?

• Share a few successes the project has experienced and to what do you attribute these successes?

SAMHSA will use the information obtained through the site visits to:

• Describe each program model and site in detail;

• Compare sites and program models in terms of the components of program process and implementation, providing interim feedback on progress as well as ultimate assessments;

• Inform the analysis of administrative/program-level and system-level site processes and outcomes for each program;

.

SAMHSA is learning more about the feasibility of integrating HIV primary care and behavioral health services. The site visit data will enable SAMHSA to learn more about what grantees are doing how they are doing it to promote SAMHSA’s overall goals for this program. In addition, the individual respondents will benefit from participating in the data collection process. The data that is obtained will allow grantees to learn from other grantees, inform strategies for service improvement and assist SAMHSA in providing appropriate technical assistance to grantees, as necessary.

Questions and answers regarding partnering organizations reflect staff perceptions and not objective data. In cases where information concerning travel time for clients is required, SAMHSA will conduct travel time estimates between partner organizations.

A.3. Use of Information Technology

Appendix A of the site visit protocol, the Grantee Self-Assessment Form, will be used to collect information on grantee service integration efforts. This form will be sent to grantees electronically through Fluid Survey, an online survey tool. This method will allow respondents to complete the form on their own time prior to the site visit. This process speeds data analysis as the data are automatically downloadable into an Excel Spreadsheet. The form will ensure an informed picture of the grantees’ integration model represented by each service delivery partner. The form asks respondents to describe the integration of services at their service site(s). Typologies of grantee programs may be developed from the data and later used to make comparisons across sites and contextualize the programmatic process data and information about the networks developed through this initiative.

A.4. Efforts to Identify Duplication

The overall evaluation strategy of the MAI-TCE utilizes three sources of data: (1) TRAC, (2) Rapid HIV Test (RHT) form, and (3) site visit interview guide. Of these three sources, only the site visit interview guide (and its associated materials) tool will be addressed within this request for OMB approval, as the TRAC tool the Rapid Test form were previously submitted for OMB review separately. TRAC and RHT provide client-level information concerning treatment and service utilization numbers and will provide data to inform the outcome evaluation. Site visits, however, focus on qualitative program-level data that will contextualize the information obtained through TRAC and RHT and provide data that is not captured through these mechanisms.

The purpose of the qualitative in-person site visits is to develop a comprehensive understanding of the following aspects of grantees’ MAI-TCE projects:

• Efforts to engage and reach their target populations

• Efforts to integrate mental health, substance abuse and HIV care

• Coordination with partners who are part of the continuum of grantee services

• Program service models

• Program implementation effectiveness and challenges

• Collect contextual qualitative data that will supplement data obtained through TRAC and RTHIV Databases

Our annual qualitative in-person site visit interviews were designed to gather and report in-depth program-level information over the course of a day-long site visit. The data we will collect are not currently available, which is why we propose to conduct these information-gathering site visits. By meeting with project staff in their own environment and discussing the issues highlighted in the interview guide, we will be able to have productive interactive discussions, learn more out the program, and make more informed observations about the context in which their project model operates.

Using qualitative methods, such as in-person data collection, allows the site visitors to discuss complex issues and to probe for more information or to explore issues that emerge during the site visit. This type of information about the MAI-TCE project does not currently exist and is best collected through an interactive in-person interview versus a structured questionnaire or form. Further, telephone interviews require similar staffing levels to identify interviewees and then coordinate the interviews and record the data. Forms require different types of staff to program surveys into an electronic format, to test them, and then manage and clean the data collected. Wherever possible our evaluation uses form-based data collection in instances where this method is more appropriate and less burdensome, such as the Self-Assessment Integration form.

As noted in our OMB application, the site visits will provide context to the information obtained through TRAC and RHT and provide data that are not captured through these mechanisms. Direct contact with project staff will provide critical information on site processes, client flow, and the impressions of grantee site staff on how these systems are working. This information will enhance our understanding of the RHT and TRAC NOMS data that will be submitted by grantee sites. Most importantly, gathering contextual information about the quantitative data that will be submitted will help clarify site-specific data reporting issues that need to be addressed.

As noted in the supporting statement, we will use the information obtained through the site visits for the following cross-site evaluation activities:

• Describe each program model and site in detail;

• Compare sites and program models in terms of the components of program process and implementation, providing interim feedback on progress as well as ultimate assessments;

• Inform the analysis of administrative/program-level and system-level site processes and outcomes for each program.

A.5. Involvement of Small Entities

Information collection will not have a significant impact on small entities.

A.6. Consequences If Information Collected Less Frequently

The information provided through the site visits will be vital to understanding of the evaluation of grantees’ activities. Site visits will be conducted twice throughout the duration of the contract period. Without collecting this data twice, SAMHSA will not be able to identify progress in the implementation and the intermediate and long-term outcomes of the funding.

A.7. Consistency with the Guidelines in 5 CFR 1320.5(d)(2)

The collection of information fully complies with the guidelines in 5 CFR 1320.5(d)(2).

A.8. Consultation Outside the Agency

A Federal registered notice was published on September 28, 2012 (Volume 78, page 59626) which solicited comments on this data collection. No comments were received. Revisions will be made if comments are received.

In April 2012, SAMHSA conducted telephone conversations with representatives from three of the 11 MAI-TCE grantee sites (the Texas Department of State Health Services, the Los Angeles County Department of Public Health, and the Philadelphia Department of Public Health). The interviews addressed initial start-up issues and planning for MAI-TCE programming for each of the respective cities. The overarching goals of the exploratory interviews were to assess, field test, and refine the Protocol. SAMHSA learned important details about the start-up activities of the three sites that informed the questions asked in the Protocol. Additionally, the interviews were helpful in determining the amount of time needed to conduct an interview.

A.9. Payment to Respondents

No payments or gifts are planned to respondents for participating in the site visit.

A.10. Assurance of Confidentiality

Data will be obtained from various individuals involved in implementing the program, including the: Project Director; Grant Coordinator; Clinical Director; Principal Investigator; Direct service staff (e.g. clinician, case manager, outreach worker); Evaluator; Behavioral Health and Primary Care Network Committee Chair; and Behavioral Health and Primary Care Network Committee members (including Co-chair, as applicable, and someone who can provide consumer perspective).

SAMHSA will likely associate particular program models with specific state programs in their reports. Therefore, the identities of the respondents will be easily recognized. However, the questions on program policies and practices and the information from respondents is part of their regular business knowledge and there are no questions of a personal nature, including opinions or the personal choices or behaviors of respondents. Thus, Abt’s IRB has deemed the proposed activities eligible for exemption as non-sensitive data collection with professional stakeholders.

A.11. Questions of a Sensitive Nature

There are no questions of a sensitive nature in the assessment form.

A.12. Estimates of Annualized Hour Burden

The total burden for the individual for site visit participation is estimated at 150 minutes maximum. Time estimates are based on experience with similar instruments in other studies of comparable organizations. In addition, parts of the Protocol were pretested (see Section A.8).

A.12.1. Number of Respondents, Frequency of Response, and Annual Hour Burden

The Protocol will have 55 respondents [Number of grantees = 11, Number of respondents per site = 5] and will take on average two and a half hours across the 5 respondents (150 minutes) to complete. The interviews will be conducted one time per yearly one-day site visit and different sections of the Protocol will be utilized with different types of respondents.

A.12.2. Estimates of Annualized Cost to Respondents for the Hour Burdens

Exhibit 1 offers an estimate of reporting burden for a sample of 55 respondents to a 150-minute Interview Guide (Appendix A). Based on U.S. Government Bureau of Labor Statistics data, we estimate an hourly wage of $43.74. Other than their time to complete the interview, there are no direct monetary costs to respondents.

Exhibit 1: Estimate of Reporting Burden: One Site Visit Round
Data Collection Tool	Number of Respondents	Responses per Respondent	Hour per Response	Total Burden Hours	Cost per Hour	Cost to Respondents
Site Visit Protocol	55	1	2.5	137.5	$43.74	$6014.25
Self- Assessment Form	55	1	.3	18.3	$43.74	$800.44
TOTAL	55*			155.8		$6814.69

Note *: the 55 respondents identified for the self-assessment are the same 55 that will be included for the site visit protocol. .

A.13. Estimates of Annualized Cost Burden to Respondents

There is no capital/startup or operation and maintenance cost to respondents involved in collecting the information.

A.14. Estimates of Annualized Cost to the Government

The total estimated cost to the Federal Government for the Site Visits to Grantees Integrating their HIV Primary Care and Behavioral Health Services data collection activity is $58,670.10. This includes the cost of developing the assessment and interview guide, conducting the site visits, and analyzing the assessment and interview responses ($53,670.10) plus 5% of a GS-13 SAMHSA employee’s (project officer’s) time at $100,000 annual salary ($5,000).

A.15. Changes in Burden

This is a new data collection.

Administrative data will be used as part of our overall effort to evaluate grantee performance. In cases where such information is viable and available we will use the information to “pre-populate” the guide to reduce participant burden.

A.16. Time Schedule, Publication and Analysis Plans

The following activities will take place within each project year:

• Year One: OMB submission and clearance.

• Years One and Two: Site visits (one per year), including site visit planning, conducting site visits, data analysis, and data reporting.

• Year Five: Final comprehensive site visit data reporting.

SAMHSA will use the site visit interview and form data to develop an understanding of the following MAI-TCE integrated primary and behavioral health care program key components:

• Primary and Secondary MAI-TCE Service Populations

• Staffing, Sub-recipients & Partnerships

• Behavioral Health & Primary Care Network Committee

• Data System Readiness, Client Tracking & Referrals

• Data Sharing Across Partners

• Data Submission Requirements

• Cultural Competence Planning, Initiatives & Implementation

Interview notes, field notes, and any secondary data obtained will be saved in an NVivo 9.0 Database designed for this study. Data coding will occur concurrently with data collection and the data will be integrated as codes in outcome analysis. The codes will aid in the identification of patterns of effective implementation of MAI-TCE goals across the sites.

SAMHSA will use the information collected to expand their understanding of the grantees’ progress with service implementation. Impressions gathered from visits with the 11 MAI-TCE grantees will be documented in internal reports and used to inform annual reporting. Over time, the data collected through site visits will provide SAMHSA, grantees, and their partners and other stakeholders with a clearer understanding of grantee service integration processes; program successes and challenges; and strategies for the replication of promising grantee programs.

A.17. Display of Expiration Date

The expiration date will be displayed.

A.18. Exceptions to Certification Statement

This submission describing data collection requests no exceptions to the Certificate for Paperwork Reduction Act (5 cfr 1320.9).

9