Site Visits with Grantees Integrating HIV Primary Care, Substance Abuse, and Behavioral Health Services
Supporting Statement
B. Collections of Information Employing Statistical Methods
B.1. Respondent Universe and Sampling Methods
The respondent universe will be key personnel and support staff across the 11 MAI-TCE program grantees. Sampling procedures will not be used for selecting interviews. Appropriate persons for interviewing will be determined in consultation with the SAMHSA leadership including evaluation staff and grantee project officers and grantee project directors for each grantee site.
B.2. Information Collection Procedures
SAMHSA will interview staff across the 11 MAI-TCE programs.. In preparation for site visits, we will consult with key grantee staff for input on who to engage during the site visit. Our experience with prior SAMHSA HIV program evaluations has shown that in general, the grantee-specific job titles may vary, but the roles and responsibilities required for the project implementation (e.g., administrative duties, clinical director, project director, evaluator, etc.) are consistent in purpose and function. As such, we will work with each MAI-TCE grantee’s Project Director to identify who among the project staff is best suited to discuss the topics in our protocol based on their functional role on the MAI-TCE project. The interview process will include an interviewer, note-taker, and interviewee(s). Detailed notes will be taken during the course of the interviews, to be reviewed, coded, and analyzed following the interview.
A mixed interview approach will be used with MAI-TCE grantees for conducting site visits. Each site visit will focus on one grantee and involve staff with in-depth knowledge of that grantee’s MAI-TCE Program (e.g., Program Director, Clinical Director) or knowledge of a specific element of that grantee’s continuum of services (e.g., clinical staff, case managers, outreach staff). Experience in conducting site visits with SAMSHA grantees has shown us that this approach to conducting site visits minimizes service disruptions, reduces burden for all programming staff, and produces in-depth qualitative information needed to conduct a comprehensive evaluation.
To determine the interview mix for each site visit, all grantees will be contacted prior to a site visit and included in all aspects of planning. We will work with grantees to determine (1) who at the grantee site is best suited for addressing questions in specific sections of the interview guide; and, (2) if the topic area (e.g., program administration, programmatic and clinical services) would best be addressed in an individual or group interview setting. At the conclusion of planning, an agenda will be drafted outlining the entire site visit, who will be interviewed, the time allocated for each section, and what will be covered in either a group or individual setting.
This project requires no sampling procedures because SAMHSA will interview key personnel and select support staff at each of the 11 MAI-TCE grantee sites. Furthermore, no weights will be calculated for analyzing and reporting the site visit results. The final products, including internal reports for SAMHSA, will contain the descriptive results of the site visits. The reports will provide findings using simple frequencies and other descriptive statistics.
B.3. Methods to Maximize Response Rates
SAMHSA is conducting the site visits with staff using updated contact information, and therefore expects fewer non-responses due to an inability to locate sample members. The Site Visit Protocol was developed with consideration to length and reading level so it is appropriate for staff to complete. As SAMHSA is conducting site visits with a universe of respondents, the main concern will be maximizing the interview and self-assessment form response rates in order to be able to assume accuracy of results. SAMHSA anticipates achieving a 90 percent interview response rate to minimize non-response bias and obtain a full picture of the grantees.
B.4. Tests of Procedures
SAMHSA conducted telephone conversations with representatives from three of the 11 MAI-TCE grantee sites to test the questions asked at site visits. The interviews addressed initial start-up issues and planning for MAI-TCE programming for each of the respective cities. One of the overarching goals of the exploratory interviews was to assess, field test, and refine the site visit questions. SAMHSA learned important details about the start-up activities of the three sites that informed the questions asked in the Protocol. Additionally, the interviews were helpful in determining the amount of time needed to conduct an interview.
The self-assessment form was piloted with the Project Directors at two grantee sites. We requested the directors complete the form and provide feedback on the content, ease of use, and the amount of time required to complete the form. Feedback from the directors indicated that the form was simple to use and required little time (approximately ten minutes). Their feedback suggested a few improvements to the response options the form allowed. In response, we clarified the language for each question and added open-ended response options improve answer flexibility.
In terms of the supporting tables that collect information on: (1) MAI-TCE partner organizations; (2) funding sources; (3) MAI-TCE services; (4) staffing and staff training & development; and, (5) the Behavioral Health & Primary Care, these items were utilized in two prior evaluations of SAMHSA HIV programming. Prior use indicated that the tables are very effective in collecting in-depth programmatic information that enhances understanding of grantee programmatic structures and their continuum of services. We have also learned that providing these instruments, which are completed and submitted by each grantee prior to any site visit; reduces the burden of the site interview process and time required while on-site. Based on the above understanding, the tables included in this OMB package have been refined to focus solely on MAI-TCE projects and collect data relevant to this programming effort.
B.5. Statistical Consultants
Darren Fulmore, Ph.D.
Cross-Site Evaluation Lead and Abt’s COTR
Public Health Advisor
Center for Substance Abuse Treatment
1 Choke Cherry Road
Rockville, MD 20857
Phone: 240.276.2824
Email: Darren.Fulmore@samhsa.hhs.gov
Liza Solomon, MHS, DrPH.
Project Director
Principal Associate, Abt Associates
4550 Montgomery Ave., Suite 800 North
Bethesda, MD 20814
Phone: 301.347.5785
Email: liza_solomon@abtassoc.com
Douglas Fuller, Ph.D.
Project Manager
Associate Scientist, Abt Associates
4550 Montgomery Ave., Suite 800 North
Bethesda, MD 20814
Phone: 301.634.1846
Email: douglas_fuller@abtassoc.com
Dana Hunt, Ph.D.
Principal Associate, Abt Associates
Phone: 617.349.2733
Email: dana_hunt@abtassoc.com
Cristina Booker, MPH
Associate, Abt Associates
Phone: 617.349.2681
Email: cristina_booker@abtassoc.com
List of Attachments
Attachment A: MAI-TCE Evaluation Site Visit Protocol
Attachment B: SAMHSA MAI-TCE Program Service Integration Site Self-Assessment Form
| File Type | application/msword |
| File Title | Abt Single-Sided Body Template |
| Author | Administrator |
| Last Modified By | DHHS |
| File Modified | 2013-06-05 |
| File Created | 2013-06-05 |