F_Program Managers Staff and Partners Interview Guide - SUD Programs_final 11.8.19

Attachment F – Program Managers, Staff, and Partners Interview Guide – SUD Programs

OMB Control No: ____-____

Expiration Date: __/__/____

Program Managers, Staff and Partners Interview Guide for SUD Programs

This document covers the key topics to be covered in individual interviews with program managers, staff, and partners at six programs integrating substance use disorder (SUD) treatment and recovery services with employment services. Each topic indicates which staff will respond (i.e., program managers, program staff, and partners). The purpose of these interviews, which will occur once at each of the six participating programs, is to understand implementation of this relatively new approach to serving individuals with SUD. A table at the end of the document summarizes which staff (e.g. managers, staff, and partners) will be included in each topic of the interviews.

Introductory statement for staff: The Building Evidence on Employment Strategies for Low-Income Families (BEES) study is being conducted by MDRC/Abt/MEF Associates, under contract to the U.S. Department of Health and Human Services. The study aims to build evidence on the effectiveness of innovative programs designed to boost employment and earnings among low-income Americans. As part of this evaluation, evaluation staff members will visit six programs that combine substance abuse treatment and/or recovery services with employment services for low-income adults with substance use disorder (SUD). The goal of the study is to document best practices, challenges, and lessons for both policymakers and program administrators. In conducting site visits to each of the programs, we are talking to project directors and staff, as well as partner organizations. We are here to learn about the service delivery of your program model and understand how it operates. Our aim is to learn from your experiences, not audit or judge your programs. This interview will take up to 90 minutes.

Privacy Statement: Before beginning the interview, I (we) want to thank you for agreeing to participate in the BEES study. I (we) know that you are busy and we will try to be as focused as possible. We have many questions and are going to talk to many different people, so please do not feel as though we expect you to be able to answer every question. And, we understand that your participation in this discussion is voluntary and you may choose to not answer some questions.

Information collected will be kept private to the extent permitted by law. The views you express will be kept private, and nothing we publish in this evaluation will identify you by name. Though we take notes during these interviews, information is never repeated with the name of the respondent. When we write our reports and discuss our findings, information from all interviews is compiled and presented so that no one person can be identified. We also ask that you refrain from sharing anything we discuss today with others to help us ensure your privacy and the privacy of others we are interviewing. Do you have any questions before we begin?

The Paperwork Reduction Act Statement: This collection of information is voluntary and will be used to understand programs that aim to improve employment outcomes for low-income adults. Public reporting burden for this collection of information is estimated to average 90 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number and expiration date for this collection are OMB #: XXXX-XXXX, Exp: XX/XX/XXXX. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to Dan Bloom (MDRC); 200 Vesey Street, 23^rd Floor, New York, NY 10281-2103.

1. Respondent Information

• Name. title, organization/affiliation, length of involvement with the organization and with the program

• Educational background and prior work experience

• Overall role/job responsibilities

2. Program Context and Environment—Program Managers and Partners

1. Lead organization background—Program Managers

• History of organization/agency

• Other programs and services offered by organization/agency

• Organization’ budget for most recently completed program year

• Organization’s major sources of funding (e.g., funding from federal/state/city agencies, foundations, private contributions, fee for service, etc.)

• Organization’s total # of paid staff: _______ # of people, _______ # of FTEs

• Types of individuals served or targeted

• Annual number of participants served overall

2. Geographic area served—Program Managers and Partners

• Program’s service area, possibilities include: portion of a city/county, single county, multiple counties, other (regional effort)

• Size of service area (in terms of geography and population)?

• Characteristics of this area (e.g., is the service area geographically large; urban, rural, mixed)

3. Area demographics—Program Managers and Partners

• Demographics of the service area, including employment rates, education levels, race and ethnicity, income levels, poverty status, immigrants)

• Changes in demographics over time

• Use (e.g. hospitalization, arrests, etc.) and history of substance abuse disorder, including opioids, in areas served by program

4. Local economic conditions—Program Managers and Partners

• Major industries and employers

• Recent economic trends (e.g. occupations experiencing earnings and job growth)

• Unemployment levels, job availability, wage levels

• Other factors that affect economy (e.g., natural disaster, companies coming or going, policing practices/criminal justice involvement of community members)

5. Community and government supports—Program Managers and Partners

• Community and government supports commonly used by program participants (e.g., TANF, SNAP, housing assistance, Medicaid)

  • Services provided

  • Formal and informal coordination with community and government support program

• Other organizations that provide services to program’s target population

3. Program Goals and Structure—Program Managers and Partners

1. Program goals—Program Managers

• Goals of the program, including changes over time

• Reasons for implementing/developing the program, particularly integrating SUD treatment and employment services

• When developed and changes since initial implementation

• Hypothesized impacts of the program on target population (immediate, short-term, long-term)

• Source of the program model (i.e., is it based on existing program that has been operated elsewhere)

2. Resources and capacity—Program Managers

• Resources required to operate the program

  • Funding sources and amount (e.g. Medicaid, SAMHSA grants, other)

  • Sources of curriculum/materials

  • Facilities (classrooms, labs, program offices)

  • Technology (laptops)

  • Housing

  • Treatment and recovery services (e.g. counselors, medical professionals, peer support specialists)

  • Other

• Annual budget and funding sources

• Future funding, stability of funding

3. Management and staffing—Program Managers

• Number of staff and positions

  • Titles

  • Required experience and qualifications

  • Primary responsibilities

  • Location of staff members

  • Full-time employees, part-time employees

• Staffing/organizational structure (e.g., type of organization (SUD treatment provider, non-profit organization), program departments, how the program is situated/managed within the larger organization or institution, who has authority over the program).

• Staff development and trainings (e.g. required trainings, how often, and provided by whom)

• Staffing shortages and staff turnover

• Performance measures for staff

4. Partnerships and organizational linkages--Program Managers and Partners

• Partnerships that provide training, treatment, recovery and mental health services

• Type of organizations

• Length of the partnership

• Historical relationship

• Nature of partnership

• Factors that influence the development and maintenance of these partnerships

• Type of arrangement: Contracts, MOUs, type of contracts (performance-based, fixed price, etc.)

• Coordination mechanisms: staff responsible, regular meetings, data exchange

• Strengths and challenges with partnership

• Changes in partnership over time

5. Target population and program eligibility—Program Managers and Staff

• Target population (SUD-related criteria, ex-offender status, socio-economic and demographic characteristics, including age, educational attainment, work history, marital status, children)

• Program eligibility requirements (specific requirements to be met prior to enrollment)

4. Program Service Components—Program Managers and Staff

1. Description of services provided—Program Managers and Staff

• Treatment and recovery services (treatment for substance use or opioid use disorder, mental health)

  • Medication-assisted treatment (medication provided; by whom)

  • Residential treatment

  • Outpatient treatment

  • Monitoring and support for adherence to treatment

  • Drug testing

  • Therapy/Counseling

• Occupational training (industry, credentials)

  • Use of peer support specialist training

  • Tailoring to accommodate those with SUD

• Basic skills instruction

• Work-based training services

  • Subsidized employment

  • Unpaid work experience

  • Paid internships

• Support services

  • Case management and counseling

  • Housing assistance

  • Financial support

  • Academic advising

  • Career advising

  • Child care

  • Work supports (tools, uniforms)

• Job readiness skill development (i.e. “soft” skills)

• Job search and placement assistance

• Referrals to other organizations and services

2. Service delivery structure—Program Managers and Staff

For each service listed above:

• Service sequencing

• Service provider and location

• Length and schedule of service component (e.g., total hours; length in weeks/months)

• Content and services provided

• Organizational partners in delivering service

3. Steps in Service Delivery—Program Managers and Staff

• Outreach/Recruitment

  • Outreach and recruitment strategies (e.g., website, social media, word-of-mouth, fliers, referrals, community events, other agencies/programs)

  • Past strategies used

  • Effectiveness of strategies

• Application and Enrollment Process

• How individuals apply to the program (i.e., online, on-site, hotline)

• Application process steps and information collected (in what order and by whom)

• Length of application process

• Use of waitlist

• Obtain copy of application

• Assessment and Participant Employment/Treatment Plans (including referrals)

• How assessment is conducted

• Information collected from the applicant

• Academic and non-academic screening tools used

• Mental health diagnoses

• Substance use screening

• Development and content of employment/treatment plan

• ASAM score

• Program Orientation

• Orientation or information sessions

• Information provided to participants after enrollment

• Participant Monitoring

• Use of employment/treatment plan to guide services

• Frequency and mode of monitoring

• Length of interactions and topics covered

• Actions taken if problems/issues

• Attendance or participation requirements

• Consequences for non-compliance

• Drug testing

• Program Completion

  • How completion defined; point at which a person completes the program

  • Completion rate: reasons for high or low completion

• Follow-up Services

  • Services provided to participants once they’ve left the program and for how long

  • Job retention/advancement services

  • Follow-up data collected on participants

4. Data collection, performance management, and program outcomes—Program Managers

• Data collected on participation in services and outcomes

• Data collected from program partners

• Performance measures in place to track the usage and quality of the services delivered

• Measures for tracking program performance

• Information collected from participants about their experiences in the program

• Recent trends in the performance of the program

5. Operational Challenges and Successes—Program Managers, Staff and Partners

• Ways program has exceeded or fallen short of goals

• Promising approaches developed by program, particularly in terms of combining treatment/employment services

• Program’s key successes

  • Factors that contributed to this success

• Challenges the program encountered and how they were overcome

• Challenges related to outreach, service retention, relapse and completion

• Areas for improvement; plans for changes and modifications

• Sustainability and prospects for future funding

• Plans to scale up, replicate in other locations

The following table provides a summary of interview topics and potential respondents for each.

Summary of Interview Topics and Respondent Type

Topic	Respondent Type
	Program Managers	Program Staff	Partners
Project Context and Environment Lead organization background Geographic area Served Area demographics Local economic conditions Community & Government Supports	X		X
Program Goals and Structure Program goals Resources and capacity Management and Staffing Target population and program eligibility	X		X
Program Service Components Description of services provided Service delivery structure Key steps in service delivery Data collection, performance management, and program outcomes	X	X
Operational Challenges and Successes	X	X	X

11