Expanding Evidence on Replicable Recovery and Reunification Interventions for Families (R3)

Instrument 2

R3 Discussion Guide for Virtual Site Selection Visits

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN: Through this information collection, ACF is gathering information to conduct a feasibility study of one or more promising family recovery and reunification interventions that use recovery coaches. Public reporting burden for this collection of information is estimated to average six hours per grantee, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0356 and the expiration date is 06/30/2021. If you have any comments on this collection of information, please contact Kimberly Francis, R3 Project Director, at Kimberly_Francis@abtassoc.com or (617) 520-2502.

Instrument 2: R3 Discussion Guide for Virtual Site Selection Visits

Interviewer Instructions: This guide is intended to build on Instrument 1. Please review notes from prior calls and customize the topics below, eliminating any redundancies.

Introduction

Thank you for taking the time to meet with the R3 study team. As a reminder, R3 is a study of recovery and reunification interventions that use recovery coaches for families involved in the child welfare settings. Abt Associates and its partners are conducting the study on behalf of the Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS). The study will help us determine the feasibility of conducting a rigorous evaluation of one or more potentially replicable and scalable interventions. This feasibility study is the first step toward the goal of conducting a rigorous evaluation of recovery coaching interventions.

We are holding these [phone calls/video conferences] with you because the study team and ACF think that your program may be a good fit for the feasibility study. During the [phone calls/video conferences] we’ll expand on our recent phone discussions to learn more about your program, let you know more about the R3 project and answer your questions, and get your input about the potential to move forward with participating in the feasibility study. We plan to hold a few calls, and these calls will require no more than a combined eight hours of your time. Lastly, please know that your participation in all discussions with our team is completely voluntary. Should you choose to participate, your responses will be discussed internally among the study team and the funding agency (the Administration for Children and Families) but that, to the extent allowable by law, individual identifying information will not be disseminated publicly.

Topics for Discussion

Program administration

• Review information obtained about the organization during previous conversations.

• What organizations and partners operate the program? What are their roles? (e.g., child welfare agency; courts; community-based family services organization; behavioral health service providers; substance use treatment providers; other state/local agencies)

• Is there a specific technical assistance provider with whom they work?

• What are the program’s funding sources?

Program design

• Review the components of the program, including any recent or planned adjustments or enhancements to the program.

• What are the program goals overall and for each component?

• How does recovery coaching fit into the program model?

• Does the program have a written logic model or theory of change?

Program locations and settings

• Review and confirm where the program operates (e.g., statewide; one county; multiple counties; multiple offices per county)

• Review the number of locations for enrolling parents. To what extent does the program differ by location? (e.g., referral process; program services; partners involved; level of support from partners and other key stakeholders)

• How is the program coordinated across multiple locations?

Service environment

• Identify other related programs or services available to the target population within organization or the broader community. If multiple communities, review for each.

• What are the “usual services” if participants were not offered this program?

Target population

• What is the target population?

• At what point in the child welfare system does the program identify eligible parents/families? (e.g., at risk of entering CW system; report of CA/N; children already removed from home)

• How are eligible parents/families identified and referred? Identify key referral sources.

• What are the eligibility criteria?

Enrollment

• Once eligible parents/families are identified and referred, what is the enrollment process?

  • What are the program intake and enrollment steps?

  • Where does intake and enrollment take place (e.g., number and type of enrollment locations)?

  • At what point in the intake and enrollment process is eligibility determined? How is eligibility determined? (probe for clinical substance use assessment vs. other method)

• Which staff and/or organizations facilitate enrollment?

Program capacity

• What is the maximum number of parents/families the program is able to serve per year? Per month?

• How many parents/families does the program actually serve per year? Per month?

• How many parents/families do you project to serve in the future?

Recovery coach services

• Which organization provides the recovery coaching services?

• What is the recovery coach’s role?

• What services does the recovery coach provide?

• How does the recovery coach coordinate with other members of the team, e.g., the child welfare case worker, the SUD treatment provider, and the court?

• Where does the recovery coach meet with parents/families?

• How frequently does the recovery coach meet with parents/families?

• What education, experience, or certification are recovery coaches required to have?

  • How have you recruited and retained staff? Has recruitment been easy/difficult?

• What training and support do recovery coaches receive once they’re hired?

Evaluation and data

• Has the program been evaluated? Is an evaluation underway?

• Who conducted the evaluation?

• What was the evaluation design?

• What was your experience with the evaluation? If there were challenges, how were they resolved?

• What data sources were used for the evaluation?

  • What data sharing agreements do you have in place?

• [If current program has not been evaluated]: Does your organization/its staff have experience with program evaluation? Describe.

• What data are currently collected on parents/families enrolled in the program?

  • Do you have an MIS (management information system) in place to track services, outputs, and/or outcomes?

• Do you have any programmatic or evaluation materials you could share with us? (e.g., written protocols, manuals, training or curriculum materials, policies and procedures, and evaluation reports)

Program implementation

• Discuss practices for monitoring program implementation, including data sources, and the identification and delivery of technical assistance

• Do you use any tools to track fidelity to the program model? How is that information used?

• What have been the program successes?

• What have been the primary challenges? How were these challenges addressed?

• How has the program evolved over time?

Current operations and future plans

• Review plans to operate the program in the future

• Review status of funding to continue or expand the program

• Discuss interest in participating in a feasibility study, and potential future rigorous evaluation. Confirm understanding of any barriers, concerns. Review preliminary draft Memorandum of Understanding.

• Do you have any other comments or advice for project team?

Next steps

Thank you for speaking with us today. The study team is holding these discussions with a small group of program sites. Based on our discussions with you and the information you’ve shared, we will coordinate with ACF to select the final group of program sites that will be invited to participate in the feasibility study. We may follow up with additional questions, and will keep you informed on next steps.

6