Exploratory telephone call semi-structured interview – program director

ATTACHMENT A

GOALS Program Director and Administrator EXPLORATORY Telephone Interview Guide

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB # for this collection is 0970-XXXX and it expires XX/XX/XXXX.

Introduction for the Respondent

As you may know, Mathematica Policy Research, Inc. is conducting the Goal-Oriented Adult Learning in Self-Sufficiency (GOALS) Study on behalf of the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services. This study will document how programs like [Name of the program] use frameworks informed by psychological concepts to provide employment and training services to low-income families.

As part of the GOALS study, we are interviewing program directors and administrators to better understand the extent to which programs provide services designed to strengthen goal-oriented skills. For the purposes of this study, we define goal-oriented skills as a set of skills that involve managing oneself and one’s resources in order to identify, work towards, and achieve a given goal. We are conducting these exploratory calls to learn about the types of services provided as part of [name of the program] and your experience with administering the program. The interview will last approximately 60 minutes. Your participation in this study is important and will help us understand more about the services provided to low-income families.

Your responses will be kept private and used only for research purposes. They will be combined with responses from other staff and no individual names will be reported. Information you provide will not be shared with other program staff, including your supervisor. Only the study team will have access to the information you provide during the interview.

Participation in the study is voluntary. You can choose not to answer a question and may stop the interview at any time.

Do you have any questions before we begin?

Note to the researcher: This individual telephone interview guide will be used to collect the information gathered during GOALS project. The benefit of a standard template is that we will gather information on similar topics across interviewers. The blue bolded headers are considered qualitative codes used to label the data. The questions will be used to gather the information during the interviews. These questions help guide the interview and remind you of the content we are trying to collect under each main topic area. It’s also important to note that service systems vary across programs. As a result, some sections may include more detailed information than other sections.

Telephone Interview Topics and Questions

Community context

• What are the demographic characteristics of the community? How would you describe the local economy?

• How would you describe the local service environment?

Program philosophy, purpose, and goals

• What do you hope to achieve with your program (e.g., organizational and client goals)?

• Why and how did the program come about? What was the initial design process? Who was involved? How long did it last?

• How would you describe the initial implementation of the program? To what extent did the hopes for the program match what actually happened? What worked well during the initial implementation? What were the main challenges? How were they addressed?

• How has the program evolved over time? What are some of the main lessons?

• To what extent did you and others use research to inform the program design? If so, please describe.

• For purposes of this study, we are defining ‘goal-oriented skills’ as a set of skills that involve managing oneself and one’s resources in order to identify, work towards, and achieve a given goal. What does ‘goal-oriented skills’ mean to you? What skills are encompassed in ‘goal-oriented skills’? What are some examples?

• How does a focus on the development of goal-oriented skills fit into your program model? What are the ‘goal-oriented skills’ your program seeks to develop among participants or clients?

• Are there other ways your program would foster the development of skills that help participants attain their [short- and long-term] goals?

Funding and program costs

• What are the amount and types of funding to support the program?

• What is the overall cost of operating the program? What is the cost per participant?

Administrative and staffing infrastructure

• What is the administrative staffing structure? What are their roles and responsibilities?

• Who are the primary program champions?

• What is the program staffing structure? What are the roles and responsibilities of different positions?

• What is the process for training staff (e.g., how, when, what)? What are the ongoing efforts to train staff?

• What are the lessons learned with administering and staffing your program?

Target population and client characteristics

• Who is the target population? What are the eligibility criteria?

• What are the characteristics of program participants?

Core program components and services

• What are the core program components? What purpose do they serve in general and in strengthening goal-oriented skills?

• Assessment (e.g., types of tools, process for assessing clients, how the information is used)

• Case planning

• Case management

• Coaching

• Peer mentoring

• Use of incentives

• Group activities

• Mental health counseling

• Vocational education

• Job retention services

• Other

• How is goal-oriented skill development incorporated into or part of the service delivery process? What program components focus on goal-oriented skill development? What tools or resources are used to support the development of goal-oriented skills?

• What are the most common services provided to participants? What additional services are needed?

• What is the order in which services are provided (case flow)? How do clients progress through the service delivery process? Who provides each of the different types of services?

• How successful is the program with encouraging program participation and progress? What is the role of motivation and incentives in serving clients?

• What types of work and personal supports are provided to program participants? What services do participants use most often? What additional services are needed?

Coordination and linkages with other services within the community

• What community partners are involved with the program? How are they involved? What is their level of involvement?

• What services, resources, and supports are available through community partners?

Outcome measurement and evaluation

• What is the process for measuring program success?

• What short- and long-term outcome measures are used? How are they defined? Who was involved in the process for selecting the outcome measures and defining them?

• What measures are used that may reflect progress on improving goal-oriented skills? (For example, are their measures directly related to the skills? Are their measures related to behavioral outcomes as a result of achieving the goals?)

• What evidence has been used to inform the measurement of program objectives and outcomes? How else has the research been used to inform the program?

• What evaluation efforts have been completed? If completed, how was the program evaluated? Who conducted the evaluation? What were the findings? How were evaluation findings used?

• If an evaluation has not been completed, then is an evaluation in the works or being planned? If so, what are the evaluation plans?

• What additional information would you like to learn from a formal evaluation?

Program successes, challenges, and lessons

• What are the program successes?

• What has been the long-term progress in goal-directed behaviors as a result of the program?

• What have been the primary challenges? How were these challenges addressed?

• What are the lessons from operating the program?

• What is the sustainability of the program? What factors might influence the sustainability?

• How easily might the program be replicated? Scaled up? What is required to implement the program?