Instrument 4: HPOG 2.0 National Evaluation in-person implementation interviews

Instrument 4-E OMB # 0970-0462

Expiration Date xx/xx/xxxx

Instrument 4-E: Sustainability

Site Visit Preparation for Interviewers

• Review Responses from telephone interviews. The telephone interviews with grantees include information about the HPOG program’s approach to sustainability. The exhibit below highlights the interview questions associated with this topic in the telephone interview protocol.

Exhibit 1: Interview Topics to Review in Preparation of Site Visits

Interview Topic Areas	Questions
Sustainability	8.1 – 8.8b

• Review the Screening Interview to Identify Respondents for the HPOG 2.0 National Evaluation First-Round Telephone Interviews. Create a list of the local HPOG program staff, key partners, and stakeholders with knowledge and expertise about Sustainability. Always interview the program director and include others on the list as well. This may include program coordinators, job developers, curriculum developers, instructors, tutors, partners or other stakeholders that have a role to play in sustainability.

• Customize interview topics by the type of respondent. All topic areas should be covered when interviewing program directors. Based on your list of respondents, tailor the topics and/or questions to the role and level/type of involvement of each respondent.

• Customize interview topics with information collected from the telephone interviews. Use the italicized notes on which interview questions to customize for selected topics in the sustainability guide to prepare in advance of the site visit.

HPOG 2.0 Implementation Study

Site Visit Guide on Sustainability

Introductory statement to [Pre-fill with Respondent Type]: Thank you for taking the time to talk to us today. As you may know, the Administration for Children and Families (ACF) of the U.S. Department of Health and Human Services has funded the 2^nd Generation Health Profession Opportunity Grants (HPOG 2.0). ACF has also funded an evaluation study of HPOG 2.0; we are members of the research team. We are visiting HPOG 2.0 grantee programs to document in greater detail program design and implementation. The major purpose of this site visit is to gather more information about your HPOG 2.0 program’s approach to longer-term sustainability [name of local HPOG program] once the grant ends. We plan to use the information to produce research reports that describe the HPOG 2.0 initiative and analyze its results and effectiveness.

In preparation for our discussion today, we have worked with representatives from your program to identify the most appropriate staff, partners, and stakeholders to talk to us about sustainability. We have also reviewed responses about sustainability from telephone interviews we previously conducted with [name of local HPOG program] staff, management and partner organizations in [month and year of telephone interview], as well all other information about the [name of local HPOG program] program we already have available.

The interview will take about 60 to 90 minutes to complete. Interview questions are structured around five key topics:

1. Sustainability planning,

2. Partners working for sustainability,

3. Working for sustainability,

4. Prospects for sustainability, and

5. Lessons for the field

Before we begin, I would like to assure you that all of your responses will be kept private and used only for this research study. Your name will not appear in any written reports we produce. Also, the interview is voluntary and you may choose not to answer any specific question and may end the interview at any time.

According to the Paperwork Reduction Act (PRA), an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this collection is 0970-0462 and it expires on xx/xx/xxxx. If you have comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, please send them to [Contact Name]; [Contact Address]; Attn: OMB-PRA (xxxx-xxxx).

Do you have any questions before we begin? I would first like to start by giving you and/or each individual a minute to introduce yourselves. Please tell me:

• Your name:

• Your role within [name of local HPOG program]

I have some topics I would like to cover, but these interviews are meant to be exploratory. Please feel free to bring up additional points that you would like to make about sustainability.

1. Sustainability Planning

Let’s begin by discussing your plans for sustainability. According to information collected from the telephone interview, we understand that your program [has implemented or plans to implement] the following strategies (e.g. leveraging funds, maintaining key partnerships, etc.) to sustain HPOG programming: [Pre-fill with information from questions 8.2, 8.3a, 8.3b, 8.5, 8.8a, and 8.8b from the telephone interview protocol under Sustainability].

1. Perspectives on sustaining HPOG (e.g., assessing which components of the program are worth sustaining)

2. Timing of sustainability planning (e.g., beginning of HPOG, end of year one, etc.)

   1. Rationale for sustainability planning at the allotted time

3. Process and strategies for developing post-grant sustainability plans

   1. Best planning strategies for sustainability

2. Partners Working for Sustainability

Now let’s discuss how your partners are involved in sustainability. As we understand from the telephone interview, [Pre-fill from question from questions 8.3b and 8.4 from the telephone interview protocol under sustainability if the grantee has contacted partner agencies or programs for a joint effort in finding and securing funding (8.3b) or assigned staff or hired a consultant to pursue potential future funding or other support (8.4)].

4. Type of institutions (e.g., partners, other public and private social service or educational institutions, etc.) currently, or may eventually be involved in sustainability planning

   1. Engagement of new partners in sustainability planning (e.g., new partners who were not involved in HPOG program planning and development)

      1. Description of outreach efforts to engage new partners

      2. Contribution of new partners in sustainability efforts (e.g., ideas, resources, etc.)

5. Rationale for institutional involvement in sustainability planning (e.g., Why did the other institutions involved join in the effort to secure continued funding?)

6. Description of partners’ roles and responsibilities in sustainability efforts

7. Best partners in sustainability efforts

   1. Examples of effective strategies by these partners

3. Working for Sustainability

As we understand from the telephone interviews, staff interviewed believe the approach described [is/will be: Pre-fill with information from question 8.6a from the telephone interview protocol under Sustainability on respondents views on whether the approach is effective in sustaining HPOG programming after the HPOG 2.0 funding ends].

8. Confirmation and description of strategies pursued in efforts to ensure sustainability

   1. Description of strategies used to sustain funding

      1. Discuss if these strategies include using PAGES data on program activity and results or emphasizing that your program is participating in a rigorous evaluation

9. Effective strategies for gaining recognition and support in the community and elsewhere for the HPOG program

10. Successful approaches in leveraging and/or raising non-HPOG funding for current program operations

    1. Type and contribution from partners, agencies, or institutions supporting program operations

    2. Description of process for raising additional funds

    3. Assessment of whether additional funding will carry over when the HPOG grant period ends

5. Prospects for Sustainability

Now let’s discuss your prospects for sustainability. According to the telephone interview, [Pre-fill with information from questions 8.7a and 8.7b from the telephone interview protocol under Sustainability as to whether respondents are optimistic about future funding and sustainability].

11. Rating of program’s prospects for sustained operations

    1. Reasons for rating on program prospects for sustainability

12. Program changes required to support operations when the grant period ends

    1. Examples of program changes needed

6. Lessons for the Field

Finally, we would like your general reflections for the purposes of sharing lessons with the field.

13. Best strategies to ensure sustainability

14. Areas for improvement

15. Advice for those designing and sustaining programs like [name of HPOG program]

Thank you for your time and your thoughtful responses. Please feel free to contact us if you have additional information you’d like to communicate.

Abt Associates Sustainability ▌pg. 2