Instrument 4: HPOG 2.0 National Evaluation in-person implementation interviews

Instrument 4-B OMB # 0970-0462

Expiration Date xx/xx/xxxx

Instrument 4-B: Basic Skills Training Provision

Site Visit Preparation for Interviewers

• Review Responses from telephone interviews. The telephone interviews with grantees include information about the HPOG program’s approach to basic skills training. The exhibit below highlights the interview questions associated with this topic in the telephone interview protocol.

Exhibit 1: Interview Topics to Review in Preparation of Site Visits

Interview Topic Areas	Questions
Basic Skills	5.1 – 5.6

• Review the definition of basic skills that is included in PAGES. Basic skills training is the combination of one or more courses or activities designated as basic skills training by a grantee. These courses or activities may include: Adult basic education, Adult secondary education, College developmental education, and English language acquisition. Basic skills training may be delivered prior to healthcare training or concurrent with healthcare training. Basic skills training that is integrated with healthcare occupational training will be listed with healthcare occupational training activities and should not be included as a distinct basic skills training.

• Adult basic education: A class or instructional program which teaches basic skills including reading, mathematics, and writing, provided to adults with skills at or below 8th grade level and which does not charge college tuition.

• Adult secondary education (e.g. GED, HiSET, TASC): A class or instructional program which teaches secondary education material to adults with skill levels between 9th and 12th grade levels and which does not charge college tuition. Such classes typically prepare students for testing to receive a high school equivalency credential such as GED, HiSET, or the Test for Assessing School Completion (TASC).

• College developmental education: Class or series of classes offered by a college and charging tuition which are designed to raise participants’ math, reading, or writing skills to enable them to succeed in college-level work.

• English language acquisition: A class or instructional program to help adult English language learners to improve their proficiency in the English language.

• Review the Screening Interview to Identify Respondents for the HPOG 2.0 National Evaluation First-Round Telephone Interviews. Create a list of the local HPOG program staff, key partners, and stakeholders with knowledge and expertise about Basic Skills. Always interview the program director and include others on the list as well. This may include program coordinators, curriculum developers, instructors, tutors, partners or other stakeholders that have a role to play in the HPOG program’s basic skills intervention.

• Customize interview topics by the type of respondent. All topic areas should be covered when interviewing program directors. Based on your list of respondents, tailor the topics and/or questions to the role and level/type of involvement of each respondent (e.g., instructors should not receive the same questions as partners).

• Customize interview topics with information collected from the telephone interviews. Use the italicized notes on which interview questions to customize for selected topics in the basic skills training guide to prepare in advance of the site visit.

HPOG 2.0 Implementation Study

Site Visit Guide on Basic Skills Training Provision

Introductory statement to [Pre-fill with Respondent Type]: Thank you for taking the time to talk to us today. As you may know, the Administration for Children and Families (ACF) of the U.S. Department of Health and Human Services has funded the 2^nd Generation Health Profession Opportunity Grants (HPOG 2.0). ACF has also funded an evaluation study of HPOG 2.0; we are members of the research team. We are visiting HPOG 2.0 grantee programs to document in greater detail program design and implementation. The major purpose of this site visit is to gather more information about your HPOG 2.0 program’s approach to basic skills training. By basic skills training, we are referring to courses or activities that may include: Adult basic education, Adult secondary education, College developmental education, and English language acquisition. We plan to use the information in our research reports that describe the HPOG 2.0 initiative and analyze its results and effectiveness.

In preparation for our discussion today, we have worked with representatives from your program to identify the most appropriate staff, partners, and stakeholders to talk to us about basic skills training. We have also reviewed responses about basic skills training from telephone interviews we previously conducted with [name of local HPOG program] staff, management and partner organizations in [month and year of telephone interview], as well all other information about the [name of local HPOG program] program we already have available.

The interview will take about 90 minutes to complete. Interview questions are structured around seven key topics:

1. Program design and start-up,

2. Recruitment and intake,

3. Education and training activities,

4. Support services tailored/targeted to this group,

5. Partnerships,

6. Participant progress and outcomes, and

7. Lessons for the field

Before we begin, I would like to assure you that all of your responses will be kept private and used only for this research study. Your name will not appear in any written reports we produce. Also, the interview is voluntary and you may choose not to answer any specific question and may end the interview at any time.

According to the Paperwork Reduction Act (PRA), an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this collection is 0970-0462 and it expires on xx/xx/xxxx. If you have comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, please send them to [Contact Name]; [Contact Address]; Attn: OMB-PRA (0970-0462).

Do you have any questions before we begin? I would first like to start by giving you and each individual a minute to introduce yourselves. Please tell me:

• Your name

• Your role within [name of local HPOG program]

I have some topics I would like to cover, but these interviews are meant to be exploratory. Please feel free to bring up additional points that you would like to make about basic skills training and services and how it contributes to your HPOG program.

1. Program Design and Start-up

Let’s first talk about how [name of local HPOG program] designed and implemented the basic skills components of your HPOG program.

1. Rationale for addressing basic skill needs within your program

2. Selection of occupation(s) for training this population, local healthcare employer needs, and education/skill levels needed

3. Use of evidence-based models to improve basic skill levels

4. Differences in approach to service delivery for individuals with low basic skills as a part of HPOG compared to approach prior to HPOG

5. Capacity of the local/regional system to serve this population

   1. Initial partner recruitment to support individuals with low basic skills

6. Any upfront policy or practice challenges, including the following:

   1. Licensure or certification for occupations requiring a high school credential

   2. Employer preferences to hire job candidates with a high school diploma

   3. College/training provider enrollment policies that require a high school credential

   4. Internal processes that hinder basic skills instruction, including resistance from instructors to include basic skills instruction in occupational training

7. Start-up activities such as curriculum development, teacher training/professional development, assessments/testing, and GED preparation

2. Recruitment and Intake

Next, I will ask you about how individuals with basic skill needs are recruited and enrolled in the program. According to the information collected from the Evaluation Design and Implementation Plan (EDIP) that we developed along with [name of local HPOG program] to guide the integration of the research study into program operations, we understand [name local HPOG program] uses the following recruitment mechanisms: [Pre-fill with information from question 2 and 3 from the HPOG Site Team Program Information Form under Outreach and Recruitment].

8. Confirm recruitment sources

9. Main recruitment sources for individuals with low basic skills (e.g., current participants, partner referrals, etc.)

10. Targeting outreach to recruit individuals with low basic skills

11. Assessments of basic skills (e.g., TABE, English language assessments, college entrance exams like COMPASS, etc.)

12. Effectiveness of assessments

13. Criteria for placement in basic skills services (e.g., classes, tutoring, etc.)

14. Enrollment in program (e.g., any policy changes needed for those without a high school credential)

15. Other enrollment criteria for those with basic skill needs (e.g., low-skilled individuals targeted, etc.)

16. Financial aid for those with basic skill needs (e.g., use of ability to benefit provision, etc.)

17. Case management (e.g., development of initial plans for participants with basic skill needs to be successful, etc.)

3. Education and Training Activities

According to information we collected from the telephone interview and the HPOG data system, PAGES, [name of local HPOG program] is using the following strategies to accommodate individuals entering your program with very low basic skills: [Pre-fill with information from questions 5.3, 5.4a, and 5.4b from the telephone interview protocol under Basic Skills Training and information from PAGES].

18. Confirm strategies and approach

19. For each strategy being used, discuss the following:

1. Staff responsible

2. Structure and approach

3. Frequency and intensity

4. Course/training sequencing (e.g., how time to completion is adjusted for those with basic skill needs) and duration

20. Any adaptations for those with low basic skills (e.g., work-based learning opportunities, etc.)

4. Support Services Tailored/Targeted to this Group

We also want to understand support services that are tailored or targeted to this group. Let’s discuss:

21. Career navigation and employment placement

22. Academic counseling

23. Tutoring (e.g., individually or by group)

24. Communication between support service staff and instructors with respect to participants with basic skill needs

5. Partnerships

According to the telephone interviews, we understand that your partners play the following roles in the HPOG program with respect to services for those with low basic skills: [Pre-fill with information from question 2.2 from the telephone interview protocol under Partnership Characteristics and Roles]. Let’s discuss:

25. Roles of other organizations in basic skills instruction

26. History of the partnership (e.g., when it started, how it evolved, etc.)

27. Communication between [name of local HPOG program] and partners

28. Extent of negative perceptions of individuals with low basic skills by colleges and employers

    1. Strategies to overcome negative perceptions

29. What works in the partnership

30. Areas for improvement

6. Participant Progress and Outcomes

Next, we want to understand how you measure progress for your students with low basic skills and how this compares to how you measure progress for other students who are not basic skills students.

31. Process for measuring progress among individuals with low basic skills

32. Process for measuring progress among current students who are not basic skills students

33. Results to date (e.g., evidence on outcomes for those needing basic skills instruction)

7. Lessons for the Field

Finally, we would like your general reflections for the purposes of sharing lessons with the field.

34. Effectiveness of basic skills training approaches

35. Challenges with delivery of basic skills training

36. Areas for improvement

37. Advice for those designing programs like [name of HPOG program]

Thank you for your time and your thoughtful responses. Please feel free to contact us if you have additional information you’d like to communicate.

Abt Associates Basic Skills Training ▌pg. 2