Attachment A2_Matrix of Questions that Appear in the PRS and Protocols

Attachment 2: Matrix of Questions that Appear in the PRS and Protocols

Question	Protocol (Question Number)	Justification for including in Protocol
Please tell us your first name and what tribal HPOG program are you enrolled in	Participant Focus Group/Interview Guide (Introductory question)	The PRS will provide NORC with the number of enrollees in each program, and that data will be used for the official count of program enrollees. Beginning the focus groups/interviews with this lead-in question is meant to serve as an introduction and contextual foundation for the remainder of the questions. From here, the protocol focuses on the program participants’ experiences and perspectives, which is information that cannot be gathered through the PRS.
Are supports or services provided to help you stay enrolled in the tribal HPOG program? What services or supports do you receive? Which, if any, are most helpful?	Participant Focus Group/Interview Guide (Question 7); Program Completer Interview Protocol (Question 5); Program Non-Completer Interview Protocol (Question 4)	The PRS will provide data on what services are received by each program participant. The focus group/interview question is included to garner information on the support services that are most helpful and why. The initial questions of “what services are provided” and “which services were received” are used to introduce the topic, enabling NORC to then ask participants about their perceptions of the services, and what worked best. By asking the question of current participants, program completers, and program non-completers, the evaluation will be better positioned to understand what services supported success, and why, as well as services that may have been lacking.
What program did you participate in? Why did you choose this program? Competencies to be developed (i.e., skill requirements of the target occupation) Pre-requisites Start and end dates. Program location Convenience Accessibility	Program Completer Interview Protocol (Question 1a); Program Non-Completer Interview Protocol (Question 1a)	The PRS will provide NORC with the number of enrollees in each program, and that data will be used for the official count of program enrollees. In addition to using this question to serve as an introduction and contextual foundation for the remainder of the questions, it also provides additional information that cannot be gleaned from the PRS. Specifically, the question seeks to understand why the participant chose the program, not just that they choose it. By asking this question, the evaluation will be able to explore what factors facilitate and support program enrollment. The sub-points provide examples of contextual factors that may support a participants decision to enroll in a certain program, and can be used as probes to find out if they had any effect on the program selection. So while the PRS does capture when each participant starts a program, it does not capture if that start date had any bearing on participation in the program.
Did you earn a certificate, degree or licensure? If so, what certificate, degree or licensure did you earn?	Program Completer Interview Protocol (Question 8a); Program Non-Completer Interview Protocol (Question 7)	The PRS will provide NORC with the number of enrollees in each program to receive a certificate degree or licensure, and that data will be used for the official count in the evaluation. This question is included in the interview to serve as an introduction to the topic. It is followed up by important questions regarding the components of the program which supported success, contextual factors outside the program that facilitated success, and the participant’s plans moving forward. As such, the interview discussion provides perceptions and reasoning behind the data collected through the PRS. Furthermore, by asking this question of program non-completers, additional information is gathered on the usefulness of the program beyond receiving a degree or licensure (e.g., the transferability of credits and skills learned while enrolled in the program).
Did you have a job prior to participating in the program? If yes, were you working in a healthcare field? Are you currently employed? Please indicate whether you are: Employed full-time Employed part-time Serving internship Did you continue to receive any support after you completed the program? If so, describe. Examples are job retention and advancement supports (e.g., mentoring, peer support groups)	Program Completer Interview Protocol (Question 9a-c)	The PRS will provide NORC will data on the rate of employment 6 months after participants complete a program, and that data will be used as the official rate in the evaluation. While the interview questions are similar to the data collected in the PRS, they provide an opportunity for respondents to elaborate on their outcomes from program participation, and express any experiences derived from the program and its components. Furthermore, the questions provide an introduction to the topic of employment, therefore serving as a conceptual foundation for more in-depth questions regarding how their experiences in the program supported their employment (e.g., “Did the program help you find this job?”, “Do you think you will be able to advance in the future?”). As such, but asking these questions in the interview, the evaluation will gain important information on the perceived strengths and weaknesses of the program in regards to employment, while also providing feedback on why components were successful or not.
You started but did not finish the program. Please tell me about your decision.	Program Non-Completer Interview Protocol (Question 1c)	The PRS will provide NORC with the number of program participants who exit the program before completion, and that data will be used for the official count in the evaluation. This question is included in the interview to gather contextual factors from participants on why they did not complete the program.