End of Training Implementation Survey

APPENDIX F. END-OF-TRAINING IMPLEMENTATION SURVEY

This page has been left blank for double-sided copying.

Connecting Adults to Success: Evaluation of Career Navigator Training

End-of-Training Implementation Survey

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is [xxxx-xxxx]. The time required to complete this information collection is estimated to average 10 minutes. If you have any comments concerning the accuracy of the time estimate or suggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202. If you have comments or concerns regarding the content or the status of your individual submission of this form, write directly to: U.S. Department of Education, Institute of Education Sciences, 550 12th Street, SW, Washington, DC 20202.

INSTRUCTIONS SCREEN

Before you get started, here are a few helpful tips:

• To answer a question, click the box to choose your response or type your answer.

• To continue to the next webpage, click the "Next" button.

• To go back to the previous webpage, click the "Back" button. Please note that this command is only available in certain sections.

• For security purposes, you will be timed out if you are idle for longer than 30 minutes.

Please click the “Next” button below to begin or close this webpage to exit.

ALL

[ADULT EDUCATION PROVIDER SITE] [TRAINING NAMe]

The U.S. Department of Education is sponsoring a study of career navigation training called Connecting Adults to Success: Evaluation of Career Navigator Training. Mathematica is conducting this study for the Department in partnership with Manhattan Strategy Group and Social Policy Research. As a reminder, [ADULT EDUCATION PROVIDER SITE] agreed to participate in the study.

The purpose of the study is to understand the effects of training for career navigators on learners’ education and employment outcomes.

• Career navigators may also be referred to as career coaches, career advisors, career consultants, career counselors, college navigators, transition coordinators, or case managers.

• Learners may also be referred to as clients or customers.

The study will compare learners enrolled in adult education services in two different groups of providers, one where career navigators receive training in the fall of 2022, and one where career navigators receive training two years later. Your provider was assigned to a group based on a random process, like a coin flip. All career navigators at your site are in the same group.

Today, you are being asked to take a short survey about the [TRAINING NAME] you just completed. Your participation in this survey is important and will help us learn about the different experiences of career navigators. This survey will take about 10 minutes to complete, depending on your responses. Participation in the survey is completely voluntary and you may choose to skip any question you prefer not to answer. If you are unsure of how to answer a question, please give the best answer you can rather than leaving it blank.

The researchers conducting this study follow the confidentiality and data protection requirements of the U.S. Department of Education’s IES (The Education Sciences Reform Act of 2022, Title I, Part E, Section 183). All your responses will be kept private and used only for research purposes. Your responses will be combined with the responses of other respondents and no individual names will be reported. While there are no direct benefits to participants, your participation will help us learn about career navigators and how career navigation training relates to student outcomes. There are no known risks associated with your participation.

If you have any questions about the study, please contact Mathematica’s survey director, Lisbeth Goble, at XXX-XXX-XXXX or email her at XXX@mathematica-mpr.com.

 By clicking this box, you are confirming that you understand that the information you provide will be kept private and used only for research purposes. You further understand that your answers will be combined with the responses of other navigators so that no individuals will be identified.

ALL

[TRAINING NAME]

Q1. About how many hours did you spend in the training platform (i.e., Moodle) preparing for and completing the training? This could include time spent completing training activities in the training platform or accessing other resources listed in the training platform. Please only include time spent working on training tasks, rather than the total time between the start and end of the training. Your best guess is fine.

______________ HOURS

ALL

[TRAINING NAME]

Q2. Outside of time spent working on the training in the training platform (i.e., Moodle), about how many additional hours did you spend preparing for and completing the training? This could include time spent reading the training manual, conducting research activities, or completing any training homework. Please only include time spent working on training tasks, rather than the total time between the start and end of the training. Your best guess is fine.

______________ HOURS

The following questions will ask you about different topics you may have covered in your training.

ALL

[TRAINING NAME]

Q3. Which of the following topics were covered during [TRAINING NAME]?

Select all that apply

• Characteristics of the adult education population 1

• Policy and programmatic context of adult education programs 2

• Understanding of characteristics of adult learners in the local area 3

• Cultural competency—the ability to understand, communicate with, and effectively interact with people across cultures 4

• Equity, including racial equity, digital equity, and differential access to resources 5

• Trauma-informed care (for example, recognizing the presence of trauma symptoms and the role trauma may play in a learner’s life) 6

• Prioritizing and tailoring services for learners with different needs 7

• Active listening (for example, listening attentively, paraphrasing and reflecting back, withholding judgement) 8

• Motivational interviewing (for example, directive and learner-centered approach to help learners change behavior) 9

• Resources available to learners inside and outside of the organization, including the types of resources that might be useful and how to find and access them 10

• How to create and maintain strong relationships and referral partnerships inside and outside of the organization 11

• Gathering and understanding information about available careers, labor markets, and in-demand industries 12

• Gathering and understanding information about education and training programs or courses 13

• Career explorations and mapping out career pathways 14

• Coaching learners to develop their own planning skills 15

• Goal setting, pursuit, and monitoring 16

FILL ROW IF ITEM SELECTED IN Q3

Q4. For each of the following topics, please indicate to what extent you agree with the following statement:

The information we covered on this topic will be useful to my career navigator role.

	STRONGLY AGREE	AGREE	DISAGREE	STRONGLY DISAGREE
a. Characteristics of the adult education population	1 🔾	2 🔾	3 🔾	4 🔾
b. Policy and programmatic context of adult education programs	1 🔾	2 🔾	3 🔾	4 🔾
c. Understanding of characteristics of adult learners in the local area	1 🔾	2 🔾	3 🔾	4 🔾
d Cultural competency—the ability to understand, communicate with, and effectively interact with people across cultures	1 🔾	2 🔾	3 🔾	4 🔾
e. Equity, including racial equity, digital equity, and differential access to resources	1 🔾	2 🔾	3 🔾	4 🔾
f. Trauma-informed care (for example, recognizing the presence of trauma symptoms and the role trauma may play in a learner’s life)	1 🔾	2 🔾	3 🔾	4 🔾
g. Prioritizing and tailoring services for learners with different needs	1 🔾	2 🔾	3 🔾	4 🔾
h. Active listening (for example, listening attentively, paraphrasing and reflecting back, withholding judgement)	1 🔾	2 🔾	3 🔾	4 🔾
i. Motivational interviewing (for example, directive and learner-centered approach to help learners change behavior)	1 🔾	2 🔾	3 🔾	4 🔾
j. Resources available to learners inside and outside of the organization, including the types of resources that might be useful and how to find and access them	1 🔾	2 🔾	3 🔾	4 🔾
k. How to create and maintain strong relationships and referral partnerships inside and outside of the organization	1 🔾	2 🔾	3 🔾	4 🔾
l. Gathering and understanding information about available careers, labor markets, and in-demand industries	1 🔾	2 🔾	3 🔾	4 🔾
m. Gathering and understanding information about education and training programs or courses	1 🔾	2 🔾	3 🔾	4 🔾
n. Career explorations and mapping out career pathways	1 🔾	2 🔾	3 🔾	4 🔾
o. Coaching learners to develop their own planning skills	1 🔾	2 🔾	3 🔾	4 🔾
p. Goal setting, pursuit, and monitoring	1 🔾	2 🔾	3 🔾	4 🔾

FILL ROW IF ITEM SELECTED IN Q3

Q5. Which three topics of the course do you think you will applymost often in your role as a career navigator? (Adapted from NCTN end of training survey Q19)

	Select three
a. Characteristics of the adult education population	|__|
b. Policy and programmatic context of adult education programs	|__|
c. Understanding of characteristics of adult learners in the local area	|__|
d Cultural competency—the ability to understand, communicate with, and effectively interact with people across cultures	|__|
e. Equity, including racial equity, digital equity, and differential access to resources	|__|
f. Trauma-informed care (for example, recognizing the presence of trauma symptoms and the role trauma may play in a learner’s life)	|__|
g. Prioritizing and tailoring services for learners with different needs	|__|
h. Active listening (for example, listening attentively, paraphrasing and reflecting back, withholding judgement)	|__|
i. Motivational interviewing (for example, directive and learner-centered approach to help learners change behavior)	|__|
j. Resources available to learners inside and outside of the organization, including the types of resources that might be useful and how to find and access them	|__|
k. How to create and maintain strong relationships and referral partnerships inside and outside of the organization	|__|
l. Gathering and understanding information about available careers, labor markets, and in-demand industries	|__|
m. Gathering and understanding information about education and training programs or courses	|__|
n. Career explorations and mapping out career pathways	|__|
o. Coaching learners to develop their own planning skills	|__|
p. Goal setting, pursuit, and monitoring	|__|

ALL

[TRAINING NAME]

Q6. To what extent do you agree with the following statements. (NCTN end of training survey)

	Select one per row
	STRONGLY AGREE	AGREE	DISAGREE	STRONGLY DISAGREE
a. I was satisfied with the amount of time I spent on this course	1 🔾	2 🔾	3 🔾	4 🔾
b. I found it easy to navigate the training platform	1 🔾	2 🔾	3 🔾	4 🔾
c. I know more about career navigation than I did before taking the course	1 🔾	2 🔾	3 🔾	4 🔾
d. The instructor’s contributions to course activities added value to my learning	1 🔾	2 🔾	3 🔾	4 🔾
e. The instructor was responsive to my questions or issues regarding the [TRAINING NAME]	1 🔾	2 🔾	3 🔾	4 🔾
f. I will be able to apply what I learned to my work helping adult learners navigate their further education/training and careers	1 🔾	2 🔾	3 🔾	4 🔾

ALL

[TRAINING NAME]

Q7. How satisfied are you with the quality of the training you received from [TRAINING NAME]? (KIPP Leadership Survey)

 Very satisfied 1

 Somewhat satisfied 2

 Somewhat dissatisfied 3

 Very dissatisfied 4

ALL

[TRAINING NAME]

Q8. How likely would you be to recommend [TRAINING NAME] to a colleague? (Adapted from KIPP leadership survey)

 Definitely recommend 1

 Probably recommend 2

 Probably not recommend 3

 Definitely not recommend 4

Q8 = 3 OR 4

[TRAINING NAME]

Q9. What is the most important reason why you would not recommend [TRAINING NAME] to a colleague?

Thank you for your time!