Appendix J_Interview Guides

Appendix J

Interview Guides

Appendix J

Interview Guides

OMB #: 0970-XXXX

Expiration Date: XX/XX/XXXX

Program Staff Interview Guide: Case Managers, Job Developers, Job Coaches

Subsidized and Transitional Employment Demonstration (STED)

Enhanced Transitional Jobs Demonstration (ETJD)

Staff Role and Background

1. What agency do you work for?

2. Please give a brief description of your role in the agency

3. Which aspects of the program did you work on?

[Prompts]

• Recruiting participants;

• Assessing participants through formal tests or interviews;

• Providing or helping participants access job-readiness training or other training

• Recruiting and enrolling employers

• Placing participants in subsidized jobs

• Providing case management

• Visiting worksites and communicating with employers during the subsidized job

• Providing or helping participants access support services

• Helping participant to find and prepare for unsubsidized employment

Overall Goals

4. In your own words, what are the primary goals your program is trying to achieve? What are its main objectives?

5. What are the key ingredients that lead to the program’s successes? [probe: what is it about the program that helps people get (better) jobs than they could get on their own.]

6. Describe the subsidy structure and other features of the subsidy. [Probe: duration; wage level; fringe benefits and UI; employer contribution; employer of record; special features and incentives; interaction with TANF policies.]

Recruitment and Intake

7. Let’s assume I am a potential participant. Walk me through the set of activities that happen to inform me of the study and recruit me into it.

[Prompts]

• How will I have learned about the study and the random assignment process?

• What will I have seen / heard during the orientation/initial set of activities (what are people told, what activities occur)?

• What staff have I worked with up to this point?

8. What is the nature of your recruitment or referral process?

[Prompts]

• What is the mix of mandatory versus voluntary clients?

• What is the mix of direct referrals versus walk-ins?

9. To what extent do you “market” the program and/or the study? To whom? How?

10. What kinds of eligibility criteria are in place and why?

11. What proportion of those referred show up for an orientation/the initial set of activities? Do you have a sense of why people do not sign up for the project? Explain?

12. Do you face any barriers to recruitment?

• If so, what are those barriers?

• How have you tried to overcome these barriers? Are these strategies working?

13. How many people have you recruited (i.e., shown up for orientation/initial activities) since [START DATE]? How many of these people were enrolled/entered research study? What is the reason why people did not participate in the research study?

14. How many slots are available at a point in time? Are all slots usually filled?

Initial Services

15. Let’s assume I’m a study participant who has just been assigned to your program. Please walk me through, step by step, what happens to me after random assignment. I want to understand the overall flow through the program, what activities I’ll do in what sequence and for how long, and the types of staff involved in providing these services. (Probe for which services happen before assignment to a subsidized job, which during, and which after).

(Ask 16- 19 only if not covered in prior answers):

16. Once the individual has been randomly assigned, how is he notified and what are the next steps? For example, when does he meet with the program staff to do intake?

17. What happens to participants who do not show up for the activities that occur prior to placement in subsidized jobs?

• How much of a problem is attendance at this stage?

• Are there any incentives to participate in the pre-placement activities? Explain.

18. For any or all of the various program components, is a specific curriculum used? If yes, how was this chosen and why? (probe for each component separately: job readiness, job search, anything else)

19. If no, how was the structure and activities of your organization’s programming decided upon?

20. What types of information do you collect about participants?

• When does this happen?

• How formal is the process (e.g., are specific assessment tools used or is it more general questions?)

• What is the information used for?

Assessment

21. What methods do you use to assessment participants? Describe the assessment process. [probe: standardized assessment; format – tool or interview (group or individual)].

22. What information is learned through the assessment -- basic skill levels, employment interests, barriers, other? How is this information used?

Job Readiness Classes

23. What types of pre-employment activities are offered to participants in preparation for their job search?

• What fraction of participants get each of these?

• Do they participate in a group? How many in a group?

24. How long do job readiness activities last?

• Do you follow a standard classroom structure (e.g., a new class starts each week)? Is the program open-entry and exit? Or something else? Explain…

25. Is a standard curriculum used for any or all of the job readiness activities?

• <If yes> What materials do you use?

How did the organization decide to use this curriculum?

• <If no or a mix> How did the organization decide how to structure the activities?

26. What topics are covered (probe: developing good work ethic; appropriate workplace clothing; understanding employers’ expectations; communication skills; getting along with coworkers; problem solving; handling difficult situations or anger management; stress management; time management.)

• Are any of the following areas of training provided: computer training; customer service; basic reading/math/writing skills; GED; ESL?

• Which of the following job search topics are covered: job search skills; resume preparation; interview skills?

27. What happens to participants who do not show up for job readiness activities?

28. How much of a problem is attendance at this stage?

29. Which staff are involved in the provision of job readiness activities?

30. On average, how many hours of training do participants receive prior to being placed in the subsidized job?

31. What were the most significant challenges your agency faced in providing job readiness training?

32. How often do case managers or other staff meet with or contact participants when they are in job readiness activities? Does this vary over time?

33. How many participants does each case manager work with at a time?

Subsidized/Transitional Jobs

34. Please describe the type(s) of transitional jobs opportunities available to participants.

• How and when were these positions developed?

• Who is responsible for finding these positions?

• How difficult is it to find subsidized positions with employers/organizations?

• (probe for need for more slots, need for more diverse employment opportunities, etc)

• Are there any criteria that employers must meet?

• Do any of the TJs offer the potential to convert to permanent FT jobs?

35. What factors were considered in matching participants with subsidized jobs?

36. Were all participants who completed the initial activities placed in subsidized jobs? If not, why?

37. How soon after entry into the program do participants start working?

• What types of jobs/tasks do they perform?

• <If multiple TJ providers are used>: How is the decision made about which type of job a participant is assigned to? (probe for slot availability vs. participant interests vs. some other decision making process)

• What is the pay rate?

• How long does the assignment last? How is that determined (probe for whether or not there issues with the number of slots available and if so, what are they)?

38. What happens if participants don’t show up for their assignment?

• How do you learn that participants haven’t shown up?

• How much of a problem is this?

• Who is responsible for monitoring attendance and other types of progress/benchmarks?

39. How long do participants stay in a transitional job, relative to their assignment?

• What are the reasons that people leave a TJ before their assignment ends (probe for extent to which people find unsubsidized employment vs. just do not show up vs. recidivate)

40. What is the nature of supervision on the job (e.g., who provides and in what manner)?

• How are issues about job performance communicated to other program staff?

• How are issues about job performance communicated to the participant? How are the issues addressed?

• What types of issues do the participants face related to job performance? (probe for specific examples and how the problems were resolved)

41. What types of issues do participants need the most help with when it comes to on-the-job performance?

42. Why do some participants not complete their subsidized jobs?

43. How much and what types of contact occur between your organization and the TJ provider(s)?

Probes:

• Who connects with whom?

• How often are you in contact?

• What is the purpose of the contact?

• Is the contact formalized (for example, in regularly scheduled meetings or on an ad-hoc basis?)

• Do you discuss issues/needs of individual clients?

• What kind of information did you obtain about individual workers?

• Do you keep each other informed on participants’ progress?

44. How often do case managers (other staff??) meet with or contact participants when they are working in the subsidized job? Does this vary over time?

45. Are these meetings in a group or one-on-one? Where do they take place?

46. Are there problems with no-shows to these meetings?

47. What types of issues are discussed?

48. What types of issues do participants need the most help with when it comes to their performance on the job?

49. Do participants have needs/barriers that you/the program is not well-equipped to help them with?

Worksite visits

50. For each participant, how many visits did you/your agency conduct to the worksite during the first two weeks of the subsidized job? How many after the first two weeks?

51. How often do you/your agency communicate with worksite supervisors? What issues are discussed?

Employer Recruitment for Subsidized Positions

52. How did your agency recruit employers? How many (and which) staff members were responsible for recruiting employers?

53. What challenges did you face in recruiting employers?

54. What reasons did employers provide for NOT wanting to participate?

55. What type of employer was more likely to agree to participate?

56. Did you target particular types of employers?

57. Did you set specific criteria that employers had to meet?

58. Were there specific characteristics about the placement or the employer that you looked for?

59. Did any employers leave the program? Explain.

Transitions out of TJs

60. What is the overall process for transitioning participants out of transitional jobs and into regular employment?

61. How often does the subsidized employer retain the participant in an unsubsidized job? What factors account for this?

62. At what point are participants considered ready to work with job placement/job development staff for regular/unsubsidized employment? Who decides? Based on what criteria?

63. What happens if participants don’t show up for job search activities?

• How much of a problem is this?

• Who is responsible for monitoring attendance and other types of progress/benchmarks?

64. How long do/can participants stay in job search activities?

• What are the reasons that people leave the job search component earlier than usual (probe for extent to which people find unsubsidized employment vs. just do not show up vs. recidivate)

65. What types of job development do staff do? (e.g., working with specific employers to identify specific job openings? Attending job fairs? Other types of activities?)

• To what extent is job development tailored to individual participants vs. generic strategies to identify openings?

• How are participants matched with open jobs?

66. Are there specific industries or employers that are being targeted for job development?

• If yes, why these industries?

• What are typical starting wages?

• How was that relationship developed?

67. In what ways do job developers/other staff use participants’ placement in a transitional job when marketing specific participants to employers or explaining the program to employers? Explain.

• In your opinion, does putting people in TJ first makes it easier to get them a permanent job, get them a better permanent job, or not matter? Why? How?

68. What types of activities and services are available to participants who are looking for unsubsidized employment? (probe for help with applications/resumes, helping clients prepare for interviews at specific firms? Following up with employers to see how interviews go?)

• To what extent do participants use these services? Why or why not?

• How often would a job developer typically meet with a participant?

69. What kind of post-employment and employment advancement services are available?

• Which staff provide them and for how long?

• How many participants receive the various types of services and how often?

70. How closely are participants followed once they leave the transitional job? By whom and how? (probe for whether this is formal or informal follow-up)

71. What types of challenges have participants faced thus far in terms of finding and keeping jobs? What is the program doing in an attempt to address these challenges?

72. How would you characterize the local labor market and the opportunities for workers like the program participants?

73. Are there incentives for participants to find and/or stay in unsubsidized jobs? What is the schedule for incentives or bonuses and how much money is offered?

• How many clients are reaching the various milestones?

• What is the reaction of the clients to the retention bonus system?

• Walk through how the incentive system is implemented (documentation, payment system, etc.)

Bonus Payments

74. Did you provide any bonus payments or other rewards to participants who achieved specific benchmarks? Describe. How many participants received these rewards?

Provision and availability of other services:

75. What other services do participants receive that your organization provides?

• How many participants receive various services and how often?

• How is need for these services determined?

76. What other services do participants receive that are provided by other agencies/organizations?

• How many participants receive various services and how often?

• How is need for these services determined?

• How do participants find out about these services? (probe for formal referrals vs providing information about the service vs some other mechanism)

• How and in what ways do you monitor whether or not participants used these services?

77. What types of services do participants need that this program or your partners do not provide? Explain… (probe for whether this is due to lack of funding, non-existence of this type of service, lack of partners who provide this service, etc).

Collaboration:

78. What is the nature of the contact between you and (the other major collaborators) in this effort, in particular, the partner site?

79. Has the relationship changed since the beginning of the project? If yes, how and why?

80. What are the strengths of the collaboration?

81. What kinds of barriers to collaboration have been experienced with the various partners? How have you tried to overcome these barriers? Have these strategies working?

82. What are the gaps in services for the population you serve?

Program Administration

83. How many staff were actively working on running the program?

84. Did you hire additional staff to run this program? How many?

85. Did you restructure existing staff to run this program?

Organizational Capacity-

86. Tell me about [name of organization]:

• When was it founded?

• What is its mission?

• (If relevant) What populations has it traditionally served?

87. Please explain the overall structure of the organization—if you can provide an organizational chart that would be very helpful.

• How many FTEs work for the organization?

• How many staff work with this program?

88. What were the reasons your organization wanted to participate in this demonstration? Explain. What proportion of the organization’s budget is devoted to this initiative?

89. Organizationally, what have been the greatest benefits to participating in this demonstration?

90. Organizationally, what have been the greatest challenges to participating in this demonstration?

OMB #: 0970-XXXX

Expiration Date: XX/XX/XXXX

Program Staff Interview Guide: Director

Subsidized and Transitional Employment Demonstration (STED)

Enhanced Transitional Jobs Demonstration (ETJD)

Organizational Capacity

1. Please explain the overall structure of the agency/organization—if you can provide an organizational chart, that would be very helpful.

• How many FTEs total?

• How many staff work with this program?

2. What is the organization’s experience with subsidized jobs?

3. What is the background of the staff who work with the program?

4. Were any new staff hired or brought into the program? If yes, for what positions and why?

5. What proportion of the organization’s budget is devoted to this initiative?

6. What are your current sources of funding for this program?

Overall Goals

7. In your own words, what are the primary goals your program is trying to achieve?

8. What are the key ingredients that lead to the program’s successes? [probe: what is it about the program that helps people get (better) jobs than they could get on their own.]

Program Experiences

9. What were the reasons your organization wanted to participate in this demonstration? Explain

10. Organizationally, what have been the greatest benefits to participating in this demonstration?

11. Organizationally, what have been the greatest challenges to participating in this demonstration?

12. Do you feel there are gaps in the services provided or the program design?

Context

13. What other initiatives are operating or in development?

Interview Guide: Employers

Subsidized and Transitional Employment Demonstration (STED)

Enhanced Transitional Jobs Demonstration (ETJD)

Employer Background

1. Tell me about your company, such as the type of work you do, the number of employees you have, and a little bit about the history of the organization.

Local Economic Context

2. How is the local economy doing? How has the local economic climate changed over the past two years?

3. What broad economic trends are affecting your business?

4. Have there been other significant changes in the local economy (e.g. industries closing/new business starting) that have made a difference for employment opportunities in your area?

Program Involvement

5. Describe your involvement with this program. Do you hire participants? Provide training?

6. How did you hear about the program?

7. What were the reasons your organization wanted to participate in this program? Explain

8. How much communication do you have with program staff? With whom do you typically communicate? What is the nature of the communication?

Subsidized Job Placements

8. How many program participants have you placed in subsidized positions?

9. Please describe the type(s) of jobs in which participants work.

   1. How and when were these positions developed?

   2. Do any of the TJs offer the potential to convert to permanent FT jobs?

10. What is the nature of supervision on the job (e.g., who provides and in what manner)?

11. How are issues about job performance communicated to program staff?

12. How are issues about job performance communicated to the participant? How are the issues addressed?

13. What types of issues do the participants face related to job performance? (probe for specific examples and how the problems were resolved)

14. What happens if participants don’t show up for their assignment?

15. How much of a problem is this?

16. Who is responsible for monitoring attendance and other types of progress/benchmarks?

17. What types of issues do participants need the most help with when it comes to on-the-job performance?

18. How much and what types of contact occur between your organization and the program staff?

Probe for:

• Who connects with whom?

• How often are you in contact?

• What is the purpose of the contact?

• Is the contact formalized (for example, in regularly scheduled meetings or on an ad-hoc basis?)

• Do you discuss issues/needs of individual clients?

• Do you keep each other informed on participants’ progress?

8. How long do participants stay in the subsidized job? What are the reasons that people leave before their assignment ends (probe for extent to which people find unsubsidized employment vs. just do not show up).

9. Have you kept any participants on in unsubsidized jobs? If not, why not?

Partnership with the Program

8. Has the relationship between your organization and the program changed since the beginning of the project? If yes, how so?

9. What are the strengths of the partnership?

8. What challenges do you face in partnering with this organization?

Benefits and Challenges

8. What have been the greatest benefits to participating in this program?

9. What have been the greatest challenges to participating in this program?

8. Is there anything else that would be useful for the research team to know?

OMB #: 0970-XXXX

Expiration Date: XX/XX/XXXX

Interview Guide: Referral Partners

Subsidized and Transitional Employment Demonstration (STED)

Enhanced Transitional Jobs Demonstration (ETJD)

1. Please describe your current relationship with the program.

2. How long have you worked with [lead agency]? What was the nature of your relationship before the study began? How has it changed?

Referrals and enrollment

3. How does the referral process work?

4. What is your role in determining eligibility and appropriateness for services? When/how are these decisions made?

5. Please describe what type of client you would be likely to refer to the program. What kind of client would you be unlikely to refer?

6. How selective do you feel you have been with regard to the people that you are referring to the program? (Probe for job readiness, harder to serve population, etc).

7. What materials, if any, do you use to market the program?

8. Since the study began, have you faced any challenges with the referral and recruitment process? If so, what are those challenges? How have you tried to overcome them? Are these strategies working?

Random Assignment/Evaluation

9. What role do you have in the random assignment notification?

10. What services do those assigned to the control group receive? Are control group participants being referred to other programs?

11. How do you track the control group members?

Other

12. Are there additional services available in your community to assist clients with specific needs? What are these programs? What is your relationship with these programs?

13. [Ask staff to provide a summary of the organizations a control group member could access.] What are these programs? How long have they existed?

14. How has the program compared with your expectations prior to beginning random assignment? Any surprises or unexpected challenges? If so, do you have plans to address these challenges?

OMB #: 0970-XXXX

Expiration Date: XX/XX/XXXX

Interview Guide: Referral Partners (Parole and Corrections Officers)

Subsidized and Transitional Employment Demonstration (STED)

Enhanced Transitional Jobs Demonstration (ETJD)

Collaboration

1. What is the nature of the contact between you and [staff from Program Name]?

2. Has the relationship changed since the beginning of the project when the funding was awarded? If yes, how and why?

3. What kinds of barriers to collaboration have been experienced with the various partners? How have you tried to overcome these barriers? Have these strategies working?

4. What are the gaps in services for the population you serve?

Relationships with the Criminal Justice System

5. What is the relationship between the organization providing the TJ and the state and local criminal justice system, including parole agencies? (Cover issues related to funding, referrals, reporting, etc.)

6. To what extent is the criminal justice system aware of this initiative and supportive of it? To what extent is the criminal justice system providing direct referrals to the project?

7. To what extent does [Program Name] report information back to the criminal justice system agencies? If so, what do they report back?

The Criminal Justice Context

8. What is the context of the state and local criminal justice system? To what extent would you say the DOC is interested in providing employment programming and help to assist former prisoners?

9. What other initiatives are operating or in development, and how might this affect the TJ demonstration?

10. What types of laws and regulations exist around parole violations that could affect operations of the demonstration?

11. How quickly do parole agencies violate people? Do they follow a graduated sanctions policy, or are they quick to send someone back to prison?

12. How much do parole agencies have an interest in providing “services” to clients rather than just providing “supervision”?

Corrections Questions

13. How did you find out about [Program Name]? Why did you decide to participate?

14. What are the characteristics of people you consider appropriate to refer to [program name]? What types of people would you consider inappropriate for the program?

15. Have you referred people who did not show up at the program? How did you find out? What happened?

16. To what extent does the DOC provide employment programming and help to assist former prisoners? (probe for the extent to which parole provides “services” to clients as opposed to “supervision”)

17. Without mentioning names, can you tell me whether any of your clients in this program have been sanctioned? For what? Do you know whether their employer found out? What did you do about it?

18. More generally, can you tell me about your agencies policies for responding to violations of conditions of supervision? Compared to other parole agencies you might be familiar with, how quickly does your agency respond to violations of supervision? How quickly do you send someone back to prison? Do you follow a graduated sanctions policy?

19. What types of laws and regulations exist around parole violations that could affect operations of the demonstration?

J-1