Appendix Q.1. Summary of requested changes – revised May 2022

Appendix Q.1. Summary of requested changes

Please find below a summary of the changes being requested in May 2022 to information collection activities under OMB #0970-0545:

Baseline Survey (Instrument 1. Baseline survey – revised)

We propose the following changes to the baseline survey. These proposed changes do not impact the burden estimates provided in the original ICR submission.

1. We removed most items from a question on potential employment challenges (C6) as these items are not needed for the impact study. The revised question now only asks about COVID-19-related challenges.

First Follow-up Survey (Instrument 3. First follow-up survey – revised)

We propose the following changes to the first follow-up survey. These proposed changes do not impact the burden estimates provided in the original ICR submission, given site-specific skip patterns within the survey.

1. We added skip patterns tailored for each program currently selected for inclusion in the NextGen Project.

2. We made minor revisions to skip patterns, and the wording of some items, for clarity.

3. We moved all employment and earnings questions to the beginning of the survey from Section B to Section A. This information is critical for the impact analysis so we would like to collect this information first.

4. We added items to collect information on the effect of COVID-19 on outcomes. These include a series of items to ask respondents if they experienced any workplace changes as a result of the pandemic (item A02a); an item to ask if the physical, mental, or emotional condition that limits work is related to COVID-19 or its effects (A22a); a series of items to ask about employment challenges related the pandemic (A23); an item to ask about service receipt related to meeting any workplace requirements for the pandemic (B01m); and an item to ask about their vaccination status (C10a).

5. To simplify the employment series, we removed some items from the list of benefits available at the respondent's main job (A16) and dropped the industry and occupation items for the respondent's main job.

6. To reduce burden, we removed most items from a question on potential employment challenges (A23). The revised question now only asks about COVID-19-related challenges.

7. For one NextGen Project program, we added a series of questions about confidence in ability to seek employment (A25), at the suggestion of program staff.

8. We made modifications to ask about the types of service receipt inclusive of all settings (B01), rather than services received in one-on-one versus group settings. One question was added to ask about attendance to any group support meetings (B02). This item is only for one NextGen program that includes a group support intervention.

9. We made modifications to the types of service receipt items on the survey (B01). These refinements are intended to better match the types of services participants may receive under programs selected for the NextGen Project.

10. The service receipt dosage questions were removed to simplify the reporting of service receipt. These items include start and end dates, frequency, and length of typical sessions.

11. We revised the item asking about the service location to only ask about the location where respondents received the most services (B03), rather than asking about all locations. This change will simplify the reporting of service receipt.

12. Rather than ask questions about education, training, and employer-provided training in a separate series of questions about each specific program, these items were simplified to ask about education and training programs generally (B04 to B05b). Questions about employer-provided training were removed to reduce burden. These questions were combined with non-employer-provided job training into one set of training questions (B8 to B8b), as this reduces the number of questions for respondents. Additionally, questions collecting details on the specific type of education or training program, start and stop dates, and place of services have been removed as education and training is not a focus area for the selected NextGen programs. Items were added to ask if sample members are currently participating in education and training programs (B04a and B05a). Finally, items asking if participants received a certificate or degree from specific education or training programs were modified to ask about their receipt more generally (B04c and B05b).

13. A similar set of education and training questions were developed for one NextGen Project program that works with young adults. The study participants from this program will not be asked items B04 to B05b. Instead, they will be asked items B06 to B08b. These items ask about high school, General Educational Development programs, and any other education or training programs.

14. We revised the recall period for mental health service receipt to the last six months instead of since the date they were randomized for the study (B10, B10a, and B10b). This shortens the recall period, which should increase accuracy of responses. It also better matches the time frames over which the study programs provide mental health services.

15. We added new questions on service receipt for physical medical conditions within the last six months (B11 and B11a) and receipt of mental health services for any respondents' children in the last six months (B12 and B12a). These items will capture important outcomes for one NextGen Project program since the program includes an intervention component focused on ensuring greater access to physical and mental health services for its participants and their children.

16. Since housing is not a focus of most of the specific programs selected for the NextGen Project, we removed items asking about the type of rental housing and number of days spent homeless. We also moved the remaining housing questions to an earlier location in Section C (C01, C01a, and C02).

17. To simplify an item asking about household benefit receipt (C03), we removed some response options: worker's compensation; short term disability; Section 8 housing choice vouchers; and veteran's benefits. We do not expect many NextGen Project participants to report receipt of these items.

18. For one NextGen Project program serving young adults, we added a series of questions that ask about financial help received from parents, relatives, friends, or neighbors and the amount received, if so (C04b to C04e1).

19. We made modifications to the items about emergency support (C05 and C06) to ask how many people someone can turn to for help, to better measure social support outcomes.

20. We added items to collect information on social trust, including: how many people do respondents have to talk to for advice (C07a); if people can generally be trusted (C07b); and if they trust most people in their neighborhood (C07c).

21. We added the Healthy Families Parenting Inventory's parenting efficacy subscale (C08) for use by one NextGen Project program that aims to improve mothers’ belief in their parenting skills.

22. We added items to ask about confidence in their future financial situation (C09a and C09b) for use by one NextGen Project that aims to increase youths’ belief in their ability to become self-sufficient.

23. We added a question to see if participants were currently covered by health insurance (C10b).

24. We added the Center for Epidemiologic Studies Depression Scale Revised (CESD-R) (C22a) for use by one NextGen Project program that serves mothers with symptoms of depression. For study participants in this program, this more in-depth depression scale will be used instead of the shorter K-6 Distress Scale (C22).

25. We made revisions to the criminal justice series of questions to better collect information on these outcomes for one program that focuses on participants recently released from jail or prison. The prior version of the survey asked about arrests; number of arrests; number of arrests leading to conviction; if incarcerated since the study randomization date; time spent in incarceration; and if currently on parole or probation. With the new proposed revisions, the section rewords the parole question to ask if currently under court-ordered supervision (C38) and adds new items to ask how many convictions were felony convictions (C42) and if any of the reported incarcerations were for violating the terms of court-ordered supervision (C44).

26. We added item D02d for use by one NextGen Project program that aims to foster a trusted adult connection with participants.

27. To reduce burden, we removed some items asking program group study participants about their satisfaction with the program.

We have updated the question-by-question justification for the follow-up surveys to reflect the changes proposed above and included it with this change request submission (Appendix D. Question-by-question justification for follow-up surveys – revised).

Second Follow-up Survey (Instrument 4. Second follow-up survey – revised)

The second follow-up survey includes all the changes, as applicable, that were made to the first follow-up survey. These proposed changes do not impact the burden estimates provided in the original ICR submission, given site-specific skip patterns within the survey.

Follow-up Survey Reminders and Notifications (Appendix G. Follow-up survey reminder and notifications – revised)

We propose changes to follow-up survey reminders and notifications to adapt them for the proposed revised survey tokens of appreciation and prepaid experiment. The revised notifications are largely similar to the initial submission, with changes to increase the gift card amount from $40 to $50 for the first follow-up survey, adapting language on the advance letter (page G1) for the proposed prepaid token experiment, and updating the token amount to $55 for the experiment. We also included additional reminders that will be used during data collection. These include two additional emails (G4 and G5) and two additional text messages (G7 and G8). Finally, we added a thank you letter or email to send to participants upon completion of the survey (G12). The revisions and new notifications do not add burden to this ICR request.

NextGen Project Recruitment (Appendix G.1. NextGen Project recruitment materials)

We are using additional materials to assist in the recruitment of individuals for the NextGen Project. These materials are tailored to programs that have agreed to join the NextGen Project. They include informational flyers; appointment reminders; marketing flyers; recruitment flyers; and a video storyboard used to make a video to advertise the study and promote attendance at the informational meetings.

These materials also include an 18th Birthday Mailer for participants under 18 years old at the time of enrollment within the Bridges from School to Work (Bridges) program. The study’s institutional review board (IRB) has requested we provide these participants a chance to opt-out of the study when they turn 18. This mailed package will include an 18th birthday card and a copy of the study’s adult informed consent form that includes opt-out instructions.

Finally, the study’s IRB has requested Bridges participants be given a chance to opt-out of study participation, following the end of the second follow-up survey. We included a letter to be sent to all study participants at this program after we complete the second follow-up survey.

The materials do not add burden to this ICR request.

The materials tailored for the NextGen Project programs, as included in Appendix G.1, include:

1. Program-specific recruitment flyers and materials

   1. Bridges parent flyer (Appendix G1.1): At the first informational meeting about the study, parents of potential Bridges participants are provided this flyer about Bridges and the NextGen Project.

   2. Bridges student flyer (G1.2): Bridges distributes this flyer to students, with less detailed information than the two-page parent flyer given to parents. This flyer is tailored to each Bridges subsite.

   3. Individual Placement and Support for Adults with Justice-Involvement (IPS JI) appointment reminder (G1.3): For each of the five mental health centers under IPS JI, this document is given to potential participants at informational meetings to remind them of their enrollment appointment at the mental health center.

   4. IPS JI marketing flyer (G1.4): For each of the five mental health centers under IPS JI, this flyer is tailored and posted in physical locations to advertise the study and promote attendance at informational meetings.

   5. IPS JI recruitment flyer (G1.5): For each of the five mental health centers under IPS JI, this flyer is tailored and distributed during informational meetings with potential participants and provides more details about the study.

   6. IPS JI video storyboard (G1.6): At the IPS JI subsites, this storyboard will be used to make a video to advertise the study and promote attendance at the informational meetings. The video narrative will change slight depending on where the video airs (as indicated this in the storyboard). Some images in the storyboard will be substituted with site-specific images and video footage in some of the sites.

   7. IPS JI social media messages (G1.7): These messages are used to reach a broader population of potential participants. They are posted on social media websites and include a link to a web-version of the marketing flyer or video.

   8. Families Achieving Success Today (FAST) marketing flyer (G1.8): This flyer is posted in physical locations at FAST subsites to advertise the study.

   9. FAST recruitment flyer (G1.9): A FAST participant recruitment flyer for each of the TANF program provider offices. This flyer is distributed by TANF program provider staff to potential participants and provides more details about the study.

   10. FAST outreach letter (G1.10): This outreach letter serves as another way to communicate the opportunity to enroll in the study for eligible participants and ensure a one-on-one meeting with an enrollment coordinator is scheduled. It is sent to eligible participants’ home addresses and is only sent to eligible study participants that enrollment coordinators have not otherwise had success reaching.

   11. Western Mass MOMS marketing flyer (G1.11): This flyer assists in the recruitment of study participants. This flyer is provided to referral agencies to give to prospective study participants. People who are interested in participating in the study are invited to an enrollment interview where they are provided more information about the study.

   12. Philadelphia Workforce Inclusion Network services (Philly WINs) marketing flyer (G1.12): For each of the four PA CareerLink® centers under Philly WINs, this flyer is distributed to PA CareerLink® staff to advertise the study and provides information about eligibility criteria and center staff’s role.

   13. Philly WINs recruitment flyer (G1.13): For each of the four PA CareerLink® centers under Philly WINs, this flyer is posted in physical locations to advertise the study to potential participants and promote enrollment.

2. Bridges 18th Birthday Mailer (G1.14A and G1.14B): For Bridges participants under 18 years old at the time of enrollment, the study’s IRB has requested the study provide them a chance to opt-out of the study when they turn 18. This mailed package will include an 18th birthday card and a copy of the study’s adult informed consent form that includes opt-out instructions.

3. Bridges end of study letter (G1.15): For Bridges participants, the study’s IRB has requested they be given a chance to opt-out of study participation, following the end of the second follow-up survey. This letter will be sent to all Bridges study participants after we complete the second follow-up survey. The letter would thank them for their participation so far and inform study participants that Mathematica has completed the survey data collection as part of the NextGen Project. It will remind them that further research may be conducted by Mathematica or the Social Security Administration using federal administrative data through 2040. It will also remind study participants that they have the option to leave the study and provide information on how to do so.