Supporting Statement B_URM 2019-2-11_clean

We will administer the online surveys to all State Refugee Coordinators (SRCs) from 15 states with Unaccompanied Refugee Minors (URM) programs (Appendix A), URM program directors from all 22 programs (Appendix B), and child welfare administrators from the 14 sites with private custody arrangements (Appendix C). Accordingly, we estimate a maximum sample size of 51 respondents in total. We will work with the Office of Refugee Resettlement (ORR) and the resettlement agencies to determine appropriate points of contact at each site.

B.1.2. Site visits

Given that only six of the 22 programs will be chosen, the selected sites cannot provide full representativeness of the characteristics of URM programs or of the individuals served by them. Instead, the sites selected, as a group, will inform the Office of Planning, Research, and Evaluation (OPRE), ORR, and other audiences about the variety of ways programs operate in different conditions, and have the potential to offer useful lessons for improving programs and crafting future evaluations. The following factors, developed through consultations with stakeholders and federal staff, will be considered to ensure that sites selected contain the characteristics and variety needed to understand different aspects of the URM Program:

Resettlement agency. Selected sites should include some programs affiliated with Lutheran Immigration and Refugee Service (LIRS) and some with U.S. Conference of Catholic Bishops (USCCB).
Custody arrangement: Selected sites should include a mix of programs with public and private custody arrangements. If possible given other considerations, the sites with public custody arrangements should include at least one site where the custody arrangement is with the state and one where it is with the county, as programs in states with different arrangements may face different challenges and involve a different array of stakeholders.
Program size. The set of sites selected should contain some variety in terms of program size by including at least one or two small- or moderate-sized programs.
Types of services provided (including continuum of care): The six sites should include both programs with more extensive services and less extensive services.
Relative proportions of refugees and Unaccompanied Alien Children (UACs) among the populations served. Given the different issues involved in serving refugees and UACs, we would like to include both programs that have served large numbers of UACs and programs that have not.
Refugee and immigrant populations: Several factors related to the populations both served by the URM programs and in the communities where the programs are located would be useful to consider. These include:
- The overall number of refugees resettled in the community (that is, adult and child refugees). Ideally the six sites could include variation in the extent to which the communities in which the URM programs are operating are experiencing the presence of recently arrived refugees in general.
- The national origins of the youth served by the local URM program. The six sites should include a mix of refugees with different national origins.
- The variety of national origins in the area, either of refugees or immigrants overall. If possible, this information may also allow us to include variety among the sites in the extent to which there is an existing co-ethnic population in the community to the youth served, which may affect the ability of programs to facilitate cultural connections for youth.
Geography: If possible, we would to like to include programs in both larger cities and smaller cities, as programs in more or less urban areas operate in different conditions.
Program tenure: Programs that have been operating for a longer period may have more perspective on strategies for serving youth that have been useful in different contexts. However, it may be useful to include one to two newer programs where current staff may have a perspective on the experience operating a new or recent program.

We expect information on most or all of these items will come either from ORR or from publicly available sources such as Census Bureau data.

Once the sites have been selected, we will work with the point of contact at each site (likely the URM program director) to identify respondents for each of the interviews we plan to conduct (see B.2.3).

B.1.3. Focus Group Participants

Once the six sites have been selected for site visits, the research team will work with the URM program staff to identify and convene URM youth and foster families to participate in focus groups. Discussions with youth who were part of the URM program and with foster parents who had worked with the program will ensure that researchers obtain the important perspective of those served by the program in each site, and have the potential to provide useful lessons that ORR, states, and program administrators can draw on to improve programs or guide future policy. Focus groups with these two groups will promote discussion on how they interacted with the program, what services they received, and their overall satisfaction with the program’s services.

We will aim for each focus group discussion to include about nine participants. Participants in the youth focus group will consist of youth aged 16 and over who entered the URM program within seven to 24 months of the focus group. Focus groups with foster parents will consist of adults who recently served as foster parents for youth who were part of the URM program. Participants in each group will represent a convenience sample; the study team will work with the URM program in each site to identify individuals who may be appropriate for the group and to invite them to participate. We will provide sites with written guidance and information about the goals of the focus groups to facilitate the selection of participants (Appendix N). Sites will be instructed to recruit URM youth who are willing to speak to the topics included in the Focus Group Guide for URM Youth (Appendix I), and to make it clear to URM youth that their participation is voluntary.

B2. Procedures for Collection of Information

B.2.1. Surveys

The team will contact all online survey respondents via email (see Communications to Survey Respondents in Appendices K-M) to inform them about the survey and to provide background about its goals and objectives. We will send a unique electronic link to each designated respondent. This correspondence will contain additional instructions about distributing the survey to appropriate staff internally, consent and privacy, due date, and who to contact with questions or concerns. The online survey will be available for one month to ensure survey participants have ample time to complete the survey.

While some questions may be asked of both SRCs and URM program directors where appropriate to gauge differences in perspectives, the team will avoid unnecessary redundancy to minimize participant burden. For instance, questions about inter-agency collaboration might be posed to both types of respondents, while items related to foster parent recruitment for URM youth have been reserved for URM program directors, as they are the most knowledgeable on this topic. Additionally, the research team will merge existing data on program characteristics (e.g., location of program, custody arrangement) with survey responses during analysis, which will reduce the length of the survey, as we will not need to ask respondents for information we already possess.

The Survey for State Refugee Coordinators (Appendix A) will take up to 30 minutes to complete for each person, while reading the introductory email, reminder emails, and, if necessary, follow-up communication (Communications to Survey Respondents: State Refugee Coordinators in Appendix K) may require up to 10 additional minutes for a total respondent burden of up to 40 minutes (0.67 hours). We anticipate the SRCs may include additional respondents in their answers and have accounted for an average of 2.5 respondents per state in our burden estimates (one SRC and between one and two additional staff members).

The Survey for URM Program Directors (Appendix B) will take up to 45 minutes to complete. All respondents will also be required to read the introductory email containing the survey link (Communications to Survey Respondents: URM Program Directors in Appendix L), and some individuals may also receive follow-up communication by phone or email to remind them to complete the online survey (Appendix L); together, these communications may add up to 15 additional minutes for a total respondent burden of up to 60 minutes (1 hour). We have accounted for URM Program Directors to include one or two additional staff members in their responses and have included 2.5 respondents per program in our burden estimates (one Program Director and between one and two additional staff members).

The Survey for Private Custody Child Welfare Administrators (Appendix C) will take up to 30 minutes to complete, with communication with the research team and additional follow up of up to 10 minutes (Communications to Survey Respondents: Private Custody Child Welfare Agency Administrators in Appendix M), for a total respondent burden of up to 40 minutes (0.67 hours). We expect that child welfare agency administrators may include one additional person in their response and have included an average of 1.5 respondents per site in our burden estimates.

State Refugee Coordinators (SRCs), URM Program Directors, and Private Custody Child Welfare Administrators will also receive thank you emails notifying them that their surveys were received by the study team (Appendices K through M, respectively). The time required to read these emails has been factored into the additional time survey respondents may require for follow-up communications with the research team.

B.2.2. Phone Interviews

The research team will conduct Interviews with Child Welfare Agency Administrators (Appendix F) via phone for sites with public custody arrangements that will not be visited during the site visits (see B.2.3 below). The team will contact potential respondents by email to identify a date and time during which to have this conversation.

With a total of eight public custody sites in six states, and an assumption that the research team visits at least one public custody site, there would be a maximum of seven phone interviews with up to two respondents from the child welfare agencies (for a total of 14 respondents). Interviews will last 60 minutes (1 hour).

B.2.3. Site visits

Using the criteria mentioned above, the study team will select sites with a variety of program characteristics. Once the six sites have been finalized, the study team will work with the sites to identify dates for the site visits based on availability of local staff. The research team will work with sites to schedule interviews with URM program managers and staff, child welfare agency administrators, and local community partners that provide services to URM youth. The interviews will last between 60 and 90 minutes and will use discussion guides, included in Appendices D-H. The site visits will include the following interviews:

1 90-minute Interview with the URM Program Manager (Appendix D)
1 90-minute Interview with URM Program Case Managers (Appendix E)
1 60-minute Interviews with other URM Program Staff (Appendix E)
1 60-minute Interview with Child Welfare Agency Administrators (Appendix F)
4 60-minute Interviews with Community Partner [General] Staff (Appendix G)
1 60-minute Interview with Community Partner [Education] Staff (Appendix H)

We anticipate one or two respondents for the Interview with URM Program Manager for an average of 1.5 respondents (Appendix D), between two and four respondents each for the Interview with URM Program Staff and the Interview with Program Case Managers for an average of three respondents from each group (both using Appendix E), two respondents for each of the Interviews with Community Partners [General] and the Interview with Community Partner [Education] (Appendices G and H, respectively), and two respondents for the Interview with Child Welfare Agency Administrators (Appendix F).

The research team plans to conduct one Focus Group with URM Youth (Appendix I) and one Focus Group with URM Foster Families (Appendix J) at each site. These focus groups at each site will be comprised of seven to ten URM youth and seven to ten URM foster parents, respectively; we estimate nine respondents from each group in our burden estimate. The focus group discussions will last 90 minutes.

The table summarizes these estimates of the total number of respondents to all site visit information collection. It outlines both the estimated number of interviews per site visit as well as the estimated number of respondents per interview.

Exhibit 1: Estimated Number of Respondents for Site Visits

Interview/Focus Group	Instrument	Number of Interviews Per Site Visit	Estimated Number of Respondents per Interview/Focus Group	Total Number of Respondents Per Site Visit	Total Number of Respondents for Six Site Visits
URM Program Manager	Appendix D	1	1-2 (average 1.5)	1.5	9
URM Program Case Managers	Appendix E	1	2-4 (average 3)	3	18
URM Program Staff	Appendix E*	1	2-4 (average 3)	3	18
Child Welfare Agency Administrators	Appendix F†	1	2	2	12
Community Partners [General]	Appendix G	4	2	8	48
Community Partners [Education]	Appendix H	1	2	2	12
URM Foster Families	Appendix I	1	7-10 (estimated 9)	9	54
URM Youth	Appendix J	1	7-10 (estimated 9)	9	54

*Note: Appendix E includes both Interviews with Program Staff and Program Case Managers, for a total of 36 respondents in the burden estimate table (See A.12.1).

†Note: Appendix F will also be used for phone interviews with Child Welfare Agency Administrators in public custody sites (see B.2.2). Therefore, the burden table includes both the phone interviews (14 respondents) and site visit interviews (12 respondents) in the total number of respondents (26) associated with Appendix F.

Focus groups will be conducted by research team members in English or Spanish where appropriate, and Youth Focus Group protocols (Appendix I) will be developed in both languages. The research team also has staff who speak Amharic and Farsi, who may assist in facilitation and translation when necessary. The language used for each group will be English if enough youth recently served by the program in the site are sufficiently proficient in English; if not, the groups will be conducted in a language common enough among those youth to fill a focus group. In sites where focus groups must be conducted in languages other than English, Spanish, Amharic, or Farsi, we will work with local organizations to identify an appropriate local facilitator. That facilitator will translate the Focus Group Guide for URM Youth (Appendix I), conduct the group, and provide the team with translated notes. The research team will provide both written guidance to the facilitator and hold a discussion with the facilitator in advance of the group, to ensure the facilitator knows how to conduct the group discussion and provide notes to the team afterwards.

Additionally, all facilitators will receive training on research involving human subjects and vulnerable populations. The procedures used to administer verbal informed consent and conduct the focus groups have been developed with attention to the needs of URM youth, who may be susceptible to strain from discussion of certain topics given their past experiences and traumas, and to avoid the potential for coercion (or the appearance of coercion) to participate in the focus group. To help put youth at ease and lessen the chance they will feel pressured to participate, a facilitator will administer verbal assent/consent with youth in a one-on-one setting, apart from the other youth and out of the earshot of program staff. In addition to administering the consent individually, the general introduction to the group will repeat that participation is voluntary and that youth do not need to answer any questions that they do not want to. Further, the focus group protocols do not ask about particularly sensitive topics such as their experiences before arriving in the U.S.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

B.3.1 Expected Response Rates

Online surveys: The Survey of State Refugee Coordinators (Appendix A) will be distributed to SRCs from 15 states with URM programs, and the Survey of URM Program Directors (Appendix B) will be distributed to URM Program Directors from all 22 programs. We anticipate all URM program directors and SRCs will be responsive to the survey, for a response rate of 100 percent.

The Survey of Private Custody Child Welfare Agency Administrators (Appendix C) will be distributed to child welfare administrators from the 14 sites with private custody arrangements. We expect child welfare administrators to be less responsive to the survey than the other groups will. Conservatively, we estimate that 75 percent of the private custody child welfare administrators will respond.

Phone interviews: The research team anticipates that the public custody child welfare administrators will be more responsive to requests for phone interviews (Interviews with Child Welfare Agency Administrators in Appendix F) than private custody child welfare administrators to requests for surveys, due to collaboration between the URM program and child welfare agency in public custody arrangements. We expect a 100 percent response rate for these interviews.

Site visits: As the research team plans to engage with sites throughout the planning and recommendation process, the study anticipates 100 percent participation among sites where we conduct visits. However, if any sites entirely decline to participate, we will identify a site with similar characteristics as a back-up. We will plan with management at each site to find a time that works for most or all of staff. Because the research team will be working closely with URM program staff to identify participants for Interviews with URM Program Managers and Staff (Appendices D and E) and for Interviews with Community Partners [General and Education] (Appendices G and H), we expect a response rate for these groups to be near 100 percent, as well.

Attendees for Focus Groups with URM Youth (Appendix I) and Focus Groups with URM Foster Parents (Appendix J) will be identified and recruited with the help of local URM program staff. We will provide written guidance to URM programs to assist with their recruitment efforts (Appendix N). These instructions include guidance to seek participants who are interested in participating (and emphasize that it is important to be clear that participation is voluntary), explain the topics to be discussed in the focus groups, and describe the $30 gift cards that will be used as a thank you and to cover incidental expenses. Based on the efforts described and the ability to identify additional participants if needed, we expect to reach our goal of recruiting 7-10 participants per site for each focus group. Given the diverse nature of the populations served by the URM programs, we are not expecting to get full representativeness.

B.3.2 Dealing with Nonresponse

For all online Surveys of State Refugee Coordinators (Appendix A), URM Program Directors (Appendix B), and Private Custody Child Welfare Agency Administrators (Appendix C), the research team will include an initial email from ORR, which will indicate their involvement and support of the study. We anticipate this will be helpful in generating a 100 percent response rate. In addition, the research team will conduct follow up to encourage those who have not responded. While the online survey is open, the research team will monitor response rates and send up to two reminder emails (Appendices K, L, and M) or make a phone call to individuals who have not completed the survey.

Non-response bias is possible if we do not achieve our target response rate of 100 percent. Regarding the surveys, for the State Refugee Coordinators and URM Program Directors, given the nature of the questions on the survey, and the fact that we are asking questions about a program, we do not expect those who respond to the survey to be different in a meaningful way from those who do not respond. For the Private Custody Child Welfare Agency Administrators, non-response bias might be more problematic, and their non-response could reflect their limited involvement with the URM Program. As a result, there is a risk that we miss these perspectives more systematically. As described in the previous paragraph, we will send regular reminder emails and make phone calls to individuals to prompt them to complete the survey.

With regards to the site visits, as noted in B.1.2, the sites selected for site visits cannot provide full representativeness of the URM programs even with a 100 percent response rate. However, while we anticipate a 100 percent response rate on the phone interviews with public Child Welfare Agency Administrators (Appendix F); in-person interviews with URM Program Mangers, Staff, Community Partners [General and Education], and Child Welfare Administrators (Appendices D-H); and focus groups with URM Youth and URM Foster Parents (Appendices I and J), it is possible that there will be non-response bias if we are not successful in achieving this goal. To avoid non-response bias, we will work with the sites in arranging the visits to try to obtain 100 percent response rate, and given that we will arrange our site visits in cooperation with the programs, we expect that we will be able to conduct interviews at times that all interviewees are available. If needed, we can schedule follow-up calls with those with whom we are unable to meet in person due to unforeseen circumstances. With regards to the focus groups, we will work with sites to find times and locations that will be convenient for potential participants and offer incentives that will help offset costs of transportation and rearranging work schedules, in order to avoid nonresponse bias by only including respondents who more financially able to attend a focus group. We will share information about the focus groups before visits (Appendix N), so that sites may begin outreach to families and youth well in advance and allow attendees time to coordinate their attendance.

We acknowledge that nonresponse could affect the perspectives we are able to capture in our study. We would expect that, if we are unable to obtain a 100 percent response rate, those with whom we are unable to conduct an interview may be less likely to be involved in the URM program and may be less knowledgeable about the program and URM youth. This could create bias in our findings, as we would be missing the perspective of those who work less closely with the program.

B.3.3 Maximizing Response Rates

For the Surveys of State Refugee Coordinators (Appendix A), URM Program Directors (Appendix B), and Private Custody Child Welfare Agency Administrators (Appendix C), the engagement of ORR and the resettlement agencies will help in promoting participation. This engagement will inform respondents about the study and help to maximize the number of respondents to complete the survey.

For site visits, the research team will engage with the SRCs and national resettlement agencies to gain contextual information, learn about approval processes, and connect with URM program staff. The study team will also work with SRCs and resettlement agencies to identify and engage with key contacts at each site to promote participation.

B4. Tests of Procedures or Methods to be Undertaken

The research team will pilot the Survey of State Refugee Coordinators (Appendix A) and the Survey of URM Program Directors (Appendix B) with three of each type of respondent and ask for their feedback, including how long it took them to complete, whether the questions were clear, etc. We propose piloting the Survey of State Refugee Coordinators (Appendix A) and the Survey of URM Program Directors (Appendix B) in early 2019. If changes are made based on this pretesting, we will submit the updated surveys to OMB for review.

B5. Individual(s) Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Exhibit 2: Individuals Consulted on Statistical Aspects and Collecting and/or Analyzing Data

Design of the survey:

Name	Affiliation	Contact
Lyn Morland	McGill University	lynmorland@gmail.com
Elzbieta Gozdziak	Georgetown University	emg27@georgetown.edu
Tom Crea	Boston College	creat@bc.edu
Jennifer Berenson	Catholic Family Center (Rochester, NY)	jberenson@cfcrochester.org
Charles Shipman	Arizona Immigrant and Refugee Services	cshipman@azdes.gov

Involved in Data Collection and Analysis:

Name	Affiliation	Contact
Sam Elkin	MEF Associates	sam.elkin@mefassociates.com
Kim Foley	MEF Associates	kimberly.foley@mefassociates.com
Liza Rodler	MEF Associates	liza.rodler@mefassociates.com
Sarah Catherine Williams	Child Trends	swilliams@childtrends.org
Heather Wasik	Child Trends	hwasik@childtrends.org
Lauren Supplee	Child Trends	lsupplee@childtrends.org
Lyn Morland	McGill University	lynmorland@gmail.com
Tiffany McCormack	Office of Planning, Research, and Evaluation	tiffany.mccormack@acf.hhs.gov
Gabrielle Newell	Office of Planning, Research, and Evaluation	gabrielle.newell@acf.hhs.gov
Catherine Marie Lawrence	Office of Planning, Research, and Evaluation/Business Strategy Consultants	Catherinemarie.Lawrence@acf.hhs.gov

9 Supporting Statements for OMB Clearance Request Part B |

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	OPRE OMB Clearance Manual
Author	DHHS
File Modified	0000-00-00
File Created	2021-01-15

Supporting Statement B_URM 2019-2-11_clean