Implementation Grants to Develop a Model Intervention for Youth/Young Adults With Child Welfare Involvement At-Risk of Homelessness:
Phase II
OMB Information Collection Request
0970-0445
Supporting Statement
Part A
April 2018
Submitted by:
Office of Planning, Research and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officer:
Mary Mueggenborg
This Information Collection Request (ICR) is a request for a time extension on OMB Control Number 0970-0445 (ICR Reference Number 201704-0970-001), which was reinstated with change on July 24, 2017. The current OMB clearance expires on July 31, 2018. Due to delays in the work by grantees, the ICR will not have been started by that time. The grantees who are the focus of this ICR are receiving an additional year of support. The research questions posed in the ICR are interested in understanding the experience of grantees near the end of their grant period.
There are no changes to the ICR other than the timing of the site visits. All research questions, instruments, and processes remain unchanged from the ICR package submitted for reinstatement with change that was granted on July 24, 2017.
A1. Necessity for the Data Collection
The Office of Planning, Research and Evaluation (OPRE), Administration for Children and Families (ACF), U.S. Department of Health and Human Services (HHS) seeks approval for data collection activities with grantee and partner organizations that are receiving implementation grant funds through the Children’s Bureau (CB) in ACF. These activities will include (1) communication related to planning site visits, and (2) individual interviews or small-group interviews with key informants during site visits to be used for a process study of these implementation grants. The process study will assess the barriers and facilitators grantees experienced designing and implementing comprehensive model interventions to address homelessness among youth ages 14 to 21 currently or formerly in foster care. The process study will also assess how a multiphase grant effort supported grantees in implementing and preparing for rigorous evaluation of their model interventions.
To improve the well-being of youth and young adults with child welfare involvement who are at risk of homelessness, CB is supporting the development and implementation of comprehensive, integrated service models based on the youth framework from the U.S. Interagency Council on Homelessness (USICH).
There is currently no solid evidence showing what works to provide stability for this vulnerable population. To date, efforts to address homelessness and unstable housing among former foster youth have been limited, and approaches to these problems have not been rigorously tested. The U.S. Department of Housing and Urban Development (2012, p. 28) has noted: “The lack of rigorous evaluations of housing programs for young people who have aged out of foster care means that we do not know whether any of the programs prevent homelessness or otherwise reduce housing instability.”
Experts agree that approaches to the problems of homelessness and housing instability must engage multiple human services systems. To fully address the problem, there must be a comprehensive, multisystem approach—ideally, including the criminal justice system—and long-term programming (Kroner 1999). The evidence to date indicates that approaches to preventing homelessness among former foster youth must be adapted to the difficult array of situations they face.
Policies and programs to address the challenges of former foster youth lag behind other efforts launched by HHS and other federal agencies in building and sustaining comprehensive, integrated, evidence-based services and supports. On September 30, 2013, ACF announced the award of planning grants for the first phase of a new multiphase grant effort to develop model interventions for youth with child welfare involvement who are at risk of homelessness (HHS-2013-ACF-ACYF-CA-0636). These “Planning Grants to Develop a Model Intervention for Youth/Young Adults with Child Welfare Involvement at Risk of Homelessness” (“YARH grants”) were awarded to 18 grantees who used the two-year planning grant period to develop multicomponent interventions that support better outcomes for youth with child welfare involvement in four critical domains: (1) stable housing, (2) permanent connections to caring adults, (3) education and employment, and (4) social/emotional well-being.
Of the 18 grantees that received YARH planning grants funded under the first phase of YARH (YARH 1), CB recently awarded three-year implementation grants to six grantees that demonstrated the capacity to engage in Phase II implementation work (HHS-2015-ACF-ACYF-CA-0961). This second phase of grant funding (YARH 2) will enable the six grantees to implement the model interventions developed in their first two-year planning phase and prepare for a rigorous evaluation of those interventions. In addition, OPRE has awarded a technical assistance contract to Mathematica Policy Research to help the YARH 2 implementation grantees develop rigorous evaluation plans and to conduct a process study of the grantees’ implementation processes. The process study is the focus of this Information Collection Request (ICR).
Data collection for a process evaluation of the 18 Phase I YARH grantees was completed under a prior ICR. The data collection included quantitative data (a survey of the grantees on organizational readiness and partnerships), and qualitative data (reviews of grant applications and semi-annual reports that grantees submitted to CB, and notes from two-day site visits to the grantees). All data collection approved for YARH 1 is complete. The data provided insight into how grantees experienced the first planning phase of this multiphase competitive grant opportunity, the benefits and challenges of grants structured to support program planning, and the implications of the grantees’ experiences for future multiphase grant efforts. The currently proposed data collection for YARH 2 will build on earlier data collection for the YARH process study to understand grantees’ experiences as they move from the planning phase to an implementation phase of the multiphase YARH grant program, and the benefits and challenges of grants structured to support program implementation.
The YARH process study data collection is legislatively authorized by section 105(b)(5) of the Child Abuse Prevention and Treatment Act of 1978 (42 U.S.C 5106(b)(5)), as amended by the CAPTA Reauthorization Act of 2010 (Pub. L. 111-320).
A2. Purpose of Survey and Data Collection Procedures
Overview of Purpose and Approach
This ICR describes the data collection activities for a process study that will document intervention implementation activities and experiences, technical assistance, and outcomes of the implementation phase for YARH grantees. The process study will be descriptive and will provide ACF and the field with a better understanding of the implementation factors and supports needed to develop, and put into operation, comprehensive, integrated, evidence-based service models that can be rigorously evaluated. The process study will include (1) information from interviews conducted with grantee implementation team members and key partners in late 2018 and early 2019 (upon OMB approval), and (2) documents submitted to the federal government to meet the grant-reporting or contractual requirements for the larger YARH project. Data analyses will (1) describe how grantee activities were designed and put into operation to achieve outcomes; (2) identify adaptations to intervention design and activities, as well as changes to target populations that occurred during implementation; and (3) obtain an understanding of the stakeholders’ experiences to assess implementation supports and challenges.
The process study will help ACF better understand what the YARH 2 grantees have accomplished during the implementation period, as well as the extent to which they are in a position to rigorously evaluate their model interventions. Data collected for the process study will be used for two main purposes: (1) to assess grantees’ individual and common experiences designing and implementing the comprehensive services model, and (2) to assess how a multiphase grant effort supported grantees in implementing and preparing for rigorous evaluation of their model interventions. Researchers will gather data through document review (for example, grant applications and semiannual progress reports) and interviews during site visits to all six grantees.
Research Questions
The research questions for the process study address four key areas:
1. Model intervention design
What outcomes did grantees identify for their model interventions, and why were those outcomes selected?
How did grantees refine the three target populations identified in the YARH funding announcement?
Youth in foster care, ages 14-17.
Young adults who were in or are in foster care, ages 18-21.
Homeless youth with foster care histories, up to age 21.
What are the critical components of each model intervention?
How did these elements of intervention design change between Phase I and Phase II, and why?
2. Implementation experience
What did grantees do to organize and promote intervention implementation?
How did the Phase I planning period facilitate intervention implementation in Phase II?
What facilitators and challenges did grantees experience related to refining the target population, intervention components, and data collection procedures? How did grantees overcome these challenges?
What facilitators and challenges did grantees experience during the implementation phase? How did grantees overcome implementation challenges?
What facilitators and challenges did grantees experience demonstrating readiness for full implementation and planning for a rigorous summative evaluation?
3. Implementation progress
How was the use of resources for pre-implementation and initial implementation activities associated with grantees’ progress in completing evaluation technical assistance activities?
Which implementation activities required the most resources, and why? Which activities required the fewest resources, and why?
How did the implementation activities help grantees prepare for full implementation?
4. Multistage grant process
How did planning the model intervention in Phase I help grantees prepare for implementation in Phase II?
How are the Phase II activities helping grantees implement their model interventions and prepare for a rigorous summative evaluation?
What about the Phase II activities is challenging to grantees?
The discussion guide that is the focus of this ICR will gather information about implementation experiences that will help answer questions about implementation grant activities and outcomes and planning for evaluation.
Study Design
The process study will include the full population of six grantees receiving Phase II YARH grants. Because of the small number of grantees, we propose that all grantees be included in the process study. There are two sources of data for the process study: (1) document reviews, and (2) site visits.
The process study team will review documents that the grantees submit as part of the normal grant administration process. These documents include the grant application, semiannual progress reports, and presentations (if applicable). The process study team will also review records of technical assistance provided to grantees. The team’s document review will provide information on what the grantees plan to do, what they accomplish, and the resources provided by the technical assistance contractor.
In addition, the process study team will conduct site visits to each of the six grantees. Site visits will last approximately two days and include individual and small-group interviews. Researchers will gather information to understand grantees’ implementation experience and help fill in missing information about intervention design from the document review.
The process study will meet the needs of ACF by providing an in-depth study of how the implementation grants supported local communities in implementing innovative approaches to providing services to youth at risk of homelessness and preparing to evaluate the effectiveness of these services. The information obtained through the process study can be used to inform decisions related to future government investments in programs seeking to develop comprehensive service models for at-risk youth and young adults. The process study is limited in that it will reflect the experiences of only the six communities awarded the YARH implementation grants. Other communities may undertake similar comprehensive implementation efforts in a different manner, and require other technical assistance and support.
Universe of Data Collection Efforts
Current Request
ACF seeks Office of Management and Budget (OMB) approval for the following data collection activities to inform the previously identified areas:
Communications for Site Visit Planning (Attachments A and B). The study team will work with the project director of each grantee to plan the site visit that will occur in late 2018 and early 2019, upon OMB approval. We anticipate scheduling an initial 30-minute conference call with the project director to discuss the purpose of the site visit, identify people to participate in individual interviews, and begin scheduling the actual site visit. Additional communication may be needed to develop the actual site visit, which we anticipate will require no more than an additional 30 minutes of time from the project director.
Discussion Guide for Interviews (Instrument 1). This discussion guide will ensure that questions are asked consistently across grantee sites, to ensure that information is collected efficiently and completely, and will facilitate comparability of data during analysis. The interviews will last no more than one and a half hours, depending on the number of participants in the interview and their roles on the implementation team. Instrument 1 contains a list of topics.
Site visits will be conducted in late 2018 and early 2019, upon OMB approval. One researcher will visit each grantee for no more than two days. The timing of the site visit, and the interviews conducted during the site visit, will be coordinated with grantees to ensure minimal disruption to the work of implementation team members.
A3. Improved Information Technology to Reduce Burden
We will plan the site visit with the grantee project director. To minimize burden, we will use conference calls and emails to the extent possible.
The interviews will be conducted either individually or in a small group. Because of the nature of the interviews, it is not appropriate to use information technology such as computerized interviewing.
If necessary to minimize burden and collect comprehensive information, we will conduct small-group interviews, rather than individual interviews. Each group interview will include staff at the same or similar levels. For example, one group interview may be held with two or three frontline workers, such as caseworkers or outreach specialists. A separate group discussion may be held with supervisors of frontline staff. If there is only one staff member in a particular level, however, an individual interview will be conducted. We anticipate that staff at each of these levels will have different perspectives and thus may have different experiences with the implementation team. Group interviews will allow us to reduce the length of time spent at the site, while still obtaining comprehensive and in-depth information from staff with a range of experiences.
A4. Efforts to Identify Duplication
The information collection requirements for this process study have been carefully reviewed to determine what information is already available from existing studies and program documents and what must be collected for the first time. Although the information from existing sources improves our understanding of intervention design and implementation, ACF does not believe that it provides enough information on how comprehensive service models are developed and put into operation.
A5. Involvement of Small Organizations
Many of the respondents will be from small entities. The data collection was designed to minimize burden on small entities by identifying as few respondents as necessary from each small entity.
The site visit and interviews will be scheduled in collaboration with program staff to minimize disruption of daily activities. If feasible, the site visitor will conduct small group interviews with multiple staff. Individual interviews may be necessary for smaller entities or when staff schedules do not align for group interviews.
A6. Consequences of Less Frequent Data Collection
Rigorous evaluation of innovative initiatives is crucial to building evidence of what works and how best to allocate scarce government resources. This process study represents an important opportunity for ACF to learn about activities associated with successfully developing comprehensive service models for youth and young adults who have had contact with the child welfare system and are at risk of homelessness, and designing rigorous evaluations to measure the impacts of those services.
Not collecting information for the process study would limit the government’s ability to document the kinds of activities implemented and how those activities can be successfully implemented with federal funds, as well as to measure their effectiveness. Data from this initial information collection offer an opportunity to determine whether the cost and time associated with this phase produce high-quality, comprehensive service models and rigorous evaluation designs.
The site visit interviews are a one-time collection effort. If the site visits are not conducted, the evaluation team will be limited in its ability to examine themes seen in the document reviews.
A7. Special Circumstances
There are no special circumstances for the proposed data collection efforts.
A8. Federal Register Notice and Consultation
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and OMB regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on February 1, 2018, in Volume 83, Number 22, pages 4657-4658, and provided a 60-day period for public comment. A copy of this notice is included as an attachment. No substantive comments were received during the 60-day notice period.
Dr. Allison Metz at the National Implementation Research Network is serving as a consultant to the study. She provides consultation on instrumentation for the process study and on using an implementation science approach to guide grantee implementation activities. Discussions about the study have been conducted with federal staff in ACF and HUD as part of agency collaboration on the youth framework model from USICH. ACF recently initiated a monthly advisory call for the initiative with federal staff, including Sarah Hunter and Todd Shenk from the Office of the Secretary, HUD.
A9. Incentives for Respondents
No incentives for respondents are proposed for this information collection.
A10. Privacy of Respondents
Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. Participants will be informed that interviews will be recorded and that their information will be kept private to the extent permitted by law.
As specified in the contract, the Contractor shall protect respondent privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information. The Contractor has developed a Data Safety and Monitoring Plan. The Contractor shall ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor, who perform work under this contract/subcontract, are trained on data privacy issues and comply with the above requirements. All Mathematica employees are required to sign a confidentiality pledge upon hire and are reminded of their obligations to confidentiality during ongoing corporate security awareness training.
As specified in the evaluator’s contract, the Contractor shall use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The Contractor shall securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Processing Standard. The Contractor shall: ensure that this standard is incorporated into the Contractor’s property management/control system and establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology (NIST) requirements and other applicable Federal and Departmental regulations. In addition, the Contractor must submit a plan for minimizing to the extent possible the inclusion of sensitive information on paper records and for the protection of any paper records, field notes, or other documents that contain sensitive or personally identifiable information that ensures secure storage and limits on access.
Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individuals’ personal identifier.
A11. Sensitive Questions
There are no sensitive questions in this data collection.
A12. Estimation of Information Collection Burden
Table A.1 summarizes the proposed annualized estimated reporting burden for the data collection instruments. The individual and small-group interviews will be conducted with 10 people at each of the six sites. The annual burden is estimated to be 90 hours.
Table A.1. Estimate of Burden and Cost for the YARH Process Study—Current ICR
|
Number of Respondents |
Annual Number of Responses per Respondent |
Average Burden Hours per Response |
Annual Burden Hours |
Average Hourly Wage |
Total Annual Cost |
Instrument 1: Discussion guide: individual and small-group interviews |
60 |
1 |
1.5 |
90 |
$34.07 |
$3,066 |
Estimated Annual Burden Total |
|
|
|
90 |
|
|
Total Annual Cost
We estimate the average hourly wage for staff at the grantee organizations, $34.07, to be the average hourly wage of “social and community service managers” as determined by the U.S. Bureau of Labor Statistics Occupational Employment and Wage Statistics for 2016 (U.S. Department of Labor 2017).
A13. Cost Burden to Respondents or Record Keepers
One person at each of the six sites will participate in communications for site visit planning. The time for these communications and planning is estimated at 6 hours (see Attachments A & B).
|
Number of Participants |
Number of Responses per Participant |
Average Burden Hours per Response |
Annual Burden Hours |
Average Hourly Wage |
Total Annual Cost |
Attachments A & B: Communications for site visit planning |
6 |
1 |
1 |
6 |
$34.07 |
$204 |
We estimate the average hourly wage for staff at the grantee organizations, $34.07, to be the average hourly wage of “social and community service managers” as determined by the U.S. Bureau of Labor Statistics Occupational Employment and Wage Statistics for 2016 (U.S. Department of Labor 2017).
A14. Estimate of Cost to the Federal Government
The total cost for the data collection activities under this current request will be $651,056.
A15. Change in Burden
This request is for an extension to the second phase of information collection related to the Planning Grants to Develop a Model Intervention for Youth/Young Adults with Child Welfare Involvement at Risk of Homelessness. The request includes continued use of Instrument 1: Discussion guide: individual and small-group interviews. We moved the time for communications for site visit planning (Attachments A & B) from section A12 to section A13 to account for participant time, but to make it clear that these are not information collections.
A16. Plan and Time Schedule for Information Collection, Tabulation, and Publication
Process Study Analysis Plan
The first step of the process study includes a review of information already available on grantees. The next step—which is the focus of this ICR—will be to conduct site visits that include interviews with grantee staff. Atlas.ti, or a similar software program, will be used to support analysis of data collected during the site visits.
In the process study report, which will be based on all these data collection efforts, researchers will describe the design of grantees’ model interventions, changes to those designs, factors influencing changes to design, target populations, changes to target populations, factors influencing target population changes, implementation experiences, factors contributing to implementation experiences, and the extent to which grantees developed evaluable interventions. The data will be reported in a comprehensive report, with two distinct volumes to present the findings for the two overarching research objectives. As described earlier, our first research objective is to understand grantees’ individual and common experiences designing and implementing their innovative comprehensive service array intended to prevent homelessness among youth and young adults with child welfare involvement. The primary audience for the first research objective is child welfare stakeholders and the implementation science research community. Our second research objective is to understand how having a multiphase grant effort supported grantees in implementing and preparing for rigorous evaluation of an innovative service array intended to prevent homelessness among youth and young adults with child welfare involvement. The primary audience for the second research objective is federal and state funders. The report will integrate data from all data sources, including the document review and site visits, to address the research objectives and provide a comprehensive understanding of the YARH grantees and the implementation of their model interventions.
Table A.2 shows a schedule for the process study.
Table A.2. Schedule for the Process Study
Activity |
Datea |
Submit OMB package for site visit semistructured interviews |
April 2018 |
Conduct two-day site visits to grantees |
December 2018 – March 2019 |
Analyze data collected during site visits |
March 2019 – May 2019 |
Draft process study report |
May 2019 – August 2019 |
Revise process study report based on comments from ACF |
August 2019 – September 2019 |
aThe actual start date depends on OMB approval and Grantee schedules.
A17. Reasons Not to Display OMB Expiration Date
All instruments will display the expiration date for OMB approval.
A18. Exceptions to Certification for Paperwork Reduction Act Submissions
No exceptions are necessary for this information collection.
References
Kroner, Mark. Housing Options for Independent Living Programs. Washington: CWLA Press, 1999.
U.S. Department of Housing and Urban Development. “Housing for Youth Aging Out of Foster Care: A Review of the Literature and Program Typology.” Washington, DC: HUD, April 2012.
U.S. Department of Labor. “Occupational Employment and Wages, May 2016.” Washington, DC: DOL, March 2017.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OPRE OMB Clearance Manual |
| Author | DHHS |
| File Modified | 0000-00-00 |
| File Created | 2021-01-21 |