YARH Supporting Statement B revised 3 28 14

Planning Grants to Develop a Model Intervention for Youth/Young Adults with Child Welfare Involvement At-Risk of Homelessness

OMB Information Collection Request

New Collection

Supporting Statement

Part B

March 2014

Submitted by:

Office of Planning, Research and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

7th Floor, West Aerospace Building

370 L’Enfant Promenade, SW

Washington, DC 20447

Project Officer:

Maria Woolverton

B1. Respondent Universe and Sampling Methods

In September 2013, the Children’s Bureau (CB) within the Administration for Children and Families (ACF) awarded grants to 18 organizations to support the planning and development of comprehensive model interventions for youth/young adults with child welfare involvement who are at risk of homelessness (“YARH grantees”) and rigorous evaluation designs to measure the outcomes of those interventions.

The overall objective of the YARH process study is to document the activities and outcomes of the two-year planning period. All 18 grantees that received funding will be included in the process study.

In this Information Collection Request (ICR), clearance is sought for five instruments associated with the YARH process study. The sampling approach for the study is the administration of a survey sampling form prior to the start of data collection (Attachment 1). The project director for each grantee will be asked to complete the sampling form to identify the planning team members and administrators at key community partners who should be asked to complete the Survey of Organizational Readiness and Partnership (Attachment 2). The survey will be administered at all 18 grantee sites twice: once in year 1 and once in year 2, to planning team members—who are likely to be administrators and directors of the grantee agency or other agencies serving youth and young adults with child welfare involvement and/or at risk of homelessness—and administrators from key community partners.

Site visits will be conducted at all 18 grantee sites once, in early 2015. Planning will be done collaboratively with the project director for each grantee (Attachment 6). Discussion guides for individual interviews (Attachment 4) and focus groups (Attachment 5) will be used in semi-structured discussions with planning team members and key community partners who are not represented on the planning team. The scheduling of site visits and discussions will be done collaboratively with the grantee project director (see communication in Attachment 3). Each site visit will include discussion with no more than five individual interviews (90 individual respondents total) and no more than twelve group respondents (ideally three groups of four people; or 216 respondents total).

B2. Procedures for Collection of Information

The data collection procedures for the Survey of Organizational Readiness and Partnership in the YARH process study are described below:

• Survey Sampling Form The project director for each grantee will receive an email with a file attachment containing a table. They will be asked to identify up to 20 possible respondents for the baseline (or follow-up) survey. In addition to providing the name and organization of the individual, they will be asked to provide an email address and phone number for the individual (see Attachment 1).

• Survey of Organizational Readiness and Partnership. The web-based survey (see Attachment 2) will collect systematic information from a larger number of planning team members than would be possible during on-site interviews. Topics in the survey include prior experience with planning comprehensive integration of services and planning evaluations, perceptions of the collaboration between the planning team members, and perceptions of organizational readiness for change. Respondents identified in the sampling survey will be invited to participate in the survey by email, which will be followed by a series of three reminder emails to non-respondents (see Attachment 3).

The data collection procedures for site visits include use of a Discussion Guide for Individual and Focus Group Discussions. These will be conducted in-person in the form of semi-structured conversations. Site visits will be conducted in winter 2015. The initial contact with grantee project directors will be made by members of the project team via e-mail and conference call (see Attachment 3 for recruitment materials including a draft of the initial e-mail and Attachment 6 for the script for the initial conference call). We anticipate it will take an hour to schedule the site visit with an initial 30-minute phone call followed by an additional 30 minutes of email or phone conversations. Team members will describe the process study and purpose of the site visit to the project director. To the extent possible, the semi-structured discussions will be held with groups of staff at similar levels within their organizations. For example, a discussion may be held with planning team members how are front-line staff in the organizations including case workers and outreach specialists. A second group discussion may be held with supervisors of the front-line staff. If a grantee has only a single staff member in a particular level, however, an individual discussion will be held. Scheduling of discussions will be done collaboratively with the project director to ensure minimal disruption to operations. The discussion will be led by members of the Mathematica Policy Research team which has been awarded the contract for the process study. Project staff will take notes during the discussions. We anticipate that discussions will average 1.5 hours and not more than 306 respondents will participate.

Information for the process study will be descriptive. In general, it will not involve formal hypothesis testing.

Unusual Problems Requiring Specialized Sampling Procedures

There are no unusual problems requiring specialized sampling procedures.

Periodic Cycles to Reduce Burden

There will be only one cycle of baseline data collection followed by a single follow-up data collection approximately eight months later.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Expected Response Rates

Based on previous experiences with organizations serving at-risk youth, we anticipate response rates to the survey of approximately 80 percent, because grantees will be asked to encourage their planning team members and partners to complete it. CB will send an advance letter (Attachment 3) to the 18 grantees, informing them of the survey, its purpose, and the estimated time to complete it. The response rate will be calculated using the standard formula from the American Association for Public Opinion Research (AAPOR).

Dealing with Nonresponse

The process study is a descriptive study. Researchers will report the response rate but will not use any statistical methods to adjust for nonresponse, as the whole population will be surveyed (that is, all individuals identified as being a member or representing a key partner organization for each of the 18 YARH grantees). Every effort will be made to decrease nonresponse by providing periodic reminders. Reminder emails will be sent to nonrespondents after the survey has been available for two weeks. Four weeks after the survey is made available, nonrespondents will receive telephone calls from Mathematica Policy Research offering to complete the survey during the call or at a mutually convenient time (see Attachment 3 for the proposed communication with respondents). Moreover, based on prior experience asking similar questions with similar populations, the process study team does not anticipate significant item nonresponse on the survey.

Maximizing Response Rates

To maximize response rates and data reliability for the Survey of Organizational Readiness and Partnership, we will take the following steps:

• Administer the survey to respondents identified by the grantees. A survey sampling form will be completed by each project director. They will identify no more than 20 people to whom the survey should be sent. Each identified individual should be either an active member of the planning team or staff at a key organization who is an active partner in the planning process.

• Use a well-tested survey common to all grantees. While the Survey of Organizational Readiness and Partnership has been tailored to the specific circumstances of the YARH evaluation, it includes elements from the Evidence-Based Home Visiting Partner Survey—a CB initiative that has received OMB approval (OMB No. 0970-0375), was administered twice, and is completed.

• Use a straightforward, low-burden survey. The Survey of Organizational Readiness and Partnership is designed to be easy to complete. The questions use clear and straightforward language. The average time required for the respondent to complete the survey is estimated to be one hour, as the survey includes primarily close-ended items.

• Administer the survey using a web-based system. Administering the Survey of Organizational Readiness and Partnership via the web will maximize the reliability of the data entered by respondents through skip-pattern logic and checks for consistency and validity.

We expect to obtain a high response rate among sites for the site visits and semi-structured discussions. Several factors will help ensure a high rate of cooperation among respondents. First, senior staff of the study team who are familiar with, and known to, the grantee will contact the project director. Additionally, we aim to interview stakeholders who are heavily invested in the planning grant activities. We anticipate that respondents will be eager to engage in these discussions. Since the project team will work collaboratively with grantees to schedule the site visit and interviews, we expect a high participation rate.

B4. Tests of Procedures or Methods to be Undertaken

The process study team will carefully test the web-based Survey of Organizational Readiness and Partnership with nine or fewer respondents to ensure that the instrument is working properly.

B5. Individuals Consulted on Statistical Aspects, Collecting Data, or Analyzing Data

Preliminary input on statistical methods was received from the National Implementation Research Network at the University of North Carolina at Chapel Hill, and Mathematica. The following is a list of individuals who were consulted:

Dr. Allison J. Metz

National Implementation Research Network

Frank Porter Graham Child Development Institute

The University of North Carolina at Chapel Hill

Campus Box 8185

Chapel Hill, NC 27599-8185

Dr. M.C. Bradley

Mathematica Policy Research

1100 1st St. NE, 12th Floor

Washington, DC 20002

Ms. Lisa Klein Vogel

Mathematica Policy Research

1100 1st St. NE, 12th Floor

Washington, DC 20002

Dr. Matthew W. Stagner

Mathematica Policy Research

111 East Wacker Dr, Suite 920

Chicago, IL 60601

The team is led by Maria Woolverton, federal project officer at ACF. Additional ACF staff consulted includes Caryn Blitz, Ph.D. and Akilah Swinton, Ph.D.

Further input on analytic approaches may be sought from additional staff at these organizations, and also from outside consultants.

5