SSB - Title IV-E Prevention Services Clearinghouse

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Title IV-E Prevention Services Clearinghouse

Formative Data Collections for ACF Research

0970 - 0356

Supporting Statement

Part B

DECEMBER 2021

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers:

Christine Fortunato, Laura Nerenberg, and Jenessa Malin,

Office of Planning, Research, and Evaluation

Part B

B1. Objectives

Study Objectives

The objectives of the Title IV-E Prevention Services Clearinghouse (hereafter referred to as the Prevention Services Clearinghouse; see SSA section A1 for additional information about the Prevention Services Clearinghouse) data collection are to:

• Conduct expert consultations to inform potential revisions to the Prevention Services Clearinghouse Handbook of Standards and Procedures

• Conduct developer queries to obtain information about program or service manuals, manual citations, and feedback on the program or service descriptions that appear on the website

The purpose of this information collection is to inform improvements to the Prevention Services Clearinghouse resources and systematic reviews of research on programs and services intended to provide enhanced support to children and families and prevent foster care placements.

Generalizability of Results

The data collection described here is intended to provide internally valid information regarding descriptions of programs being considered for review and information on the clarity and rigor of the standards and procedures that guide the review, not to promote statistical generalization to other sites or service populations.

Appropriateness of Study Design and Methods for Planned Uses

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.

The data collection plans described here are appropriate to gather information to inform improvements to the Prevention Services Clearinghouse. Collecting information directly from program developers and experts will allow the project team to ensure accuracy of information used in the program review process and strengthen the Handbook of Standards and Procedures. The activities are in-line with the Prevention Services Clearinghouse’s charge to provide objective, rigorous, and transparent information on evidence-based programs and services that may be eligible for funding under Title IV-E of the Social Security Act as amended by the Family First Prevention Services Act of 2018.

B2. Methods and Design

Target Population

For the expert consultation sessions, the target populations will be comprised of individuals with substantial knowledge or experience in one of several topic areas including: parenting, mental health, substance use, child welfare, program adaptations, evidence standards, and equity.

The target population for developer queries are developers of programs and services that are being considered for review, are currently undergoing review, or have been reviewed by the Prevention Services Clearinghouse.

For both data collection activities, the team will use non-probability, purposive sampling to identify potential respondents who can provide information on the key constructs under examination. Because participants will be purposively selected, they will not be representative of the population of experts or program developers involved with programs and services intended to provide enhanced support to children and families and prevent foster care placements. Instead, we aim to obtain variation in participants’ expertise and knowledge to better understand the programs under review and the methods appropriate for reviewing them.

Respondent Recruitment

For each of the several expert consultation sessions, the project team has developed a list of recommended experts aligned to the specific topic to be approved by ACF. The consultants recommended for each group have both content and methods expertise relevant to the topics assigned to the group. In addition, each group is expected to have a diversity of opinions about the topics at issue for the group.

The respondents for the developer queries are determined by the programs and services considered for or selected for review by the Prevention Services Clearinghouse.

B3. Design of Data Collection Instruments

Development of Data Collection Instruments

The project team has developed an expert consultation discussion guide for each of several topic areas including: parenting, mental health, substance use, child welfare, program adaptations, evidence standards, and equity. The guide (Instrument 1) covers questions that will be asked of individuals across groups. Prior to the consultation sessions, the team will share a fillable pdf feedback form (Instrument 2) that includes excerpts from the Handbook and legislation followed by the topics/questions that will be discussed in the session. The project team will ask the consultants to review the materials and draft notes and comments in the form before the session. This feedback form and our request to complete the form prior to the session is intended to streamline the data collection during the sessions and maximize group time.

The project team has created two developer query templates that streamline data collection to only those pieces of information necessary to generate an accurate program review. These templates include: (1) a program description review request sent prior to publishing a program review on the Prevention Services Clearinghouse website (Instrument 3); (2) manual requests, including a version for use before the Prevention Services Clearinghouse has added a program or service to the working list of programs and services under review and a version to be used after a program or service is added to the working list of programs and services under review (Instrument 4). For each developer query, one of these templates is tailored by project staff to the specific program or service and developer.

B4. Collection of Data and Quality Control

The project team will collect all data. Emailed invitations to the expert consultants will include information about the time commitment, responsibilities of the participants, and details about honoraria. Prior to the sessions, the team will secure participation and non-disclosure agreements.

The developer queries and any responses will be saved on Abt’s secure servers and will be accessible only by members of the project team.

During expert consultation groups, one or more members of the project team will facilitate the discussion while additional project team members will take notes and ask for clarification where necessary, aiming to capture the information as close to verbatim as possible. The team members who guide the discussions will be responsible for reviewing the notes to check for accuracy and identify any missing information. Expert sessions will be recorded (with the permission of the participants) to supplement the information in the notes. In addition, expert consultants will be able to use the fillable pdf feedback form they received prior to the session throughout the discussion to take notes, and the team will collect these forms at the end of the session. The notes from the session and a summary of the comments provided in the fillable forms will be summarized in a final memo for ACF.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

The developer queries and expert consultation group sessions are not designed to produce statistically generalizable findings and participation is wholly at the respondent’s discretion. Response rates will not be calculated or reported.

NonResponse

As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated.

B6. Production of Estimates and Projections

The data will not be used to generate population estimates, either for internal use or dissemination.

B7. Data Handling and Analysis

Data Handling

All data will be collected through email, fillable pdf form, or videoconference discussions, and final versions of notes from the discussions and recordings of the sessions will be saved on Abt Associates’ secure servers. The project team will use the original notes documents for reference, supplementing the notes with details from recordings as needed.

Responses to developer queries will be reviewed by the project team. Information provided will be incorporated into the program or service details that are posted on the Prevention Services Clearinghouse website.

Data Analysis

The data for this information collection will be in the form of emails from program developers, feedback forms from the expert consultations, and notes and recordings from the expert consultations. The queries and consultations conducted as part of this data collection will be used for internal purposes, including to assist the project team in carrying out objective and rigorous systematic reviews of programs and services for the Prevention Services Clearinghouse and to inform the possible revision to the Handbook of Standards and Procedures. The information will be interpreted in conjunction with existing documentation the team collects through a systematic scan of literature on programs and services under review and background information on systematic review methodologies. The team will assess and synthesize the information to identify any next steps.

Data Use

The information collected is meant for internal purposes; to inform potential updates to materials related to the Prevention Services Clearinghouse. As such, information collected under this request will not be shared directly, but will inform documents or presentations that are made public, such as Fact Sheets or the Handbook of Standards and Procedures; resource documents such as a Reporting Guide for Study Authors; background materials for technical workgroups; or program descriptions posted on the Prevention Services Clearinghouse website.

B8. Contact Persons

Name	Title/Affiliation	Email
Sandra Wilson	Principal Associate, Abt Associates	Sandra_Wilson@abtassoc.com
Suzanne Kerns	Professor, University of Colorado	Suzanne.Kerns@cuanschutz.edu
Jen Norvell	Associate, Abt Associates	Jen_Norvell@abtassoc.com
Michelle Thompson	Research Analyst, Abt Associates	Michelle_Thompson@abtassoc.com
AshLee Smith	Senior Analyst, Abt Associates	AshLee_Smith@abtassoc.com
Meaghan Glenn	Associate, Abt Associates	Meaghan_Glenn@abtassoc.com

Attachments

• Instrument 1: Consultation Discussion Guide

• Instrument 2: Consultation Feedback Form

• Instrument 3: Program description Review Request

• Instrument 4: Manual Request

6