TIES_Supporting Statement B_Clean

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Testing Identified Elements for Success in Fatherhood Programs (Fatherhood TIES)

OMB Information Collection Request

New Collection

Supporting Statement

Part B

October 2023

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers:

Katie Pahigiannis

Rebecca Hjelm

B1. Objectives

Study Objectives

The Testing Identified Elements for Success in Fatherhood Programs (Fatherhood TIES) project will test the “core components” of fatherhood programs. Core components are the essential functions, principles and elements that are judged as being necessary to produce positive outcomes. The project is engaging five Fatherhood–Family-Focused, Interconnected, Resilient and Essential (Fatherhood FIRE) grant recipient organizations to implement and test one of three core components: targeted program content (parenting); systems navigation (e.g., family court, child support, child welfare); and goal setting and engagement supports to help the father to achieve his goals. Targeted program content will be implemented by one Fatherhood FIRE grant recipient; the other two interventions will be implemented by two Fatherhood FIRE grant recipients each.

Fatherhood TIES will enroll up to 3,000 fathers across the grant recipient organizations to participate in the study. Using the information gathered across several different data sources, the evaluation will include an implementation study and an impact study. The implementation study will describe who participated in fatherhood program services, how services operated, what fathers thought about services, and the challenges that program staff faced when implementing TIES services. The implementation study will provide lessons for the field on key elements for successful program implementation and barriers to overcome when implementing these core components.

The impact study will use experimental research methods to rigorously evaluate whether promising core components can bring about positive outcomes for fathers and their families. The specific outcomes of interest include employment and earnings, father-child relationship quality and co-parenting relationship quality.

Fatherhood TIES includes two phases of data collection: Phase 1 includes consent (Appendix #1), baseline survey (Instrument #1), a program information and management tool (Instrument #2), reflections from staff (Instrument #3), reflections from fathers (Instrument #4); Phase 2 will include additional implementation study data collection such as semi-structured interview protocols and the nine-month follow-up survey that will be used primarily for the impact analysis. The purpose of this current document is to cover consent of participants to enter the study, to collect baseline information, and to obtain information about program processes and outcomes to support the study team’s understanding of implementation and how it could be improved. A future request will be submitted for Phase 2 data collection efforts and will include the follow-up survey and semi-structured interview protocols.

Generalizability of Results

Overall, Fatherhood TIES will be designed not to promote statistical generalization to other sites or service populations, but to be internally valid for the grant recipient organizations involved in the study.

This study will include an implementation study for each of the five Fatherhood TIES grant recipient organizations. The implementation study is intended to present an internally valid description of the service population and the implementation of the interventions in chosen Fatherhood TIES grant recipient organizations, not to promote statistical generalization to other sites or service populations.

Fatherhood TIES will include randomized controlled trials involving each of the five Fatherhood TIES grant recipient organizations. The randomized study is intended to produce internally valid estimates of the intervention’s causal impact, not to promote statistical generalization to other sites or service populations. If an impact study is not feasible at any of the grant recipient organizations, the study team will pursue a quasi-experimental impact design. This non-randomized impact approach is intended to produce estimates of the intervention’s impact in chosen Fatherhood TIES grant recipient organizations, not to promote statistical generalizations to other sites or service populations.

Appropriateness of Study Design and Methods for Planned Uses

Conducting both an impact and implementation study will allow the team to meet its goals of testing the effectiveness of core components in producing positive outcomes for fathers who engage in core component interventions. The impact study will include a distinct service contrast to determine the effectiveness of the core components.

The primary sources for the impact analysis are the baseline survey (Instrument #1), a nine-month follow up survey (to be shared in Phase 2), and National Directory of New Hires data. Fatherhood TIES participation outcomes will be measured using existing data from nFORM, an existing data collection system required of all Responsible Fatherhood grant recipient organizations (OMB #0970-0566). Program staff record information about father participation in services in nFORM; it is a standardized source of participation information that can serve as context to the analysis of other outcomes and qualitative implementation study data collection.

The implementation study will provide useful feedback from both fathers and program staff on the core components and how they were implemented. Methods for collecting information from staff and fathers include reflection forms (Instruments #3 and #4) and semi-structured interviews (to be shared in Phase 2). Additional details about the impact and implementation study design will be included in the Phase 2 submission.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.

B2. Methods and Design

Target Population

The target population for Fatherhood TIES are fathers. Specifically, those who are eligible for Fatherhood FIRE grant recipient organization services. These individuals must be 18 years or older and have children expected or up to age 24 years old. They could be biological fathers who identify as married or unmarried, expectant fathers, adoptive fathers, stepfathers or individuals who live with children who are acknowledged as father figures (i.e., grandfathers or foster fathers). Fathers could be low-income, non-custodial or custodial single fathers, receiving TANF, previously received or eligible to receive TANF, or participating in Head Start or Healthy Start programs. Fathers may also have had a history of involvement with the criminal justice system or speak English as a second language.

Site Selection

Fatherhood TIES used a mix of existing data sources and individualized discussions with nine Fatherhood FIRE grant recipient organizations to select study sites. First, the study team used existing Fatherhood FIRE grant recipient organization enrollment data from nFORM to identify the top 25 programs enrolling the largest number of fathers annually. Then, the team reviewed additional information from nFORM such as program descriptions and aggregate data describing participant demographics, program content, program mode, location and implementation experiences. The combination of these two steps narrowed the pool of potential sites to nine organizations with the strongest fit for TIES.

A representative from nine organizations were invited into conversation with the study team. The team provided an overview of the proposed study, offered to answer questions, and guided staff through design brainstorming activities. The study team used these individualized discussions to recommend five grant recipient organizations as study sites. All five have agreed to participate in the Fatherhood TIES study.

B3. Design of Data Collection Instruments

Development of Data Collection Instruments

Fatherhood TIES study enrollment will build upon each fatherhood program’s existing data collection processes. This includes their use of the nFORM management information system and surveys that were developed for federal Fatherhood FIRE grant recipient organizations (under OMB #0970-0566). The baseline survey (Instrument #1), which will total no more than 22 minutes, will be administered through a separate web-based system. The baseline survey is designed to collect details about fathers that is not already collected in nFORM for the purpose of improving the sensitivity of the impact analysis; the nine-month follow up survey will use many of the same questions asked at baseline (Instrument #1).

The program information and management tool (Instrument #2, called the TIES Table) is based on previous program information and management tools used by the study team. The content noted in the headers will be tailored to the need of each Fatherhood TIES study site.

The reflection for staff (Instrument #3) and reflection for fathers (Instrument #4) is based on the study team’s experience with a similar approach in the Strengthening the Implementation of Responsible Fatherhood Programs (under OMB # 0970-0566). Instruments #3 and #4 were developed with input from Fatherhood FIRE grant recipient program staff and former fatherhood program participants.

Table B.1 provides more information about these data collection instruments.

Table B.1. Data Collection Instruments

Instrument	Details of data collection
nFORM	nFORM extract will be accessed and downloaded on a regular basis. The nFORM extract, in a csv format, includes individual-level information on contact information, program participation, and survey response nFORM operational reports will be accessed and downloaded on a regular basis. These reports will include summary level data on program participation
Baseline survey (Instrument #1)	Baseline questions aim to collect information that are not currently collected via the survey developed for the Fatherhood FIRE grant recipient organizations. Specifically, the survey questions ask about involvement in the criminal justice system, parenting and co-parenting measures and mental health. The survey questions will be accessed through MDRC’s Qualtrics system. The data will be stored and accessed via MDRC’s FedRAMP-certified cloud-based environment.
Program information and management tool (Instrument #2)	An Excel file will be custom-created for each program, to help program staff document the intended and actual service delivery of Fatherhood TIES-specific services that are not otherwise captured in the nFORM system. Program staff will enter individual-level information about services received by program participants. It will not include any personally identifiable information. Each site will submit the file and review the information with the Fatherhood TIES team approximately weekly.
Reflections from staff (Instrument #3)	Program staff will be asked to complete a short electronic, Qualtrics-based survey approximately once per cohort.
Reflections from fathers (Instrument #4)	Program participants will be asked to complete a short electronic, Qualtrics-based survey once during their workshop participation period.

The study team will work with each Fatherhood TIES grant recipient organization to fit study enrollment processes seamlessly into their existing program enrollment processes.

The baseline survey (Instrument #1) and reflections from fathers (Instrument #4) were each pre-tested with fewer than nine individuals. Overall, suggestions helped to remove unnecessary or redundant questions and to change the language of certain questions so that they were asked in more culturally responsible, understandable, and readable ways.

• The baseline survey (Instrument #1) was pre-tested with four fathers. On average, the fathers reported that they understood the questions and the wording and that they could easily read and understand them. Feedback from the fathers prompted the study team to add text to clarify the instructions and intent of questions such as why the father is asked to only respond to questions about one child.

• The reflection from fathers (Instrument #4) was pre-tested by two fathers. Based on the feedback, one key revision was reducing the number of questions asked so that fathers would be able to access and complete the reflection easily on devices like a mobile phone. Staff from three fatherhood programs also provided feedback about ways the reflection questions could be made more specific.

• The reflection from staff (Instrument #3) was reviewed by three staff from three fatherhood programs. The alignment of the questions with existing quality control efforts was noted. Every effort will be made by the study team to avoid duplicating efforts or conflicting with existing processes.

B4. Collection of Data and Quality Control

Recruitment Protocol

The Fatherhood TIES team will integrate study enrollment into each of the five Fatherhood TIES grant recipient organizations’ recruitment and enrollment processes. First, the program staff will confirm program eligibility in the nFORM system, checking that the participant is an adult and is a biological or expectant father, stepfather, or father figure raising a child who is up to age 24. After identifying prospective program participants who are eligible for study participation, program staff will administer the informed consent procedures and ask fathers to sign an electronic consent form within nFORM (Appendix #1). At two participating programs that will be implementing an intervention to support fathers’ navigation of systems (e.g., child support, child welfare, family court), program staff will confirm study eligibility by screening on participant interest in increasing time with the child and/or improving the relationships with their coparent(s). Screening will be done on these sites to better target the intervention to fathers with an interest in the support being provided. Fathers will be enrolled in the study from the time of study launch (estimated to be January 2024) for a period of approximately 21 months. The study team anticipates enrolling all eligible fathers who provide consent over the enrollment period to meet our sample size target of 3,000 fathers. Fathers who consent to be in the study will be randomly assigned into a program group or control group. The program group will have access to the individualized core components being tested in addition to the Fatherhood FIRE grant recipient’s business as usual programming. The control group will only have access to the business as usual programming

Data Collection Activities

The fathers that agree to participate in the study will be asked to complete the baseline survey (Instrument #1). This will occur within MDRC’s FedRAMP-certified cloud-based environment via Qualtrics, with access to the survey granted by the grant recipient organization program staff. If internet access is not available for any reason, the program staff will have the ability to administer the survey on paper.

Program staff will be asked to enter information about intended and actual services delivered in the program information and management tool (Instrument #2), referred to as the “TIES Table.” This is an Excel file that is customized to each program and intervention, to ensure that they are capturing and monitoring implementation of Fatherhood TIES-specific services.

Reflections from staff (Instrument #3) and reflections from fathers (Instrument #4) will take place in the middle of each Fatherhood TIES grant recipient cohort. These short surveys will be administered using Qualtrics to collect perspectives about services delivered or participated in.

Additional data collection activities will be detailed in Phase 2. These include a nine-month follow up survey of study participants and semi-structured interviews of program staff and participants.

Data Collection Monitoring

Throughout the study period, the Fatherhood TIES team will meet regularly with participating grant recipient organizations to support and ensure high-quality study enrollment and data collection processes and procedures, including to review baseline characteristics (Instrument #1) of the program and control groups and review the data compiled in the program information and management tool (Instrument #2). The team intends to meet with the program staff at a minimum once every two weeks. Additionally, the study team will receive frequent data extracts from the nFORM system, which allows the Fatherhood TIES team to have near-real-time information on the frequency of missing data or other concerning trends in the data.

The study team will also review information gleaned from reflections from staff (Instrument #3) and reflections from fathers (Instrument #4) at regular intervals to monitor program activities.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

This information collection request is for the project to obtain consent (Appendix #1) and baseline characteristics (Instrument #1) from fathers voluntarily seeking services at the five grant recipient organizations involved in the Fatherhood TIES study. Program staff will inform eligible fathers of their invitation to be a part of the Fatherhood TIES study and let them know participation is entirely voluntary. There is not a response rate associated with consent or completion of the baseline survey. The consent process is not designed to produce statistically generalizable findings. Participation in the study is wholly at the respondent’s discretion. Response rates will not be calculated or reported. The nine-month follow-up survey will be part of subsequent data collection; anticipated response rates for that survey will be detailed in the Phase 2 submission.

The data collection activities associated with Instruments #2 through #4 are for the purpose of understanding how the Fatherhood TIES interventions are implemented. For the reflections for staff and fathers (Instruments #3 and #4), participation is wholly at the respondent’s discretion. Nonetheless, response rates will be calculated and reported for Instruments #3 and #4; response rates will not be calculated for Instrument #2. The value of fathers contributing to our understanding of fatherhood programming will be emphasized in all communication.

NonResponse

As participants will not be randomly sampled and findings are not intended to be representative, non-response bias (in the form of individuals declining participation in the study) will not be calculated. Respondent demographics will be documented and reported in written materials associated with the data collection. The team will caveat analysis as appropriate.

All efforts will be made to obtain consent for participation in Fatherhood TIES (Appendix #1) and the baseline survey (Instrument #1) for all eligible fatherhood program enrollees. The nine-month follow-up survey nonresponse will be detailed in the Phase 2 submission.

The data collection associated with Instruments #2 through #4 will not be randomly sampled and findings are not intended to be representative, therefore non-response bias will not be calculated. The Fatherhood TIES team will emphasize the potential benefits of participating in data collection activities to increase motivation to participate; these benefits are not to the individual, per se, but to the fatherhood field broadly. To encourage fathers to provide insights about their experiences with the fatherhood programs (Instrument #4) the study team will provide each program with clear information about the data collection effort for staff to relay to fathers.

B6. Production of Estimates and Projections

The data will not be used to generate population estimates, either for internal use or dissemination.

B7. Data Handling and Analysis

Data Handling

The five Fatherhood TIES grant recipient organizations will use nFORM to collect electronic consent (Appendix #1) for Fatherhood TIES participants. Signed informed consents and nFORM data will be accessed directly via the nFORM web portal, requiring a dual authentication for access. Responses to baseline survey (Instrument #1), staff reflection (Instrument #3) and father reflections (Instrument #4) will be saved in MDRC’s FedRAMP-certified cloud-based environment. Access to the data will be granted on a need-to-know basis and only the Data Manager and study team members with a need-to-know will have access to the data. The Data Security Plan includes information on MDRC’s file logging system, data management, and file tracking. For instance, the Data Security Plan describes MDRC’s standard practices to access folders and files on a “need-to-know” basis. Also, data files with PII will be stored in folders within a FedRAMP/NIST 800-53-compliant environment. Only the Data Manager and a small number of trained programming staff will have access to them. Thereafter, most data processing for the project will be performed on analysis files that have been stripped of direct identifiers. Information about the handling of the nine-month follow-up survey data will be detailed in Phase 2.

Data Analysis

Both nFORM data and baseline survey data are designed primarily for quantitative analysis. The baseline data (Instrument #1) will be used to monitor research group assignment (that is, program or control group assignment). Baseline data will also be used for the impact analysis in conjunction with the nine-month follow-up survey data to control for baseline characteristics and baseline equivalence. The team will see to what extent scores improved after program involvement. The nFORM program participation data will be used to monitor program participation and track sample members for the follow-up data collection.

Information from the program information and management tool (Instrument #2) and reflections from staff and fathers (Attachments #3 and #4) will be analyzed periodically throughout the study period to help inform the study team’s understanding of the implementation of the Fatherhood TIES interventions. Plans to analyze the same data at the end of the study will be shared in Phase 2.

The study will be registered prior to the initiation of data collection with an appropriate registry such as the American Economic Association.

Data Use

The information gathered via the program information and management tool and reflections from staff and fathers (Instruments #2, #3, and #4) will enable the study team to determine how well the interventions were implemented and whether there were any implementation problems. This will inform the study team’s technical assistance to the programs to strengthen their interventions.

Findings from the current information collection request - as well as those that will be detailed in the Phase 2 request - will also be incorporated into documents or presentations that are made public, such as through conference presentations, website, or social media. These dissemination products may include infographics, short briefs or reports, or how-to guides and will describe how to properly interpret, analyze, and evaluate findings from the data collection, including limitations regarding generalizability and use of the study as a basis for policy.

This study will add to the body of knowledge in the fatherhood field, revealing evidence on whether program elements designed to produce positive outcomes for fathers do so.

B8. Contact Persons

Dina Israel, MDRC

Fatherhood TIES Project Director, Principal Investigator

(212) 340 – 8606

Dina.israel@mdrc.org

Michelle Manno, MDRC

Fatherhood TIES Deputy Project Director, Co-Principal Investigator

(212) 340 – 8873

Michelle.manno@mdrc.org

Meghan McCormick, MDRC

Fatherhood TIES Co-Principal Investigator

(347) 997 – 5311

Meghan.mccormick@mdrc.org

Attachments

Instrument #1 – TIES Baseline Survey

Instrument #2– TIES Table Template

Instrument #3 – TIES Reflection from Staff

Instrument #4 – TIES Reflection from Fathers

Appendix #1 – TIES Consent Materials for Fathers

Appendix #2 – IRB Approval

9