Alternative Supporting Statement for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
Healthy Marriage and Responsible Fatherhood Performance Measures and Additional Data Collection
OMB Information Collection Request
0970-0566
Supporting Statement
Part B
FEBRUARY 2024
Type of Request: Revision
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officers:
Rebecca Hjelm, Social Science Research Analyst, OPRE
Harmanpreet Bhatti, Social Science Research Analyst, OPRE
Harmanpreet.Bhatti@acf.hhs.gov
Part B
B1. Objectives
Study Objectives
The Administration for Children and Families (ACF), Office of Family Assistance (OFA), has had administrative responsibility for the federal funding of Healthy Marriage and Responsible Fatherhood (HMRF) Grant Programs since 2006. The authorizing legislation for the programs may be found in Section 403(a)(2) of the Social Security Act. OFA and the Office of Planning, Research, and Evaluation (OPRE) in ACF are proposing incorporate a minor change to the approved quarterly narrative reports and to extend approval of all other data collection activities currently approved for the HMRF grant program under this OMB number. A key objective is to facilitate the collection and reporting of quality and consistent performance measures data for all HMRF grant recipients. For further background, please see Supporting Statement A.
Generalizability of Results
The performance measure data collection and reporting described in this information collection request (ICR) is intended to present internally-valid descriptions of all HMRF programs and the populations they serve, not to promote statistical generalization to other sites or service populations.
Appropriateness of Study Design and Methods for Planned Uses
ACF requires HMRF grant recipients to collect performance measures data on their own grant operations and clients and to prepare and submit reports to ACF using those data. To facilitate these processes for grant recipients, ACF has developed a range of data collection and reporting instruments that are programmed into the nFORM (Information, Family Outcomes, Reporting, and Management) system. This web-based system allows for streamlined and standardized data collection and submission of grant recipient performance data through quarterly progress reports. nFORM is provided free of charge to grant recipients.
The performance measures design and methods for this request follow the previously approved approach that allows grant recipients to collect their own performance measures data using the nFORM data collection system (#0970-0566). nFORM was initially implemented with the 2015 cohort of HMRF grant recipients (0970-0460) to improve the quality of data and the efficiency of data collection and reporting. The performance measures, data collection and reporting instruments, and data collection system were revised in 2020 to reflect ACF’s priorities for the 2020 cohort, better align the data with the characteristics of HMRF programs and clients, and improve the grant recipient user experience. For the 2020 cohort of HMRF grant recipients, ACF also provided a continuous quality improvement (CQI) plan template to support grant recipients’ program improvement efforts. Grant recipients create plans using this template on paper and submit them to OFA.
This request includes only descriptive analyses of performance measures across grant recipients to support grant recipient and ACF program monitoring and improvement.
As noted in Supporting Statement A, this information is not intended to be used as the principle basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.
B2. Methods and Design
Target Population
There are currently 110 active grant recipients. Using the nFORM system, each grant recipient collects data on the applicable target populations associated with their grant type. These include healthy marriage (HM) adult individuals, couples, and youth; and responsible fatherhood (RF) community and reentering fathers. (Programs serving RF community fathers may serve mothers, but they are not a target population.) The nFORM system provides data at the grant recipient, program, and individual levels. For the 2020 cohort, grant recipients are estimated to enroll about 50,000 clients per year across these target populations.
Sampling and Site Selection
No sampling or site selection techniques are employed for the performance reporting system. All HMRF grant recipients are required to collect and report on their performance using the nFORM system. A sample survey cannot be used, because it would not provide the full and complete performance measures data reflecting the experiences of all clients served across the 2020 cohort of grant recipients.
B3. Design of Data Collection Instruments
Development of Data Collection Instruments
ACF is proposing to extend approval of the collection and reporting of the performance measures for the 2020 cohort of HMRF grant recipients to have standardized information on applicant characteristics, program operations, service delivery, and client outcomes. Minor updates are proposed only for the quarterly narrative reports.
These measures were initially developed in 2014 after extensive review of the research literature and grant recipients’ past measures. The performance measures, instruments, and nFORM system were revised in 2020 based on a targeted analysis of existing measures, feedback from key audiences, and discussions with ACF staff and the 2015 cohort of recipients. OMB approved the revised measures and instruments in 2021 (#0970-0566) and has approved a number of non-substantive changes since then. In an effort to gain a greater understanding of how HMRF programs influence program participants and staff at an individual level, ACF proposes to add one open text field to the quarterly narrative reports to capture information about the experiences of HMRF participants and/or staff.
All of the instruments covered by this ICR are used to achieve the objectives described in B1. Each instrument is described in Table A.1 in Part A of this ICR, including the instrument name, respondent type, content and periodicity of instruments, and mode of administration. The data from the survey instruments and service data are used to generate the quarterly reports to ACF; data from the survey instruments, service data, and quarterly reports are also used for developing and implementing CQI plans.
B4. Collection of Data and Quality Control
Who will be collecting the data? HMRF grant recipient staff use the nFORM system for most of the data collection and reporting activities described in Table A.1 in Part A. Program clients self-administer the client surveys (Instruments 1 and 4) after grant recipient staff launch the applicable survey for each client. In some cases, client surveys are administered on paper and grant recipient staff later enter the responses into nFORM. Grant recipient staff access and complete the CQI plan template outside of the nFORM system.
What is the recruitment protocol? Grant recipients are responsible for developing their own recruitment plans based on the populations they are serving. Recipients may use nFORM data regarding how clients were referred to the program to inform future recruitment strategies.
What is the mode of data collection? All grant recipients are required to use the web-based nFORM system for performance measures data collection and reporting, to improve data quality and consistency across grant recipients and reduce the burden of data collection and reporting for the instruments in Instruments 1 through 6. nFORM has a user-friendly interface accessible to authorized users from any computer with internet access, allowing for ease of data entry without purchasing or installing additional software or changing the configuration of their computers. Through nFORM, grant recipient staff generate unique survey passcodes for each client when they are ready to complete an instrument. Program clients use computers or tablets (often provided by grant recipient staff) to self-administer the applicant characteristics, entrance, and exit surveys in nFORM. This method presents several advantages over interviewer-administered surveys. It ensures greater privacy, and respondents will be less likely to give socially desirable responses, particularly with sensitive questions (Turner et al. 1998; Tourangeau and Smith 1996). It also reduces burden for grant recipient staff who would otherwise need to administer the surveys. To address possible literacy limitations, respondents have the option to wear headphones and listen to a recording of the questions, known as Audio Computer-Assisted Self-Interview (ACASI). nFORM allows grant recipients to generate the required quantitative performance measures for quarterly reporting to OFA with only the touch of a button, thereby minimizing their burden while maximizing the consistency and quality of performance data across grant recipients. Grant recipients complete the CQI plan template on paper, outside of nFORM, and submit the completed plan directly to OFA via email.
How are the data collection activities monitored for quality and consistency? The nFORM system facilitates the collection and reporting of high-quality and consistent performance measures for all 2020 HMRF grant recipients at the program and client levels. nFORM includes automatic data quality checks for numerous fields to improve data reliability. If grant recipient staff enter unusual or unlikely values in a particular field, the system will prompt users to check the value. For some fields, the response values are restricted; for others, grant recipient staff are able to override the check. In addition, nFORM includes reporting features for grant recipients that can be used to assess data quality, such as a query tool and operational reports.
What data evaluation activities are planned as part of monitoring for quality and consistency in this collection? Grant recipients are required to designate data managers who are responsible for routinely monitoring the quality and consistency of their data collection activities using nFORM. Improvement goals can be incorporated into grant recipients’ CQI plans and strategies. nFORM system users receive detailed training and technical assistance on nFORM prior to their initial use and throughout their grants. They also have continuous access to nFORM and CQI help desks to raise any questions regarding the performance measures, nFORM, or CQI. Grant recipients submit quarterly reports to ACF, which uses the data to assess grant recipients’ progress and performance.
B5. Response Rates and Potential Nonresponse Bias
Response Rates
The instruments described in Supporting Statement Part A, Table A.1 are not designed to produce statistically generalizable findings and participation is wholly at the respondent’s discretion. The data collection procedures, the nFORM system, and training and technical assistance provided to grant recipient staff described in B4 are designed to maximize response rates and data reliability. These efforts have been successful in ensuring strong response rates to date with the 2020 cohort of HMRF grant recipients. As of September 29, 2023 (the end of grant year 3), grant recipients in the 2020 cohort had completed enrollment for 103,444 clients. Of these, 94,177 (91 percent) had completed an entrance survey and 66,476 (64.3 percent) had completed an exit survey.
Non-Response
As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated. Respondent demographics will be documented and reported in written materials associated with the data collection.
B6. Production of Estimates and Projections
The data will not be used to generate population estimates, either for internal use or dissemination.
B7. Data Handling and Analysis
Data Handling
Responses in Section B4 above describe how nFORM was designed to standardize and improve the reliability of the data collected and reported by grant recipients. As described in B4, design features include automatic data quality checks during data entry and automated production of quarterly performance reports using data entered in nFORM.
Data Analysis
Descriptive statistics (such as counts, percentages, means, and modes) will be computed to summarize grant recipient and client experiences for program monitoring and improvement. These analyses include client enrollment, characteristics, services received, and outcomes. Quarterly grant recipient reports and ad hoc analyses will not be made public. Comprehensive annual reports and a final report with summary statistics across grant recipients will be made public.
Data Use
The comprehensive performance measure data collected by HMRF grant recipients is a valuable resource for program monitoring and improvement, as well as for research designed to broaden understanding of HMRF programs. For example, measuring income at enrollment and employment status at the beginning and end of the program helps grant recipients match individual clients to economic support services and also determine whether the program appears to be meeting that improvement goal across clients. If it is not, then grant recipients can develop and implement data-driven CQI plans to build more robust economic support activities for their clients. Similarly, asking youth their attitudes about sex at the beginning and end of programming allows grant recipients to target and refine services that aim to build relationship skills for individual clients and across their clients. Asking both HM and RF adults about their disciplinary practices at the beginning and end of programming helps grant recipients target and refine services aimed at improving clients’ parenting skills.
To support future research efforts, ACF and the nFORM contractor will update the data archiving plan to identify the most appropriate entity with which to archive a restricted use file of de-identified data at the end of the grants, so that the data are accessible to interested researchers.
Existing data documentation, including the nFORM user manual and data dictionary, will be updated to reflect enhancements to the system for the 2020 cohort of grant recipients. Additional documentation will be developed as part of ACF’s plan for archiving de-identified data for research purposes. This will include a communication plan with the identified archiving entity, a description of the data and supporting documentation, storage and security requirements, access information, analytic file formats, and a publishing schedule for the data.
B8. Contact Persons
The performance measures, instruments, and reports were developed by staff in OFA and OPRE in partnership with Mathematica. Staff at Mathematica were consulted on statistical methods, including:
Dr. Sarah Avellar
Principal Researcher
Mathematica
SAvellar@Mathematica-MPR.com
Inquiries regarding statistical aspects of the ICR should be directed to:
Social Science Research Analyst
Office of Planning, Research, and Evaluation
Administration for Children and Families
U. S. Department of Health and Human Services
Rebecca.Hjelm@acf.hhs.gov
Harmanpreet Bhatti
Social Science Research Analyst
Office of Planning, Research, and Evaluation
Administration for Children and Families
U. S. Department of Health and Human Services
Harmanpreet.Bhatti@acf.hhs.gov
Further consultations will be made with statistical experts in developing the data archiving plan.
Attachments
APPENDIX A: Legislative Authority
APPENDIX B: Institutional Review Board Exemption for ACF-Sponsored Analysis of Performance Measures
APPENDIX C: Assumptions for Calculations of Burden Estimates
Instrument 1: Applicant Characteristics Survey
Instrument 2: Program Operations Survey
Instrument 3: Service Delivery Data - nFORM Service Receipt
Instrument 4: Entrance and Exit Surveys
Instrument 5: Semi-annual Performance Progress Report (PPR)
Instrument 6: Quarterly Performance Report (QPR)
Instrument 7: Continuous Quality Improvement (CQI) Plan
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Pooja Curtin |
| File Modified | 0000-00-00 |
| File Created | 2025-05-23 |