SHM OMB SupportingStatement PART A final revised

SUPPORTING STATEMENT

FOR OMB CLEARANCE

SUPPORTING HEALTHY MARRIAGE (SHM)

PROJECT EVALUATION

BASELINE DATA COLLECTION

(OMB Control No. 0970-0299)

ADMINISTRATION FOR CHILDREN AND FAMILIES

OFFICE OF PLANNING, RESEARCH AND EVALUATION

370 L’ENFANT PROMENADE, S.W.

WASHINGTON, D.C.

A. JUSTIFICATION

A1. Circumstances Necessitating Data Collection

In keeping with the agency’s commitment to performance measurement and assessment, the Administration for Children and Families (ACF), Office of Planning, Research and Evaluation, undertook to evaluate the effectiveness of programs under the Healthy Marriage Initiative. In September 2003, ACF awarded a contract to MDRC to evaluate programs providing marriage education services to lower-income married couples with children. The project is called the Supporting Healthy Marriage Evaluation (SHM).

The evaluation design for SHM includes the collection of baseline data from research sample members. In May 2006, the Office of Management and Budget provided a 3-year period of approval, through May 31, 2009, for the SHM baseline data collection instruments (OMB Control No. 0970-0299).

The start up of the programs and services to be evaluated for SHM was delayed beyond the schedule that was originally projected. As a result, the intake and enrollment period for the research sample and baseline data collection are expected to continue for a longer period than originally expected. This information collection request is seeking approval for continued use of the OMB-approved baseline data collection instruments for an additional twelve month period through May 2010.

A2. How, By Whom, and For What Purpose Are Data to be Used

Purpose: SHM is the first large-scale, multi-site test of marriage education programs for low-income married couples.¹ It offers a tremendous opportunity to build knowledge on how to support healthy marriages among low-income couples. The study design utilizes the strongest known method for assessing program effects—random assignment. The multi-site structure provides an opportunity to assess a variety of approaches to marriage education. In addition, the evaluation will analyze short and longer-term program impacts. This well-designed study will illuminate the determinants of healthy marriages, and any tests that improve marital outcomes will provide important information about the causal links between any such improvements and outcomes for children, adults, and families.

The study will address the following major research questions:

1. How effective is marriage education for low-income married couples and what outcomes does it affect?

2. Who benefits the most and least from marriage education? Does marriage education work better for some groups than for others?

3. Why do some marriage education programs work better than others? What are the goals and service models of each site? The start-up challenges of the programs? What are the early lessons we can learn from these sites on designing marriage skills programs, securing program funding, building interagency partnerships, identifying and recruiting couples, and encouraging participation?

MDRC and subcontractors identified eight sites with the local capacity and commitment to implement the SHM model and participate in and support a rigorous evaluation of their programs. MDRC and subcontractor staff worked intensively with each of the eight sites to develop, refine and pilot test each proposed demonstration project and to put the random assignment and data collection protocols necessary to implement the evaluation in place. In each site, MDRC’s team² supported staff training and start-up processes and conducted an assessment of a pilot period. This ensured that each program was operating as planned and that the flow of clients through the program was consistent with both program and evaluation requirements. All eight of the sites met the criteria to move from pilot to evaluation status, including their implementation of a random assignment intake procedure. The movement from pilot to evaluation status occurred on a rolling basis. As random assignment began, the project sites collected baseline information from both members of all couples who agreed to participate. Random assignment is conducted after all baseline data are collected. We expect that the last site will complete research sample enrollment in early-to-mid 2010.

Baseline data are critical for various aspects of the SHM evaluation. The data will be used to describe the characteristics of the population being served, contact the couples for follow-up surveys, assess the validity of random assignment, define key sub-groups for later analyses, assess the extent of sample attrition and response bias for follow-up surveys, and increase the precision of impact estimates by controlling for baseline characteristics and differential rates of non-response.

How Data Are Collected and By Whom:

Except for the “Self-Administered Questionnaire (SAQ),” the baseline data collection instruments are administered by trained SHM program staff in each of the eight sites. An initial set of questions is used to determine if the couple meets SHM program eligibility criteria. Those who are found eligible complete the remaining sections with SHM staff.

A3. Use of Information Technology for Data Collection to Reduce Respondent Burden

The use of improved technology has been incorporated into the data collection design wherever possible in order to reduce respondent burden. Program staff in four sites have the capability to use computers to collect and input baseline information in order to reduce respondent burden. These computerized surveys reduce burden by skipping inappropriate and non-applicable questions, thus facilitating more streamlined data collection administration. For example, respondents who are not currently working will not be prompted to answer questions concerning their employment. Also, respondents will only be prompted to answer child-related questions specifically for the number of children they have. This technology allows for an easier and quicker information collection process. Because the self-administered questionnaire (SAQ) asks sensitive questions, it is not administered by SHM program staff but completed separately by each member of the couple via paper-and-pencil.

A4. Efforts to Identify Duplication

The information to be collected by this data collection does not currently exist in a systematic format. Some SHM program sites that offer other services within the community, beyond marriage education, may serve one or both members of the couple in other programs and collect a limited amount of information of a similar nature (e.g., demographic information). However, privacy issues limit the ability to access those other data.

All data collected for the project is input into a Management Information System (MIS). The MIS cross checks all participants entered into the system to ensure that couples are only randomly assigned to the study once.

A5. Burden on Small Business

All respondents are individuals. No small businesses are involved.

A6. Consequences to Federal Program or Policy Activities if the Collection of Information is not Conducted or is Conducted Less Frequently

This submission requests continuation of previously approved baseline data collection instruments. The instruments were originally approved for the period May 2006 through May 31, 2009. If baseline data collection is not continued beyond May 31, 2009, we will not have the information required to assess the validity of the random assignment process, address non-response bias in impact analyses, or determine impacts on sub-groups of couples for the full sample.

A7. Special Data Collection Circumstances

There are no special circumstances associated with this information collection request.

A8. Federal Register Comments and Persons Consulted Outside the Agency

Federal Register Comments: The 60-day Federal Register notice was posted in the Federal Register, Volume 73, Number 193, page 57633 on October 3, 2008. No comments were received. The 30-day Federal Register notice will be published at the end of the 60-day notification period.

Persons Consulted: In developing the data collection instruments, the SHM team worked closely with researchers at Mathematica Policy Research (MPR) who are conducting an evaluation of the Building Strong Families (BSF) marriage education program for ACF. In addition, the team worked with researchers at Child Trends, a sub-contractor on the SHM project, who have extensive experience in measures of healthy marriage. Measures tested and used in prior studies were included to the extent possible (e.g., National Survey of Families and Households (NSFH); Current Population Survey (CPS), National Health Interview Survey (NHIS), the National Longitudinal Survey of Youth (NLSY), the Intergenerational Panel Study of Parents and Children, and the National Co-Morbidity Study).

Further, MDRC and Child Trends collaborated consistently with a number of experts who serve as consultants to the SHM project. They include Dr. Benjamin Karney, Professor of Psychology at UCLA and Behavioral Scientist at the RAND Corporation; Dr. Thomas Bradbury, Professor of Psychology at the University of California at Los Angeles; Drs. Philip and Carolyn Cowan, Professors Emerita of Psychology at the University of California at Berkeley; Dr. Richard Heyman, Professor of Psychology at the State University of New York at Stony Brook; and Anne Menard, director of the National Resource Center on Domestic Violence. MDRC also drew on the internal expertise of Dr. Charles Michalopoulos, Dr. Allen LeBlanc and Jo Anna Hunter, each having been involved with developing surveys and impact analysis at MDRC for numerous other projects.

A9. Justification for Respondent Payments

A $25 respondent payment was previously approved by OMB and would continue through completion of this baseline data collection.

A10. Confidentiality

Respondents will be informed that the identifying information they provide will be kept private and confidential as provided by the Confidentiality Certificate issued by HHS and other provisions of law, and that the results of the study will be presented only in aggregate form. They will be asked to read and sign an informed consent form on behalf of themselves and their children. No personal identifiable information will be used in any report. A Confidentiality Certificate was received from the National Institute of Health (NIH) on August 7, 2006 and expires on September 30, 2013 (a copy is provided in Appendix A). The Certificate of Confidentiality issued by NIH provides that: “persons so authorized to protect the privacy of such individuals may not be compelled in any Federal, State, or local civil, criminal, administrative, legislative, or other proceedings to identify such individuals.”

In addition, the following safeguards are routinely employed by MDRC to carry out privacy and confidentiality assurances, and are explained to study participants on the informed consent form they will be asked to read and complete prior to enrolling in the study:

• All staff – at MDRC, its subcontractors, and SHM consultants – sign an agreement to abide by the corporate policies on data security and confidentiality. This agreement affirms each individual's understanding of the importance of maintaining data security and confidentiality and abiding by the management and technical procedures that implement these policies.

• All data, both paper files and computerized files, are kept in secure areas. Paper files are stored in locked storage areas with limited access on a need-to-know basis. Computerized files are managed via password control systems to restrict access as well as physically secure the source files.

• Merged data sources have identification data stripped from the individual records or encoded to preclude overt identification of individuals.

• All reports, tables, and printed materials are limited to the presentation of aggregate numbers.

• Compilations of individualized data are not provided to participating agencies.

• Confidentiality agreements are executed with any participating research subcontractors and consultants who must obtain access to detailed data files.

A11. Questions of a Sensitive Nature

The instruments remain the same as previously approved by OMB. The baseline information collection instruments include some questions that will potentially be “sensitive” for respondents, including questions regarding employment and income, mental health, substance abuse, family composition, their own experiences with abuse or neglect, and participants’ attitudes and experiences regarding marriage and relationships.

To improve understanding of the low-income married population and subgroups that might be particularly affected by SHM interventions, it is important to gather these types of data. For example, impacts for subgroups such as adults with mental health problems, high levels of relationship conflict or existing substance abuse may differ due to the interdependence between these factors and marital satisfaction. Therefore, the baseline data collection obtains measures of these characteristics prior to the intervention. However, as indicated in the original submission, couples are not asked to divulge details of how these factors play out in their own lives and can choose not to answer any question. In cognitive testing of the SHM baseline instruments, we found that couples were comfortable responding to these types of questions, despite their sensitive nature.

A12. Estimates of the Hour Burden of Data Collection to Respondents

Annual Burden Estimate³

Instrument	Annual Number of Respondents	Number of Responses per Respondent	Average Burden Hours per Response	Estimated Annual Burden Hours
Eligibility Checklist	4,000	1	.08	332
Baseline Information Form	4,000	1	.15	600
Self-Administered Questionnaire	4,000	1	.18	732
Contact Information Sheet	4,000	1	.17	668
Estimated Annual Burden Hours:				2,333 hours

A13. Estimates of Capital, Operating, and Start-Up Costs to Respondents and Record Keepers

There are no direct monetary costs to respondents other than their time to participate in the study.

A14. Estimates of Costs to Federal Government

The total estimated cost for baseline instrument development, data entry and processing, and data monitoring is $1,333,188. The estimated annual cost associated with baseline data collection and processing is $354,577. The costs of reporting on the baseline data were included with the estimates submitted with instruments for implementation and impact analyses since all reporting will be in the context of future implementation or impact reports.

A15. Changes in Burden

This submission to OMB is a request for approval to continue a previously approved information collection instrument that will expire on May 31, 2009, before the information collection activities are completed. There is no change in burden. However, given the changes from ICRAS 3 to ICRAS 4, the total number of responses appears higher in this submission. This is an artifact of the ICRAS 3 burden being imported as one single instrument, when in fact there were four. However, in actuality, there are no changes in the number of respondents, responses, estimated burden per response or overall burden.

A16. Tabulation, Analysis, and Publication Plans and Schedule

There are no plans for publications exclusively intended to present analyses of baseline data. The baseline information is primarily intended for use in the implementation and impact analyses. Baseline data submitted by SHM sites are reviewed and analyzed on an ongoing basis to ensure that the programs are reaching appropriate target populations and that the data are complete. Summaries of the baseline data will be prepared within a few months after random assignment is completed in each site. Information about the characteristics of couples served by SHM programs may be included in presentations or briefings on the evaluation. The data will primarily support later impact analyses, as stated previously.

Exhibit 4 Time Schedule
Activities and Deliverables	Date
Instrument Design Complete Baseline Data Collection	June 2006 May 2010
Implementation Evaluation Report Interim Impact Report Final Impact Report	December 2010 January 2012 July 2013

A17. Reasons for Not Displaying the OMB Approval Expiration Date

We intend to display the OMB approval number and expiration data on all baseline materials.

A18. Exceptions to Certification Statement

We have no exceptions to the Certification Statement.

Appendix A

Confidentiality Certification from National Institute of Health, US Department of Health and Human Services

1 The Building Strong Families (BSF) project is another multi-site random assignment evaluation funded by the Department of Health and Human Services in 2002 as part of the Healthy Marriage Initiative. BSF is an initiative to develop and evaluate programs designed to help interested unwed parents achieve their aspirations for healthy marriage and a stable family life.

2 MDRC is the lead evaluative organization conducting the SHM project. It is joined by a team of subcontractors including Abt Associates, Child Trends, Optimal Solutions Group, and McFarland Associates.

3 We calculated the cost burden to SHM participants by dividing the average husband’s yearly earnings ($20,000) by a 35-hour work week, assuming husbands worked for 50 weeks per year.

10