OMB Part B_TDI_2019-02-06

Temporary Assistance for Needy Families (TANF) Data Innovation Project

OMB Information Collection Request

New Collection

Supporting Statement

Part B

September 2018

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer:

Girley Wright

B1. Respondent Universe and Sampling Methods

The TANF Data Innovation (TDI) project online needs assessment survey will be targeted towards TANF agency administrators and staff from US states and territories. Because this data collection is intended to inform technical assistance and training outreach to states—including ACF follow-up—we do not plan to sample states and therefore the study design does not include sampling methods. Instead, we will aim to receive responses from the universe of 54 states and territories that operate the TANF program, including Washington, DC. In order to accomplish the goal of the needs assessment--which is to provide the right technical assistance and training to states, we need specific information on the universe. The study will seek to engage the maximum possible response rate from the 54 TANF agencies by employing a targeted outreach and follow-up strategy, described in B3. We anticipate a near 100% response rate.

B2. Procedures for Collection of Information

After gaining OMB approval, the project team will contact TANF agencies in 54 states and territories to collect firsthand information about state capacities, capabilities, and their current use of data in order to determine each TANF agency’s data readiness and to identify the TTA needs that would most benefit these agencies (see Appendix B – Survey Communication Materials for e-mail and phone scripts to be used during outreach). To collect detailed, comprehensive data while respecting the time and attention of staff in each state, we propose to use an online survey tool comprised of a series of eight modules (Attachment 1 - TANF Data Innovation (TDI) Needs Assessment Survey). The first survey module assists the TANF administrator in distributing the seven content modules to appropriate staff. Each of the seven content modules targets an agency subject matter expert for specific areas of data practice, such as analysis, data sharing, or data infrastructure. Depending on the structure of agency staffing, some staff may be assigned multiple modules.

Contractors will keep individually-identifiable primary data collected as part of the needs assessment private to the extent permitted by law. State-identified responses to individual survey items will not be shared directly with HHS or with the general public. Data collection pertaining to the federal reporting survey module will only be shared with HHS in a form that does not identify individual respondents or states. This is important in order to collect honest assessments of state experiences, and it will be made explicit in our survey instruments. We believe this will also increase the response rate and reduce non-response.

The team will use a combination of qualitative and quantitative research methods to analyze the data from the needs assessment survey, with the goals of identifying themes, challenges, and opportunities across states and identifying clusters of states that share similar strengths and challenges in the use of TANF data. The analysis will categorize states’ readiness to effectively use data and identify gaps in states’ capacity to work with data maximize the efficacy of technical assistance we provide.

The team will make initial contact with state- and territory-level TANF administrators via email using information acquired from HHS. (See Appendix B Survey Communication Materials for e-mail and phone scripts to be used during outreach.) Introductory emails will be sent to all TANF administrators, requesting participation in the survey and providing instructions for accessing a webpage where the TANF administrator can assign the seven content modules to appropriate staff within the agency.

Administrator Assignment of Modules

Administrators will be asked to complete assignments within ten business days. Reminder emails will be sent at various intervals as long as assignments have not yet been submitted. An overdue notification will be sent one business day after the deadline to administrators with outstanding modules. Subsequent communication, if necessary, will be handled over the phone, with a call being placed three business days after the completion deadline. Links to the content survey modules will be sent to assigned staff members within two days of receiving the completed staff assignments.

Staff Completion of Modules

TANF agency staff who are assigned content modules will be asked to complete one or more of the seven survey modules meant to gather information about the TANF agency’s data needs and current capacities. Invitations to complete the topical modules will be sent by e-mail. Respondents will be asked to complete their assigned module(s) within 15 business days. Reminder emails will be sent at various intervals as long as the assigned modules have not been submitted. A summary reminder will also be sent to the relevant TANF Administrator when five days remain in the module completion window. An overdue notification will be sent one business day after the deadline to staff with outstanding modules. Subsequent communication, if necessary, will be handled over the phone, with a call being placed three business days after the module completion deadline.

A final communication will be sent via email to all respondents with outstanding modules and the TANF administrators of states with outstanding modules when there are five days until the close of the survey data collection period. We anticipate survey data collection will close approximately eight weeks from the launch of the survey.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

The analysis from this data collection will be purely descriptive. As such, nonresponse bias is not as serious an issue as compared to in a causal inference study. Still, our goal is to minimize nonresponse to provide quality information for internal planning purposes. Therefore, we will use several strategies to increase response rates (described below).

Maximizing Response Rates

We plan to use several strategies to maximize response rates:

• If individuals do not respond to the initial email outreach, we will send reminder e-mails at regular intervals and follow-up with them by phone, if necessary.

• The project team will communicate with TANF agency administrators about whether survey modules have been completed or not.

• Contractors will keep individually identifiable primary data collected as part of the needs assessment private. State-identified responses to individual survey items will not be shared directly with HHS or with the general public. Data collection pertaining to the federal reporting survey module will only be shared with HHS in a form that does not identify individual respondents or states. This is important to increase the response rate.

• HHS staff that are involved with the project have helped the project team identify individuals to contact.

• The project team will market the TTA opportunities available through TDI at conferences – such as the Association of Public Data Users (APDU) and TANF State Regional Administrator conferences. We expect this will motivate administrators and staff to complete the needs assessment to inform the TTA activities.

Based on the previous experience from a similar survey, the team anticipates a response rate close to 100%. In 2016, the Office of the Assistant Secretary for Planning and Evaluation (ASPE) and the Administration for Children and Families (ACF) sponsored an online survey focused on data interoperability and systems integration targeted to state human service commissioners from the 50 states and Washington DC. This survey achieved a 90% response rate in a five-week survey period (collected under OMB #0990-0421). Further, the TDI needs assessment survey questions have been pilot tested with subject matter experts and the team does not anticipate that any of the needs assessment questions will be particularly problematic, producing higher item non-response rates. Although we anticipate a near 100% response rate, a response rate as low as 80 percent or 43 states would nicely satisfy the purposes of this data collection. An 80% response rate should give us sufficient information to provide useful TA to most states and territories, but we would have to take the non-response into consideration as we analyze the data in order to get a better picture of needs for the full set of states and territories. Getting closer to a 100% response rate will require less adjustment for the missing data. The study team would describe those states that did not respond so that it would be clear which type of states were missing. The study team will use publicly available information gathered from government and related websites to describe both respondents and non-respondents. With an 80 percent response rate, we would have sufficient demographic, geographic, and political variation to inform the activities of this project.

B4. Tests of Procedures or Methods to be Undertaken

The study team used pretesting with fewer than 9 respondents to identify necessary revisions to procedures and the survey (Attachment 1). Expert pilot testers were sourced from the professional networks of the TDI team. We sought pilot testers with government TANF and human services expertise specific to the survey topics. We used former state agency staff whenever possible to avoid pilot testing with the expected survey respondents.

B5. Individual(s) Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following key people from the study team will lead the data collection and analysis of data. They will be supported by a team of junior staff analysts.

Individual	Affiliation and Position	Role
Robert Goerge	Senior Research Fellow, Chapin Hall at the University of Chicago	Lead survey data collection, Lead data analysis
Emily Wiegand	Researcher, Chapin Hall at the University of Chicago	Coordinate survey data collection, Conduct data analysis
Leah Gjertson	Researcher, Chapin Hall at the University of Chicago	Coordinate survey data collection, Conduct data analysis
Adelia Jenkins	Executive Director, Actionable Intelligence for Social Policy (AISP) at University of Pennsylvania	Review of survey, Review of analysis
Richard Hendra	Senior Fellow, MDRC	TDI Project Director, Review of survey, Review of analysis
Barbara Goldman	Vice President, MDRC	Review of survey, Review of analysis

4