TANF Case Studies_OMB SSA_new template_Clean_8.6.19_final

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

State Temporary Assistance for Needy Families Case Studies

OMB Information Collection Request

0970-NEW

Supporting Statement

Part A

July 2019

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer:

Girley Wright

Part A

Executive Summary

• Type of Request: This Information Collection Request is for a new information collection. We are requesting 3 years of approval.

• Description of Request: Descriptive case studies to collect information about innovative employment and training programs for low-income individuals, including TANF recipients. We will collect information on up to 32 innovative programs through semi-structured interviews with program staff, in-depth interviews with participants, guided case reviews, and observations. The data collected in the study are not intended to be generalized to a broader universe.

A1. Necessity for Collection

The Office of Planning, Research, and Evaluation (OPRE) in collaboration with the Office of Family Assistance (OFA) in the Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) proposes to collect information for the State TANF Case Studies project.

Study Background

Moving toward self-sufficiency can be daunting for low-income individuals and families. In addition to dealing with serious barriers to employment, such as limited education, poor work history, or health issues, they also frequently need to interact with complex TANF, workforce, and other support systems in their efforts to obtain and maintain employment. States and localities have increasingly pursued innovative and promising approaches to help clients obtain success in the workforce and increase their overall stability. Innovative and promising approaches may consist of strategies that have demonstrated empirical effectiveness or are prepared to test new interventions with sound logic models designed to demonstrate promising program components. These approaches may include innovative employment and training strategies, such as apprenticeship and career pathways programs; subsidized and transitional employment; vocational training; and goal-setting and executive function skills coaching. Innovation in workforce support often requires effective coordination with providers of wraparound services, including child care, transportation, and homeless and emergency assistance, to provide the critical supports families need for long-term success.

The goal of the State TANF Case Studies project is to expand the descriptive knowledge base of innovative employment and training programs for low-income individuals, including TANF recipients.

Legal or Administrative Requirements that Necessitate the Collection

There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.

A2. Purpose

Purpose and Use

This descriptive study will document the implementation of innovative employment and training programs for low-income individuals, including TANF recipients. The study will also examine how programs provide or link families to wraparound services in support of program completion.

The information collected under this request will improve federal, state, and local policymakers’, practitioners’, and other stakeholders’ knowledge and understanding of innovative employment and training models designed to help low-income individuals succeed in the workplace. The project will produce a single summary report consisting of summaries of site visits and case studies of each selected program. The results will serve as a source of well-documented information about the strategies TANF agencies and workforce development agencies are implementing to help TANF recipients and similar low-income individuals become economically self-sufficient. The results will benefit state TANF administrators and also help ACF and other key stakeholders in setting program and evaluation priorities.

Research Questions or Tests

This information collection will explore the following research questions to fully document and understand the selected innovative programs:

1. What innovative employment and training strategies do program sites use to help move low-income individuals and families toward self-sufficiency?

2. To what extent do selected programs incorporate wraparound services—in particular, child care assistance—to support participants’ ability to get, keep, and advance in jobs?

3. How do selected programs deliver services that integrate innovative employment and training services and additional wraparound supports, such as child care assistance?

4. What implementation challenges do innovative programs face and what do staff believe to be the most important lessons they have learned in the process?

Study Design

For this descriptive study, the project team will collect information on up to 32 programs (state and local services operated under TANF including contracted programs and workforce development agencies’ services) that implement innovative approaches to workforce development for TANF recipients and other low-income individuals. Data will be used to produce individual case site descriptions and a case study compendium.

Data will be collected with two methods: (1) in-person site visits, including semi-structured interviews with program staff, in-depth interviews with participants, guided case reviews, and observations for up to 12 comprehensive case studies, and (2) semi-structured phone interviews with program staff for up to 20 shorter case studies.

The project team will purposively select programs for inclusion in the study. The programs will represent a range of employment and training models designed to help low-income individuals and families transition to self-sufficiency through direct support of employment activities and/or indirectly, by providing supports that alleviate barriers to employment. More details about site selection are available in Supplemental Statement B.

The study is intended to be descriptive in nature, and the limitations of the study methodology will be clearly stated in all published materials. For the comprehensive case studies, site visits will be limited to 2 to 3 days each. In the time available on site, we will collect as much information as possible, focusing on activities identified as most promising or innovative by the project team during the site selection process. For the shorter case studies, we will rely on publicly-available reports and documentation to generate comparable descriptions of selected programs. Telephone interviews will focus on improving consistency of information across the selected programs, and may not capture the full set of activities for all programs. For both components of the study, information will not support causal linkage between program services and participant outcomes.

Universe of Data Collection Efforts

The project team developed three data collection instruments: (1) a semi-structured program staff interview guide for interviews with staff during in-person site visits and telephone interviews, (2) a case review guide for use on site visits, and (3) an in-depth participant interview guide for use on site visits.

Data Collection Activity/Instrument	Respondent(s)	Content and Reason for Collection	Duration and Mode
Comprehensive Case Studies
Instrument 1: Semi-Structured Program Staff Interview Guide	Program administrators and/or frontline staff from each of the comprehensive case study programs. Program administrators include staff who administer and supervise the case study program, TANF and employment and training programs, child care and other wraparound supports, and other workforce programs; partners, such as community colleges, adult basic education providers, and employers; and state decision makers, as appropriate. Frontline program staff include intake workers, case managers, job developers, and other direct service providers who work at TANF agencies and American Job Centers, employment and training providers such as community colleges, and providers of wraparound supports, such as child care subsidy frontline staff. We anticipate conducting interviews with up to 168 respondents.	These interviews will capture information about: program philosophy and the program’s essential components; implementation of employment and training strategies; linkages with child care and other wraparound supports; perspectives of staff who work with clients; and outputs and outcomes.	Duration: 60 minutes Mode: In-Person Note: We will not administer the interview guide in its entirety, instead the project team will select and ask questions that are relevant to each respondent and program.
Instrument 2: Case Review Guide	Frontline staff who work with the clients whose cases are under review. We expect two respondents (likely case managers) per program for a total of 24 respondents.	These reviews will allow us to understand the sequencing and tailoring of services. For a purposive sample of cases, reviews will capture information about: the clients’ background as well as their service goals; which program services were delivered and their frequency, timing, dosage, and intensity; the role of wraparound services; clients’ barriers; and case outcomes. Note: The case review guide does not ask for any personally-identifiable information (PII) about the client. The project team will ask frontline staff to remove any PII from the case file prior to reviewing the cases with frontline staff who work with the clients whose cases are under review.	Duration: Two 45-minutes case reviews per staff member. Mode: In-Person Note: We will not administer the guide in its entirety, instead the project team will select and ask questions that are relevant to each respondent and program.
Instrument 3: In-Depth Participant Interview Guide	Program participants We will ask program staff to identify two clients with different service needs and different program experiences. We expect two respondents per program for a total of 24 respondents.	These interviews will enable the research team to obtain a comprehensive picture of client experiences and perspectives. By conducting in-depth interviews following program staff interviews and case reviews, we will gain an in-depth understanding of how clients’ individual and family circumstances interact with services delivered and the outcomes they experience. We will conduct interviews with rigorous attention to cultural sensitivities, following approved consent and assent procedures, and obtaining detailed information on how participants view the programs’ effect within the broader context of their lives.	Duration: 90 minutes Mode: In-Person Note: We will not administer the interview guide in its entirety, instead the project team will select and ask questions that are relevant to each respondent and program.
Shorter Case Studies
Instrument 1: Semi-Structured Program Staff Interview Guide	Program administrators and/or frontline staff from each of the shorter case study programs We anticipate conducting interviews with up to 32 respondents.	For each program, the semi-structured interviews will focus on filling in information not captured by a review of available reports and documentation, including information about: program philosophy and the program’s essential components; implementation of employment and training strategies; linkages with child care and other wraparound supports; perspectives of staff who work with clients; and outputs and outcomes.	Duration: 60 minutes Mode: Phone Note: We will not administer the interview guide in its entirety, instead the project team will select and ask questions that are relevant to each respondent and program.

As part of the on-site data collection conducted for the comprehensive case studies, the project team will also conduct up to four observations (Appendix 4) of each program’s services, activities, and environment. Observations will help us to document whether and how each program follows a human-centered design approach, such as offering services in a comfortable and welcoming physical environment, promoting positive staff-client relationships, and maintaining a positive and streamlined service delivery flow. We will use a versatile observation guide that can be adapted for observations of different activities, such as intakes, referrals, a job club or other type of employment workshop, and a transitional job experience. Depending on the program design, the project team may observe a subset of the following elements: office layout, decor (including signage), intake activities, client-staff meetings (when both consent), delivery of classroom training, and job placement activity. This activity does not impose any burden on study participants.

Other Data Sources and Uses of Information

To reduce burden on participating sites, the project team will review publicly available program documents, such as policy manuals, organizational charts, annual reports, and outreach documents. Where available, the team will request copies of memorandum of agreements (MOUs) with service partners and operating budgets. Beyond the time necessary for identifying appropriate documents to share, this activity does not impose any burden on study participants.

A3. Use of Information Technology to Reduce Burden

The most efficient way to collect most of the required information is to interview program staff and participants. However, whenever possible, information technology will be used in data collection efforts to reduce burden on program staff and in-depth interview participants. For example, interviews may be conducted by teleconference or WebEx to accommodate the needs of respondents.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

The project will systematically and comprehensively collect data to describe innovative employment programs for TANF recipients and other low-income families that is not available in comparable format elsewhere.

To allocate government resources towards collecting new information and generating a comparable understanding of programs, the project team will not begin data collection with sites until after thorough review of publicly available information, as described in A2 above.

A5. Impact on Small Businesses

This study might include small, nonprofit organizations. The project team will minimize burden for respondents by restricting the interview length to the minimum required and conducting interviews at times convenient for the respondents.

A6. Consequences of Less Frequent Collection

This is a one-time request for information.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on May 10, 2019, Volume 84, Number 91, page 20632, and provided a sixty-day period for public comment. A copy of this notice is attached as Appendix 5. During the notice and comment period, no substantive comments were received.

Consultation with Experts Outside of the Study

Experts in their respective fields from Mathematica and other entities listed below were consulted in developing the design, data collection plan, and materials for which clearance is requested.

Yvette Chocolaad, National Association of State Workforce Agencies

Kerry Desjardins, American Public Human Services Association

Angela Rachidi, Mathematica

Debra Strong, Mathematica

A9. Tokens of Appreciation

Program participant interview data are not intended to be representative in a statistical sense, in that they will not be used to make statements about the prevalence of experiences in the TANF or other program population. However, at each site, it is important to speak with participants with a range of background characteristics to capture a variety of possible experiences with the site’s employment and training services. Without offsetting the direct costs incurred by respondents for attending interviews, such as arranging child care or transportation, the research team increases the risk that only those individuals able to overcome the financial barriers will participate in the study, affecting the quality of the resulting data and insights.

The in-depth program participant interviews will take place in person and during scheduled visits to the participating employment and training programs. To offset costs of participation in the 90-minute interviews, we plan to offer program participants a $20 gift card. The token of appreciation will be provided at the time of the interview, after careful explanation of the interview procedures. Any individual who chooses not to participate after receiving the detailed explanation will be provided the token so that the token is not perceived as a coercion to participate.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

We will collect program staff and participants’ names and telephone numbers for use during study recruitment. This personally identifiable information will be destroyed after completion of the interviews.

Information will not be maintained in a paper or electronic system from which it is actually or directly retrieved by an individual’s personal identifier.

To maintain participant privacy during case file reviews with frontline staff, we will ask frontline staff to remove any personally-identifiable information from the case file prior to the review. The case review guide does not ask for personally-identifiable information and members of the project team will be instructed not to collect such information.

Assurances of Privacy

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law.

As specified in the contract, Mathematica will adhere to a strict set of approaches to ensure that data and respondent privacy are protected to the extent permitted by law. All interview respondents will be notified at the beginning of each interview as is stated in the instruments (see Instruments 1, 2, and 3) that the information that they provide will never be linked to their names, that their names will never be shared in interview transcripts or any study report, and that their participation is voluntary. On-site staff and in-depth participant interviews will be conducted in private areas, such as enclosed offices or conference rooms.

Interview notes will not be shared by the project team with ACF or anyone outside of the project team, except as otherwise required by law. All project team site visitors and interviewers have received training in privacy procedures.

Data Security and Monitoring

We will store data collected in a restricted folder on Mathematica’s servers. Mathematica’s servers are located behind Mathematica’s firewall and housed in a locked data center located in Mathematica’s locked access-controlled office suite. The data is mirrored in a secure, fault-tolerant data center; only authorized Mathematica Information Technology Services staff have physical and or logical access to the data mirror. Sensitive data resides on a project-specific folder that is only accessible to Mathematica staff who have a business need-to-know, as restricted by identity-based policies and access control lists. The data is encrypted as it is stored on the server with an Advanced Encryption Standard (AES) 256-bit key, which is FIPS 140-2 compliant. The folder in which the data resides is backed up onto encrypted disks. These backups are overwritten every two months by backups of newer secure data, a process that enables compliance with secure data destruction requirements.

All project team site visitors and interviewers have received training in data security procedures.

A11. Sensitive Information ¹

Some sensitive questions are necessary in a study of programs designed to affect employment of vulnerable populations. Before starting the in-depth interviews, all respondents will be informed that their identities will be kept private and that they do not have to answer any question that makes them uncomfortable. Although some questions may be sensitive for many respondents, they have been successfully asked of similar respondents in other data collection efforts, such as Parents and Children Together (OMB control number 0970-0403) and the Evaluation of Employment Coaching for TANF and Related Populations (0970-0506).

The sensitive questions in the data collection instruments relevant for this ICR include:

• Wage rates. It is necessary to ask about earnings because increasing participants’ earnings is a key goal of these programs. During the in-depth interviews, we will ask about participants’ jobs and how much they earned.

• Challenges to employment. It is important to ask about challenges to employment to understand the subgroups the programs are serving and the types of challenges the programs might be addressing. Challenges discussed during the in-depth participant interviews include having health issues and lack of education or work skills.

• Criminal record. Prior involvement in the criminal justice system makes it harder to find employment. Hence, for the in-depth interview participants with a criminal record, it is important to discuss their views on whether and how having a criminal record affected their employment prospects and history.

• Economic hardships. The in-depth interviews may include questions about economic hardships, such as difficulty paying bills, credit problems, or needing to borrow money from friends. These outcomes reflect a lack of self-sufficiency and may be affected by the programs.

• Health. The in-depth interviews will collect information on participants’ health and whether health issues impact participants’ ability to work or sustain work. Health factors could play a major role in participants’ ability to maintain employment.

A12. Burden

Explanation of Burden Estimates

We estimate the information collection burden for each of the three instruments in Table 1 below. See sections A2 and B2 for details about each instrument and respondents.

Table 1. Annual burden hours requested under this information collection (over the 3-year period of the project)

Instrument	Total number of respondents over study	Number of responses per respondent	Average burden hours per response	Total burden hours over study	Annual burden hours	Average hourly wage	Total annual cost
Instrument 1: Semi-structured program staff interview guide	200	1	1	200	67	$33.91	$2,271.97
Instrument 2: Case review guide	24	2	.75	36	12	$33.91	$406.92
Instrument 3: In-depth participant interview guide	24	1	1.5	36	12	$7.25	$87.00
Estimated annual burden total				272	91	$29.76	$2,765.89

Estimated Annualized Cost to Respondents

The total annual cost is $2,765.89. The total estimated cost figures are computed from the total annual burden hours and an average hourly wage for staff, participants, and employers. We estimate the average hourly wage for program staff to be the average hourly wage of Social and Community Service Managers (SOC 11-9151) taken from the U.S. Bureau of Labor Statistics National Compensation Survey, 2017 ($33.91). The average hourly wage of study participants is estimated to be $7.25, the federal minimum wage.

A13. Costs

There are no direct costs to respondents.

A14. Estimated Annualized Costs to the Federal Government

The total cost for the data collection activities under this current request will be $1,118,334 over three years. The annual cost is $372,778. We estimate the cost for each study activity in Table 2 below. Estimated costs include staff labor hours; operational expenses including equipment, overhead, printing, staff support, and travel; and any other expenses which would not have been incurred without this collection of information.

Table 2. Estimated costs of this information collection (over the 3-year period of the project)

Cost Category	Estimated Costs
Instrument Development and OMB Clearance	$137,626.14
Field Work	$309,990.90
Publications/Dissemination	$670,716.81
Total costs over the request period	$1,118,333.85
Annual costs	$372,777.95

A15. Reasons for changes in burden

This is a new information collection request.

A16. Timeline

Document reviews began in the second quarter of 2019 and program consultations will take place beginning in the third quarter of 2019.

The project team will conduct the site visits to up to 12 programs over a period of six months, beginning one month after program selection. After each site visit, the project team will prepare a short summary of the visit. The summary will describe the site visit activities and briefly describe the program’s key services. All site visit summaries are expected to be completed within one month after the end of data collection.

The project team will conduct the semi-structured phone interviews with up to 20 programs over 18 months, beginning one month after program selection.

All comprehensive case study drafts will be completed within three months after the end of data collection. For programs that participate only in a phone interview with no site visit, the project team will produce up to 20 shorter case studies. The shorter case studies will be completed within six months after the end of data collection. The shorter case studies will be published on the new Pathways to Work Evidence Clearinghouse website.

Within six months after data collection is completed, the project team will produce a public-facing final report that describes the project objectives and activities, summarizes the programs and innovative approaches included in the case studies, and includes all the completed case studies.

Table 3 below provides a schedule of the data collection and publication activities.

Table 3. Data collection and reporting timeframe

	Number of months	Start date
Data collection
Document review	12	Immediately following program selection
Semi-structured program staff interviews (phone)	18	1 month after program selection
Semi-structured program staff interviews (in person)	6	1 month after program selection
In-depth participant interviews (In person)	6	1 month after program selection
Case reviews (in person)	6	1 month after program selection
Observations (in person)	6	1 month after program selection
Reporting
Case studies	21	3 or 6 months after data collection
Final report	3	6 months after data collection is complete

A17. Exceptions

No exceptions are necessary for this information collection.

Attachments

Instruments

Instrument 1: Semi-Structured Program Staff Interview Guide

Instrument 2: Case Review Guide

Instrument 3: In-Depth Participant Interview Guide

Appendices

Appendix 1: Program Recruitment Emails

Appendix 2: Participant Recruitment Emails

Appendix 3: Project Description

Appendix 4: Observation Guide

Appendix 5: 60 Day Federal Register Notice

1 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.

10