TOC Study Supporting Statement B_Non-Substantive Change Request_clean

This document discusses the planned statistical methods for the TANF Office Culture (TOC) study. This study is sponsored by the Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families (ACF) in the U.S. Department of Health and Human Services (HHS). This submission seeks OMB approval for three data collection instruments that will be used as part of the sample selection and interview process:

Leadership and Supervisor Interview Guide
Frontline Staff Interview Guide
Focus Group Interview Guide

This justification provides supporting statements for each of the five points outlined in Part B of the OMB guidelines.

B.1 Respondent Universe and Sampling Methods

The respondent universe for this study includes Temporary Assistance for Needy Families (TANF) program leadership and staff supervisors, front-line staff in TANF offices, and TANF clients in six sites. The sample goal is 21 State, County, or Regional Directors and other relevant leadership, 20 supervisors, 40 front-line staff, and 54 TANF clients across six sites. Although sites chosen will vary by geographic region as well as along an urban/suburban/rural dimension, this sample will not be representative of all TANF leadership, staff, and clients.

B.1.1 Considerations for Selecting Sites

The study team will purposefully select six sites to address the objectives of this study. They will identify a diverse array of TANF programs that exhibit characteristics that are indicative of a positive office culture. This includes the following criteria:

Clear and consistent program mission and goals. The literature review emphasized the importance of clear and well-defined organizational imperatives that are clearly communicated to staff at all levels and regularly monitored using performance benchmarks. Additionally, goals and performance measures should be outcome- as opposed to process-driven, and client- as opposed to compliance-centered. Beyond mission and vision statements alone, the research team will seek agencies whose goals for a specific program or initiative demonstrate a commitment to “vertically integrating” these goals throughout staff levels. While it is also important to verify that staff at all levels are able to articulate these shared goals and the extent to which their roles relate to them, this is something the research team will assess during site visits as opposed to a selection criterion.
Focus on innovation and learning. Research suggests the importance of organizations maintaining a commitment to program improvement by adapting to new and changing information. Agencies can use internal program data and staff insights or borrow lessons from other programs and outside research to inform their improvement efforts. Accordingly, this criterion can be demonstrated to some degree through a willingness to take part in pilots or special initiatives to improve services. However, the research teams’ field experience suggests that a constant focus on pilot programs and new initiatives can cause organizational stress. To balance the presence of innovative strategies with the organizational prioritization of innovation, the research team will seek sites that demonstrate a commitment to organizational learning by allowing staff discretion and risk taking or by using performance metrics to direct the program in new directions, as opposed to constant programmatic change.
Staff development and empowerment. The literature emphasizes the value of organizations supporting staff to feel adequately trained to be effective in their role. Investing in qualified staff, fostering constructive risk-taking among staff, and giving them autonomy to exercise discretion in their casework through decentralized decision-making systems helps to improve staff morale and client outcomes. The research team will seek organizations that invest in professional development opportunities for their staff, building the skills to independently handle the varying and complex case circumstances they face.
Client-centered office and business processes. The literature suggests organizations that design their offices and processes to ease clients’ access to services and benefits for clients and staff. Organizational characteristics that can support client accessibility include physical office space layout; efforts to minimize paperwork; and efforts to minimize the number of in-person office trips required while still maintaining opportunities for meaningful client-staff interaction. Several states have streamlined and redesigned business processes to make service delivery more efficient and accessible in order to promote an overall culture of client service. In addition, the literature emphasized the benefit of staffing arrangements to effectively manage staff workload given fluctuations in client demand. The research team will seek organizations whose processes demonstrate attention to improving client access.
Diversity: The research team will seek diversity in sites along two primary dimensions: program characteristics and nature of improvement efforts. In addition, we will select a combination of sites that include a focus on both the upfront eligibility determination and recertification process as well as services offered from welfare-to-work programs. Lastly, sites will vary along an urban/rural/suburban dimension as well as by geographic region.
Willingness to participate: The research team will only include sites that are interested in participating.

Site selection will be a multi-stage process. The study team will review an array of available sources, including agendas and presentations from recent conferences and public reports highlighting TANF programs, as well as solicit recommendations from a subset of the experts listed in A.8.2 to identify an initial 10 to 12 potential site visit locations whose TANF programs exhibit at least two of the above criteria. Sites meeting those minimum criteria will then be the starting point for a “site selection matrix.” The study team will use that matrix to summarize additional information reviewed and gathered during the site identification process. The matrix will primarily include qualitative descriptions relevant to each of the site selection criteria.

From the initial list of 10 to 12 sites, the research team will contact fewer than 10 administrators in potential sites to summarize the goals of the study, confirm the information in the site selection matrix is accurate, gather more information about sites’ organizational cultures not found during the site identification process, and gauge interest in potential participation. These conversations, combined with input from ACF and the Office of Family Assistance, will determine which six sites best meet the criteria mentioned above and will be considered the final sites for the study.

B.1.2. Focus Group Participants

Once the six sites have been selected for site visits, the research team will work with the TANF program staff to identify and convene current TANF participants to participate in focus groups. We will aim for each focus group discussion to include about six to nine participants. Focus groups participants will represent a convenience sample; the study team will work with the TANF program staff in each site to identify individuals who may be appropriate for and interested in the focus group and to invite them to participate.

B.2 Procedures for Collection of Information

Using the criteria and process mentioned above in Section B.1.1, the study team will select sites that represent exemplars of TANF organizational culture. Drawing on relationships with state and local TANF agencies, the researchers will recruit sites. Once sites have been selected, the research team will schedule site visits that include interviews with the TANF Director, County Director, or Regional Director and other relevant leadership; two supervisors; and four front-line staff in each local office, as well as a focus group with nine TANF clients.¹ After consent is provided, staff and TANF clients will be interviewed using the appropriate interview guides during site visits in each of the six sites (Appendices A and B).

In-person interviews allow for the collection of non-verbal information (for example, general attitude of the interviewee, assessments of rapport built during the interview, how the interviewee interacts with their surroundings) that other data collection methods do not. Interviewers will take detailed notes during each interview to capture both verbal and non-verbal information, and some interviews may be audio recorded. Interviewers will write summary notes after each interview to record early impressions and themes that emerged during the interview.

The research team plans to conduct one focus group comprised of TANF clients per site with approximately nine individuals. The study will include individuals who have received TANF assistance and interacted with staff at the site. The group discussions will last 90 minutes. The focus group interview guide is included in Appendix C. The research team will also use an observation checklist to document the degree to which office design and service delivery flow highlights or embodies principles that are associated with a more client-oriented setting. This supplementary document (Appendix D) will not impose any burden on staff.

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

B.3.1 Expected Response Rates

This study uses a qualitative approach to identify and describe exemplars of TANF organizational culture. As described above, the research team will select sites that will inform ACF and others about successful strategies human services offices have undertaken to improve their organizational culture as well as describe TANF programs that exhibit a positive organizational culture.

We expect the following response rates:

Program leadership, staff supervisors, and frontline staff: Because the research team will be working with staff and managers who have expressed their willingness to participate, response rates will be close to 100 percent. In the event of scheduling conflicts, the research team will conduct phone interviews with those unable to meet on-site at a time convenient to the respondent.
TANF clients: As described in Section B.1.2, the research team will seek focus group participants who are interested in participating. We will plan to recruit 8-10 individuals per site to yield focus groups of 6-9 individuals yielding response rates of about 75-90 percent.

B.3.2 Dealing with Nonresponse

Focus groups with TANF clients have the potential to suffer from nonresponse bias. Participation in the focus group may require individuals to arrange for transportation and childcare and may exclude those who are unable to overcome these barriers to attend. Offering an incentive to potential focus group participants can reduce nonresponse bias by offsetting the costs of expenses incurred by individuals. Each focus group participant will receive a $25 gift card.

B.3.3 Maximizing Response Rates

The research team will maximize response rates by drawing on relationships with state and local TANF agencies to recruit sites, work with staff to schedule interviews with those willing to participate at a convenient time on-site, and will follow-up by phone with those unable to attend in-person due to scheduling conflicts.

B.4 Tests of Procedures or Methods to be Undertaken

There are no plans to test any of the mentioned procedures or methods.

B.5 Individual Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Exhibit B-1 lists individuals who will be responsible for collecting and analyzing data.

Exhibit B-1: Individuals Consulted on Statistical Aspects and Involved in Collecting and Analyzing Data

Individual	Affiliation	Role in Study
Mary Farrell	MEF Associates	Interview and data analysis
Asaph Glosser	MEF Associates	Interview and data analysis
Angela Gaffney	MEF Associates	Interview and data analysis
Riley Webster	MEF Associates	Interview and data analysis
Carolyn Hill	MDRC	Senior Adviser
Mike Fishman	MEF Associates	Senior Adviser
Amelia Popham	Office of Planning, Research, and Evaluation	Project Officer
Lauren Deutsch	Office of Planning, Research, and Evaluation/Business Strategies Consultants	Contract Project Officer

1 Four out of the six sites will include visits to two local offices. The number of leadership staff interviewed will vary by site, from two to six.

Supporting Statements for OMB Clearance Request Part B | 5

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	OPRE OMB Clearance Manual
Author	DHHS
File Modified	0000-00-00
File Created	2021-01-15

TOC Study Supporting Statement B_Non-Substantive Change Request_clean