TOC Study Supporting Statement A_REVISED_Clean_20181017_OPRE

The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) seeks approval for data collection activities for a study aimed at identifying and describing exemplars of Temporary Assistance for Needy Families (TANF) organizational culture as well as successful strategies human services offices have undertaken to improve their organizational culture. ACF seeks OMB approval for three data collection instruments that will be used as part of the field interviews:

Leadership and Supervisor Interview Guide
Front-line Staff Interview Guide
Focus Group Interview Guide

The proposed data collection activities described in this justification will collect qualitative data about the day-to-day influence of agencies’ organizational cultures on clients and front-line staff; exemplars of productive client-oriented office settings and organizational processes and cultures; and how TANF offices that have made conscious attempts at organizational change approached the process and evaluated the effects. This justification provides supporting statements for each of the eighteen points outlined in Part A of the OMB guidelines.

A.1.1 Study Background

The TANF Office Culture (TOC) study is sponsored by ACF’s Office of Planning, Research, and Evaluation within the U.S. Department of Health and Human Services who has contracted with MEF Associates to complete the study. ACF’s Office of Family Assistance administers the TANF block grant program. Established by the Personal Responsibility and Work Reconciliation Act of 1996, TANF is guided by four purposes:

Provide assistance to needy families so that children can be cared for in their own homes;
Reduce the dependency of needy parents by promoting job preparation, work, and marriage;
Prevent and reduce the incidence of out-of-wedlock pregnancies; and
Encourage the formation and maintenance of two-parent families.

Given the flexibility of the block grant structure, states vary in how they implement their TANF programs. Examples of this variation include the amount of cash assistance provided to families, the lifetime limit on the number of months a family can receive cash assistance, the organizational structure of the TANF agency, and the distribution of TANF funds across programs designed to accomplish each of the four purposes of TANF.

The way in which state and local agencies use the flexibility of the block grant influences TANF implementation and client experiences. Everything from how clients apply for services to the way in which clients meet with program staff influence client experiences. In addition to these policies and procedures, the organizational culture of TANF offices may also affect how policies are implemented and the experiences that clients have in these offices.

Organizational culture can be defined broadly as “the way things are done.” The processes, practices, and structure of a TANF office are tightly intertwined with its culture, often transmitting cultural cues and values to staff. Likewise, TANF agency leadership, norms, and staff attitudes may shape and be shaped by organizational culture. Multiple components of an agency’s organizational culture influence the way in which clients interact with the agency and its services. From the broader vision of agency leadership, to individual staff encounters, to the overarching policies and procedures, all layers of an organization directly influence client experiences.

Despite the fact that organizational culture has the potential to affect policy implementation and client experiences in TANF offices, there is little documented information on states or localities who have succeeded in promoting a positive organizational culture in their TANF programs. In order to document exemplars of organizational culture, the TOC study will involve in-depth interviews with leadership, supervisors, staff, and clients of TANF programs in six sites. Specifically, the study will conduct interviews to learn 1) what is the day-to-day influence of the various agencies’ organizational cultures on clients and front-line staff; 2) what are exemplars of productive client-oriented office settings and organizational processes and culture; and 3) how have agencies and offices that have made conscious attempts at organizational culture change approached this process or evaluated the effects. These interviews will inform the writing of a final report. Although ACF is the primary beneficiary of the proposed data collection effort, the audience may include other federal agencies, program practitioners, state policy officials, and researchers.

A.1.2 Legal or Administrative Requirements that Necessitate the Collection

There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.

A.2 Purpose of Survey and Data Collection Procedures

A.2.1 Overview of Purpose and Approach

The TOC study is designed to identify exemplars of TANF organizational culture as well as to describe successful strategies human services offices have undertaken to improve their organizational culture. The purpose of this information collection is to highlight TANF programs with positive organizational culture practices as well as those that have successfully undertaken efforts to change their organizational culture to serve as exemplars for other TANF programs.

The study involves visits to six sites over 5 months, beginning in summer 2018 (pending OMB approval) where the research team will use semi-structured interview guides to interview TANF program leadership, supervisors, front-line staff, and conduct focus groups with TANF clients. The research team will ask leadership and supervisors to describe the general division of processes across teams or locations, clients’ experiences accessing the program, the agency’s goals and how they measure progress or performance, and any organizational culture change initiatives to assess how they perceive their agency’s organizational culture. Similarly, the research team will ask front-line staff to describe their role in the service delivery process, clients’ experiences accessing the program, their perceptions of the agency’s goals, as well as their interactions with other staff at the agency.

The research team will also use a discussion guide to gauge how TANF clients perceive agency processes, how they interact with agency staff, and their assessment of the agency’s organizational culture. The research team will compile information from all interviews in a final report to be submitted July 2019.

A.2.2 Research Questions

Researchers have long studied the role that organizational culture plays in the delivery of public assistance benefits. However, this attention became more pronounced following the passage of the 1996 Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA). PRWORA replaced the Aid to Families with Dependent Children (AFDC) program with TANF. PRWORA represented a notable shift in how the federal government funded welfare programs, the level of control states had in operating their programs, as well as the goals and priorities for welfare agencies.

The shift from AFDC to TANF emphasized that cash welfare programs were supposed to provide temporary assistance and support families’ transition to self-sufficiency. It also gave states wide latitude in how they spent their TANF block grant. This shift changed the role of welfare agencies and their staffs, including an increased emphasis on the delivery of employment services and an imperative to increase work participation to limit families’ time on TANF. Following PRWORA, scholars studied the implementation of TANF. This included studies of the way in which organizational culture contributed to or inhibited this implementation

Despite research into the role of organizational culture in welfare programs following passage of the 1996 PRWORA, less has been documented about what TANF agencies have done to promote organizational cultures that support effective implementation of TANF programs and how those organizational cultures impact clients and staff. To help fill this research gap, the research team will interview staff and TANF clients to discern:

What is the day-to-day influence of the various agencies’ organizational cultures on clients and front-line staff?

What are exemplars of productive client-oriented office settings and organizational processes and culture?

Human services agencies typically aim to change organizational culture by changing staff behavior. The literature also has examples of agencies that have streamlined and redesigned business processes to make service delivery more efficient and accessible while promoting an overall culture of client service. Despite growing research on explicit efforts by agencies to change their organizational cultures, there is limited empirical evidence of the effects of efforts to change organizational culture in TANF offices. A small number of studies have shown positive results related to efficiency in service delivery, client wellbeing and benefit receipt, staff morale and retention, and overall agency performance. Site visits and interviews will add to this body of limited research, and help answer the question:

Are there TANF offices that have made conscious attempts at organizational culture change and how have they approached this process? How have agencies and offices evaluated the effects of these changes?

A.2.3 Study Design

The study will be comprised of one round of in-person interviews with leadership, supervisors, staff, and TANF clients across six sites. The research team will draw on relationships with state and local TANF agencies as well as publicly available information to purposefully select sites that exhibit the following characteristics that are indicative of a positive office culture:

Clear and consistent program mission and goals
Focus on innovation and learning
Staff development and empowerment
Client-centered office and business processes

For a more detailed discussion of the criteria for selecting sites, see Supporting Statement B.1.1.

Sites will vary by geographic region as well as along an urban/suburban/rural dimension to obtain a diversity of sites, although not representative. Once sites have been selected, the research team will schedule site visits that include the following at each site:

interviews with the TANF Director, County Director, or Regional Director (Appendix A);
interviews with three supervisors (Appendix A);
interviews with two front-line staff (Appendix B); and
a focus group of approximately nine TANF clients (Appendix C).

After consent is provided (see Appendices A-C), staff and TANF clients will be interviewed using the appropriate interview guides during site visits in each of the six sites.

In-person interviews allow for the collection of non-verbal information (e.g., general attitude of the interviewee, assessments of rapport built during the interview, interviewees’ interactions with their surroundings) that other data collection methods do not. Interviewers will take detailed notes during each interview, capturing both verbal and non-verbal information. Interviewers will write summary notes after each interview to record early impressions and themes that emerged during the interview. These notes will not be considered project data to be analyzed but they will serve as a record of early impressions and initial (emerging) themes that will then be followed up on by a more thorough and systematic analysis of the interview data.

The research team will also use an observation checklist to document the degree to which office design and service delivery flow highlights or embodies principles that are associated with a more client-oriented setting. This supplementary document (Appendix D) will not impose any burden on staff.

The qualitative study design will result in a description of successful strategies human services offices have undertaken to improve their organizational culture as well as a description of TANF programs that exhibit a positive organizational culture. One limitation of the qualitative study design and small sample size is that results cannot be generalized beyond the TANF programs, staff, and clients interviewed. However, the main goal of the TOC study is to identify exemplars of TANF office culture which differ from other programs. For further discussion of the study design and procedures, see A.16.1 and Supporting Statement B.

A.2.4 Universe of Data Collection Efforts

Exhibit 1 below describes examples of questions in each data collection instrument that pertain to the research questions:

Data Collection Instrument	Research Question	Examples of Instrument Questions
Leadership and Supervisor Interview Guide (Appendix A)	RQ1	Are there examples of times when it is appropriate for frontline staff to diverge from typical processes or procedures? What works well about your program structure, processes, or staffing model for clients?
	RQ2	In general, how would you describe your agency’s openness to new ideas?
	RQ 3	Can you provide an example of a recent service delivery improvement the agency has implemented? How did it affect clients’ experience or outcomes? Did you conduct an evaluation or use any tools to measure the change, either quantitatively or qualitatively?
Front-line Staff Interview Guide (Appendix B)	RQ1	Are you able to use your discretion when you provide services? Generally speaking, how easy is it for clients to navigate the steps in [insert agency’s name]’s TANF program? Have you been able to make any changes to the general service delivery process to make it easier for clients? What was the effect of this change on client experiences?
	RQ2	How would you describe your agency’s culture?
	RQ3	Are there any new agency priorities associated with this change? If so, what are they?
Focus Group Interview Guide (Appendix C)	RQ1	What was your experience like?
	RQ2	How would you describe the “vibe” at [agency]?
	RQ3	Has anything changed about working with [insert agency name] since you first started coming here?

The data we will collect using these instruments are not currently available through any outside sources.

A.3 Improved Information Technology to Reduce Burden

The information from site visits will be collected through semi-structured interviews that are not conducive to information technology, such as computerized interviewing. However, some interviews may be audio recorded.

A.4 Efforts to Identify Duplication

OPRE has done previous work examining components of organizational culture in tribal TANF programs¹ as well as in home visiting program sites.² A Descriptive Study of Tribal TANF Programs documented the implementation and operation of four different Tribal TANF programs, describing their philosophies, goals, and strategies; organizational structure and management; and service delivery models. While the study wasn’t explicitly focused on organizational culture, researchers found that integrating tribal culture into the tribal TANF program helps “foster program relevance and client empowerment,” and staff and clients interviewed reported that incorporating tribal culture into the program helped clients connect with staff in their day-to-day interactions. While this study provides important tribal context for Research Question 1, its findings are not necessarily generalizable to other, state-run TANF programs. In addition, since the study was not solely focused on organizational culture, its purpose was not to highlight exemplars of productive, client-oriented Tribal TANF programs.

Another OPRE study is currently evaluating the Maternal, Infant, and Early Childhood Home Visiting Program (MIECHV) in 85 sites. Part of the evaluation involves an assessment of the organizational culture and climate of those sites to see how an agency’s organizational culture is related to providers’ work attitudes. While these findings could provide valuable information on the day-to-day influence of organizational culture on front-line staff in a home visiting program, these may not be relevant to TANF programs which have different goals, policies, procedures, and bureaucratic structures.

Research that documents exemplars of TANF organizational culture as its primary focus is limited. The TOC study data collection activities will collect information not currently available through other sources. In addition, all interview guides have been developed using guidance from the literature and expert consultations in order to maximize the content the study team is able to gather through individual questions and avoid asking multiple questions to collect the same information.

A.5 Involvement of Small Organizations

The primary organizations involved in this study will be state, county, and local TANF offices, although the specific TANF offices included will be determined subsequent to receiving OMB approval. The study may also include one or more Tribal TANF programs, which may qualify as small entities under OMB’s definition. We anticipate that no more than two of the sites included will meet the small entity qualification. For all sites, but particularly for those that are small entities, the research team will minimize burden. We will do so by providing clear guidance on procedures and by requesting only the information required to achieve the study’s objectives. Interviews with staff and participants will be scheduled according to their availability. There should be no adverse impact for any organizations participating in the study.

A.6 Consequences of Less Frequent Data Collection

This is a onetime data collection.

A.7 Special Circumstances

There are no special circumstances for this data collection.

A8. Federal Register Notice and Consultation

A.8.1 Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on January 22, 2018, Volume 83, Number 14, page 2996, and provided a sixty-day period for public comment. A copy of this notice is attached as Attachment 1. During the notice and comment period, no comments were received.

A.8.2 Consultation with Experts Outside of the Study

The study team conducted phone consultations with researchers, policymakers, and program operators who have a range of expertise in organizational culture, organizational change, and TANF program operations. The recommendations received from experts helped shape the final data collection instruments. The following outside experts contributed to the study design:

Outside Expert	Affiliation	Contact Information
Heather Hahn	Urban Institute	hhan@urban.org; 202-833-7200
LaDonna Pavetti	Center on Budget and Policy Priorities	pavetti@cbpp.org; 202-408-1080
Stuart Oppenheim	Child and Family Policy Institute of California	stuart.oppenheim@cfpic.org; 415-317-4568
Uma Ahluwalia	Montgomery County Human Services	Uma.Ahluwalia@montgomerycountymd.gov; 240-777-1198
Ken Miller	Change Innovation Agency	ken@changeagents.info; 573-353-7150
Marcia Meyers	Evans School of Public Policy and Governance, University of Washington	mkm36@u.washington.edu; 206-616-4409
Joe Soss	Humphrey School of Public Affairs, University of Minnesota	jbsoss@umn.edu; 612-626-9865
Erin Horgan	County Welfare Directors Association	ehorgan@cwda.org; 916-443-1749
Jodie Sandfort	Humphrey School of Public Affairs, University of Minnesota	sandf002@umn.edu; 612-625-3536

A.9 Incentives for Respondents

Focus group data are not intended to be representative in a statistical sense, in that they will not be used to make statements about the prevalence of experiences in the TANF population. However, it is important to secure participants with a range of background characteristics, to capture a variety of possible experiences with TANF services. As all participants will be current or former TANF recipients, the target population is, by definition, low income. Without offsetting the direct costs incurred by respondents for attending the focus groups, such as arranging child care, transportation, or time off from paid work, the research team increases the risk that only those individuals able to overcome the financial barriers to attend will participate in the study. Participants will receive a $25 gift card to account for incidental expenses such as transportation and/or child care that may otherwise prevent their participation in the study.

Studies have shown incentives’ effectiveness in increasing study participation among underrepresented populations, such as individuals from low-income³ or low-education households⁴, demographics of concern here. ACF’s Parents and Children Together Study (OMB control number 0970-0430) offered a $25 incentive payment to fathers that participated in focus groups, and 75% of parents who were approached participated. Given that previous studies have found incentives ranging from $20-$35 effective in increasing study participation among low education populations, we believe $25 is a reasonable amount for the time and cost associated with participation in these data collection activities but is not so high as to appear coercive for potential participants.⁵

A.10 Privacy of Respondents

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, whether the interview will be audio recorded, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. Language excerpted from the introductory script of the leadership and supervisor and front-line staff interview guides is below:

“Every effort will be made to keep information private to the extent permitted by law. However, because of the relatively small number of organizations participating in the study, there is a possibility that a response could be correctly attributed to you. Information you provide will not be shared with any other staff at your program or organization, including your supervisor or colleagues. Only the study team will have access to the information you provide through this interview. Your name will not be listed in any published reports, and comments will not be attributed to you. Instead, your information will be combined with information provided by others.”

Language excerpted from the consent form for the focus group interview guide is below:

“We will take notes and audio record the discussion. Only members of the research team will hear the recording. The notes and recording will be destroyed at the end of this study. We will not share any information about you with anyone outside the research team, and we will ask all participants to not discuss or share anything they have heard after leaving the discussion group. However, there is always a risk that other people in the group may reveal what was discussed in the focus group, or people outside the research team will see the information you provide. However, we will securely store the notes to minimize the risk that others see them. We will not include your name or any other identifying information in any reports from the study, and we will not use your name or any other personal information in any quotes we use. Information will be kept private to the extent permitted by law.”

This language, along with all interview guides, were approved by the Institutional Review Board at MDRC on December 13^th, 2017.

As specified in the contract, the Contractor shall protect respondent privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information. The Contractor has developed a Data Safety and Monitoring Plan that assesses all protections of respondents’ personally identifiable information. The Contractor shall ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor, who perform work under this contract/subcontract, are trained on data privacy issues and comply with the above requirements.

Information will not be maintained in a paper or electronic system from which they are actually or directly retrieved by an individuals’ personal identifier.

A.11 Sensitive Questions

There are no sensitive questions in this data collection.

A.12 Estimation of Information Collection Burden

A.12.1 Burden Hours

Exhibit 2 presents the burden on study participants completing the instruments included in this data collection request. The estimated annual burden is 43 hours.

Exhibit 2: Estimated Annual Response Burden and Annual Cost

Instrument	Total Number of Respondents	Annual Number of Respondents	Number of Responses Per Respondent	Average Burden Hours Per Response	Annual Burden Hours	Average Hourly Wage	Total Annual Cost
Leadership and Supervisor Interview Guide	24	8	1	1.5	12	$47.70	$572.40
Front-line Staff Interview Guide	12	4	1	1	4	$31.77	$127.08
Focus Group Interview Guide	54	18	1	1.5	27	$7.25	$195.75
Estimated Annual Burden Total					43		$895.23

A.12.2 Total Annual Cost

Exhibit 2 also presents the total annual cost. We estimated the total annual cost by multiplying the total burden hours by the average hourly wage for participants and staff using the Occupational Employment Statistics survey⁶ or the federal minimum wage for each instrument:

Leadership and Supervisor Interview Guide: wage rate for Social and Community Service Manager Occupations (SOC 11-9151), $34.07, plus a 40 percent adjustment for benefits, or $47.70
Front-line staff interview guide: wage rate for Community and Social Service Occupations (SOC 21-0000), $22.69, plus a 40 percent adjustment for benefits, or $31.77
Focus group guide: the minimum hourly wage, $7.25

A.13 Cost Burden to Respondents or Record Keepers

There are no additional costs to respondents.

A.14 Estimate of Cost to the Federal Government

The total contract costs for the data collection activities under this current request will be $160,227. Annual contract costs to the Federal government will be $53,409 for the proposed data collection.

A.15 Change in Burden

This is a new data collection.

A.16 Plan and Time Schedule for Information Collection, Tabulation and Publication

A.16.1 Analysis Plan

This project will generate original data from interviews with six TANF Directors, County Directors, or Regional Directors; 18 supervisors; 12 front-line staff; and approximately 54 TANF clients in six sites. The research team will examine the interview data in an integrative manner, looking for evidence of emergent themes across interviews and by possible subgroups (for example, front-line staff versus leadership; by site). Leadership, front-line staff, and TANF client interviews will be reviewed individually to assess the prevalence of common or salient themes that permeate across sites and interviewees. We will also look at sets of interviews to uncover themes across TANF offices. For instance, we will be examining if there are patterns associated with the type of TANF agency (e.g. Do welfare-to-work agencies have organizational cultures that are qualitatively different from agencies responsible for eligibility? Do TANF offices in rural areas have qualitatively different organizational cultures than those in urban areas?). The research team will use this information to inform the writing of the final report. The final report will describe the study’s purpose, primary research questions, methods, instruments, protocols, analyses, target population, and sample characteristics. The report will provide context regarding organizational culture in the public sector and will present the findings of the study. The report will discuss exemplars of TANF organizational culture, situate the findings in the context of existing research as synthesized, and propose directions for future work.

A.16.2 Time Schedule and Publications

Exhibit A-3

Data Collection or Publication Activity	Timing*
Site Visits and Interviews	Winter 2018– Spring 2019
Special Topics Briefs	2018-2019
Final Report	Fall 2019

*Exact timing is dependent on OMB approval of proposed information collection.

A.17 Reasons Not to Display OMB Expiration Date

All instruments will display the expiration date for OMB approval.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

1 Hahnn, Heather, Olivia Healy, and Chris Narducci. (2013). A Descriptive Study of Tribal Temporary Assistance for Needy Families (TANF) Programs. OPRE Report #2013-34, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Available online at https://www.acf.hhs.gov/sites/default/files/opre/tribal_tanf_final_report_for_submission.pdf

2 Duggan, Anne, Virginia Knox, Jill H. Filene, Helen Lee, Emily K. Snell, Sarah Crowne, Erika Lundquist, Phaedra S. Corso, Justin B. Ingels (2013). Revised Design for the Mother and Infant Home Visiting Program Evaluation. OPRE Report 2013-18. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Available online at https://www.acf.hhs.gov/sites/default/files/opre/mihope_full_report_april_17_3.pdf

3 Singer, Eleanor and Richard A. Kulka. (2002). “Paying Respondents for Survey Participation.” Studies of Welfare Populations: Data collection and Research Issues. Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs, edited by Michele Ver Ploeg, Robert A. Moffitt, and Constance F. Citro. Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

4 Singer, Eleanor, John Van Hoewyk, and Mary P. Maher. (2000). Experiments with incentives in telephone surveys. Public Opinion Quarterly, 64:2, 171-188

5 Berlin, Martha, Leyla Mohadjer, Joseph Waksberg, Andrew Kolstad, Irwin Kirsch, D. Rock, and Kentaro Yammoto. (1992). An experiment in monetary incentives. Proceedings of the Section on Survey Research Methods. Alexandria, VA: American Statistical Association.

6 https://www.bls.gov/oes/2016/may/oes_nat.htm

Supporting Statements for OMB Clearance Request Part A | 12

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	OPRE OMB Clearance Manual
Author	DHHS
File Modified	0000-00-00
File Created	2021-01-21

TOC Study Supporting Statement A_REVISED_Clean_20181017_OPRE