Alternative Supporting Statement for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
Racial and Ethnic Disparities in Human Services Analysis Execution Project
OMB Information Collection Request
New Collection
Supporting Statement
Part B
September 2022
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officer:
Kimberly Clum
Part B
B1. Objectives
Study Objectives
The objective of the proposed Racial and Ethnic Disparities in Human Services Analysis Execution one-time qualitative study, funded by the Office of Planning, Research, and Evaluation (OPRE) within the Administration for Children and Families (ACF), is to explore how changes made to respond to the COVID-19 pandemic in one state’s Temporary Assistance for Needy Families (TANF) program’s policies and service delivery system were experienced by different racial and ethnic groups in that state. The goal is to obtain an in-depth understanding of how TANF participants of different racial and ethnic backgrounds experienced these policy and programmatic changes by comparing those experiences within three purposively selected sites in one state and to assess whether those changes may have helped to ameliorate or potentially created new challenges around program and benefit access for different populations.
Generalizability of Results
This study is intended to present an internally-valid description of TANF policies, practices, and service delivery and the experience of those policies, practices, and service delivery for different racial and ethnic groups in the three chosen sites, not to promote statistical generalization to other sites or service populations. Publications resulting from the study will acknowledge this limitation.
Appropriateness of Study Design and Methods for Planned Uses
We have proposed selecting sites purposely and using qualitative methods to collect data, as this site selection strategy and these methods are optimal for achieving the study’s objectives. Qualitative methods are appropriate for this study because they allow for the collection of data from multiple sources and a variety of respondents, such as through interviews, focus groups, document review, and descriptive analysis of TANF caseload data, to gain an in-depth, multi-faceted understanding of how policies and processes were implemented and experienced within a real-life context (Creswell 2007). Qualitative methods allow us to explore critical questions that we would be unable to answer with quantitative methods including –
the motivations for particular changes to TANF policies and processes;
potential gaps between the intended implementation of those policy or process changes and their actual implementation;
TANF program administrator and staff members’ understanding of any challenges around their implementation; and
program participants’ experiences of these policy or process changes and their implementation.
These methods can offer additional insights into whether those changes made it easier or harder for program participants of different racial and ethnic backgrounds to access or maintain access to the program and why some participants may have experienced different benefits or challenges associated with these changes.
A purposive sample will allow us to select sites that represent the racial and ethnic diversity of the state and to better pinpoint the different experiences of particular racial and ethnic groups within the state.
The results, drawn from a subsample of individuals enrolled in TANF and individuals likely eligible but not receiving TANF, during the study period, are not designed to be representative of or generalizable to all individuals receiving TANF in the selected state or in other states, but are intended to capture variation and commonalities in study participants’ experiences. As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.
B2. Methods and Design
Target Population
To achieve the study’s objectives, we will conduct qualitative research with staff who administer TANF at the state level and at three local sites within the state. At the state level, the target population includes the senior leadership of the state’s TANF program (those leaders responsible for making TANF policy and programmatic decisions) and their core staff. For each of the three sites, the target population includes regional TANF administrators and staff, staff from community-based organizations that serve TANF-eligible individuals, staff from programs partnering with the state in the provision of tablet computers to TANF program participants or other virtual services, individuals enrolled in TANF and participating in the tablet program, and individuals likely eligible but not receiving TANF.
The research team will use nonprobability, purposive sampling to select the three sites and to identify potential respondents who can provide information on the study’s key constructs. Given that participants will be purposively selected, they will not be representative of the population of TANF administrators, staff, community organizations, or individuals enrolled in/likely eligible for TANF.
In recruiting potential respondents, we aim to obtain variation in individuals’ experiences to understand how the policy and process changes to the TANF program due to the COVID-19 pandemic were understood and experienced within a given site.
Site Selection
State selection
The project team will select one state that is willing to participate and that meets study selection criteria including: implementation of virtual services and other practice and policy changes in response to COVID-19 (e.g., providing virtual case management and job search activities, and other resources such as electronic tablets), and has a racially and ethnically diverse TANF population.
Site selection
We will identify the three sites for the study in consultation with the study state, selecting areas that allow the study to explore how pandemic-related practice and policy changes may be experienced and implemented in different contexts and by different racial and ethnic populations.
Respondent Recruitment
State and regional TANF administrators
Recruitment of state and regional TANF administrators will be purposeful. The project team will aim to talk with state TANF administrators knowledgeable about TANF policies and procedures and any changes at the state level in response to COVID-19. The project team will rely on the insight of the state-level administrators interviewed to identify regional TANF administrators to interview.
Local TANF frontline staff
Recruitment of local TANF staff will be purposeful. The project team will aim to talk with local TANF frontline staff that work directly with individuals enrolled in TANF and administer the program to them at each of the three sites. These staff may vary in their roles and responsibilities, and we will aim to recruit a group of staff with diverse perspectives on service provision and who work with participants in a variety of capacities. The project team will rely on the insights of regional and state-level administrators to identify the most suitable categories of staff to recruit for interviews and focus groups.
Community-based organizations that serve TANF-eligible individuals
Recruitment of community-based organizations will be purposeful. We will work with TANF state administrators to identify relevant community organizations that are pivotal in assisting families in applying for TANF and other benefits, or otherwise work with families experiencing poverty who may be eligible for TANF. We will also work with TANF administrators to identify program partners. We will recruit staff from these relevant organizations to participate in interviews. These staff will vary in their roles and have diverse perspectives, as a result. We will rely on the program leaders and their partners to identify which staff are most suitable to participate in the study.
Tablet providers and program partners
Recruitment of staff from programs partnering with the state in the provision of tablet computers and other virtual services will be purposeful. We will work with TANF state administrators to identify relevant program partners in the provision of tablet computers and other virtual services. We will recruit staff from these program partners to participate in interviews. These staff will vary in their roles and, as a result, their perspectives on service provision. We will rely on the program leaders to identify which staff are most suitable to participate in the study, such as those who have been working directly in administering the tablets to TANF participants and/or who have knowledge of how that administration has been implemented.
TANF program participants
Recruitment of individuals enrolled in TANF will be purposeful. The project team seeks variation in the individuals that are enrolled in TANF particularly by race and ethnicity and will recruit through TANF staff and community-based organizations.
Tablet program participants
Recruitment of individuals enrolled in TANF and receiving tablets will be purposeful. The project team seeks variation in the individuals enrolled in TANF and receiving tablets particularly by race and ethnicity. The project team will recruit through TANF staff or tablet providers.
Individuals likely eligible but not receiving TANF
Recruitment of individuals likely eligible but not receiving TANF will be purposeful. The project team will work with community-based organizations and program partners to identify individuals who may be eligible for TANF but are not receiving it and are willing to participate in an interview or focus group as part of the study. The project team will seek variation in the individuals recruited particularly by race and ethnicity
B3. Design of Data Collection Instruments
Development of Data Collection Instruments
We developed 7 data collection protocols for this study. Table A.1 in Supporting Statement A provides details about the respondent, content, purpose, mode, and duration for each protocol. The data collection protocols include a semi-structured interview protocol for state and regional TANF administrators, a semi-structure interview/focus group guide for TANF local frontline staff, an interview guide for staff at community-based organizations, an interview guide for tablet providers and program partners, an interview/focus group guide for TANF program participants, a focus group guide for tablet program participants, and an interview/focus group guide for individuals likely eligible but not receiving TANF.
All questions in the protocols are new because they reflect a new area of research and the constructs under study cannot be measured using existing instruments. We pretested the topic areas of the research instruments with fewer than 10 people to ensure that topic areas are relevant to different respondent categories.
This effort fills a gap in the knowledge base about potential differences in the ways in which different racial and ethnic populations have experienced changes to TANF policies and processes resulting from the COVID-19 pandemic. The data collection instruments address the key research questions included in Supporting Statement A. Within each semi-structured interview and focus group instrument, the questions align with the key constructs relevant to the research questions. Respondents will only answer the questions that align with their own areas of experience and knowledge.
See tables B1 - B7 below for a description of how each data collection instrument aligns with the research questions.
Table B1. Interview Guide for State and Regional TANF Administrators (Instrument A)
Protocol section |
Relevant research question(s) |
|
NA |
|
(#1) How was the TANF program affected by the COVID-19 pandemic – what policies and practices were modified and how? Do the modifications represent temporary or permanent changes, and if temporary, are they currently in effect and/or when did or will they end?
(#2a) For policy and practice changes to service delivery made in response to COVID-19: How was the policy implemented before COVID-19? How was the policy modified in response to COVID-19? What is the current policy? |
|
(#1) How was the TANF program affected by the COVID-19 pandemic – what policies and practices were modified and how? Do the modifications represent temporary or permanent changes, and if temporary, are they currently in effect and/or when did or will they end?
(#2b) For policy and practice changes to service delivery made in response to COVID-19: How is the policy/practice implemented across the state (e.g., who determines how the policy is implemented, who carries out the policy, how is the policy applied to different participants, how is the policy/practice described and understood by applicants or participants)?
(#2ci) How is implementation similar and how is it different in the three selected locations in the state? How do TANF state-level and local-level administrators and TANF staff explain, define, and implement the policy/practice? What are similarities and differences in their accounts?
(#2cii) How do TANF state-level and local-level administrators and TANF staff describe the period of modification in policy: how did the change come about, how was it implemented, what were implementation challenges, how were challenges addressed? What were implementation successes (how do they define success, and what made it successful)? |
|
(#1) How was the TANF program affected by the COVID-19 pandemic – what policies and practices were modified and how? Do the modifications represent temporary or permanent changes, and if temporary, are they currently in effect and/or when will they end?
(#2b) For policy and practice changes to service delivery made in response to COVID-19: How is the policy/practice implemented across the state (e.g., who determines how the policy is implemented, who carries out the policy, how is the policy applied to different participants, how is the policy/practice described and understood by applicants or participants)?
(#2ci) How is implementation similar and how is it different in the three selected locations in the state? How do TANF state-level and local-level administrators and TANF staff explain, define, and implement the policy/practice? What are similarities and differences in their accounts?
(#2cii) How do TANF state-level and local-level administrators and TANF staff describe the period of modification in policy: how did the change come about, how was it implemented, what were implementation challenges, how were challenges addressed? What were implementation successes (how do they define success, and what made it successful)? |
|
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful?
(#3c) How do TANF clients, and individuals who are eligible, but not enrolled, of different races and ethnicities describe the policy/practice and its effects? |
|
NA |
Table B2. Interview/Focus Group Guide for Local TANF Frontline Staff (Instrument B)
Protocol section |
Relevant research question(s) |
|
NA |
|
(#1) How was the TANF program affected by the COVID-19 pandemic – what policies and practices were modified and how? Do the modifications represent temporary or permanent changes, and if temporary, are they currently in effect and/or when did or will they end?
(#2a) For policy and practice changes to service delivery made in response to COVID-19: How was the policy implemented before COVID-19? How was the policy modified in response to COVID-19? What is the current policy? |
|
(#1) How was the TANF program affected by the COVID-19 pandemic – what policies and practices were modified and how? Do the modifications represent temporary or permanent changes, and if temporary, are they currently in effect and/or when did or will they end?
(#2b) For policy and practice changes to service delivery made in response to COVID-19: How is the policy/practice implemented across the state (e.g., who determines how the policy is implemented, who carries out the policy, how is the policy applied to different participants, how is the policy/practice described and understood by applicants or participants)?
(#2ci) How is implementation similar and how is it different in the three selected locations in the state? How do TANF state-level and local-level administrators and TANF staff explain, define, and implement the policy/practice? What are similarities and differences in their accounts?
(#2cii) How do TANF state-level and local-level administrators and TANF staff describe the period of modification in policy: how did the change come about, how was it implemented, what were implementation challenges, how were challenges addressed? What were implementation successes (how do they define success, and what made it successful)?
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)?
|
|
(#1) How was the TANF program affected by the COVID-19 pandemic – what policies and practices were modified and how? Do the modifications represent temporary or permanent changes, and if temporary, are they currently in effect and/or when will they end?
(#2b) For policy and practice changes to service delivery made in response to COVID-19: How is the policy/practice implemented across the state (e.g., who determines how the policy is implemented, who carries out the policy, how is the policy applied to different participants, how is the policy/practice described and understood by applicants or participants)?
(#2ci) How is implementation similar and how is it different in the three selected locations in the state? How do TANF state-level and local-level administrators and TANF staff explain, define, and implement the policy/practice? What are similarities and differences in their accounts?
(#2cii) How do TANF state-level and local-level administrators and TANF staff describe the period of modification in policy: how did the change come about, how was it implemented, what were implementation challenges, how were challenges addressed? What were implementation successes (how do they define success, and what made it successful)? |
|
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful?
(#3c) How do TANF clients, and individuals who are eligible, but not enrolled, of different races and ethnicities describe the policy/practice and its effects? |
|
NA |
Table B3. Interview Guide for Community-based Organizations (Instrument C)
Protocol section |
Relevant research question(s) |
|
NA |
|
(#1) How was the TANF program affected by the COVID-19 pandemic – what policies and practices were modified and how? Do the modifications represent temporary or permanent changes, and if temporary, are they currently in effect and/or when will they end?
(#2a) For policy and practice changes to service delivery made in response to COVID-19: How was the policy implemented before COVID-19? How was the policy modified in response to COVID-19? What is the current policy?
(#2b) How is the policy/practice implemented across the state (e.g., who determines how the policy is implemented, who carries out the policy, how is the policy applied to different participants, how is the policy/practice described and understood by applicants or participants)?
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)? |
|
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful?
(#3c) How do TANF clients, and individuals who are eligible, but not enrolled, of different races and ethnicities describe the policy/practice and its effects? |
|
NA |
Table B4. Interview Guide for Tablet Providers and Program Partners (Instrument D)
Protocol section |
Relevant research question(s) |
|
NA |
|
(#1) How was the TANF program affected by the COVID-19 pandemic – what policies and practices were modified and how? Do the modifications represent temporary or permanent changes, and if temporary, are they currently in effect and/or when will they end?
(#2a) For policy and practice changes to service delivery made in response to COVID-19: How was the policy implemented before COVID-19? How was the policy modified in response to COVID-19? What is the current policy?
(#2b) How is the policy/practice implemented across the state (e.g., who determines how the policy is implemented, who carries out the policy, how is the policy applied to different participants, how is the policy/practice described and understood by applicants or participants)?
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)? |
|
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful? |
|
NA |
Table B5. Interview/Focus Group Guide for TANF Program Participants (Instrument E)
Protocol section |
Relevant research question(s) |
|
NA |
|
(#2c) How is implementation similar and how is it different in the three selected locations in the state?
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)?
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful?
(#3c) How do TANF clients, and individuals who are eligible, but not enrolled, of different races and ethnicities describe the policy/practice and its effects? |
|
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)?
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful?
(#3c) How do TANF clients, and individuals who are eligible, but not enrolled, of different races and ethnicities describe the policy/practice and its effects? |
|
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)?
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful?
(#3c) How do TANF clients, and individuals who are eligible, but not enrolled, of different races and ethnicities describe the policy/practice and its effects? |
|
NA |
Table B6. Focus Group Guide for Tablet Program Participants (Instrument F)
Protocol section |
Relevant research question(s) |
|
NA |
|
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)?
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful?
(#3c) How do TANF clients, and individuals who are eligible, but not enrolled, of different races and ethnicities describe the policy/practice and its effects? |
|
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)?
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful?
(#3c) How do TANF clients, and individuals who are eligible, but not enrolled, of different races and ethnicities describe the policy/practice and its effects? |
|
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)?
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful?
(#3c) How do TANF clients, and individuals who are eligible, but not enrolled, of different races and ethnicities describe the policy/practice and its effects? |
|
Table B7. Interview/Focus Group Guide for Individuals Likely Eligible but Not Receiving TANF (Instrument G)
Protocol section |
Relevant research question(s) |
|
NA |
|
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)? |
|
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)?
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful?
(#3c) How do TANF clients, and individuals who are eligible, but not enrolled, of different races and ethnicities describe the policy/practice and its effects? |
|
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)?
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful?
(#3c) How do TANF clients, and individuals who are eligible, but not enrolled, of different races and ethnicities describe the policy/practice and its effects? |
|
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)?
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful?
(#3c) How do TANF clients, and individuals who are eligible, but not enrolled, of different races and ethnicities describe the policy/practice and its effects? |
|
(#2ciii) How do/did TANF applicants, and those who are eligible, but not enrolled, view the policy (i.e., the policy change whether still required or not)?
(#3) How do TANF clients, and those who are eligible, but not enrolled, experience the policy or practice (e.g., virtual services, distribution of devices to access remote services, online options for returning completed paperwork, job search requirements)?
(#3a) Are there aspects of the policy or practice they find challenging?
(#3b) Are there aspects of the policy or practice they find helpful?
(#3c) How do TANF clients, and individuals who are eligible, but not enrolled, of different races and ethnicities describe the policy/practice and its effects? |
|
NA |
B4. Collection of Data and Quality Control
Site teams will recruit and conduct all interviews at the state-level and at a particular site.
Training
To ensure we collect high quality data, we will train site teams to conduct the recruiting and interviewing activities. We will train the site teams in seven topics:
State and Regional TANF administrator interviews to prepare teams to implement the semi-structured interview guides with state and regional TANF administrators (Instrument A).
Local TANF frontline staff interviews/focus groups to prepare teams to implement the interview/focus group guide with local staff (Instrument B).
Community based organizations and program staff semi-structured interviews to prepare teams to implement the guides with staff at community organizations (Instrument C).
Tablet providers and program partners interviews to prepare teams to implement the guides with staff at tablet providers and program partner organizations (Instrument D).
TANF program participants interviews/focus groups to prepare teams to implement the semi-structured interview/focus group guide with TANF-enrolled individuals (Instrument E).
Tablet program participants focus group to prepare teams to implement the focus group guide with tablet program participants (Instrument F).
Individuals likely eligible but not receiving TANF interview/focus groups to prepare teams to implement the interview/focus group guide with individuals likely eligible for but not receiving TANF (Instrument G).
The Urban Institute will be collecting all necessary data. Table A1 in Supporting Statement A summarizes the data collection that will be conducted for the study.
Recruitment
Recruiting state and regional TANF administrators and frontline staff: The project team will seek recommendations from state level administrators for regional administrators to interviews based on their knowledge of TANF and virtual services and policy changes during the pandemic (Appendix B). The team will also seek recommendations from administrators and leadership at the regional and site level to identify frontline staff who work directly on these services and policy changes (Appendix C). The team will follow up with emails or phone calls to answer any questions respondents may have. If an administrator or staff member agrees to participate, the project team will follow up with an email that includes logistical details of the site visit and proposed interview/focus group times.
Recruiting staff from tablet providers and program partners: The project team will rely on state and regional TANF administrators to refer us to relevant program partners, like organizations partnering with the state in the provision of tablet computers to TANF program participants (Appendix B). We will work with program partner leadership to identify staff to interview, focusing on staff with knowledge of TANF policy changes and virtual services implementation. If a program partner agrees to participate, the project team will follow up with an email that includes logistical details of the site visit and proposed interview times.
Recruiting staff from community-based organizations: The project team will rely on state and regional TANF administrators to refer us to relevant community-based organizations serving individuals likely eligible for TANF (Appendix B). We will work with leadership at community-based organizations to determine which staff to speak with based on their knowledge of TANF and experience with TANF changes during the COVID-19 pandemic. If a community-based organization agrees to participate, the project team will follow up with an email that includes logistical details of the site visit and proposed interview times.
Recruiting TANF program participants and tablet program participants: To ensure maximum participation, the project team will work with local TANF staff to recruit program participants. Depending on the preferences of local staff, the project team will either ask local TANF staff to recruit individuals to participate in focus groups and interviews by signing them up for set times for in-person or virtual data collection (Appendix D) or ask local TANF staff to share information (Appendix B) about the study with a number for participants to call for more information. Alternatively, local TANF staff could ask individuals if they can share their information with the project team and our team will reach out to participants directly (Appendix E). The team seeks to make recruitment as straightforward for TANF recipients as possible, and as such is prepared for several methods of recruitment depending on the site.
Recruiting individuals likely eligible but not receiving TANF: The project team will rely on local TANF staff and staff community-based organizations to identify and recruit individuals who are likely eligible but not receiving TANF. To ensure maximum participation, we will either recruit and sign up individuals for interviews/focus groups directly through local staff (Appendix D), share information about the study (Appendix B) with individuals directly along with the study’s number, or have local staff ask individuals if they can share their information with the project team and our team will reach out to participants directly (Appendix E).
Collecting data
A site team will conduct in-person or virtual site visits with the three sites and conduct semi-structured interviews and focus groups to answer the study’s questions. We will be prepared to conduct data collection both in-person and via telephone and video conferencing.
For all interviews and focus groups, one member of the team will conduct the interview, and the other member will take notes. With the permission of respondents, we will also audio record the interviews for transcription. The digital recordings will be transcribed using the services of a qualified transcription service that uses a secure server to store their recordings and transcriptions. The study site team will confer after each interview and focus group (using recordings as needed) to ensure completeness of data. Throughout the data collection period, the whole site study team will conduct regular meetings to report on and exchange information and strategies and help troubleshoot challenges.
Details about each information collection follow:
We will conduct a 60-minute semi-structured interview with up to 8 state and regional TANF administrators.
We will conduct a 60-minute semi-structured interview with up to 10 local frontline TANF staff.
We will conduct a 90-minute focus group with up to 10 local frontline TANF staff.
We will conduct a 60-minute semi-structured interview with up to 6 staff at community-based organizations.
We will conduct a 60-minute semi-structured interview with up to 6 staff at tablet providers and program partners. We will conduct a 60-minute semi-structured interview with up to 40 TANF program participants.
We will conduct a 90-minute focus group with up to 20 TANF program participants.
We will conduct a 90-minute focus group with up to 10 tablet program participants.
We will conduct a 60-minute semi-structured interview with up to 15 individuals likely eligible but not receiving TANF.
We will conduct a 90-minute focus group with up to 15 individuals likely eligible but not receiving TANF.
B5. Response Rates and Potential Nonresponse Bias
Response Rates
The interviews and focus groups are not designed to produce statistically generalizable findings and participation is wholly at the respondent’s discretion. Response rates will not be calculated or reported.
Non-Response
As study respondents will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated. Respondent demographics will be documented and reported in written materials associated with the data collection. Any substantial nonresponse from staff will be documented and reported as a study limitation.
B6. Production of Estimates and Projections
The data will not be used to generate population estimates, either for internal use or dissemination.
B7. Data Handling and Analysis
Data Handling
The digital recordings will be transcribed using the services of a qualified transcription service that uses a secure server to store their recordings and transcriptions. In cases when participants did not consent to being recorded, the project team will clean the typed notes taken during the interview or focus group.
Data Analysis
The study will employ qualitative methods to analyze all interview and focus group data. The project team will follow a deductive coding process, beginning with development of a coding scheme based on key constructs drawn from study research questions. The coding scheme will be further developed by reviewing transcripts and interview notes in conjunction with the instruments and the research questions to identify key themes and topic areas that arise through different interviews. Project team members who participated in qualitative data collection will review the coding scheme to ensure that important points are not missed.
Transcripts of the interviews and focus groups will be uploaded into qualitative data analysis software. Project team members will test the coding scheme by coding one or two interviews each, before running a query to examine coder reliability. The coding team will then meet to review any questions and divergent codes. The coding team will continue coding sections of an interview until they have reached an interrater reliability of a kappa coefficient of .80 or greater (considered a high level of agreement between rates) (McHugh, 2012).
Data Use
The project team will use the collected data to inform a technical report that will describe the study design and findings. Dissemination of findings may include a report, research brief, and presentations or briefings. The report will include a detailed study methodology that will help the public understand and properly interpret the information derived from the data collection. The methodology section will include, but not be limited to, interview and focus group discussion topics, qualitative data analysis technique, and administrative data analysis techniques. As discussed in section B1, the study’s limitations will be included in all written products associated with the study. This research will enhance the understanding of how families of different racial and ethnic backgrounds experience changes in TANF policies and practices resulting from the COVID-19 pandemic.
B8. Contact Persons
The information for this study is being collected by the Urban Institute on behalf of ACF. Principal Investigator Marla McDaniel (mmcdaniel@urban.org) and Linda Giannarelli (lgiannarelli@urban.org) led development of the study design plan and data collection protocols and will oversee collection and analysis of data gathered through interviews and focus groups. The project officer is Kim Clum (Kimberly.Clum@acf.hhs.gov) of the Office of Planning, Research and Evaluation in the Administration for Children and Families.
Attachments
Appendices
Appendix A: Study Information for TANF Program Participants, Tablet Program Participants, and Individuals Likely Eligible but Not Receiving TANF
Appendix B: Recruitment Email from Research Team to State and Regional TANF Administrators, Local TANF Frontline Staff, Community-based Organizations, and Tablet Providers and Program Partners
Appendix C: Recruitment Email and Script from Research Team to Recruiters of TANF Staff
Appendix D: Recruitment Email and Script from Research Team to Recruiters of TANF Program Participants, Tablet Program Participants and Individuals Likely Eligible but Not Receiving TANF
Appendix E: Recruitment Email from Research Team to TANF Program Participants, Tablet Program Participants and Individuals Likely Eligible but Not Receiving TANF
Attachments (Instruments)
Instrument A: Interview Guide for State and Regional TANF Administrators
Instrument B: Interview/Focus Group Guide for Local TANF Frontline Staff
Instrument C: Interview Guide for Community-based Organizations
Instrument D: Interview Guide for Tablet Providers and Program Partners
Instrument E: Interview/Focus Group Guide for TANF Program Participants
Instrument F: Focus Group Guide for Tablet Program Participants
Instrument G: Interview/Focus Group Guide for Individuals Likely Eligible but Not Receiving TANF
References
McHugh M. L. (2012). Interrater reliability: the kappa statistic. Biochemia medica, 22(3), 276–282.
Creswell, J. W. (2007). Qualitative inquiry & research design: Choosing among five approaches. Thousand Oaks, CA: Sage Publications.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Clum, Kimberly (ACF) |
| File Modified | 0000-00-00 |
| File Created | 2023-08-27 |