Supporting Statement A - ACF Rural Context_012921_V3_clean

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Human Services Programs in Rural Contexts Study

OMB Information Collection Request

New Collection

Supporting Statement

Part A

February 2021

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, DC 20201

Project Officers:

Aleta Meyer, PhD

Lisa Zingman, MSPH

Part A

Executive Summary

• Type of Request: This Information Collection Request is for a new request. The request is for 2 years of approval.

• Description of Request: The Administration for Children and Families (ACF) Office of Planning, Research, and Evaluation (OPRE) at the U.S. Department of Health and Human Services (HHS) intends to collect data to capture the challenges and unique opportunities of administering human services programs in rural contexts. This descriptive study will collect data from 12 case study communities (sites) to provide ACF with a rich description of human services programs in rural contexts, as well as reveal opportunities for strengthening the capacity of human services programs to promote the economic and social wellbeing of individuals, families, and communities in rural contexts. The data collected in this study are not intended to be generalized to a broader audience. The study team do not intend for this information to be used as the principal basis for public policy decisions.

A1. Necessity for Collection

Though significant evaluation work has been carried out to improve general understanding of how human services programs improve family economic self-sufficiency, financial security, and overall wellbeing, knowledge is still lacking in regard to how these programs can best serve rural communities. While a lot of human services programs are implemented in rural contexts, our research is generally done in more populated areas. Rural contexts present unique opportunities and challenges for administering human service programs. Opportunities include access to natural resources (as both amenities and commodities), a commitment to innovation and adaptation, and a high degree of community social capital, among others. Meanwhile, challenges include providing access to economic opportunities, transportation, technology, and services; higher rates of poverty and unemployment than other areas of the country; greater distances to services; and negative cultural perceptions of public assistance. This study aims to (1) provide a rich description of human services programs in rural contexts; (2) determine the level of unmet need for human services in rural communities; and (3) identify opportunities for strengthening the capacity of human services programs to promote the economic and social wellbeing of individuals, families, and communities in rural contexts. These goals will be accomplished through a mixed-methods study that includes a literature review, secondary analysis of administrative data, and 12 site visits to rural communities (the subject of this request) to generate insights from program administrators, program staff, and staff from nonprofit or partner organizations.

There are no legal or administrative requirements that necessitate this collection. ACF is undertaking the collection at its own discretion.

A2. Purpose

Purpose and Use

The Human Services Programs in Rural Context study will investigate what challenges and opportunities exist for human service programs in rural contexts. Specifically, the study will examine several programs, including Temporary Assistance for Needy Families (TANF); Healthy Marriage and Responsible Fatherhood (HMRF); Health Profession Opportunity Grants (HPOG); and Maternal, Infant, and Early Childhood Home Visiting (MIECHV), as well early childhood development programs, family development programs, employment programs, and higher education and technical training programs funded in the community.

The goals of this study are to: develop a rich description of human services programs in rural contexts, devise a method to determine unmet need for human services programs in rural communities, and identify opportunities for strengthening the capacity of human services programs to promote the economic and social wellbeing of individuals, families, and communities in rural contexts.

This information will be used for the following purposes:

• Supplement and expand on existing research on rural communities

• Increase understanding of how human services programs operate in rural communities

• Understand how rural communities have adopted innovative strategies for addressing needs

• Determine how strategies may be replicated in other areas as a solution for expanding rural access to human services programs

• Identify opportunities for strengthening capacity to improve access to human service programs in rural communities, and to improve outcomes for families served

The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker and is not expected to meet the threshold of influential or highly influential scientific information.

Research Questions

These site visits will address 19 research questions:

1. What are the social wellbeing needs of the populations served by human services programs?

2. What are the economic wellbeing needs of the population served by human services programs?

3. What “standard” service delivery models are used by human services programs to deliver the programs in rural contexts?

4. How are human services programs in rural contexts adapting service delivery?

5. What are the key barriers and facilitators of the delivery of the four human services programs in rural contexts?

6. What is the organizational capacity of human services programs to deliver services in rural contexts?

7. In what ways do key local and “state” factors impact the capacity of human services programs to provide services in rural contexts?

8. What key federal factors impact the capacity of human services programs to provide services in rural contexts?

9. What components of organizational capacity are identified by staff from human services programs, local nonprofit organizations, or partner organizations as most critical to the delivery of human services programs in rural contexts?

10. What particular components of organizational capacity are identified by staff from human services programs as most critical to addressing unmet need in rural contexts?

11. In what ways does organizational capacity in rural contexts impact fidelity of implementation for human services programs?

12. What technical assistance (e.g., recruitment/intake, case management, curricula, data use, “other”) are human services programs in rural contexts currently receiving?

13. How are human services programs in rural contexts using data (e.g., performance measures, evaluation data) to improve their capacity to deliver services?

14. Which combinations of human services programs are found among rural counties with lower and higher levels of unmet need?

15. What are the key lessons learned about the delivery of human services programs in rural counties with significantly low concentrations (i.e., “cold spots”) of unmet need?

16. What are the most significant challenges facing the delivery of human services programs in rural counties with significantly high concentrations (i.e., “hot spots”) of unmet need?

17. What do human services staff in rural contexts perceive to be the greatest needs of the populations served by human services programs?

18. What do staff from human services programs in rural contexts perceive as the factors influencing the unmet needs of the populations served and unserved?

19. In what ways are nonprofits not funded through federally funded grants working to address unmet needs in rural contexts?

Study Design

The overall goals of this project are related but explore three unique and important aspects of the delivery of human services programs. The exploration of these three aspects will, ultimately, result in a better collective understanding of how human services programs are adapted and delivered in rural contexts and will reveal potential opportunities for enhancing service delivery.

Based on the study’s three specified goals, the study team determined that a mixed-methods design is best suited for this project (see Supporting Statement B1. “Appropriateness of Study Design and Methods for Planned Uses” for additional information). A key aspect of this design is that it will leverage the respective benefits of qualitative and quantitative data collection and analysis separately before combining and synthesizing the findings to better address the study’s three goals. To reduce the burden on respondents, this study will utilize the results of an extensive literature review (that has been completed) and analyze relevant secondary data to examine client populations. A geographic information system (GIS) analysis will generate maps depicting the distribution of human services funds by rural location and empirically identify “hot spots” of unmet need for human services in rural communities. The study team used stakeholder engagement and consulted with a technical work group (TWG) comprising human services practice experts, experts in rural contexts and research methods, and ACF program staff to develop discussion guides to focus on HMRF, TANF, HPOG, and MIECHV programs, as well early childhood development programs, family development programs, employment programs, and higher education and technical training programs being funded in the community. The study will use 12 site visits to rural communities to generate insights from program administrators, program staff, and current and potential program participants. The study team will then employ Qualitative Comparative Analysis (QCA) to conduct a systematic interpretation and explanation of the study’s quantitative and qualitative findings.

The study team will conduct site visits in 12 rural communities. In each community, we will speak with a maximum of 20 individuals at each site:

• 1 program director (who will also act as our liaison for coordinating the site visit)

• 4 other local human service program leaders (HMRF, TANF, HPOG, MIECHV, and other human services program staff [e.g., early childhood development, family development, employment, and higher education and technical training programs])

• Up to 9 staff from local human service programs (HMRF, TANF, HPOG, MIECHV, and other human services program staff [e.g., early childhood development, family development, employment, and higher education and technical training programs])

• Up to 6 staff from local nonprofit or partner organizations that support individuals that utilize human services

The study team will only visit each site one time. If the current COVID-19 pandemic makes it too difficult to travel safely, the study team will conduct these interviews virtually using the computer (if the respondents have access to internet capabilities) or by telephone (if respondents do not have access to internet capabilities).

Table 1. Data Collection Activity, Instrument, Respondent, Mode, and Duration

Data Collection Activity	Instrument(s)	Respondent, Content, Purpose of Collection	Mode and Duration
Site Visit Planning	In-Person Site Visit Planning Template (Instrument 1a); or Virtual Site Visit Planning Template (Instrument 1b)	Respondents: Program director or designee Content: 1a: Scheduling tool to slot interviewers at specific times and locations 1b: Scheduling tool to plan for virtual interviews at specific times using computers or telephone Purpose: To schedule interviews	Mode: Electronic Duration: 2 hours
Interviews of Program Directors and Leaders	Program Directors and Leaders Site Visit Discussion Guide (Instrument 2)	Respondents: Program directors and leadership from human service programs Content: Interview questions and probes to capture respondent perceptions of program planning and execution Purpose: To document leaders’ perspectives about challenges and opportunities of human services in rural contexts	Mode: Interview Duration: 2 hours
Interviews of Program Staff	Staff Site Visit Discussion Guide (Instrument 3)	Respondents: Program staff from human services programs Content: Interview questions and probes to capture respondent perceptions of program planning and execution Purpose: To document staff perspectives about challenges and opportunities of providing human services in rural contexts	Mode: Interview Duration: 1.5 hours
Interviews of Staff from Nonprofit or Partner Organizations	Nonprofit or Partner Organizations Site Visit Discussion Guide (Instrument 4)	Respondents: Staff from nonprofit or partner organizations Content: Interview questions and probes to capture respondent perceptions of program planning and execution Purpose: To document staff perspectives about challenges and opportunities of providing human services in rural contexts	Mode: Interview Duration: 1 hour

This study and its goals have been designed to yield information that will improve understanding of the facilitators and barriers for implementing human services in rural contexts. There are limitations, including:

• Human services are a broad field with lots of variability, and the findings will not be generalizable across all human services programs.

• This study will only address a subset of issues regarding how social determinants of health impact the implementation of human services (Office of Disease Prevention and Health Promotion, 2020). Additional factors not examined could likely impact how human services programs are implemented in rural contexts and whether the needs of people residing in those contexts are met.

• The study team will be using a convenience sample and the findings may not represent the full view of human services implementation in rural areas.

Limitations will be acknowledged, as appropriate, and will be noted in materials that result from the study.

Other Data Sources and Uses of Information

For this information collection, the study team have attempted to minimize the burden of data collection by maximizing the use of administrative data and existing literature. These other data sources will supplement the findings of the site visit data collection and will allow for the different types of data analysis described in Supporting Statement B7.

A3. Use of Information Technology to Reduce Burden

The study team will complete planning activities for the site visits electronically and by telephone. If the current COVID-19 pandemic makes it too difficult to travel safely, the study team will conduct these interviews virtually using the computer (if the respondents have access to internet capabilities) or by telephone (if respondents do not have access to internet capabilities. Whether completed in-person or virtually, the study team will audio record the interviews (with the consent of the respondent) to ensure we accurately capture the conversation. The study team will not ask respondents to write anything or fill anything out other than Instrument 1a or 1b by the program director.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

To reduce the burden on the respondents, the study team will maximize the use of existing secondary data, including data and information obtained from a comprehensive literature review and existing survey and administrative data sources. The existing administrative and survey data and information from the literature review was used to refine the study’s research questions and reduce the amount of information sought in the interviews. The information the study team will obtain from the interviews is not collected anywhere else.

A5. Impact on Small Businesses

No small businesses will be involved in this information collection.

A6. Consequences of Less Frequent Collection

This is a one-time data collection.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing their intention to request an OMB review of this information collection activity. This notice was published on November 30, 2020, Volume 85, Number 230, page 76579-76580; and provided a sixty-day period for public comment. During the notice and comment period, 4 sets of comments were received, which are included below and attached in Appendix Appendix A: Responses to Federal Register Notice Comments.

• Rebecca Schueller Training & Consulting, LLC

• Comment: I would like to encourage you to include human services programs located in both metro-adjacent and non-metro adjacent counties in this study.  Rural programs in non-metro adjacent counties face significant challenges not always captured by the experience of those programs located in metro-adjacent areas.  Transportation is just one example of the many challenges.  I would also like to see a recommendation that funding take this into account.  It is costly to support services over a wide geographical area, and yet most federal programs do not support full costs for organization, including flexible funding that allows for multiple types of transportation needed, including gas vouchers, bus tokens, cab and uber vouchers, paying for agency vehicles and paying for staff mileage reimbursement.  This flexibility is essential to help vulnerable and low-income rural clients access services in rural areas where there is not a strong transportation infrastructure, and particularly for nightshifts and weekend work.  

  • ACF Response: The study team recognizes the importance of looking at a diverse set of rural communities within this study and has included both metro-adjacent and non-metro adjacent counties in our study using data from the U.S. Department of Agriculture’s Rural-Urban Continuum Codes. The study team is also using American Community Survey (ACS) data on the percentage of households in a county without a vehicle, which will allow the study team to examine the transportation challenges that residents within rural counties are likely to encounter.

• Comment: It would also help if DHHS would influence Federal Dept. of Transportation to change the way public transportation formula funding is made available to support flexible use.

  • ACF Response: This recommendation is outside the scope of the project.

• Hogg Foundation for Mental Health

  • Comment: In your methodology, we question whether an “N of 1” (In-person site visit planning template) for each of the 12 sites proposed is robust enough to give you a picture of community conditions if you only have one person you are relying on to provide you a comprehensive picture of the community. It seems that you would want to survey/focus group/town hall many participants, including historically excluded community members to provide you the input you seek. So, our opinion is that surveying one person limits the true picture of the community you are wanting to learn more about.

    • ACF Response: The site visit planning templates are used to organize the various interviews that the study team will conduct during the site visits, which will likely be virtual due to the ongoing COVID-19 pandemic. During these site visits, the study team is hoping to interview up to 20 members from each rural community, with a preference for interviewing 1 program director/main point of contact, 5 other program directors from human services programs, 9 staff members from human services organizations in the community, and 6 staff from nonprofit or partner organizations. Unfortunately, focus groups and town hall meetings, will not be possible due to COVID-19 restrictions.

  • Comment: In light of COVID-19, having alternate ways versus just the in-person process to collect data (virtual platforms such as zoom, web-ex, etc.) or conference calls will be important so that community members feel safe during this engagement process.

    • ACF Response: The study team will likely be conducting these site visits virtually using the computer (if the respondents have access to internet capabilities) or by telephone (if respondents do not have access to internet capabilities).

  • Comment: We recommend that you include a hyperlink to the agency making the request. For example, in order to make a comment/opinion on (a) …whether the proposed collection of information is necessary for the proper performance of the agency's functions… one will need to review your mission/vision/purpose. This is not posted on the Federal Registry posting. Including a hyperlink would be most helpful to persons reviewing the posting. EX: whether the proposed collection of information is necessary for the proper performance of the agency's functions…

    • ACF Response: Thank you for your comment. The Office of Planning, Research, and Evaluation will consider updating its template to incorporate this feedback.

  • Comment: In order to receive the input, you seek, we recommend you include hyperlinks to the actual survey instruments you proport to use. Without access to the instruments, it is impossible to make a valued comment/opinion on the estimated time to validate or refute the time it will take to complete this proposed process.

    • ACF Response: Per the Notice, you can request the instruments by emailing OPREInfoCollection@ACF.hhs.gov and asking for the instruments to be emailed to you. As these are draft documents, they are not posted publicly at this time. In response to this comment, the ACF PRA Lead sent copies of these documents to the requester on January 15, 2021.

• California Department of Social Services

  • Comment: This is a request for copies of the proposed collection of information for the Human Services Programs in Rural Contexts Study.

    • ACF Response: The ACF PRA Lead sent copies of these documents to the requestor on December 18, 2020.

• Human Rights Campaign (HRC)

  • Comment: HRC urges ACF to design study methodologies that gather data on the unique challenges facing LGBTQ people and potential solutions in this study as well as future ones.

    • ACF Response: Thank you for the comment. While the study team is not talking directly with program participants during the site visits, inequities will be an area of focus in the interviews with program directors, staff members from human services organizations in the community, and staff from nonprofit or partner organizations. The study’s semi-structured interview instruments include questions about specific needs and gaps within communities, as well as unique characteristics about a site, which will provide the study team with an opportunity to explore challenges facing the LGBTQ community and other communities within rural contexts.

Consultation with Experts Outside of the Study

In June 2020, the project held a TWG meeting in which the study team received feedback on the research questions and site visit criteria, and offered suggestions for sites to be included in the study from our subject matter experts and human service program stakeholders:

• Dr. Carolyn Yvette Barnes

• Ms. Rebecca Fausett

• Dr. Cynthia Fletcher

• Ms. Christy Hicks

• Dr. Diane McEachern

• Ms. Jackie Newson

• Dr. Ann Tickamyer

• Ms. Marnie Werner.

A9. Tokens of Appreciation

This information collection will not utilize any tokens of appreciation.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

This study will collect the names and professional contact information for individuals involved in implementing human service programs (e.g., program leadership, staff). The study team will collect this information in order to invite them to speak with us. We will not maintain this information in a paper or electronic system from which we actually or directly retrieve data using an individuals’ personal identifier.

Assurances of Privacy

The study team will keep collected information private to the extent permitted by law. We will inform respondents of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. As specified in the contract, we will comply with all federal and departmental regulations for private information.

Data Security and Monitoring

This information collection will not collect sensitive information. The study team will protect respondent privacy to the extent permitted by law and will comply with all federal and departmental regulations for private information. We have developed a System Security Plan that assesses all protections of respondents’ information. OPRE will ensure that all contractor employees, subcontractors (at all tiers), and employees of each subcontractor, who perform work under this contract/subcontract, are trained on data privacy issues and comply with the above requirements.

The study team will collect oral consent for interviews by way of a script at the beginning of instruments 2, 3, and 4 (instruments 1a and 1b are the planning documents and do not require consent). We will audio record interviews if participants consent to being recorded. In addition, study team members will take notes during the interview. Interview notes and recordings may contain the names of professional staff. Therefore, the study team will take notes on laptops with full disk encryption. We will transfer audio recordings from the recording device to the encrypted laptop and delete recordings from the recording device before leaving the facility where the interview occurs. Ultimately, the study team will transfer, store, and analyze the notes and audio recordings on a secure encrypted server in project folders to which only members of the study team have access. The contractor will transcribe audio recordings of interviews. In the final interview transcripts and notes used to analyze interview data, the contractor will redact interviewees’ names.

A11. Sensitive Information ¹

The study team will not collect any sensitive information from interview subjects or collaboration survey respondents.

The contractor’s IRB has determined that the study is eligible for “Exempt with limited IRB review.” (Appendix B., Institutional Review Board approval)

A12. Burden

Explanation of Burden Estimates

The study team calculated the burden estimates by drawing upon past experience conducting interviews of similar format and length. Twelve program directors (1 from each site) will spend up to 2 hours assisting us in planning the site visit. We will interview 5 program directors and other leadership from each site (60 total interviews), up to 9 staff from each site (108 total interviews), and 6 staff from nonprofit or partner organizations from each site (72 total interviews). We will interview 20 individuals at each site, for 240 total interviews (see table 2).

Estimated Annualized Cost to Respondents

To calculate annualized costs to respondents, we used the following wage data from the Bureau of Labor Statistics May 2019 data. https://www.bls.gov/oes/current/oes_stru.htm

• Program directors and other leadership: Social and Community Service Managers, Job Code 11-9151

• Site staff: Social and Human Service Assistance, Job Code 21-1093

• Nonprofit or Partner Organizations Staff: Community and Social Service Specialists, Job Code 21-1099

The total respondent cost is $4,778.

Table 2. Burden Estimates and Annualized Cost to Respondents

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total Burden (in hours)	Annual Burden (in hours)	Average Hourly Wage Rate	Total Annual Respondent Cost
Site Visit Planning Template (Instrument 1a or 1b)	12	1	2	24	12	$35.05	$420.60
Program Directors and Leaders Site Visit Discussion Guide (Instrument 2)	60	1	2	120	60	$35.05	$2,103.00
Staff Site Visit Discussion Guide (Instrument 3)	108	1	1.5	162	81	$17.81	$1,442.61
Nonprofit or Partner Organizations Site Visit Discussion Guide (Instrument 4)	72	1	1	72	36	$22.55	$811.80
Total				378 hours	189 hours		$4,778.01

A13. Costs

There are no additional costs to respondents.

A14. Estimated Annualized Costs to the Federal Government

The total cost for the data collection activities under this current request will be $226,883.57. The cost was estimated using the project budget developed by OPRE and the contractor.

Table 3. Estimated Annualized Costs to the Federal Government

Activity	Detail	Estimated Cost
Survey design Instrument development Pilot and user testing OMB clearance	FTE time Operational expenses (such as equipment, overhead, printing, and staff support) Other expenses which would not have been incurred without this collection of information	$12,109.84
Survey administration	FTE time Operational expenses (such as equipment, overhead, printing, and staff support) Other expenses which would not have been incurred without this collection of information	$177,772.46
Analysis and initial dissemination	FTE time Operational expenses (such as equipment, overhead, printing, and staff support) Other expenses that would not have been incurred without this collection of information	$37,001.27
Total costs		$226,883.57

A15. Reasons for changes in burden

This is a new information collection request.

A16. Timeline

Data collection will occur within a two-year period. Our project timeline is as follows, dependent on the timing of the OMB approval.

Table 4. Timeline

Activity	Timing
Conduct literature review	January 2020	May 2020
Compile and analyze existing administrative data to determine unmet need	January 2020	June 2021
OMB Approval
Conduct 12 case studies	Months 1-6, following OMB approval
Analyze interview data	Months 6-8, following OMB approval
Conduct QCA	Months 8-11, following OMB approval
Synthesize all data findings	Months 8 to 12, following OMB approval
Final report	Months 10-15, following OMB approval

A17. Exceptions

No exceptions are necessary for this information collection.

Attachments

Instrument 1a: In Person Site Visit Planning Template

Instrument 1b: Virtual Site Visit Planning Template

Instrument 2: Program Directors and Leaders Site Visit Discussion Guide

Instrument 3: Staff Visit Site Visit Discussion Guide

Instrument 4: Nonprofit or Partner Organizations Site Visit Discussion Guide

Appendix A: Responses to Federal Register Notice Comments

Appendix B: Institutional Review Board Approval

Appendix C: Initial Email to Program Directors– In Person

Appendix D: Initial Email to Program Directors - Virtual

Appendix E: Email from Program Directors to Respondents-In Person

Appendix F: Email from Program Directors to Respondents-Virtual

References:

Office of Disease Prevention and Health Promotion. (2020). Social determinants of health. https://www.healthypeople.gov/2020/topics-objectives/topic/social-determinants-of-health

Sosulski, M. R., & Lawrence, C. (2008). Mixing methods for full-strength results: Two welfare studies. Journal of Mixed Methods Research, 2(2), 121–148. doi: 10.1177/1558689807312375

1 Examples of sensitive topics include (but not are not limited to) social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating, and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships (e.g., family, pupil-teacher, employee-supervisor); mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities that indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); and immigration/citizenship status.

14