SSA - Formative GenIC - ECE RISE

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Child Care Research and Evaluation Capacity Building Center Application for Receiving Individualized Support for Evaluation Capacity Building (ECE-RISE)

Formative Data Collections for ACF Research

0970 - 0356

Supporting Statement

Part A

October 2021

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Alysia Blandon

Part A

Executive Summary

• Type of Request: This Information Collection Request is for a generic information collection under the umbrella generic, Formative Data Collections for ACF Research (0970-0356).

• Progress to Date: We conducted a needs assessment to understand the research and evaluation capacity-building needs of State, Territory, and Tribal Child Care and Development Fund (CCDF) Lead Agencies (information collection approved under the same umbrella generic, Formative Data Collections for ACF Research (0970-0356)). This request will build on that work.

• Description of Request:

The Child Care Research and Evaluation Capacity Building Center will collect information to inform selection of agencies to participate in Receiving Individualized Support for Evaluation Capacity Building (ECE-RISE). The information will be collected through an application form, self-assessment, and interview which will gauge CCDF Lead Agencies’ interest, level of commitment, and ability to sustain the work after the individualized support ends. We do not intend for this information to be used as the principal basis for public policy decisions. The data collected are not intended to be generalized to a broader population.

• Time Sensitivity: To stay on track with our project timeline and allow participants to receive research and evaluation supports for as long as possible, we hope to begin data collection as soon as possible.

A1. Necessity for Collection

The Administration for Children and Families (ACF), Office of Planning, Research, and Evaluation (OPRE) is offering individualized support for evaluation capacity building to Child Care and Development Fund (CCDF) Lead Agencies through a new opportunity called Receiving Individualized Support for Evaluation Capacity Building or ECE-RISE. ECE-RISE will help CCDF Lead Agencies increase their ability to conduct and consume rigorous and policy-relevant research and evaluations as well as disseminate the information they learn from their research. OPRE has contracted with the Urban Institute and Mathematica to facilitate these efforts. Agencies interested in participating in ECE-RISE will be asked to complete an application form, self-assessment, and interview in order to assess their interest and determine whether they are a good fit for the opportunity.

A2. Purpose

Purpose and Use

This proposed information collection meets the following goal of ACF’s generic clearance for formative data collections for research and evaluation (0970-0356):

• inform the provision of technical assistance.

The Application for ECE-RISE will gather information to assess CCDF Lead Agencies’ (1) interest in building capacity to lead and conduct research and evaluation, (2) level of commitment to the process of intensive support, and (3) ability to sustain the work after the intensive support ends. The self-assessment will describe the current research and evaluation capacity of interested agencies, including strengths and challenges related to conducting, partnering to obtain, and using research and evaluation.

Information gathered will be used to identify and inform the selection of select Lead Agencies that may benefit from intensive research and evaluation capacity-building supports. The Child Care Research and Evaluation Capacity Building Center will then provide tailored support to selected Lead Agencies as they carry out their own research project over an 18-month period. This is a project funded by ACF to both build the capacity of the CCDF Lead Agencies themselves but also to inform future research and evaluation capacity building efforts for ACF-funded programs.

The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information.

Guiding Questions

The guiding question is:

• What are the research and evaluation capacities of the CCDF State, Territory, and Tribal Lead Agencies that apply to ECE-RISE?

Study Design

The ECE-RISE application process will involve three steps: (1) the ECE-RISE application form; (2) the ECE-RISE self-assessment that will provide information about how each applicant Agency currently does or uses research to inform CCDF policy, management, or practice; and (3) the ECE-RISE interview which will be conducted with up to four agencies selected based on their application form and self-assessment.

All CCDF State, Territory, and Tribal Lead Agencies will be eligible to participate in ECE-RISE. The ECE- RISE effort and the associated study design builds on previous data collection and Needs Assessment efforts (completed in the spring or summer of 2021 and information collection approved under the same umbrella generic, Formative Data Collections for ACF Research (0970-0356)). All State, Territory, and a sample of Tribal CCDF Lead Agencies were invited to respond to a survey as part of the Needs Assessment study. The ECE-RISE self-assessment is identical to that survey and will only be completed by agencies that did not already complete the survey for the needs assessment study. All CCDF entities, including those tribal CCDF lead agencies not in our needs assessment sample, will have the opportunity to complete the application form and self-assessment. However, we expect that the opportunity will be of greatest interest to agencies that have less in-house capacity to pursue research projects. ECE-RISE capacity building activities are intended to reach CCDF Lead Agencies that may have less in-house capacity to pursue research projects. We will conduct outreach to ensure that agencies that may benefit the most from ECE-RISE are aware of the opportunity, are encouraged to participate, and are provided support and receive answers to questions during the application period. We describe the outreach and advertising process further in section B2.

We are planning for a 12-week application period starting from when application materials are posted (about one week after receiving OMB clearance) to when applications are due.

The information generated from the application form, self-assessment, and interview will not be analyzed to make any generalizable statements about the whole population of CCDF Lead Agencies. Rather, it will be used to identify agencies that might be well-suited for individualized support. We will review and score each application form to assess the relevance of an Agency’s questions of interest, the feasibility of exploring the questions in an 18-month period, the composition of their Agency team and partners, their commitment to and Agency support of participating in ECE-RISE, and their Agency goals for the work and the strengths they bring to the opportunity. We will then analyze the responses from each applicant Agency’s self-assessment completed at the time of application to the ECE-RISE opportunity or their responses to the Needs Assessment Survey. We will use the results to identify applicant Agencies with research and evaluation capacity that is neither (1) too high, and not in need of intensive support, or (2) too low and lacking resources, staff knowledge and skills, and agency supports and infrastructure to be able to carry out a research project in an 18-month period.

Based on combined scores from the application form and self-assessment, up to four agencies will be selected to participate in interviews with the ECE-RISE team to further discuss details of their application and interests. We will conduct 90-minute semi-structured interviews by web conference with key staff from each of the four agencies selected for interviews. Using specific criteria, interviewers will provide final ratings of the four agencies based on the interviews and recommend two agencies for participation in ECE-RISE.

We will then work with the selected CCDF Lead Agencies over an 18-month period, providing each with tailored, research and evaluation supports. ¹ The table below provides a high level overview of the planned data collection activity, purpose of the collection and mode and duration.

Exhibit A1. Study Design Overview

Data Collection Activity	Respondent, Content, Purpose of Collection	Mode and Duration
Application form	Respondents: State, Territory, and Tribal agency staff Content: Agency’s interest in capacity building activities; Agency’s identification of potential research questions and projects to pursue, Agency’s identification of staff and resources to commit to the project; Agency’s partners in research and evaluation; Agency’s research capacity and barriers to conducting research; Purpose: Assess research and evaluation capacity and needs	Mode: Fillable PDF Duration:30 MIN
Self-assessment	Respondents: State, Territory, and Tribal agency staff that did not complete the Needs Assessment Survey under the Child Care Research and Evaluation Capacity Building Center Needs Assessment study. Content: Agency structure and role in child care system; Agency’s use of existing research and data; Agency’s research capacity and barriers to conducting research; Agency’s interest in capacity building activities Purpose: Assess research and evaluation capacity and needs	Mode: Web with paper option to mail or scan Duration:30 MIN
Interview	Respondents: State, Territory, or Tribal agency staff Content: Potential approach to answering proposed research questions; confirming areas of need; partners and internal strengths; measuring success of participation in ECE-RISE; openness to capacity building process; looking beyond ECE-RISE Purpose: Gather additional information about agency needs and interests	Mode: Web conference call Duration:90 MIN

The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker and is not expected to meet the threshold of influential or highly influential scientific information.

Other Data Sources and Uses of Information

This information collection builds on information collected for the Child Care Research and Evaluation Capacity Building Center Needs Assessment study, which was conducted under the same umbrella generic, Formative Data Collections for ACF Research (0970-0356).

A3. Use of Information Technology to Reduce Burden

Application form

The application form will be provided in a fillable PDF format to allow for electronic submission of responses.

Self-assessment

The self-assessment will be fielded through Confirmit, a web-based survey platform that includes many features that minimize burden, including skip logic to avoid irrelevant questions and pre-filling of information from previous responses into subsequent question stems to reduce the need to recall prior responses.

Interview

Interviews will be conducted virtually, using the WebEx platform, to facilitate participation of agency staff from States, Territories, and Tribes without the need for travel. WebEx allows individuals to see each other through video cameras while maintaining a secure environment. After we obtain permission from each participant, we will record all interviews to ensure that we capture information accurately at one time point.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

We will use information already collected as part of the Child Care Research and Evaluation Capacity Building Center Needs Assessment study (as described in section A2, under Other Data Sources and Uses of Information) and will only ask agencies to complete the ECE-RISE self- assessment if they have not already responded to the survey previously. There are no sources of the other information requested as part of this request.

A5. Impact on Small Businesses

No small organizations are affected by the information collection.

A6. Consequences of Less Frequent Collection

This is a one-time data collection.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the overarching generic clearance for formative information collection. This notice was published on October 11, 2017, Volume 82, Number 195, page 47212, and provided a sixty-day period for public comment. During the notice and comment period, no substantive comments were received.

Consultation with Experts Outside of the Study

We shared sections of the self-assessment with external experts to ensure that we used appropriate and relevant language.

A9. Tokens of Appreciation

No tokens of appreciation will be provided to participants.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

To ensure the correct identity of respondents, the application form and self-assessment will collect the names of respondents and their State, Territory, or Tribal Agencies. Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individuals’ personal identifier.

Assurances of Privacy

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. Interviews will be recorded with the permission of the respondents, and no one besides the research team will listen to the recording. If respondents want to say anything that they prefer not to be recorded, they can ask the interviewer to pause the recorder. Recordings and interview notes will be destroyed after the study. As specified in the contract, the Contractor will comply with all Federal and Departmental regulations for private information. Information collected will be shared with ACF to inform discussions related to the selection of applicant agencies for interviews and participation in ECE-RISE.

Data Security and Monitoring

The study team has developed a data security and monitoring plan that assesses all protections of respondents’ personally identifiable information. The Urban Institute and Mathematica will ensure that all its employees and consultants who perform work under this contract are trained on data privacy issues and comply with the requirements. All project staff, including Urban Institute and subcontractors from Mathematica will sign the Urban Institute’s staff confidentiality pledge agreeing to follow the Urban Institute’s guidelines for data security.

As specified in OPRE’s contract, Urban and Mathematica will use Federal Information Processing Standard (currently, FIPS 140-2) compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The Urban Institute and Mathematica will securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Processing Standard. Urban and Mathematica will (1) ensure that this standard is incorporated into the company’s property management and control system; and (2) establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology requirements and other applicable federal and departmental regulations. In addition, the Urban Institute and Mathematica must submit a plan for minimizing to the extent possible the inclusion of sensitive information on paper records and for protecting any paper records, field notes, or other documents that contain sensitive or personally identifiable information to ensure secure storage and limits on access.

A11. Sensitive Information ²

The information collection materials do not request any sensitive information.

A12. Burden

Explanation of Burden Estimates

Exhibit A2 summarizes the estimated reporting burden and costs for each of the study tools included in this information collection request. The estimates include time for respondents to review instructions, search data sources, complete and review their responses, and transmit information. Figures are estimated as follows:

Application form (Instrument 1). All CCDF Lead Agencies are eligible to apply to ECE-RISE. However, we expect that the opportunity will be of greatest interest to agencies that have less in-house capacity to pursue research projects. Interested agencies will fill out the application form. We anticipate the application will take about 30 minutes to complete. This burden estimate assumes that it will take about 30 minutes to complete the application and that about a third of State and Territory Agencies, and up to 10 Tribal Agencies, may apply.

Self-assessment (Instrument 2). This burden estimate assumes that all agencies completing an application form will also complete the self-assessment, except if they have already completed the Needs Assessment Survey under the Child Care Research and Evaluation Capacity Building Center Needs Assessment study. Completion time for the self-assessment based on the Needs Assessment survey completion times is 30 minutes.

Interview guide (Instrument 3). The study team expects to select four agencies to participate in interviews. We will conduct four 90-minute long interviews, with each group having about 3 participants. This burden estimate is based on 12 participants and the 90-minute length of the focus group.

Estimated Annualized Cost to Respondents

All respondents are members of state, local or tribal government. Estimates for these respondents are based on Bureau of Labor Statistics, Occupational Employment Statistics (2020) estimates for the mean hourly wage of Social and Community Service Managers (Occupational code 11-9151): $36.13

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total Burden(in hours)	Average Hourly Wage Rate	Total Annual Respondent Cost
Application form	28	1	.5	14	$36.13	$505.82
Self assessment	28	1	.5	14	$36.13	$505.82
Interview	12	1	1.5	18	$36.13	$650.34
Total				48		$1734.24

A13. Costs

There are no additional costs to respondents.

A14. Estimated Annualized Costs to the Federal Government

Cost Category	Estimated Costs
Field Work	$64,841
Publications/Dissemination	$0
Total costs over the request period	$64,841

A15. Reasons for changes in burden

This is for an individual information collection under the umbrella formative generic clearance for ACF research (0970-0356).

A16. Timeline

Exhibit A3 shows the timeline for ECE-RISE.

Exhibit A3. ECE-RISE timeline

Tasks	Date
Application posted	Completed 1 week after OMB clearance
Applications due	Ends 12 weeks from posting of application
Analysis and scoring of applications	Begins 1 day after application deadline
OPRE review of application materials	Completed 4 weeks after applications are due
Selection of four agencies for interviews	Completed 2 weeks after submission of application materials to OPRE
Interviews completed	Completed 2 weeks after selection of agencies for interviews
Interview summaries and recommendations finalized	Completed 2 weeks after completion of interviews
Final selection and notification of two agencies	Completed 2 weeks after submission of recommendations to OPRE

A17. Exceptions

No exceptions are necessary for this information collection.

Attachments

Instrument 1: Application form

Instrument 2: Self-assessment

Instrument 3: Interview Guide

Appendix A: Project information sheet

Appendix B: ECE-RISE outreach materials

References:

Bureau of Labor Statistics, U.S. Department of Labor. “Occupational Employment Statistics.” [www.bls.gov/oes/]. May 2020.

1 The research and evaluation efforts will be carried out and led by each of the selected CCDF agencies and their associated staff. There will be no federal involvement in the agency-led projects and as such those projects are not subject to the Paperwork Reduction Act.

2 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.

10