SSA - VIQI GenIC

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Variations in Implementation of Quality Interventions

Formative Data Collections for ACF Research

0970 – 0356

Supporting Statement

Part A

JUNE 2022

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers:

Ivelisse Martinez-Beck

Amy Madigan

Part A

Executive Summary

• Type of Request: This Information Collection Request is for a generic information collection under the umbrella generic, Formative Data Collections for ACF Research (0970-0356).

• Progress to Date: This information collection request builds on research conducted in the Variations in Implementation of Quality Interventions (VIQI): Examining the Quality-Child Outcomes Relationship in Child Care and Early Education Project, which received approval under the Generic for Pretesting Activities (OMB #0970-0356) and an OMB Information Collection Request (OMB #0970-0508) to conduct a pilot study, as well as an impact evaluation and process study. The project conducted a year-long pilot study in 2018-2019. Lessons learned from the pilot informed the study design and updates to data collection instruments and installation activities focused on teacher professional development for the impact evaluation. The impact evaluation and process study are currently being conducted in 2021-2022, with follow-up data collection on classrooms, administrators, teachers, and children being collected through late May/early June 2022.

• Description of Request: This information collection request is for in-depth semi-structured interviews with early care and education (ECE) center administrators and state and local ECE stakeholders to understand the effect of the COVID-19 context on the workforce and work of ECE centers participating in the VIQI project. Data collected through these interviews are not intended to be generalized to a broader population. Rather, these interviews will help ACF to gather contextual information that is not publicly available about the ECE landscape and workforce in the localities and centers participating in the project, and this information will help inform the interpretation of the research that is ongoing. We do not intend for this information to be used as the principal basis for public policy decisions.

• Time Sensitivity: Because the implementation portion of the project is ending in May 2022 and some ECE centers are closed for the summer, we need to follow-up with center administrators as early in summer 2022 as possible.

A1. Necessity for Collection

The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) launched the Variations in Implementation of Quality Interventions (VIQI): Examining the Quality-Child Outcomes Relationship in Child Care and Early Education Project in 2016. VIQI is a research study sponsored by the Office of Planning, Research, and Evaluation (OPRE) and conducted through a contract with MDRC and its subcontractors, Abt Associates, RTI International, University of Virginia, and MEF Associates.

During the 2021-2022 school year, the VIQI project supported centers in implementing two ECE curricular and professional development models, and collected quantitative data on implementation, classroom quality, and child outcomes. The research team heard anecdotally about how the ECE landscape changed throughout the 2021-2022 school year in unanticipated ways due to the constantly changing pandemic, including closures of classrooms, changes in enrollment, and changes in the workforce and challenges in hiring. The proposed data collection is for interviews with center administrators and state and local ECE stakeholders to uncover details that are not publicly available about the ways the ECE landscape and workforce was different this year than in prior years. This information will inform and help contextualize the implementation and impact findings of the VIQI impact evaluation and process study and to better understand the contexts in which the study took place. 

There are no legal or administrative requirements that necessitate this collection. ACF is undertaking the collection at the discretion of the agency.

A2. Purpose

Purpose and Use

This proposed information collection meets the following goals of ACF’s generic clearance for formative data collections for research and evaluation (0970-0356):

• inform the development of ACF research

• maintain a research agenda that is rigorous and relevant

• ensure that research products are as current as possible

The purpose of the information collection is to better understand the center-, locality- and state-level influences on the workforce and work of ECE centers participating in the VIQI impact evaluation and process study during the 2021-2022 school year, in light of the ongoing COVID-19 pandemic. Specifically, information will be gathered to detail (1) the experiences, successes and challenges faced by centers who participated in the VIQI project and how those experiences compared to prior years, and (2) the state and local context during the 2021-2022 school year that may have affected participating ECE centers – for example, economic and employment conditions, the COVID-19 pandemic, and changes to ECE standards or requirements.

Information will be collected through interviews with center administrators and state and local ECE stakeholders. The information will be used to contextualize the implementation and impact findings of the VIQI impact evaluation and process study and to better understand the contexts in which the study took place. Although the main use of the information collected is not for publication, information will contribute to public materials that describe the overall findings from the VIQI project and may inform resources used for technical assistance in ECE.

The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information.

Research Questions or Tests

Interviews with center administrators who participated in the VIQI impact evaluation aim to address the following overarching research questions:

1. How has the COVID-19 pandemic affected center operations (such as center or classroom closures and instruction), staffing, funding, and enrollment during the VIQI project?

   1. How do these aspects of center operations, staffing, funding, and enrollment during the VIQI project compare to prior years?

2. What are administrator perspectives on the facilitators and barriers to implementation of the professional development and curricula provided by the VIQI project?

Interviews with state/local systems administrators who work in the states or localities that participated in the VIQI impact evaluation aim to address the following research questions:

1. What are aspects of the local and state contexts that may influence ECE center operations, and implementation of the professional development and curricula provided by the VIQI project, specifically related to:

   1. Early care and education systems-level changes, such as changes to curricula, standards, monitoring requirements, or funding,

   2. The COVID-19 pandemic, and

   3. Employment and economic conditions.

Study Design

We will hold semi-structured interviews in-person, by phone or by video conference with up to 130 center administrators and up to 20 state and local ECE stakeholders

Instruments	Respondent, Content, Purpose of Collection	Mode and Duration
Center administrator interview protocol (Instrument 1)	Respondents: up to 130 center administrators that oversee ECE centers participating in VIQI Content: COVID-19 adaptations to programming and instruction Center funding, enrollment, and functioning Staffing changes and hiring Staff well-being For intervention centers only: Curriculum and professional development implementation Purpose: to understand how the 2021-22 school year was different from prior years and to how the COVID-19 context affected their center’s programming, funding, and staffing.	Mode: Phone or video conference Duration: up to 60 minutes
State and local ECE stakeholder interview protocol (Instrument 2)	Respondents: up to 20 administrators that oversee Head Start and/or ECE Content: Changes to ECE systems (e.g., curricula, professional development, funding) Changes to ECE landscape due to COVID-19 Economic and employment conditions in state/city Purpose: to understand what was occurring at the state and local levels and how the COVID-19 context affected the ECE centers and overall workforce during the 2021-2022 school year	Mode: Phone or video conference Duration: up to 60 minutes

Other Data Sources and Uses of Information

The data collected through these semi-structured interviews will be used to address the research questions in concert with other available sources of information, including information collected as part of the VIQI project (OMB #0970-0508) such as data on curriculum and professional development implementation, classroom quality, children’s outcomes, to provide context for the implementation and impact findings.

The research team will also search for publicly available guidance documents online that may describe COVID-related policies and practices in place during the study timeline within the states and localities that participated in the VIQI project.

A3. Use of Information Technology to Reduce Burden

Invitations to participate in interviews will be sent by e-mail, and interviews will be conducted in-person or by phone or video call, based on the respondent’s preference. Our data collection approach aims to obtain this information efficiently while minimizing respondent burden. 

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

Although the team will search for online documents that may provide contextual information about COVID-19 ECE policies and practices in place, there are no other existing data sources that can provide details about how the pandemic affected the centers participating in the VIQI project during the impact evaluation in order to inform interpretation of the implementation and impact findings from the study.

A5. Impact on Small Businesses

As we will be selecting administrators who are employed at organizations that vary in size, we expect that a small number will be employed in small businesses that provide ECE services. Burden will be reduced for all participants, including employees of small organizations, by scheduling discussions at times convenient to respondents and restricting discussions to 60 minutes. 

A6. Consequences of Less Frequent Collection

This is a one-time data collection. 

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the overarching generic clearance for formative information collection. This notice was published on November 3, 2020, Volume 85, Number 213, page 69627, and provided a sixty-day period for public comment. During the notice and comment period, no substantive comments were received.

Consultation with Experts Outside of the Study

Not applicable.

A9. Tokens of Appreciation

We will not offer any tokens of appreciation. 

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

Personally identifiable information is limited to the name and contact information. This information will be used for contacting purposes only.

Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individuals’ personal identifier.

Assurances of Privacy

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information (including any audio recording of interviews) will be kept private to the extent permitted by law. See Section B4 in Supporting Statement B for more information on how this information will be conveyed. As specified in the contract, the Contractor will comply with all Federal and Departmental regulations for private information.

Although this request does not include sensitive questions, due to the sensitive nature of the larger VIQI project, the impact evaluation obtained a Certificate of Confidentiality. The study team has been approved for this Certificate. The Certificate of Confidentiality helps to assure participants that their information will be kept private to the fullest extent permitted by law. The study team has also received IRB approval for this study.

Data Security and Monitoring

As specified in the contract, the Contractor shall protect respondent privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information. The Contractor has developed a Data Safety and Monitoring Plan that assesses all protections of respondents’ PII. The Contractor shall ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor, who perform work under this contract/subcontract, are trained on data privacy issues and comply with the above requirements. 

As specified in the evaluator’s contract, the Contractor shall use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The Contractor shall securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Processing Standard.  The Contractor shall: ensure that this standard is incorporated into the Contractor’s property management/control system; establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology (NIST) requirements and other applicable Federal and Departmental regulations.

A11. Sensitive Information ¹

No sensitive information will be collected. 

A12. Burden

Explanation of Burden Estimates

We estimate up to 130 center administrators and up to 20 state and local ECE stakeholders will participate in interviews that will be limited to 60 minutes. Each informant will participate in one interview. The protocols are designed to allow the interviewer to tailor the discussion based on the respondent’s background; not all questions may be asked of all respondents. 

Estimated Annualized Cost to Respondents

We assume that respondents will work as administrators in ECE centers or state and local ECE systems. According to the salary averages reported in May 2021 National Occupational Employment and Wage Estimates by the Bureau of Labor Statistics the average hourly wage for education and child care administrators in preschool and child care centers and programs is $25.87 (11-9031 Education and Childcare Administrators, Preschool and Daycare; https://www.bls.gov/oes/current/oes119031.htm). 

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total/annual Burden (in hours)	Average Hourly Wage Rate	Total Annual Respondent Cost
Instrument 1: Center administrator interview protocol	130	1	1	130	$25.87	$3,363.10
Instrument 2: State and local ECE stakeholders interview protocol	20	1	1	20	$25.87	$517.40
Total				150		$3,880.50

A13. Costs

There are no additional costs to respondents.

A14. Estimated Annualized Costs to the Federal Government

Cost Category	Estimated Costs
Field Work	$175K
Total costs	$175K

A15. Reasons for changes in burden

This is for an individual information collection under the formative generic clearance for ACF research (0970-0356).

A16. Timeline

We anticipated completing interviews over about three months following OMB approval. Ideally we will begin data collection in June 2022.

Data will inform study findings, which will be released in public materials on a rolling basis beginning potentially in November 2022.

A17. Exceptions

No exceptions are necessary for this information collection.

Attachments

Instrument 1: Center administrator interview protocol

Instrument 2: State and local ECE stakeholders interview protocol

Appendix A: Interview outreach

1 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.

10