Instrument 1: Landscaping protocol with Stakeholder Agencies (staff burden in Head Start (HS) grantee and community-based child care agencies)

VIQI Landscaping Protocol with Stakeholder Agencies and Related Materials

July 2020

Landscaping Protocol with Stakeholder Agencies

and Related Materials

The purpose of the Landscaping Protocol with Stakeholder Agencies is to better understand the feasibility of conducting the VIQI project in various localities, the structure of local early care and education programs, and the ways in which the VIQI team will be able to assess variation in initial quality across centers. An agenda will be sent to stakeholders prior to the discussion and a semi-structured protocol will be used to guide the discussion. This attachment includes:

1. VIQI Landscaping Discussions Sample Agenda

2. Semi-Structured Protocol

1. VIQI Landscaping Discussions Sample Agenda for [DATE AND TIME]

1. Introductions

2. Review agenda for call and provide brief overview of VIQI project

3. Any questions/initial reactions regarding plans for the project

4. Group discussion

Note: topics and questions listed on the agenda will vary depending on the expertise of the informants and the project team’s extant knowledge base. For example, agenda items for a conversation with an informant at the local level who has expertise in professional development might include:

• We know the COVID-19 pandemic has affected centers in different ways, so we’d like to understand what that might look like in [LOCALITY]). How has COVID-19 affected ECE settings and initiatives related to quality improvement and professional development?

• We are exploring how initiatives to improve quality and teacher professional development might influence the level of quality in early care and education (ECE) settings. What are important recent, current, or upcoming initiatives that focus on teacher professional development as a way to improve quality in [name of locality]?

• What does in-service training for ECE teachers entail (across Head Start and child care settings)?

• Do teachers receive mentoring or coaching?

5. Next steps/follow-up

This collection of information is voluntary and will be used to understand the landscape of early care and education throughout the country. Public reporting burden for this collection of information is estimated to average 90 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-0508 and the expiration date is XX/XX/XXXX. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to [Contractor Contact Name]; [Contractor Contact Address].

2. Semi-Structured Protocol

Guidelines for Use of this Protocol in Landscaping Discussions. This protocol will be used to guide a series of semi-structured landscaping discussions with state and local early care and education (ECE) informants. In each module, illustrative questions and prompts provide examples of the types of questions that will be asked.

Modules will cover the following topics:

Module A. ECE Program/Center Structure and Population Served

Module B. Quality and Quality Measures

Module C. Curricula and Professional Development

Module D. Data Infrastructure

Module E. Feasibility of Design Options

Module F. Follow-up/Next Steps

Facilitators will not cover all of the questions in this protocol during every discussion. We are asking informants for a discussion that is expected to last approximately one hour. We may also conduct a second discussion to follow-up and clarify information emerging from the initial discussion. In total, we expect that any given participant will be engaged in a maximum of 1.5 hours of discussion. (For localities that we already have existing relationships with or already have detailed information about ECE programming, quality improvement and professional development initiatives underway, especially due to COVID-19 pandemic, we expect that the discussion will be shorter.) Facilitators will select and edit questions/prompts from different modules as needed, depending on: 1) the expertise of the informants with whom they are speaking; and 2) the study team’s extant knowledge base. Additional questions and prompts (not included in this protocol) may be asked as non-standardized follow-up prompts and to clarify responses provided by informants.

Members of the VIQI team will serve as facilitators, and will:

1. Introduce members of the team and ask informants to introduce themselves and describe their roles and responsibilities. [Tailor and personalize introduction, if we have already spoken and have an existing relationship with key contact.]

2. Provide a brief overview of the VIQI project and summarize the purpose of the call/visit.

Standard introductory language to be used by facilitators (not for distribution):

• The Administration for Children and Families’ Office of Planning, Research and Development is working with MDRC and its partners on a major new study that will examine how different dimensions and levels of quality in early childhood classrooms relate to children’s developmental outcomes. We plan to partner with multiple localities to conduct this study.

• In the planning stages for the [insert year] of the VIQI project. To help us plan, we are reaching out to leaders in the early childhood field to:

  • share more details about the study;

  • ask questions to help us better understand the landscape of ECE services; and

  • discuss potential opportunities and challenges that our study team should consider in planning for this study.

• We know the COVID-19 pandemic has affected centers in different ways, so we’d like to understand what that might look like in [LOCALITY]).

• This discussion should take a maximum of 1.5 hours. [If more is known about a particularly locality, add: Given that we know a fair amount about [LOCALITY], it is likely that this conversation will be shorter.]

• Participation is voluntary, and responses will be kept private to the extent permitted by law.

• An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is ________________and the expiration date is ________________.

3. Answer clarifying questions about the purpose of the discussion and study plans.

4. Confirm information that was obtained in previous communications (only in cases where the team is reconnecting with a locality that was contacted for VIQI during the 2019-2020 school year).

5. Select questions and prompts from the following modules as needed based on the study team’s relationship with the informant, what is already known about the locality and ECE programming, and the expertise of the informants.

Module A. Early Care and Education Program/Center Structure and Population Served

This module focuses on the structural characteristics of early care and education (ECE) services in specific localities. Illustrative questions and possible prompts are provided below. Note: Some questions and prompts in this module also appear in Module D (“Data Infrastructure”), but the prompts serve different purposes across the two modules. In Module A, the prompts serve to gather high-level, first-hand insights from experts regarding the landscape of ECE services in different localities. In Module D, the prompts serve to learn more about the type and depth of information being stored in existing data sources. Asking the prompts in Module D allows us to identify specific data sources that might be helpful to explore to develop a more detailed understanding about the characteristics of child care programs and centers nationally and in different localities.

Introduction to Module A. The VIQI study will be conducted in collaboration with multiple localities. By “locality,” we mean “metropolitan areas.” We will be looking for localities where there are large numbers of Head Start and child care centers serving children from low-income backgrounds, and where there is a documented wide range of quality among centers across settings. In this initial planning phase, we would like to develop a better understanding of the landscape of ECE services in [locality X], including the structural characteristics of ECE programs and centers, the populations of children and families served, and how this has been impacted by the COVID-19 pandemic

Illustrative Questions and Prompts

• Who are the key stakeholders involved in oversight of child care and Head Start programs in [locality X]? ^*

• Possible prompts:

  • What are the major nonprofit organizations that operate multiple ECE centers?

  • Who determines which curricula are used?

  • Who determines what professional development teachers receive (particularly when it comes to instructional strategies and implementation of curricula)?

  • Who are the other key decision-makers or influencers? (e.g., funders, teachers’ unions, heads of early childhood coalitions or advocacy groups)
    
    

• Are there large concentrations of certain types of ECE programs in [locality X]?*

  • Possible prompts: large, multi-site non-profit organizations, for-profit organizations, faith-based organizations
    
    

• What is the age make-up of most ECE classrooms?*

  • Possible prompts:

    • Are two’s, three’s, and four’s in mixed age classrooms or separated based on birth date?

    • Is this different across different ECE settings? (Head Start, child care?)

    • Are three’s included in public pre-k programs?
      
      

• We need to develop a detailed understanding of the characteristics of Head Start and child care programs and centers in [locality X]. Do you have answers to the following questions, or can you tell us how to obtain the answers? *

  • Number of Head Start and child care programs and centers

    • Number of centers per program

    • Number of classrooms per center

  • Length of day

  • Demographic data about children served by ECE programs and centers in a given locality

    • The number of children ages two, three, and four

    • The proportion of children who are from low-income families

    • The proportion of children who are English language learners

    • The proportion of children who have special needs
      
      
      

• What guidance or mandates have been issued by the state or locality on how centers should change their usual operations in response to COVID-19? (e.g. number of children per classroom, the ratio of teachers to preschool-aged children, priority populations of children served)

Module B. Quality and Quality Measures

This module aims to understand how states and localities might assess and measure ECE quality on an ongoing basis. It further aims to understand how this information is stored and maintained. Note that some of the questions in this module will overlap with the questions in Module D (Data Infrastructure), but this module is designed to focus the discussion on measures of quality, while Module D is designed to gather information about a broader set of data that can inform the VIQI study. Illustrative questions and possible prompts are provided below.

Introduction to Module B. We would like to ask for your insights about common ways in which the quality of ECE programs and centers is described and measured, and about state/local trends in quality improvement, and whether the COVID-19 pandemic has impacted this.

Illustrative Questions and Prompts

• The field has identified several basic dimensions of classroom quality that are hypothesized to influence children’s outcomes – namely structural quality (how centers and classrooms are designed and configured) , process quality (quality of children’s interactions with teachers and others in the classroom, including warmth, sensitivity, and overall classroom management and organization, and instructional quality (in terms of intentional teaching of children through organized activities, including scope and sequence of activities within specific domains).

  • Possible prompt:

    • Are there other aspects of quality that locality X (or the state) is particularly interested in?
      
      
      

• What are important recent, current, or upcoming initiatives to improve ECE quality at the state and local levels? How has this been impacted, if at all, by the COVID-19 pandemic?

  • Possible prompts:

    • Any major recent/upcoming changes to ECE curricula?

    • Other initiatives led by foundations or regional/state/local early childhood coalitions?
      
      
      

• Which extant data sources (Quality Rating and Improvement Systems [QRIS], other) can be used to determine levels of quality among ECE programs and centers in [locality X]?*

  • Possible prompts:

    • What data/management information systems (MIS) are used to facilitate monitoring of child care and Head Start programs and centers in [locality X]?

      • Tracking as part of community-based organizations’ services

      • Tracking as a part of a local ECE initiative

      • Other tools and measures used to monitor curriculum implementation

    • How would you describe the variation in levels of quality among ECE programs (and centers within those programs) that are included in these monitoring systems? Are there many high AND low quality classrooms?*

      • What about the programs and centers that are NOT included in these monitoring systems?

      • How would you describe the variation in levels of quality by type of ECE program (e.g. Head Start, child care center)?

      • What are the reasons for the variation in quality?

Module C. Curricula and Professional Development

This module focuses on common curricula, curricular enhancements, professional development (PD) initiatives, and other interventions that [locality X] is using in Head Start and child care centers. Illustrative questions and possible prompts are provided below.

Introduction to Module C. In the planning and design phase of the VIQI study, we are gathering information about the curricula, curricular enhancements, professional development initiatives, and other interventions that localities are using with three and four year-olds in Head Start and child care centers. We would like to ask for your thoughts about this topic in general and in the context of COVID-19, and any advice you can offer regarding other stakeholders or resources that might help us with this effort.

Illustrative Questions and Prompts

Curricula and curricular enhancements

• What curricula and curricular enhancements are commonly used within Head Start centers in [locality X]? Within child care centers? *

  • Possible prompts:

    • Different curricular/interventions for ages two, three and four?

    • Curricular enhancements focused on particular domains (e.g., math, language and literacy, social-emotional skills)?

Professional Development (PD)

• We are exploring how professional development (or PD) might influence the level of quality in an ECE center or classroom. What are important recent, current, or upcoming initiatives that focus on teacher PD as a way to improve quality in [locality X]? How has this been impacted, if at all, by the COVID-19 pandemic?
  
  

• Have training requirements changed in response to COVID-19?

• What does in-service training for ECE teachers entail?

  • In Head Start? In child care settings?

  • How often, how long, and on what topics?
    
    

• Do teachers receive mentoring or coaching?

  • How often, how long, by whom, and on what topics?

  • Does it involve a classroom visit and/or a sit-down meeting?
    
    

• Do teachers have shared planning time?

  • How often, how long, and with whom?
    
    

• What are some of the challenges that states and localities face with respect to PD for ECE teachers? Has the COVID-19 pandemic influenced this in any way?

Module D. Data Infrastructure

This module is designed to explore the types of data collected on the state and local levels that can inform the VIQI study. Illustrative questions and possible prompts are provided below. Note: Some questions and prompts in this module also appear in Module A (“ECE Program/Center Structure and Population Served”), but the prompts serve different purposes across the two modules. In Module A, the prompts serve to gather high-level, first-hand insights from experts regarding the landscape of ECE services in different localities. In Module D, the prompts serve to learn more about the type and depth of information being stored in existing data sources. Asking the prompts in Module D allows us to identify specific data sources that might be helpful to explore to develop a more detailed understanding about the characteristics and child care programs and centers nationally and in different localities.

Introduction to Module D. One of the goals of the VIQI study is to understand the effects of different dimensions and thresholds of quality on child outcomes and to understand how these effects might vary depending upon the characteristics of the participating ECE programs and the populations they serve. We would like to work with localities that have a large number of ECE centers that represent a wide range of quality. We would like to understand, from your perspective, what types of data already exist that might help our team to better understand the landscape of these programs (and centers within those programs) in [locality X].

Illustrative Questions and Prompts

• What data sources do you think would be helpful in this endeavor? (At the state level? At the local level?) *

  • Possible prompts:

    • Quality Rating and Improvement Systems (QRIS)

    • Monitoring data collected by local agencies with oversight of ECE programs

    • Are there any initiatives or efforts underway to build an ECE data infrastructure?

  • Possible prompts for each data source mentioned:

    • Does this data source include names and characteristics of Head Start or child care centers?

• At what level: Individual centers? Programs?

• Does the data source provide information on the following features of programs, centers and population characteristics?

• Structural characteristics

  • Number of centers in a given locality

  • Number of classrooms per center

  • Length of day

  • Grouping of two, three and four-year-olds: are they usually in mixed classrooms, or grouped by age?

  • Language(s) of instruction

• Demographic data about children served by ECE programs in [locality X]

  • The number of children ages two, three, and four

  • The proportion of children who are from low-income families

  • The proportion of children who are English language learners

  • The proportion of children who have special needs

• Characteristics of administrators, teachers, and other staff

  • By center or program

• Curricula and Professional Development

  • Curricula and curricular enhancements used by individual centers

  • Information about professional development (how long, how often, on what topic, etc.)

• How are these data maintained?

  • How frequently are the data updated in this data source?

  • What is the catchment area covered by this data source?

  • What is the level of penetration for this data source?

    • Does the level of penetration differ for Head Start or child care programs?

    • Is participation in this data source voluntary or mandatory?

• How complete are these data? Are there concerns about missing fields of information in this data source? What might these missing fields be?

• How accurate is the information included in the data source?

• How are these data maintained?

• How can these data sources be accessed? What is the process for accessing the data?

  • Are ECE management information systems (MIS) within localities usually linked to each other? To other MIS systems (e.g., k-12)? If so, for what purposes?

    • Possible prompt: Are there unique IDs that are used to follow children’s progress from child care through the K-12 system?

• How are MIS data structured to describe quality (e.g., at the center or program level? At the state or local level?)

  • What are the primary measures that are used to assess classroom quality, teacher quality, and child outcomes in ECE programs and centers in [locality X]?

  • Possible prompts:

    • What are the key dimensions of these measures?

    • On classroom quality, is there anything collected besides the ECERS or the CLASS?

Module E. Feasibility of Design Options

The questions in this module are designed to elicit additional advice from informants about any topics or issues that have not already been covered in the discussion. Illustrative questions and possible prompts are provided below.

Introduction to Module E. We would like to know whether there are any other words of advice you might have for us as we embark on this study – including any additional thoughts about opportunities or challenges we should have on our radar screen, or any other questions you think we should consider.

Illustrative Questions and Prompts

• We have a particular interest in recruiting a sample of ECE programs and centers that includes good representation of programs and centers that are both higher and lower in initial quality to stratify the sample. What opportunities/challenges in [locality X] would come with trying to do so?

  • Possible prompts:

    • How might the COVID-19 pandemic affect the distribution of quality across ECE programs and centers?

    • How might the existing distribution of initial quality across ECE programs and centers affect our ability to do so?

    • Might this vary by setting type?

    • Might this vary specifically for ECE programs and centers serving 3- and 4-year olds?

• What are your thoughts about the challenges we may encounter (either in site recruitment, or implementation of quality interventions)?

  • Possible prompt:

    • Do you have any advice for us regarding possible solutions to those challenges?

    • Given the COVID-19 pandemic, are there any new practices or mandates that we should be mindful of?
      
      

• Is [locality X] (or the ECE programs/centers within [locality X]) participating (or planning to participate) in any other research projects?

Module F. Follow-up/Next Steps

This module focuses on wrapping up the discussion and laying out next steps. Illustrative questions and possible prompts are provided below.

Introduction to Module F. In the last few minutes of this discussion, we would like to discuss next steps, make plans to follow up on items that we have discussed today, and address any other questions you might have for us.

Illustrative Questions and Prompts

• Are there other particular localities that you think might have the following characteristics:

  • A range of program quality

  • Large numbers of Head Start and child care centers serving children from low-income backgrounds

  • Localities that have strong commitment to data collection and quality
    
    

• Do you have suggestions for key contacts in these localities?
  
  

• We want to make sure we speak with the people who can best inform our landscaping and study design. Based on the nature of our questions and interests, who else would you suggest we speak with to inform the study?

  • Possible prompts:

    • Other stakeholders in the ECE system

    • Other key staff in state/local child care/Head Start programs

    • Experts on curricula and interventions, quality and quality improvement, professional development

    • Experts on ECE data infrastructure
      
      

• (If appropriate): Would your office be willing to sign a letter of introduction addressed to (insert names of stakeholders as appropriate) describing the VIQI study and asking them to speak with someone from our team about the VIQI study?
  
  
  

Participation is voluntary, and responses will be kept private to the extent permitted by law. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-0508 and the expiration date is 05/31/2021.

* Starred questions are only relevant when speaking with stakeholders in NEW localities that have not been previously engaged in VIQI recruitment discussions.

3