SSB Pretesting GenIC - HS2K

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Further Understanding Children’s Transitions from Head Start to Kindergarten:

A National Descriptive Study (HS2K)

Pre-testing of ACF Data Collection Activities

0970 - 0355

Supporting Statement

Part B

JANUARY 2026

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Kathleen Dwyer, PhD (OPRE)

Part B

B1. Objectives

Study Objectives

The kindergarten transition takes place within a context of diverse organizational infrastructures and local community needs, and a smoother transition can help children build upon the early development and gains they made while in HS. However, very little is known about how leaders and staff across multiple educational organizations—HS (HS) programs, elementary schools, and local education agencies (LEAs)—engage with each other and collaborate to create environments that lead to smooth transitions from HS to kindergarten for children and families.

This proposed pilot study would support plans of the Office of Planning, Research, and Evaluation (OPRE) at the Administration for Children and Families (ACF) under the U.S. Department of Health and Human Services (HHS) to conduct a nationwide study to better understand how local systems (HS and LEA) work together to build collaborations that support staff, children, and families. OPRE has contracted with NORC at the University of Chicago to conduct the national descriptive study with a primary goal to examine the variation in systems-level transition approaches and their associations with outcomes. Surveys will capture (1) how both the HS and K-12 systems structure kindergarten transition supports in service of creating more positive experiences for children and their families, and (2) the extent to which various types of cross-system collaboration efforts occur in different types of geographic locales. OPRE aims to better understand these objectives through six nationwide surveys. The resulting insights from this study will fill knowledge gaps about the HS to kindergarten (HS2K) transition and will enable the field to better understand how cross-system collaboration may differ and how different structures may play a role in that variation. Survey data collected from HS programs for the national descriptive study will be archived and made available to the public for secondary data analysis. Findings from the national descriptive study may inform decisions around what types of transition supports may benefit HS and LEA systems.

This pilot study is intended to test the feasibility of the study design, recruitment methodology, and survey design. The HS2K team will seek to recruit respondent groups at three levels within each system: 1) HS Program Directors and LEA administrators, 2) HS Center Directors and Elementary School Principals, and 3) HS lead teachers and Kindergarten lead teachers. Results of the pilot study will help to determine, if necessary, how to improve the study design and recruitment strategies to successfully implement the national descriptive study.

The information collected during the pilot study is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker. The information collected for the pilot is to inform the study design and procedures.

Generalizability of Results

The pilot study will provide an opportunity for the research team to mimic the national descriptive study’s sampling plan and data collection methodology on a smaller scale, however, the results of the pilot study will not be generalizable. The national descriptive study, planned for early 2027, is intended to produce nationally-representative estimates of HS programs, HS centers, and teachers that directly provide HS services to children who will be transitioning to kindergarten the next school year and that have established relationships with their local LEAs. The sample is also designed to yield nationally representative estimates of LEAs, elementary schools, and kindergarten lead teachers in elementary schools located near HS programs that have established relationships with their local LEAs.

Appropriateness of Study Design and Methods for Planned Uses

The HS2K study design is well-aligned with its primary objective: to examine how HS programs and LEAs collaborate to support HS to kindergarten transitions. The multi-level sampling strategy—targeting HS program directors, LEA administrators, HS center directors, elementary school principals, HS lead teachers, and kindergarten lead teachers—reflects the organizational complexity of the transition process and enables analyses across individuals and systems.

By mimicking the national descriptive study’s sampling and data collection procedures in the pilot, the HS2K team ensures methodological consistency and tests feasibility before full-scale implementation. The pilot study results are not intended to be nationally representative, but instead will be used solely to refine study methods, materials, and procedures. As noted in Supporting Statement A (SSA), this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.  

B2. Methods and Design

Target Population

The pilot study will have the same target populations, sampled at a smaller scale, as the planned national descriptive study described below and will mimic and test the methods and design of the national study, but at a smaller scale.

The HS2K National Descriptive Study will survey six respondent groups at three levels – 1) HS Program directors and LEA administrators, 2) HS center directors and elementary school principals, and 3) HS lead teachers and kindergarten lead teachers. This will enable analyses with different units of analysis, including HS Programs and LEAs, HS centers and elementary schools, teachers, and cross-system dyads (e.g., HS program and LEA pairs).

The initial sample for the nationally representative descriptive study is HS programs operated by grant recipients or delegate agencies that directly serve children and families who have established relationships (per the HS Program Information Report [PIR]) with at least one local education agency (LEA) within their service area (82% of Programs in 2024).¹ The PIR is used to identify eligible HS programs and to provide contact information (address, phone number, and email) for the eligible HS programs. This population of interest also includes the Migrant and Seasonal program type and the Tribal Government or Consortium (American Indian and Alaska Native [AIAN]) agency type. The HS2K team would exclude programs that provide only Early HS services (for pregnant mothers and families with children from birth to age 3) since they do not engage in kindergarten transition activities.

The target populations for the samples associated with HS centers, HS lead teachers, LEAs, elementary schools, and kindergarten lead teachers are defined as follows: all HS centers within eligible HS programs; all lead teachers in eligible HS centers; all LEAs that have an established relationship with an eligible HS program; all elementary schools that maintain a formal relationship with a HS program; and all kindergarten lead teachers employed in those eligible elementary schools. For each of these groups, the unit of analysis aligns directly with the target population: HS centers, HS center lead teachers, LEAs, elementary schools, and kindergarten lead teachers, respectively. The remaining study samples will originate from the nationally representative sample of HS programs.

First, the HS2K team will randomly sample up to two LEAs whose boundaries overlap with the HSHS Program’s service area. The LEAs will be identified using LEA geocoded data from National Center for Education Statistics (NCES) 2023-24 Elementary/Secondary Information System (ElSi) Common Core of Data (CCD) and Private School Survey (PSS)^2,3 and HS Early Childhood Learning & Knowledge Center (ECLKC) 2024 HS Center Location Datasets⁴. According to the 2024 PIR, 39% of HS programs have agreements with only one LEA, 11% have an agreement with two, and the remaining 50% have agreements with three or more LEAs. For each sampled LEA, the team will randomly sample a geographically associated HS center; and for each sampled HS center, the team will randomly sample one HS lead teacher who serves children transitioning to kindergarten per sampled HS center. If two LEAs are sampled, but there is only one HS center that is associated with the HS program, then only that HS Center and only one HS lead teacher associated with that HS center will be sampled. The HS2K team will also identify one elementary school per sampled LEA that is most geographically proximate (i.e., closest) to the sampled HS Center and randomly sample one kindergarten lead teacher per sampled elementary school.

The HS2K team will gather information about kindergarten transitions via six different surveys, one each for: HS Program Directors, HS Center Directors, HS Lead Teachers, LEA administrators, Elementary School Principals, and Kindergarten lead teachers. As noted above, the HS2K team will randomly select HS programs and HS centers nested within those programs to participate in the study. Because no existing national lists of HS lead teachers or kindergarten lead teachers exist, the team will collect contact information about these individuals from HS Center Directors and Elementary School Principals to create the sample frame for each of those two surveys.⁵

Sampling

The pilot study will follow the same sampling and methodology plan at a smaller scale as the national descriptive study to ensure they are feasible. The primary difference will be the number of programs sampled. A brief description of the sampling plan for the pilot study is provided first, followed by a more detailed plan for the 2027 national descriptive study that the pilot study will mimic.

Pilot Study

The HS2K team will select an initial sample of 20 HS programs and use administrative data to determine whether LEAs, HS centers, and elementary schools can be easily and accurately identified. The team will use the HS Early Childhood Learning & Knowledge Center (ECLKC) 2024 HS Center Location Datasets⁶ and the ACF’s Active Centers File to identify HS centers and the National Center for Education Statistics (NCES) 2023-24 Elementary/Secondary Information System (ElSi) Common Core of Data (CCD) and Private School Survey (PSS)^7,8 to identify LEAs and elementary schools. From the initial pilot sample, the HS2K team will select a subsample of five HS Programs (and their associated LEAs, centers, and schools) to pilot the recruitment strategy. The HS Programs will be purposively selected to ensure that the stratification characteristics of interest (e.g., rurality, agency type, program type) are adequately represented. The team will then continue to recruit LEAs, HS centers, elementary schools, HS lead teachers, and kindergarten lead teachers. The team will attempt to recruit 5 HS programs and collect data from 5 program directors; for each of the 5 HS programs, the team will attempt to recruit one or two LEAs; for each of the HS programs, the team will recruit 1 HS center and attempt to collect data from a total of 10 HS center directors and 10 HS lead teachers; for each of the recruited LEAs, the team will recruit 1 elementary school and collect data from one elementary school principal associated with each LEA and one kindergarten teacher associated with each participating school principal. The team believes that five HS Programs is the minimally sufficient number of cases to determine if our sampling strategy works or needs to be revised, as well as to determine if the methodology proposed is feasible.

Additional details about the sampling plan for the planned national descriptive study (which the pilot study will mimic) can be found below.

National Descriptive Study

Creation of the Sample Frame

The sample frame, or the list of HS programs considered eligible to be sampled for the study, will include all HS programs—that is, grant recipients and delegate agencies that directly provide HS services to children who will be transitioning to kindergarten the next school year and that have established relationships with their local LEAs (per the PIR). Programs that provide only Early HS services will be excluded since they do not engage in kindergarten transitions. The study team will use three sources of administrative data to create the sample frame and to determine which HS programs to sample for the study:

1. The HS Program Information Report (PIR), ACF’s source for administrative data on HS.⁹ The study team will use the latest available PIR to develop the sample frame. Given the PIR publication schedule, the HS2K team will likely use the PIR data published for programs operating during the prior program year (2025-2026).

2. Grant recipient and delegate agency locations and contacts obtained from the Active Centers file downloaded from the HS Enterprise System (HSES).

3. A list from the Office of HS (OHS) with the Agency ID for current grant recipients, the 2025-2026 grant number associated with that account, and the current (2026-2027) grant number.

The team will start with the PIR and combine grant numbers that are associated with the same HS program (i.e., some HS programs have two or more grant numbers in the PIR), then link the grant numbers in the PIR and the grant recipients’ and delegate agencies’ locations (which will range from a single site in one state to multiple sites across multiple states) to create a list of HS programs and HS centers operating in the 2026-2027 program year.

Sampling Procedure

The sample frame will include HS programs that provide only HS services, or both Early HS and HS services, and will exclude HS programs that do not have an established relationship with a LEA. The sample frame will include all eligible grant recipients and delegate agencies that directly provide HS services. There are four reasons that programs may be excluded from the sample frame.

1. Grant recipient does not operate programs directly: Because the study is intended to provide information about kindergarten transitions, the sample frame would exclude grant recipients that do not operate any HS programs directly. Agencies that are awarded HS grants may provide direct services or delegate service delivery to other entities. There are five classifications used in the HS PIR: 1) Grant recipient that directly operates program(s) and has no delegates; 2) Grant recipient that directly operates programs and delegates service delivery; 3) Grant recipient that maintains central office staff only and operates no program(s) directly; 4) Grant recipient that delegates all of its programs; it operates no programs directly and maintains no central office staff; and 5) Delegate agency. The sample frame would exclude classifications 3 and 4 from the sample. Because delegate agencies are included in the sample frame, the sample frame would include all agencies that operate HS programs.

2. Grant recipient does not have a relationship with a LEA: The sample frame would exclude grant recipients or delegate agencies that do not have an established relationship with a LEA per the PIR (using PIR questions D.7 and D.7.b, which ask whether the recipient has a formal relationship with an LEA and if so, how many LEAs). An existing relationship is necessary to answer the study’s research questions, which focus on cross-system kindergarten transition practices, policies, professional support, and perspectives.

3. Early HS programs (EHS) : The sample frame would exclude EHS programs since the focus of this study is on kindergarten transitions and EHS programs do not serve children who are transitioning to kindergarten the next year. As such, the HS program must serve children who are transitioning to kindergarten after they complete the current program year.

4. Transitional management: The sample frame would exclude HS programs under transitional management. The Community Development Institute (CDI) runs these programs and are listed as the grantee in the PIR.

The sampling unit would be HS programs as defined above, located in the 50 states, the District of Columbia, Puerto Rico, and other U.S. Territories. The sample frame will be used to draw a nationally representative sample of 90 HS programs (the selected sample). The study team will select this sample using stratified random sampling. The study team will stratify the sample frame of HS programs in the 50 states and the District of Columbia using three characteristics – agency type, rurality, and program type, as described further below – and then randomly select programs within each stratum. HS programs in the U.S. Territories will be included as a separate stratum, and will not be stratified by agency type, rurality, and program type. The selection of characteristics used to stratify the programs was based on knowledge gleaned from case studies conducted in earlier stages of the HS2K project and consultations with OPRE and OHS. These strata are defined as follows:

• Agency type: OHS categorizes HS programs into one of seven agency types: Community Action Agency (CAA), school system, charter school, private/public non-profit (non-CAA), private/public for-profit, government agency (non-CAA), or tribal government or consortium (AIAN).¹⁰ In the HS2K case studies, they learned that co-location of HS programs within elementary schools was associated with differences in kindergarten transitions.¹¹ This co-location is most often observed among the school system and charter school agency types, compared to all other HS program agency types¹²

• Rurality: The number and density of LEAs and elementary schools that could coordinate kindergarten transitions with a HS program may vary based on whether they are located in rural or urban settings. The study team will categorize HS center-based programs based on rurality using the 6-point Urban-Rural classification based on the 2013 National Center for Health Statistics (NCHS) Urban-Rural Classification Scheme for Counties.¹³ From those descriptors, the study team would create a 2-point rurality indicator (where large central metro, large fringe metro, medium metropolitan, and small metropolitan areas are classified as non-rural, and nonmetro micropolitan and nonmetro noncore areas are classified as rural). The study will sample programs from each of these two categories such that the distribution of HS programs by rurality is nationally representative. Programs in U.S. Territories do not have NCHS classifications and cannot be categorized using this methodology and since very few HS programs are expected to be sampled in the U.S. territories, HS programs in the U.S. territories will be included as a separate stratum.

• Program type: To ensure the inclusion of Migrant and Seasonal HS programs, program type will be stratified into two categories: Migrant and Seasonal HS, all other eligible HS programs including AIAN HS.

For HS programs in the 50 states and the District of Columbia, the sample frame will be stratified into 8 strata (school system/all other agency types, rural/non-rural, Migrant and Seasonal HS vs other HS programs), and HS programs in U.S. Territories will be classified into a separate stratum. Thus, the sample frame of HS programs will be stratified into a total of nine strata, and a total of 90 HS programs will be sampled using a stratified random sample to provide a nationally representative sample of HS programs. Region will not be used as a stratification variable; however, the study team will systematically sort each stratum by region to help ensure geographic representation across the United States.

If any of the sampled HS programs have known barriers to participation, such programs will be replaced with a sampled HS program from the same stratum prior to beginning data collection. Barriers include Institutional Review Board (IRB) review periods that do not fit within the data collection timeline. Additionally, the study team anticipates that HS programs from two agency types—tribal governments or consortiums (AIAN) and school systems—will have additional tribal or school district-level research approval processes that will need to be completed before fielding any of the six HS2K surveys. The team expects the approval processes to be lengthy and might not be completed in a time frame that aligns with the HS2K study timeline.

For each sampled HS program, the study team will identify LEAs (after excluding any LEAs with known barriers for recruitment) that are associated with the HS program and sample up to 2 LEAs, for a total of up to 180 LEAs. If all LEAs associated with a sampled HS program have known barriers to recruitment, the HS program will be replaced.

For each sampled LEA, the team will sample an associated HS center (up to 180) and one HS lead teacher (up to 180) who serves children transitioning to kindergarten per sampled HS center.

The team will also identify one elementary school per sampled LEA that is most geographically proximate to the sampled HS Center and randomly sample one kindergarten lead teacher per sampled elementary school (up to 180 HS centers and 180 kindergarten lead teachers.)

The sample frames for the HS Lead Teacher and Kindergarten Lead Teacher surveys will be created using lists of staff provided by HS Center Directors and Elementary School Principals in their respective surveys. The total number of HS programs (N=90) was chosen because it is sufficiently statistically powered to produce a nationally representative sample of HS programs (i.e., our project goal). The sample sizes for the other five respondent groups flow down from the HS program sample.

B3. Design of Data Collection Instruments

Development of Data Collection Instruments

Each set of survey instruments aims to collect unique, but complementary, information about how HS and K-12 systems structure kindergarten transition supports and their cross-system collaboration efforts. Table B3 aligns the project objectives with the survey instruments (per section B1).

The objective for each survey is to obtain information about the types of approaches to and supports for the kindergarten transition within each system, as well as the degree of collaboration between HS and LEA systems. The surveys all include a focus on kindergarten transition perspectives, policies, professional supports, and practices (i.e., the 4Ps). These 4Ps were identified under the previous HS2K contract as core mechanisms that shape a child’s transition experience, highlighting the way the HS and LEA systems work both independently and collaboratively to support kindergarten transitions¹⁴. Perspectives refer to visions, values and beliefs participants may have regarding kindergarten transitions. Policies refer to explicit documentation of organization regulations, roles, standards, procedures and guidance. Professional supports refer to learning opportunities, professional development, and relationship building participants may receive to support their work surrounding transitions. Practices refers to concrete activities designed to support the kindergarten transition. For each P, items were developed that address each role’s experience both within and across systems.

The surveys were drafted to capture alignment within systems (vertical) and across the two systems (horizontal). Thus, to meet the study objectives, there is intentional overlap across many survey items to examine each domain of interest at the three levels within each system and to triangulate findings across respondents during analysis (i.e., HS program directors and LEA administrators, HS center directors and elementary school principals, and HS lead teachers and kindergarten lead teachers).

As noted, survey items aim to gather information on kindergarten transition policies, professional supports, practices, and perspectives. We use multiple items for each of the sections to capture the complexity of each of the Ps. Additional items in these surveys capture demographic characteristics of the respondent and program, and barriers and facilitators to implementing coordinated kindergarten transition practices, policies, and professional supports.

Instrument development was initiated during the foundational HS2K project¹⁵ which included a design options task for a national descriptive study¹⁶. Instrument development has continued during this project. The four previously developed surveys were updated, and two teacher survey instruments are newly developed for the national descriptive study. Many of the survey items included in the instruments are newly developed to capture concepts that were not addressed in previous large-scale data collections. However, some items were modified or adapted from existing data sources including HS Family and Child Experiences Study (FACES) (OMB # 0970-0151); National Center for Early Development and Learning (NCEDL) Kindergarten Transition Project; HS Program Information Report (PIR, OMB #0970-0427); Pre-Elementary Education Longitudinal Study (PEELS) (OMB #1850-0809); National Center for Early Development and Learning’s (NCEDL) Kindergarten Transitions Project (and related adaptations); Pre-Kindergarten Transition Practices Survey (OMB # 1820-0616); Teachers’ Perceptions on Transition (TPOT); and National Early Intervention Longitudinal Study (NEILS). For example, we used items from FACES for the demographic and background questions that would be relevant to the participants, programs, and schools in our sample. We also adapted the initial list of transition practices centers and schools may engage in from TPOT, FACES, and the Pre-Kindergarten Transition Practices Survey.

The HS2K team conducted cognitive interviews on some of the program and administrator surveys developed under the first HS2K contract, and engaged in expert consultation on those items’ development. Under the current contract, we have adapted those surveys based on findings from the multi-case study that was conducted concurrent with the survey design work on the first HS2K contract. We are cognitively testing those refined surveys to inform refinements to survey questions. These pre-testing activities asked the same question of fewer than 10 individuals.

B4. Collection of Data and Quality Control

Who will be collecting the data (e.g., agency, contractor, local health departments)?

The HS2K team will collect data from HS Program directors, LEA administrators, HS Center Directors, elementary school principals, and HS and Kindergarten lead teachers. The pilot study will mimic the national descriptive study’s survey administration plan, at a smaller scale, to determine if the proposed methodology is effective.

What is the recruitment protocol?

HS Programs and LEAs: Recruitment will occur in multiple stages for each sample type. After the HS2K team samples HS programs and LEAs, they will start by recruiting the HS programs into the study. Recruitment will occur through a series of mailings (i.e., initial letter, postcard, and follow-up letter), emails and phone calls to explain the study and gain their cooperation. Utilizing multi-modes helps to ensure that the sample member receives survey materials, as oftentimes either the mailing address or email is out of date. Once a HS program agrees to participate, recruitment of the associated LEAs will begin through a series of similar mailings and phone calls. Additionally, HS programs will assist in recruitment of the LEAs by notifying their key contact person at the LEA about the study and letting them know that NORC will be contacting them about their participation. Recruitment materials for each sample can be found in the appendices (Appendix B: HS Program Directors, Appendix C: LEA administrators, Appendix D: HS Center Director, Appendix E: Elementary School Principal, Appendix F: HS Lead Teacher, and Appendix G: Kindergarten Lead Teacher.)

HS Program administrators will complete the HS Program Administrator survey and answer a series of questions about both of the sampled LEAs. This will increase the likelihood that a matched response across the HS Program and LEA administrator level will be obtained. Both LEAs (should they both choose to participate), will complete the LEA administrator survey using the same HS Program as the survey referent (they will answer questions specifically about their coordination with the same HS program.)

HS Centers and Elementary Schools: Once the HS Program is successfully recruited, the HS2K team will begin recruitment at the HS Center level. Likewise, once the LEA agrees to participate, they will begin recruiting one elementary school associated with the sampled HS Center. The HS Center Director will complete their survey using the sampled Elementary school as the survey referent. The Elementary school principal will complete their survey using the sampled HS center as the survey referent.

Recruitment for both of these respondent groups will also occur through mail, email, and phone prompts. The HS2K team will also plan to work with the HS Program and LEA respondents to help gain cooperation from the HS Center and elementary school level. Respondent materials included in the appendix of SSA include draft email text that sample members can use to discuss the study with other sample member groups.

HS Lead Teachers and Kindergarten Lead Teachers: As previously mentioned, HS lead teachers and Kindergarten lead teachers will be sampled from a roster of teachers provided by the HS Center and elementary school respondents. Teachers will be sent a similar series of mailings and will be prompted by phone and email (if email is available.) If necessary, the team will work with the HS center director and elementary school principal to gain teacher cooperation to complete the survey.

What is the mode of data collection?

While the data collection approach for each sample type is customized based on their unique characteristics and related obstacles, all share a broad design that is based on NORC’s previous successful methodologies. The overall data collection approach will use gaining cooperation strategies which conserve costs, increase response rates, and minimize overall respondent burden. The HS2K team will use a mixed-mode strategy escalating from low cost, less intrusive modes to higher cost modes requiring more contact with respondents. The HS2K team will conduct as many interviews as possible through self-administered web-based surveys and over the phone with telephone interviewers (TIs) as needed.

Data collection will begin with first gaining cooperation with the HS program. The HS2K team will utilize a carefully constructed three-step mail outreach sequence for the entire sample based on the Dillman method (Dillman, Smyth, & Christian, 2014). This sequence will begin with a mailing of the official survey invitation that introduces the study to the respondent. This letter will describe the study, highlight the importance of the study, and explain how sample members were selected. It will also provide recipients with the necessary information to access the web survey. A survey invitation will also be sent via email with the same information. The initial invitation will be followed by two additional mailings: 1) a postcard approximately two weeks later to any non-respondents reminding them of the survey and requesting participation and 2) a follow-up letter sent after another two weeks that stresses the importance of their response and provides the web access information again. Approximately two weeks after the initial mailing TIs will begin calling and emailing HS programs to gain their cooperation if they have not already completed the survey. Once cooperation of the HS program is gained (at least indicating that they plan to complete the survey), the team will utilize the same sequence of mailings (with slightly altered text) to recruit the LEAs and HS center directors. Once the LEA has agreed to participate, the HS2K team will reach out to elementary school principals to complete their survey, again using a combination of mailings and phone recruitment/prompting.

Given that HS Programs will agree to participate at different times during the data collection period, Mailings will be sent in batches as HS Programs agree to participate (though may not yet have completed the survey) throughout the data collection period. Every two weeks the team will review which HS programs have agreed to participate and send a new batch of initial invitation letters out to LEA administrators and HS Center Directors. Likewise, the HS2K team will also review which HS Centers and elementary school principals have completed in the past two weeks, and send the sampled teachers invitation letters to participate.

TIs will contact non-responders by phone and emails as necessary to gain cooperation and address any respondent questions or concerns, encouraging them to complete the web survey, or completing the survey with the sample member if they are unable to do so themselves. TIs will also collect updated contact information such as phone numbers and email addresses, when possible, to be utilized for further prompting if necessary.

Informed by surveys of similar scope, the HS2K team will employ tablet technology to support TI phone prompting for HS2K survey recruitment and completion. This will be particularly beneficial in allowing interviewers to email web survey URLs while talking with sample members, and for using their tablets to deliver technical support to respondents to complete the survey.

NORC will utilize NORCSuite Mobile (NS-Mobile), a mobile version of CM-Field, a NORC proprietary case management system (CMS) to support data collection operations. This system can integrate with multiple databases and data collection tools and is designed to operate securely from remote locations. The HS2K team will customize NS-Mobile to meet the needs of the data collection, displaying information not only about the selected sample member and related contact information, but also about the survey status (including breakoffs), historical call records, and integrated prompting activities (e.g., email prompting). For example, when prompting a sample member, the interviewer will know if the respondent has accessed the web survey and, if so, how much of the survey has been completed. The interviewer can use this information to customize outreach and facilitate survey completion. The team will leverage its data management capacity to support timely access to project case management and cost data, enabling better informed project management decision making. This system will help to ensure that interviewers work cases efficiently and effectively.

How are the data collection activities monitored for quality and consistency (e.g., interviewer training).

A series of consistency and range checks will be built into the surveys to prevent invalid responses from being recorded. The HS2K team will thoroughly test programmed surveys prior to the start of data collection. This testing will include a review of the data to confirm that responses are being recorded as expected. Throughout data collection, they will monitor the functioning of the questionnaire to detect potential technical issues and possible misinterpretation of questions by respondents. They will produce regular data quality assurance reports that collate questionnaire data across variables of high analytic value to help identify such issues and allow us to take corrective action.

In addition to questionnaire functioning, the project monitors data collection progress carefully throughout the field period to achieve good response rates and representative data. Daily production reports will show how data collection is progressing across each respondent group and enable them to identify problem areas and take remedial action quickly when needed.

To monitor telephone interviewer performance specifically, the HS2K team will develop a series of performance metrics built from multiple data sources (e.g., item non-response rates, questionnaire completion rates, case prompting rates). These metrics, produced weekly, show where interviewers are deviating from the norm on key performance measures, thereby revealing areas where re-training or special coaching may be needed.

What data evaluation activities are planned as part of monitoring for quality and consistency in this collection, such as re-interviews?

A key methodology to be evaluated in the pilot study is utilizing respondent groups who have agreed to participate to help gain the cooperation of other sample members. Members of the project’s advisory board, consisting of HS program and center directors indicated that their counterpart on the LEA side (superintendent, elementary school principal) would be more likely to participate if they reached out to them first to let them know about the study and to expect materials from NORC about how to participate. Given this, the HS2K team will work with the HS respondent group, both at the program and center director levels to gain cooperation with the LEA respondent groups, along with our normal recruitment methods or mailings, emails, and telephone interviewer prompts.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

Whereas the pilot study results are not designed to produce statistically generalizable findings, response rates will be calculated to inform the design of the national descriptive study.

NonResponse

As findings from the pilot study are not intended to be representative, non-response bias will be qualitatively assessed to inform the design of the national descriptive study.

B6. Production of Estimates and Projections

Not Applicable for the pilot study.

B7. Data Handling and Analysis

Data Handling

Procedures to minimize errors in the data begin with designing questionnaires which will collect accurate data and which have skip patterns that minimize situations where respondents are unable to navigate appropriately. Questionnaire programming builds on this through techniques such as ranges for numeric items, presenting in words any numbers entered, and the inability of data entry of invalid codes for fixed-coded items. Variable creation and analysis also have strict quality assurance protocols with review at a specifications development stage, review of statistical programming code, and then verification of analytic output (including against comparison data when available).

Data Analysis

Not applicable for the pilot study. The goal of the pilot is to determine if our data collection procedures are sufficient to gain cooperation with each sample type. The survey data collection will not be utilized for analysis.

Data Use

The goal of the pilot is to determine if our data collection procedures are sufficient to gain cooperation with each respondent type. The survey data collection will be used for internal planning purposes for the future national descriptive study. Additionally, it may be possible that information learned through the pilot could be shared in publications for contextual information about the study design and the full descriptive study.

B8. Contact Persons

Marc Hernandez (Hernandez-marc@norc.org)

Caroline Lancaster (Lancaster-caroline@norc.org)

Ganesh Nada (Ganesh-nada@norc.org)

Lekha Venkataraman (Venkataraman-lekha@norc.org)

Kathleen Dwyer (Kathleen.dwyer@acf.hhs.gov)

Attachments

Instrument A: HS Program and LEA Administrator Survey

Instrument B: HS Center and Elementary School Principal Survey

Instrument C: HS and Kindergarten Lead Teacher Survey

Appendix A: HS2K Survey Consent Information

Appendix B: HS2K Recruitment Materials for HS Program Directors

Appendix C: HS2K Recruitment Materials for LEA Administrators

Appendix D: HS2K Recruitment Materials for HS Center Directors

Appendix E: HS2K Recruitment Materials for Elementary School Principals

Appendix F: HS2K Recruitment Materials for HS Lead Teachers

Appendix G: HS2K Recruitment Materials for Kindergarten Lead Teachers

1

2https://nces.ed.gov/ccd/elsi/

3 https://nces.ed.gov/programs/edge/Geographic/SchoolLocations

4 https://headstart.gov/about-us/article/head-start-service-location-datasets?redirect=eclkc

5 This clustered, multi-stage approach to sampling teachers is less efficient than a simple random sample, in the statistical sense that estimates from a simple random sample would be more precise (that is, they would have smaller standard errors). The approach used does not, however, affect the ability to generalize estimates to the population of interest, and it has logistical advantages (by reducing burden on programs).

6 https://headstart.gov/about-us/article/head-start-service-location-datasets?redirect=eclkc

7 https://nces.ed.gov/ccd/elsi/

8 https://nces.ed.gov/programs/edge/Geographic/SchoolLocations

9 The HS PIR is approved under OMB# 0970-0427. All grant recipients are required to submit the PIR for HS and Early HS programs each year. The PIR provides information on child, staff, and family demographics, program characteristics, and services.

10 2021-2022 Head Start Program Information Report (hhs.gov), p. 4, accessed May 26, 2022. The PIR form allows grant recipients and delegate agencies to indicate only one choice for the description of agency type.

11 Gordon, Molly F., Hafford, C., Barrows, M.R., Ehrlich Loewe, S.B., Cook, K.D., Gutwein, M., Carrazza, C., Feldman, R.C., Halle, T.G., & Kauerz, K. (2023). Understanding Cross-System Transitions from Head Start to Kindergarten: A Comparative Cross Case Study of Head Start and K-12 Partnerships, OPRE Report # 2023-247, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

12 Agency type is often, but not always, mutually exclusive in practice. Overlap is seen with Head Start programs where the grant recipient directly operates programs and delegates service delivery. For example, the grant recipient may be a non-profit organization and the delegate agency may be a school. Grant recipients with delegates will be assigned to a single agency type based on their agency type only.

13 https://www.cdc.gov/nchs/data_access/urban_rural.htm

14 Ehrlich, S. B., Cook, K. D., Thomson, D., Kauerz, K., Barrows, M., Halle, T., Gordon, M., Soli, M., Schaper, A., Her, S., & Guerra, G. (2021). Understanding Children’s Transitions from Head Start to Kindergarten: A Literature Review and Theory of Change, OPRE Report # 2021-128, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

15 Under the previous contract, (Understanding Children’s Transitions from Head Start to Kindergarten (HS2K), 2019-2023, the HS2K Team developed draft versions of four administrator surveys: 1) Head Start Program administrator, 2) LEA administrator; 3) Head Start center administrator and 4) Elementary school administrator. https://acf.gov/opre/project/understanding-childrens-transitions-head-start-kindergarten-hs2k-2019-2022

16 Stacy B. Ehrlich Loewe, Sarah Kabourek, Andrew Schaper, Kyle DeMeo Cook, Christina Padilla, Mitchell R. Barrows, Sara Amadon, Tamara Halle, Kristie Kauerz, Cristina Carrazza, Gretchen Streett, Sarah Her, Lekha Venkataraman (2022). Design Options for Head Start to Kindergarten Transitions Descriptive Study, OPRE Report #2022-203, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

11