School Pulse Panel
OMB# 1850-0969 v.13
Supporting Statement Part B
National Center for Education Statistics
Institute of Education Sciences
U.S. Department of Education
February 2024
revised May 2024
Contents
B1. Respondent Universe and Sample Design and Estimation 1
B2. Procedures for the Collection of Information 3
B3. Methods to Maximize Response Rates 6
B5. Individuals Responsible for Study Design and Performance 6
The U.S. Census Bureau will collect the School Pulse Panel (SPP) data on behalf of the National Center for Education Statistics (NCES) within the Institute of Education Sciences (IES) at the U.S. Department of Education. Data collection will be a self-administered, online survey. The survey is estimated to require, on average, 30 minutes of school staff time each month. The sampled school will be offered a reimbursement of $200 each month that they complete a survey over the course of 11 months between August 2024 and June 2025. The reimbursement will be paid out monthly in the form of a debit card. If a school district does not permit its schools to receive any form of incentive, the reimbursement will be sent to a point of contact in the district or the reimbursement will be withheld. Principals, or the school staff most knowledgeable about their school environment and experiences within the school, can complete the survey. No classroom time is involved in the completion of this survey.
The resulting data will provide aggregate estimates for public schools across the nation. The goal will be to have responses from at least 1,200 schools each month in order to report out nationally representative estimates. To achieve this, a stratified sample design will be used to select approximately 4,000 U.S. public schools. In addition, a reserve sample of 4,000 replacement schools will be selected to boost the number of responses if a notable number of schools from the initial sample do not respond. The sample is designed to provide nationally representative estimates of public primary, middle, and high schools, considering the type of locale (urbanicity) and racial/ethnic student enrollment.
The sampling frame for the School Pulse Panel is derived from the National Teacher and Principal Survey (NTPS) 2023-24 frame, which itself was largely derived from the 2021-22 Common Core of Data (CCD), the file of public schools supplied annually by state educational agencies to NCES. The sample frame used for the 2024-25 SPP was updated with the 2022-23 CCD to account for school openings and closures between survey administrations. Only public schools in the 50 states and the District of Columbia will be included in the School Pulse Panel sampling frame. A universe collection (115 public schools) from the Outlying Areas (Guam, Northern Marianas, U.S. Virgin Islands, American Samoa) will be administered as well. Certain types of schools are excluded, including closed schools, target services, adult correctional facilities, private schools, and schools with the highest grade of kindergarten or lower. Regular public schools, charter schools, alternative schools, special education schools, vocational schools, ungraded schools, virtual schools, and schools that have partial or total magnet programs are included in the frame. For sample allocation purposes, strata are defined by instructional level, and the sample is sorted to achieve a unique sort by geographic region (Census region), percent minority enrollment, type of locale (urbanicity), school district, size of school (student enrollment), charter status, and unique school identifier.
Tables 1-3 show the estimated expected distribution of the public-school sampling universe for the School Pulse Panel by school level and region (Table 1), school level and urbanicity (Table 2), and by school level, percent minority enrollment (Table 3), and school level and size (enrollment [Table 4]), respectively. At the time of submission, the update to the 2024-25 SPP frame using the 2022-23 CCD has started but has not been completed.
Table 1. Respondent universe for the SPP sample, by school level and region, based on the estimated 2024-25 School Pulse Panel Frame
Region |
Elementary |
Middle/Combined/Other |
High/Grade 9-11 |
Total |
Northeast |
8,282 |
2,956 |
3,313 |
14,551 |
Midwest |
12,343 |
4,544 |
6,045 |
22,932 |
South |
18,136 |
7,555 |
7,755 |
33,446 |
West |
13,001 |
4,859 |
5,290 |
23,150 |
Total |
51,762 |
19,914 |
22,403 |
94,079 |
Table 2. Respondent universe for the SPP sample, by school level and urbanicity, based on the estimated 2024-25 School Pulse Panel Frame
Urbanicity |
Elementary |
Middle/Combined/Other |
High/Grade 9-11 |
Total |
City |
15,468 |
5,109 |
5,955 |
26,532 |
Suburb |
17,616 |
6,438 |
5,975 |
30,029 |
Town |
5,677 |
2,979 |
3,530 |
12,186 |
Rural |
13,001 |
5,388 |
6,943 |
25,332 |
Total |
51,762 |
19,914 |
22,403 |
94,079 |
Table 3. Respondent universe for the SPP school sample, by school level and percent minority enrollment, based on the estimated 2024-25 School Pulse Panel Frame
Percent Minority |
Elementary |
Middle/Combined/Other |
High/Grade 9-11 |
Total |
0 to less than 25 |
14,113 |
5,422 |
6,566 |
26,101 |
25 to less than 50 |
11,095 |
4,584 |
4,226 |
19,905 |
50 to less than 75 |
9,280 |
3,825 |
3,663 |
16,768 |
75+ |
17,274 |
6,083 |
7,948 |
31,305 |
Total |
51,762 |
19,914 |
22,403 |
94,079 |
Table 4. Respondent universe for the SPP school sample, by school level and size (enrollment), based on the estimated 2024-25 School Pulse Panel Frame
School Size |
Elementary |
Middle/Combined/Other |
High/Grade 9-11 |
Total |
Less than 300 students |
13,629 |
5,711 |
8,446 |
27,786 |
300-499 students |
20,568 |
4,141 |
3,560 |
28,269 |
500-999 students |
16,688 |
7,883 |
4,482 |
29,053 |
1,000+ students |
877 |
2,179 |
5,915 |
8,971 |
Total |
51,762 |
19,914 |
22,403 |
94,079 |
Sample Selection and Response Rates
A stratified sample design will be used to select approximately 4,000 U.S. public schools. The sample will be designed to provide cross-sectional national estimates of primary, middle/combined, and high schools, considering the type of locale (urbanicity), percent minority enrollment, school size (student enrollment), and geographic region (Census region). Note that combined schools will be grouped with middle schools for the purposes of measurement and estimation.
There are two stages of sample selection. A base sample of 4,000 schools will be drawn as an initial stage as well as another reserve sample (second stage) of 4,000 schools with similar characteristics to our base sample. In case we do not get the necessary number of schools to participate out of the base sample, we plan to reach out to the reserve sample schools to complete the panel.
Roughly 115 Outlying Areas (OA) schools will be a part of the SPP data collection. It is planned for the collection of these schools to be a universe collection, so they are separated from the sample of stateside schools. All study operations will be identical for the collection from these schools, except that OA schools will not receive the initial letter that is sent to schools each month because, based on proximity to the mail facility, they are not delivered in a timely fashion.
Recruitment for the panel study began in February 2024, where districts that require applications and approval for their schools’ participation in the study were contacted. For those schools and districts that do not have this requirement, recruitment will begin in July of 2024. Learning from the 2023-24 SPP collection and response patterns, the goal for the 2024-25 collection will be to obtain commitments from 1,200 schools to participate in the study throughout the duration of the 11 monthly collections (roughly 30 percent response rates each month).
Sample Design for the School Pulse Panel
The main objective of the School Pulse Panel sampling design is to obtain nationally representative estimates that can be disaggregated by various school characteristics. For sample allocation and sample selection purposes, strata were defined by instructional level. In addition, geographic region (Census region), percent minority enrollment, type of locale (urbanicity), school district, size of school (student enrollment), charter status, and unique school identifier were used as implicit stratification variables by sorting schools by these variables within each stratum before sample selection. The explicit stratification and the first three implicit stratification variables (region, percent minority enrollment, and locale) are priorities for evaluation for this panel. Schools are proportionally allocated to the different sampling strata according to the overall U.S. public school population.
Weights will be attached to each surveyed school so that the weighted data will represent the population. The final weight for completed cases will be composed of a sampling base weight and adjustments for nonresponse and coverage. Nonresponse weighting adjustment cells for the SPP data will be defined by the three grade-level categories (stratum). Coverage adjustments will be made along the following school characteristics: Census region, percent minority enrollment, locale, and grade-level categories. The final, adjusted weights will be raked so that the sum of the weights matches the number of schools derived from the School Pulse frame.
Data from the Outlying Areas will be weighted separately from the main SPP sample. A simplified weighting procedure that only includes a nonresponse adjustment within the three grade-level categories will be used.
A set of 50 replicate groups will be created by sorting the sample cases in their order of selection, then systematically dropping out 1/50 of the sample cases. For the first replicate group, records 1, 51, 101, 151, … are dropped. For the second replicate group, records 2, 52, 102, 152, … are dropped, etc. A factor of 50/49 is then applied to the remaining sample cases, and the full weighting procedure is repeated to create a set of 50 jackknife replicate weights to be used for variance estimation.
Standard errors of the estimates will be estimated using jackknife replication. Replicate codes that indicate the replicate group to which each sample unit belongs will be developed, as will the weights for all replicates that were formed in order to calculate variances.
Special contact districts require that a research application be submitted to and reviewed by the district before they will allow schools under their jurisdiction to participate in a study. Districts are identified as “special contact districts” prior to data collection because they were flagged as such during previous cycles of the Schools and Staffing Survey (SASS), NTPS, the School Survey on Crime and Safety (SSOCS), or the 2023-2024 SPP. Special contact districts are also identified during data collection when districts indicate that they will not complete the survey until a research application is submitted, reviewed, and approved.
Once a district is identified as a special contact district, basic information about the district is obtained from the NCES Common Core of Data (CCD). The basic information includes the NCES LEA ID number, district name, city, and state. The next step is to search the district’s website for a point of contact and any information available about the district’s requirements for conducting external research. Some districts identified as being a special contact district from the previous cycle may be incorrect and staff will verify whether a given district has requirements for conducting external research before proceeding.
The following are examples of the type of information that will be gathered from each district’s website to prepare a research application for submission to this district:
Name and contact information for the district office or department that reviews applications to conduct external research, and the name and contact information of the person in charge of that office;
Information about review schedules and submission deadlines;
Whether application fees are required, and if so, how much;
Whether a district sponsor is required;
Whether an online application is required, and if so, the link to the application, if possible;
Information about research topics and/or an agenda on which the district is focusing;
The web link to the main research department or office website; and
Research guidelines, instructions, application forms, District Action Plans, Strategic Plan or Goals, if any.
Recruitment staff will contact districts by phone and email to obtain key information not listed on the district’s website, (e.g., requirements for the research application, research application submission deadlines, etc.). OMB approval to contact these districts was obtained in January 2024 (OMB#1850-0969 v.12)
SPP staff developed a generic research application that covers the information typically requested in district research applications. Staff will customize the generic research application to each district’s specific requirements that need to be addressed or included in the research application (e.g., how the study addresses key district goals, or inclusion of a district study sponsor), or submit the generic application with minimal changes to districts that do not have specific application requirements.
Using the information obtained from the district website or phone or email exchanges, a district research request packet will be prepared. Each research application will include the following documents, where applicable:
District research application cover letter;
Research application (district-specific or generic, as required by the district);
Study summary;
FAQ document;
Special contact district approval form;
Participant informed consent form (if required by the district);
SPP Project Director’s resume;
Copy of questionnaires; and
Application fee (if required by the district).
Where required or requested, applications will include draft 2024-25 SPP questionnaires. The SPP content that most closely matches the 2024-25 SPP that has been fielded in 2023 may be provided as an exemplar of questionnaires to be administered in school year 2024-25. Additionally, SPP staff will recommend district staff to reference this package to review draft items provided in the item bank in Attachment B. Other information about the study may be required by the district and will be included with the application or provided upon request.
Approximately one week after the application is submitted to the district (either electronically or in hard copy, as required by the district), SPP district recruitment staff will contact the district’s research office to confirm receipt of the package and to ask when the district expects to review the research application and when a decision will be made. If additional information is requested by the district (e.g., the list of sampled schools), recruitment staff will follow up on such requests and will be available to answer any questions the district may have throughout the data collection period.
Some districts charge a fee (~$50-200) to process research application requests, which will be paid as necessary. Special district operations began by contacting up to 100 “certainty” special contact districts for which, due to their size, it was certain that at least one school from their jurisdiction would be randomly sampled. Other special contact districts were contacted after the sample was drawn in spring of 2024.
The school and district precontact notification includes mailing a two-sided, full-color letter to sampled schools to introduce the survey, to promote survey recognition, and to verify the school’s mailing address.
A school screener survey will be sent via mail and email to principals to obtain point of contact information for the delivery of the monthly data collections. This information will collect principal information, designated point of contact information, and reimbursement mailing address information.
The School Pulse Panel data collection will begin in August of 2024. The U.S. Census Bureau, acting as a contractor for NCES, is responsible for the data collection operation. For each month, each sampled school will be sent a letter and an email notifying them of the survey, which will include information necessary to complete the online questionnaire. Respondents will have a two-week window to respond to the survey. Reminder emails will be sent during the data collection window. Schools in the Outlying Areas will not receive the initial letter.
The SPP estimates will be released on the SPP dashboard (https://nces.ed.gov/surveys/spp/)
within 6-8 weeks after the end of each monthly data collection. Due to the expedited development of this study and release of data, NCES considers the SPP data collections as experimental. Experimental data products are innovative statistical tools created using new data sources or methodologies. Experimental data may not meet all of NCES’s quality standards but are of sufficient benefit to data users in the absence of other relevant products to justify release. NCES clearly identifies experimental data products upon their release. A nonresponse bias analysis will be conducted at the conclusion of the 2023-24 collection to determine if substantial bias is found as a major concern.
Drawing the Sample
The sample of schools was drawn in spring of 2024. Many districts (known as “special contact districts”) require research applications to be submitted and reviewed for approval before contact can be made with schools within their districts. Special contact district outreach was cleared in January 2024 (OMB# 1850-0969 v.12) and recruitment began in February 2024.
School Communication
The School Pulse Panel will be conducted via a self-administered web-based survey instrument. A clerical operation prior to data collection will be conducted to obtain e-mail addresses for all the sampled school principals, and these e-mail addresses are used to contact the school principals for the screener and throughout the monthly data collections. For each month, an initial letter and initial email are sent requesting their participation in the study and to inform schools about reimbursements of $200 a month offered to schools over the course of the school year for their continued participation in the study. Reminder e-mails are sent throughout the two-week data collection window.
An example of the types of letters and e-mails that will be sent to school principals throughout the SPP data collection are included in Appendix A.
During the monthly data collections, Census Bureau staff will initiate phone calls with nonrespondents, reminding them to complete their questionnaire.
Refusal Conversion for Schools That Will Not Participate
If a school expresses strong concerns about participating at any time during data collection, these concerns will be directed to the Census Project Director (and possibly to NCES) for formal assurance. All materials will include the project’s toll-free number. In addition, initial emails will include information about why the participation of each sampled school is important and how respondent information will be protected.
NCES is committed to obtaining a high response rate in a short data collection window for the School Pulse Panel survey that allows for reporting national-level estimates monthly. In general, a key to achieving this is to track the response status of each sampled school, with telephone follow-up, as well as follow-up by e-mail, of those schools that do not respond promptly. To help track response status, survey responses will be monitored through an automated receipt control system. Telephone interviews will be conducted only by Census Bureau interviewers who have received training in general telephone interview techniques as well as specific training for SPP.
As part of the development of the SPP instruments, the monthly instruments will regularly undergo cognitive testing and expert review. New items proposed in this package will go through cognitive testing with 20-30 school personnel. This testing will occur in late spring and early summer of 2024. A generic package wase submitted for review and approval in April 2024, prior to recruitment for testing with 20 – 30 school personnel. For any new items that are developed that are not included in the generic package, quarterly monthly instruments with the new items will be posted for 30-day comment and may simultaneously undergo cognitive testing with 8-9 school personnel. Change requests reflecting modifications to items based on feedback from cognitive testing will be submitted and approved prior to each monthly collection.
Several key staff responsible for the study design and performance of the School Pulse Panel. They are:
Erin Tanenbaum, National Center for Education Statistics
Rebecca Bielamowicz, National Center for Education Statistics
Ryan Iaconelli, National Center for Education Statistics
Chris Chapman, National Center for Education Statistics
Andrew Zukerberg, National Center for Education Statistics
Cassandra Logan, U.S. Census Bureau
Elke McLaren, U.S. Census Bureau
Aimee Donnelly, U.S. Census Bureau
Alfred Meier, U.S. Census Bureau
Mandi Martinez, U.S. Census Bureau
Kathleen Kephart, U.S. Census Bureau
Jonathan Katz, U.S. Census Bureau
Kristin Flanagan, Westat
Jim Green, Westat
Lou Rizzo, Westat
Robert Finnegan, ETS
Laura Jerry, ETS
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Kathryn.Chandler |
| File Modified | 0000-00-00 |
| File Created | 2024-07-25 |