Emergency Memo

Memorandum United States Department of Education

Institute of Education Sciences

National Center for Education Statistics

DATE: July 27, 2021

TO: Dominic Mancini, Acting Administrator, Office of Information and Regulatory Affairs

FROM: Peggy Carr, NCES

THROUGH: Carrie Clarady, NCES

SUBJECT: Progress in International Reading Literacy Study (PIRLS 2021) Main Study Data Collection Emergency Justification and Change Memo (OMB# 1850-0645 v.16)

The primary purpose of this memo is to demonstrate the need for an emergency review under 5 CFR 1320.13(a) to continue recruiting schools for participation in a Fall 2021 data collection after substantive changes were required to an already approved and finalized data collection plan. In summer 2021, NCES was notified by the coordinating international organization the International Association for the Evaluation of Educational Achievement (IEA) that teacher questionnaire data from the United States would not be included in the Progress in International Reading Literacy Study (PIRLS) international report or international database. At the same time, IEA requested changes to the school questionnaire to more accurately solicit information about the 2020-2021 school year. The need for immediate clearance is due to the time sensitivity of this data collection, as normal clearance procedures would not allow NCES to follow the mandates set by the sponsoring international organization and make the required changes to the data collection while also respecting the timeline specified for this data collection. NCES will publish a Federal Register Notice soliciting 30 days of public comment on this collection concurrent with data collection. This memo also documents all of the changes being made to this collection to bring it in line with the standards set by the international organization.

PIRLS is coordinated by IEA, an international collective of research organizations and government agencies that create the assessment framework, the assessment instrument, and background questionnaires. The IEA decides and agrees upon a common set of standards and procedures for collecting and reporting PIRLS data, and defines the studies’ timeline, all of which must be followed by all participating countries. As a result, PIRLS is able to provide a reliable and comparable measure of student skills in participating countries. In the U.S., the National Center for Education Statistics (NCES) conducts this study, with support from U.S. Department of Education contractors, and works with the IEA to ensure proper implementation of the study and adoption of practices in adherence to the IEA’s standards. Participation in PIRLS allows NCES to meet its mandate of acquiring and disseminating data on educational activities and student achievement in the U.S. compared with foreign nations [The Educational Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. §9543].

PIRLS is an international assessment of fourth-grade students’ achievement in reading. PIRLS reports on four benchmarks in reading achievement at grade 4 (Advanced, High, Medium, and Low) and on a variety of issues related to the education context for the students in the sample, including instructional practices, school resources, curriculum implementation, and learning supports outside of school. Since its inception in 2001, PIRLS has continued to assess students every 5 years (2001, 2006, 2011, and 2016), with the next PIRLS assessment, PIRLS 2021, being the fifth iteration of the study. Participation in this study by the United States at regular intervals provides data on student achievement and on current and past education policies and a comparison of U.S. education policies and student performance with those of the U.S. international counterparts. In PIRLS 2016, 58 education systems participated. The U.S. will participate in PIRLS 2021 to continue to monitor the progress of its students compared to that of other nations and to provide data on factors that may influence student achievement.

In preparation for the PIRLS 2021 main study, all countries were asked to implement a field test in 2020 in order to evaluate new assessment items and background questions, to ensure practices that promote low exclusion rates, and to ensure that classroom and student sampling procedures proposed for the main study are successful. In selecting a school sample for this purpose, it is important to minimize the burden on schools, districts, and states, while also ensuring that the field test data are collected effectively. PIRLS staff will work to help respondents understand the study’s value relative to the burden imposed, and to ensure a high level of school participation.

Data collection for the field test in the U.S. occurred from March 1 through April 15, 2020 and involved a sample of 45 public schools and about 1,650 students (selecting two classes from each school). The U.S PIRLS 2021 main study involves a nationally-representative sample of 290 schools and about 9,280 students. Main study data collection was originally scheduled to be completed in Spring 2021, but due to the COVID-19 pandemic the main study has been delayed and will be conducted from September through October 2021.

The submission describing the overarching plan for all phases of the data collection, including the 2021 main study, and requesting approval for all activities, materials, and response burden related to the field test recruitment was approved in April 2019 with a change request in September 2019 (OMB# 1850-0645 v.11-12), while the submission describing all aspects of the field test and recruitment for the main study was approved in October 2019 (OMB# 1850-0645 v.13). The submission for all aspects of the PIRLS 2021 main study, including data collection activities, with an accompanying 30-day public comment period was approved in May 2020 (OMB# 1850-0645 v.14) with a change request in February 2021 (OMB# 1850-0645 v.15).

As previously mentioned, in summer 2021, NCES was notified by the IEA that teacher questionnaire data from the United States would not be included in the PIRLS international report or international database. IEA also requested changes to the school questionnaire to solicit information about the 2020-2021 school year. The exceptional circumstances of the 2021 PIRLS administration in the United States and these other countries (assessing fifth-grade students at the beginning of the academic year rather than fourth-grade students at the end of the academic year) present challenges for reporting and interpreting some PIRLS questionnaire data. This issue impacts other Northern Hemisphere countries administering the PIRLS teacher questionnaire to the teachers of fifth grade students in the fall of 2021. Due to the exclusion of teacher questionnaire data from international reporting and limitations in its use for national analysis, the U.S. PIRLS 2021 administration will no longer include a teacher questionnaire component. In accordance with the IEA’s guidance, the school questionnaire has been modified to more adequately characterize the impact of the pandemic on students in countries assessing students at the beginning of fifth grade rather than at the end of the fourth grade. Note, for example, that the school questionnaire now asks questions about resources available to 4^th grade students. The aim is to evaluate students at the beginning of their fifth-grade year, in light of what was available to them throughout their fourth-grade year. These changes will facilitate the inclusion of U.S. data in international reports that include findings from the school questionnaire.

The following revisions were made to the documents of this package.

Global changes:

• All references to teacher questionnaire administration in the main study have been removed

  • References to school and teacher questionnaires have been updated to refer to only schools

    • School administrators and teachers of the selected classrooms will also be asked to complete questionnaires.

  • Descriptions of data collection procedures related to the teacher questionnaire have been removed, including references to procedures for collecting teacher information and student-teacher linkage information.

    • School coordinators will be asked to provide school information, class lists, and student lists along with associated teacher information through the e-filing system.

  • All references to the teacher payment have been removed

    • Teachers will receive $20 as a thank you for completing the questionnaire.

Part A

Introduction (p.3)

With the COVID-19 pandemic and under the guidance from IEA, this submission request is to update changes in the sampling population and data collection materials related to the changes. The submission to update COVID-19 related changes in the sampling population and data collection materials related to the changes was approved in February 2021. Due to the COVID-19 pandemic and under guidance from the IEA, this submission includes further changes associated with the cancellation of the teacher questionnaire and modifications to the school questionnaire, and modifications recruitment and data collection materials. The supporting statements Parts A and B describe all aspects of the study; Appendices A1 and A2 provide the main study communication materials and additional materials related to the data collection activities, respectively; Appendix B provides the non-response bias analysis plan for the main study; Appendix C provides the questionnaires for the Main Study. In the case that the final approved U.S. adapted versions of the PIRLS 2021 main study questionnaires differ from those in this submission, the final versions including any updates for the Main Study will be submitted to OMB for approval as a change request in May 2021.

A.2 Purposes and Uses of Data (p. 4)

Teacher Questionnaire. Reading teachers of students in selected classes will be asked to complete a teacher questionnaire, which will include questions about teacher preparation and experience, reading instruction, instructional resources and technology, instructional time, instructional engagement, and classroom assessment. These questionnaires will be offered online, with a paper-and-pencil backup. Although the Teacher Questionnaires were administered in the PIRLS 2021 Field Test, they will not be included as part of the Main Study administration in Fall 2021.

A.3 Improved Information Technology (Reduction of Burden) (p. 5)

The process of collecting teacher and student-teacher linkage information has been streamlined to improve user flexibility and efficiency. Teacher information is now collected through the MyPIRLS website on the “Submit Class List” page during the e-filing process. On that page, school coordinators are asked to enter a complete and current list of all of their school’s fifth-grade classes, including the student roster for each listed class as well as the name and email address of the associated reading teacher. Excel templates of the student list are posted on the “Submit Student List” page for the school coordinators to provide student information and link teachers to the students of each class listed on the “Submit Class List” page. This data collection procedure eliminates the need for the previously used Student-Teacher Linkage Form (STLF). Furthermore, information about the associated teachers is securely stored in Westat’s e-file system, which is connected to the database for the MyPIRLS website and is updated with teacher questionnaire participation status on a regular basis, making it easy and efficient for test administrators to track questionnaire status, and eliminating the need for the Teacher Tracking Form (TTF). This approach centralizes the information to be shared with the school coordinators in one secured online location through the MyPIRLS website, rather than having multiple forms the school coordinators have to verify and confirm at different stages of the data collection phase.

A.9 Payments or Gifts to Respondents (p. 7)

In the PIRLS Field Test, teachers were Teachers will be offered a $20 Amazon gift card for completing the PIRLS teacher questionnaire; in the PIRLS main study, there will be no teacher questionnaire. Historically, participation is high among school administrators without offering incentives; therefore, no incentive will be offered for completion of the school administrator questionnaire.

A.12 Estimates of Burden (p. 10)

Based on the estimated hourly rates for principals/administrators, school coordinators, teachers, and parents of $46.85, $29.25, $29.25, and $24.34, respectively², and the federal minimum wage of $7.25 as the hourly rate for students, and based on the estimated total of 8,008 7,024 burden hours for PIRLS main study data collection, the associated total estimated respondent burden time cost is $92,316 $77,654.

Table A.1. Burden estimates for PIRLS 2021 Field Test and Main Study.

Activity	Sample size	Expected response rate	Number of respondents	Number of responses	Per respondent (minutes)	Total burden (hours)
Field Test Recruitment
Contacting Districts	45	1	45	45	10	8
Contacting Schools	45	1	45	45	20	15
District IRB Staff Study Approval	5	1	5	5	120	10
District IRB Panel Study Approval	30	1	30	30	60	30
Parental notification	1,650	1	1,650	1,650	10	275
Total Field Test Recruitment			1,775	1,775		338
Field Test Data Collection
Student
Assessment digitalPIRLS	1,650	0.85	1,403	1,403	80	1,871
Directions	1,650	0.85	1,403	1,403	10	234
Student Questionnaire	1,650	0.85	1,403	1,403	40	936
School Staff
School Administrator	45	0.95	43	43	40	29
Teacher (2 per school)	90	0.95	86	86	40	57
School Coordinator	45	1	45	45	240	180
Total Field Test Data Collection			1,577	2,980		1,436
Total Field Test			3,352	4,755		1,774
Main Study Recruitment
Contacting Districts	290	1.00	290	290	10	49
School Recruitment (Original Schools)	290	0.70	203	203	20	68
School Recruitment (Replacement Schools)	80	0.50	40	40	20	14
District IRB Staff Study Approval	30	1.00	30	30	120	60
District IRB Panel Study Approval	180	1.00	180	180	60	180
Parental notification	11,600	0.80	9,280	9,280	10	1,547
Total Main Study Recruitment			10,023	10,023		1,918
Main Study Data Collection
Student
Assessment digitalPIRLS	7,280	0.85	6,188	6,188	80	8,251
Bridge study paperPIRLS assessment	2,000	0.85	1,700	1,700	80	2,267
Directions	9,280	0.85	7,888	7,888	10	1,315
Student Questionnaire	9,280	0.85	7,888	7,888	35	4,602
School Staff
School Administrator	243	0.95	231	231	35	135
Teacher (2 per school)	486	0.95	462	462	30	231
School Coordinator	243	1.00	243	243	240	972
Total Main Study Data Collection			8,824 8,362	16,712 16,250		7,255 7,024
Total Main Study			18,847 18,385	26,735 26,273		9,173 8,942
Total Burden for Field Test and Main Study			22,199 21,737	31,490 31,028		10,947 10,716
Total requested in this submission			13,375	14,778		2,939

Note: Burden in grey font represents carried over burden approved in previous submissions; all burden for this project was requested and approved in v.14 of this package. The black font represents the updated numbers for which burden approval is sought in this submission. Burden for student assessments is shown italicized font and are not included in the burden total because cognitive assessments are not subject to PRA. The totals in the “Number of respondents” column do not include duplicate counts of respondents. The total request for this submission serves as a final adjustment of burden requests for this package over submissions v.13-v.15.

A.15 Program Changes or Adjustments (p. 11)

The apparent increase in burden time is because this submission is for all aspects of the PIRLS 2021 main study data collection, while the last approved burden was for the all aspects of the PIRLS 2021 field test as well as recruitment for the main study. The main study data collection involves a much larger sample of participants who complete longer instruments. The decrease in burden time is because this submission includes the cancellation of the teacher questionnaire component of the study.

Part B

B.1 Respondent Universe (p. 1)

The PIRLS 2021 Field Test completed data collection in early spring 2020. The respondent universe for the PIRLS 2021 Field Test was all students enrolled in grade 4 with a mean age of at least 9.5 years of age during the 2019-2020 school year. While the 2021 Main Study Data Collection was originally scheduled for the parallel population in Spring 2021, due to the COVID-19 pandemic and under the guidance of IEA, the main study assessment period for the U.S. will be delayed and students will be assessed at the beginning of fall 2021 for school year 2021-2022, instead of spring 2021 for school year 2020-2021. Due to this change, the respondent universe for the PIRLS 2021 main study will be all students enrolled in grade 5 that have a mean age of at least 10.5 of age during the 2020-2021 school year. The universe for the selection is all public schools in 15 populous states for the field test, and all public and private schools across all 50 states and the District of Columbia for the main study. The field test selected a sample of 45 schools, with the goal of obtaining participation from a minimum of 40 schools. The main study will select a sample of 290 schools to be nationally representative, with the goal of obtaining participation from a minimum of 243 schools. Within sampled schools, approximately 1,650 students for the field test and 9,280 students for the main study will be selected for participation by drawing a random sample of two classes (or including all eligible students if the school has only two or less eligible classes). All selected students in the field test will participate in an electronically administered PIRLS using the digitalPIRLS assessment platform. Of the 9,280 students in the main study, a sample of 7,280 students will participate in an electronically administered PIRLS using the digitalPIRLS assessment platform, and a sample of 2,000 students will take PIRLS on paper in order to allow for a bridge analysis with paper-based trend lines. Only intact classes of grade 4 students for the field test and grade 5 students for the main study will be assessed. School administrators (and teachers in the field test only) of the selected classrooms will also be asked to complete questionnaires.

B.2 Statistical Methodology (pp. 3-5)

Main Study Sampling Plan and Sample

Because of the fall assessment schedule and under the guidance of IEA, the school sample design for the main study must be a probability sample of schools that fully represents the entire fifth grade population in the United States. At the same time, to ensure maximum participation, it must be designed to minimize overlap with other NCES studies involving student assessment that will be conducted around the same time. The main study will take place in the fall of 2021, in the following school year after the NAEP 2021 assessment. NAEP 2021 will assess several thousand schools nationally at grades 4 and 8. To be fully representative, the PIRLS grade 5 sample may include some schools that will have participated in the Main NAEP 2021 at grade 4. However, this number will be kept to a minimum. Additionally, under the guidance of the IEA, the teacher questionnaire will not be administered in the United States as teacher questionnaire data from fifth grade teachers in the fall semester cannot be meaningfully reported alongside data from fourth grade teachers in the spring semester.

In order to assess the minimum required 5,000 students from 150 schools for the digitalPIRLS plus 1,500 students from 50 schools for the bridge study paperPIRLS, we will sample 290 schools and about 9,280 students. For each original sample school, two replacement schools will also be identified. The sampling frame will be obtained from the most current versions of NCES’s Common Core of Data (CCD) and Private School Survey (PSS) files, restricted to schools having grade 5, and eliminating schools in Puerto Rico, U.S. territories, and Department of Defense overseas schools. The sample will be stratified according to school characteristics such as public/private, Census region, poverty status¹ (as measured by the percentage of students in the school receiving free or reduced-price lunch in the National School Lunch Program [NSLP]). This will ensure an appropriate representation of each type of school in the selected sample of schools. The process used to determine school eligibility, student eligibility, and student sampling is described below.

Schools will be selected with probability proportional to size (PPS) sample, where the measure of size is based on the number of estimated students at grade 5. A PPS design ensures that all students have an approximately equal chance of selection because the same sample size will be selected from each school, regardless of the size of the school. It also improves cost-efficiency by increasing the number of students per school.

Once cooperation is obtained from the schools, the class and student lists will be collected from the participating schools electronically using a secure electronic filing process (as explained in Part A). Electronic filing provides advantageous features, such as efficiency and data quality checks. Schools will access the electronic filing system through the secure MyPIRLS website. School coordinators will be asked to provide school information, class lists, and student lists along with associated teacher information through the e-filing system. Due to the cancellation of the teacher questionnaire, teacher and student-teacher linkage information will not be collected. Parental notification materials (i.e., Facts for Parents about PIRLS and sample parent/guardian notification letters; see Appendices A1 and A2) will be provided to the school coordinators to send to the parents or guardians of the sampled students in selected classes. The school coordinators will collect parental consent forms and submit them to the PIRLS staff.

The process of collecting teacher and student-teacher linkage information has been streamlined to improve user flexibility and efficiency and reduce burden for the data collection process. School coordinators are asked to provide teacher information through the MyPIRLS website on the “Submit Class List” page during the e-filing process by entering a complete and current list of all of their school’s fifth-grade classes, including the student roster for each listed class as well as the name and email address of the associated reading teacher. Excel templates of the student list are posted on the “Submit Student List” page for the school coordinators to provide student information and link teachers to the students of each class listed on the “Submit Class List” page. This data collection procedure eliminates the need for the previously used Student-Teacher Linkage Form (STLF). Furthermore, information about the associated teachers is securely stored in the e-filing system, which is connected to the database for the MyPIRLS website and is updated with teacher questionnaire participation status on a regular basis, making it easy and efficient for test administrators to track questionnaire status, and eliminating the need for the Teacher Tracking Form (TTF). This data collection approach centralizes the information to be shared with the school coordinators in one secured online location through the MyPIRLS website, rather than having multiple forms the school coordinators have to verify and confirm at different stages of the data collection phase.

Westat staff and the test administrators will work with the school coordinators to monitor the participation status for the school and teacher questionnaire completion through the MyPIRLS website. Automated reminder emails will be sent, as needed, to schools and teachers who have not yet begun to complete their questionnaires. School coordinators will notify the Westat PIRLS team or the test administrators in cases where principals or teachers of the sampled classes are unable to complete the questionnaire. Principals may appoint a member of school staff (typically the assistant principal or the school coordinator) to complete the school questionnaire as described by IEA in the survey operation procedures.

B.3 Maximizing Response Rates (pp. 5-6)

When a school agrees to participate in PIRLS, we request that the school designate a School Coordinator to facilitate the school’s participation. The School Coordinator will notify teachers and students in selected classrooms about PIRLS, explain the importance of participating, and explain test day activities. Additionally, the School Coordinator may be asked to encourage sampled teachers to complete the teacher questionnaire.

Our approach to maximizing teacher recruitment is to:

• Send letters and informational materials to teachers;

• Provide the option of an electronic or hard-copy questionnaire;

• Offer a $20 incentive for participation; and

• Have the test administrator speak to the teacher on the day of the student session.

Appendix A1

PIRLS 2021 Main Study School Letter [First Tier/Second Tier, Optional COVID-19 Clause]

Should there be a conflict on this date, a PIRLS representative will work with you to identify an alternative time. Within the next few weeks, I will ask you to identify a school coordinator and will provide detailed information about the assessment. PIRLS representatives will provide significant support to schools, bring all necessary materials, including tablets and all equipment, and administer the assessment.

Note: Because some contact materials were already sent to schools with the previously approved contact materials, there are now two versions of many contact materials in this package – those that mention the teacher questionnaire (already sent) and those that do not (those that will be sent to continue recruitment upon approval of this package). The contact materials marked “REVISED” at the end of Appendix A1 are new additions to the package, and they differ from their previously approved counterparts that appear earlier in Appendix A1 ONLY in the ways listed below.

REVISED PIRLS 2021 Main Study School District Letter [First Tier/Second Tier, Optional COVID-19 Clause]

REVISED PIRLS 2021 Main Study Private School District Letter [First Tier/Second Tier, Optional COVID-19 Clause]

REVISED PIRLS 2021 Main Study School Letter [First Tier/Second Tier, Optional COVID-19 Clause]

REVISED PIRLS 2021 Main Study Private School Letter [First Tier/Second Tier, Optional COVID-19 Clause]

REVISED letters omit bullet point referring to teacher payment:

• Teachers will receive $20 as a thank you for completing the questionnaire.

REVISED PIRLS 2021 Main Study Brochure Content: 8 ½ X 11in Tri-fold

Inside panels:

What is PIRLS?

The Progress in International Reading Literacy Study (PIRLS) is an international assessment and research project designed to measure reading achievement at the fifth-grade level, as well as school and teacher practices related to instruction. PIRLS was previously assessed in 2001, 2006, 2011, and 2016 with the United States participating in all past assessments. In 2021, PIRLS will involve students from more than 50 countries, including the United States. For the first time, PIRLS will be completely electronic and administered on a tablet with a keyboard. This new format for the assessment is called digitalPIRLS.

Back panels:

Other information collected by PIRLS

PIRLS is more than an assessment of student knowledge in reading. PIRLS also considers the context in which learning occurs. Students, teachers, and schools are asked about a variety of aspects of the environments in which content is taught, learned, practiced, and applied. In this way, PIRLS provides each country with a rich source of information on the factors related to reading achievement.

REVISED PIRLS 2021 Main Study FAQ

What is PIRLS?

The Progress in International Reading Literacy Study (PIRLS) is an international assessment and research project designed to measure trends in reading achievement at the fifth-grade level as well as school and teacher practices related to instruction. Since 2001, PIRLS has been administered every 5 years. PIRLS 2021, the fifth study in the series, will involve students from more than 50 countries, including the United States. For the first time, PIRLS will be completely electronic and administered on a tablet with a keyboard. This new format for the assessment is called digitalPIRLS.

How isare the teacher and school questionnaires administered?

The teacher and school questionnaires are is administered either online from a secure website or via a hardcopy form. Teacher questionnaires take about 30 minutes to complete and ask teachers questions about their experience, available resources, and instructional practices. School questionnaires take about 35 minutes to complete and ask about school practices and resources.

What are the benefits?

The nation as a whole benefits from PIRLS by having a greater understanding of how the reading knowledge and skills of U.S. students compare with students from other countries. To thank participating schools and individuals for their time and effort, we offer tokens of appreciation: schools that participate in PIRLS will receive $200 and the school coordinator (staff person designated to assist with the study) will receive $100. Teachers who complete a survey will receive $20 Amazon gift cards, and eEach student who participates will receive a small gift.

REVISED Summary of School Activities: PIRLS 2021 Main Study

IES Logo Summary of School Activities: PIRLS 2021
	Summer/Fall 2021	August-October 2021 Prior to assessment day	October-November 2021 Assessment day	Benefits
School coordinator	Confirm an assessment date convenient for your school Register on www.MyPIRLS.com and complete the Provide School Information (PSI) page	Reserve assessment location for the PIRLS sessions Confirm dates and location with Westat PIRLS staff Provide class list, student list, and contact information for grade 5 teachers through the secure PIRLS portal (www.MYPIRLS.com) Notify teachers, selected students, and students’ parents of the study and benefit of participating Work with Westat staff to identify students needing accommodations Coordinate the principal’s completion of the school questionnaire (online or hardcopy) Coordinate the teachers’ completion of the teacher questionnaire (online or hardcopy) Collect parental consent forms where required and submit to PIRLS staff	Confirm spaces for assessment are problem-free Collect completed school and teacher questionnaires (if not completed online) and give to Westat PIRLS staff Ensure all sampled students attend the assessment sessions Meet with Westat PIRLS staff and provide feedback about the assessment process	Receive a $100 personal check
Teachers of eligible courses		Complete Teacher Questionnaire and return to school coordinator prior to assessment day (if not completed online)		Represent the United States in the international study Receive a $20 Amazon gift card
Westat’s assessment staff	Work with the school to confirm assessment dates Help school coordinator with assessment details and logistics Protect school and student confidentiality	Call the school coordinator to discuss assessment day location(s) and student participation Select classroom sample and notify school of selected classes Provide the school and teacher questionnaires to the school coordinator for distribution (if not completed online)	Administer assessments from start to finish Furnish all assessment equipment or materials (e.g., tablets, pencils, and test booklets) Meet with the school coordinator to debrief at the end of the assessment Maintain security of all materials	Ensure quality and uniformity of data collected across the United States

Appendix A2

Note: Because some contact materials were already sent to schools with the previously approved contact materials, there are now two versions of many contact materials in this package – those that mention the teacher questionnaire (already sent) and those that do not (those that will be sent to continue recruitment upon approval of this package). The contact materials marked “REVISED” at the end of Appendix A2 are new additions to the package, and they differ from their previously approved counterparts that appear earlier in Appendix A2 ONLY in the ways listed below.

REVISED PIRLS 2021 Main Study Email to School Coordinator (SC)

(yellow highlight for mail merge; red for customization)

You can refer to the attached MyPIRLS Registration and Provide School Information (PSI) Guide and Instructions for Submitting Class and Student Lists to assist you through the immediate next steps, and please call us if you have any questions or need additional assistance. We can be reached at [PIRLS E-File phone number] or by email at [PIRLS E-File email]. We have also attached/linked a PIRLS Brochure and Summary of School Activities, which will provide you with an overview of PIRLS, as well as a list of upcoming tasks and a timeline for their completion.

PIRLS 2021 Update – Teacher Questionnaire Canceled

The teacher questionnaire component of PIRLS has been cancelled in the United States due to the unique circumstances of the 2021 assessment (assessing fifth-grade students at the beginning of the year rather than fourth-grade students at the end of the year).

Students will be asked to complete a reading assessment and background questionnaire. For the first time, PIRLS will be completely digital, and administered on a tablet with a keyboard that we provide. Some students will take a paper and pencil assessment in order to bridge the new digital version of PIRLS to the previous paper and pencil version. A school administrator and the teachers of selected classes will also be asked to complete a questionnaire. They will be contacted separately, and their responses will be collected through a web-based application or hard-copy survey. Please prompt the administrator and teachers to complete the questionnaire as soon as they receive it.

Attachments:

MyPIRLS Registration and Provide School Information (PSI) Guide

Instructions for Submitting Class and Student Lists

PIRLS Brochure and Summary of School Activities

REVISED PIRLS 2021 Main Study School Questionnaire Letter – for Hardcopy Mailings

We are asking you to complete a 35-minute questionnaire to provide insight into the practices and resources at your school. The survey is designed to be completed by you or a staff person you designate who can provide information about the characteristics of the school, its enrollment, resources, policies, and learning environment. This questionnaire and the PIRLS assessment were originally intended to be administered at the end of the 2020-2021 school year, but were postponed because of the pandemic. When answering questions, please use the 2020-21 academic year as reference. The questionnaire may be completed by hardcopy or online using a secure website. To access the questionnaire online, please use the web address and unique study ID and password provided in this letter. If completing the questionnaire by hardcopy, please use the enclosed business reply envelope to return the completed questionnaire to us. You may also give the completed hard copy questionnaire to the PIRLS school coordinator at your school.

REVISED MyPIRLS Registration and Provide School Information (PSI) Guide

Provide School Information (PSI)

The first step in MyPIRLS is to complete the Provide School Information page. Click the Provide School Information link at the left-hand menu to access this page.

On the PSI page, the information on the left is the current information available for your school. Please make updates in the fields on the right if the current information on the left is blank or incorrect. Make sure to complete both the “School Contact Information,” and “School Characteristics,” and “COVID-19” sections. Then click Save.

[Deleted screenshot]

[New screenshot]

REVISED MyPIRLS Provide School Information (PSI) Page

[Replaced screenshot]

REVISED Automated Email for MyPIRLS School Coordinator registrant

In September late August, you will need to provide a complete and current list of all of your school’s fifth-grade classes for the PIRLS team to randomly select two classes to participate in PIRLS. You will then be asked to submit lists of all students in the selected reading classes. We will email you in early September late August with instructions about how to submit the class list, student list, and teacher list through www.MyPIRLS.com.

REVISED MyPIRLS Home Page

1. Provide School Information—Fall 2020

2. Submit Class List—late August thru September 2021

3. Submit Student List—September 2021

4. Prepare for Assessment—September 2021

REVISED Instructions for Submitting Class and Student Lists- Grade 5

Instructions for Submitting the Class List

The PIRLS team needs a complete and current list of all of your school’s fifth-grade reading classes in order to randomly select reading classes to be assessed. Typically, two classes are sampled in each school, and all students in the selected classes are assessed. Submit your class list; then, the PIRLS team will email you the list of selected classes. After you receive the list of selected classes, you will be asked to submit a student list naming all students in the selected classes only.

1. Navigate to the Submit Class List page and form.

Go to www.MyPIRLS.com and log in. On www.MyPIRLS.com, select Submit Class List from the navigation menu on the left. The table on the page provides a description of all information that is needed. Scroll down to enter information for each fifth-grade class in the form at the bottom of the page.

[Deleted screenshot]

[New screenshot]

2. Enter class information.

1. Class Name: Each class name must be unique and descriptive, e.g. “Mrs. Johnson’s 5th grade class”. Again, each row must have a different class name.

2. Number of 5th grade students in the class: Use current and accurate counts. Do not include students in other grade levels.

3. Class Exclusion Status: Include all classes in the list, even if the students in those classes are typically excluded from your state testing program or take alternate assessments. If all students in a class should be excluded because no students in the class would be able to take the assessment with or without the allowed accommodations, you can indicate this by selecting the appropriate Class Exclusion Status code. Only use this field if every student in a class cannot take the PIRLS assessment with or without the allowed accommodations.

4. Name of Reading Teacher: Include the name of the reading teacher.

5. Email address of Reading Teacher: Include the email address for the reading teacher. We will use this information to send a questionnaire to the teacher of the selected class(es).

Table of Class List Field Descriptions

The table on the submit class list page provides a description of the information that is needed.

✶Class Name (each class name must be unique)	Record the class name that is typically used by your school to refer to the class. For example, it may be that your school uses the grade plus a letter for the class name (5a, 5b, etc.), the grade plus a number (5.1, 5.2, etc.), the teacher name, the class period (Period 1, Period 2, etc.), the class location (Room 7, Room 8, etc.), or some other combination of these items. It is important that unique class names are entered because these names will be used to inform the Test Administrator of which classes will be tested.
✶Number of 5th-grade students in each class	Enter the number of fifth-grade students in each class. In the case of multi-grade classes (e.g., students from more than one grade level in the same class), only the fifth-grade students should be counted as a class in the list. For example, if the class has 20 students including 5 fourth-graders, 7 fifth-graders, and 8 sixth-graders, enter “7” as the Number of Students in this class.
Class Exclusion Status (if applicable; if at least one 5th-grade student in this class is able to be assessed, do not use this field) Accommodations Guide	1 = Students with functional disabilities; i.e., students who have physical disabilities in such a way that they cannot perform in the PIRLS testing situation. Students with functional disabilities who are able to perform should be accommodated in the test situation, within reason, rather than excluded.  2 = Students with intellectual disabilities; i.e., students who are considered, in the professional opinion of the school principal or by other qualified staff members, to have severe intellectual disabilities or who have been tested as such. This category includes students who are emotionally or mentally unable to follow even the general instructions of the test. Students should not be excluded solely because of poor academic performance or normal disciplinary problems. It should be noted that students with dyslexia, or other such learning disabilities, should be accommodated in the test situation, within reason, rather than excluded.  3 = Non-native language speakers; i.e., students who are unable to read or speak the language(s) of the test and would be unable to overcome the language barrier in the test situation.  If all students in the excluded class do not belong to the same exclusion category, please identify the category corresponding to the majority of students.
✶Name of Reading Teacher	Name of the reading teacher for the class.
✶Email address of Reading Teacher	Email address of the reading teacher for the class.

Class List Form

Enter information for each fifth-grade class in the form at the bottom of the submit class list page. You can hover your mouse cursor over the Information symbol to get more information about each field and its requirements. The white asterisks indicate which fields are required. Enter class details; then, click Save after each row. The row will not save if an error occurs. If you see an error message, please correct the text in that row, and save it again to activate a new row. Continue to click Save and make sure to click Save after you enter the last class. Do not click Finished until after you click Save one last time and there are no error messages.

Tips:

• Use the Save button to save information you have entered for each class.

• If you need to delete a class, click the   at the beginning of a saved class row.

• Use the Print button to print a list of the classes you included for reference.

• Each class name must be unique. Do not just type in “Grade 5 class” several times. We must be able to differentiate between classes, so please use different class names for each reading class with at least one 5th-grade student.

• Click the Finished button when you have entered and saved all fifth-grade reading classes in the form.

• Go to the Submit Student List page and submit the student list if you have less than 50 fifth-grade students enrolled in your school. If you have more than 50 fifth-grade students, wait until you receive an email from us with the list of randomly selected classes.

[Deleted screenshot]

[New screenshot]

Instructions for Submitting the Student List

Along with the list of classes, PIRLS needs a complete and current list of all students in the classes that are randomly selected to participate. You can only submit the student list after you have submitted the class list.

1. Prepare the Excel file.

• Teacher Name: Reading teacher of the named class.

• Teacher Email: Reading teacher’s work email address.

It is preferred that you include column headers as the first row in your student list (see templates for examples in the Documents section on https://www.MyPIRLS.com). However, student lists without column headers will be accepted. If you cannot submit your student data with this information in an Excel file, please call or email the PIRLS Help Desk at [PIRLS E-File phone number] or [PIRLS E-File email]. Please save this file with your school name and district in the file name.

[Deleted screenshot]

[Updated screenshot]

REVISED Submit Class List Page

 

Submit Class List

✶Name of Reading Teacher	Name of the reading teacher of the class
✶Email address of Reading Teacher	Email address of the reading teacher of the class

REVISED Submit Student List Page

[Deleted screenshot]

[Updated screenshot]

REVISED MyPIRLS Prepare for Assessment Page

Please note below:

Each STF has the list of students in the selected class and their teachers.

Questionnaire Completion Statuses

Please review the completion status for each questionnaire, and prompt the administrator and teachers to complete the questionnaires as needed.

Recipient	Questionnaire Type	Status
[Principal Name]	School	[OSS Status]
[Teacher Name]	Teacher	[OSS Status]
[Teacher Name]	Teacher	[OSS Status]

REVISED Student Tracking Form (STF) Email

• Questionnaire Statuses – Please review the completion status for each questionnaire, and prompt the administrator and teachers to complete the questionnaires as needed.

REVISED Online Questionnaire Login Screen Text

(for the Principal and Teachers)

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this voluntary information collection is 1850-0645. The time required to complete this information collection is estimated to average 35 minutes per principal, and 30 minutes per teacher, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have any comments or concerns regarding the accuracy of the time estimate(s), suggestions for improving the form, or questions about the status of your individual submission of this form, write directly to: the Progress in International Reading Literacy Study (PIRLS), National Center for Education Statistics, Potomac Center Plaza (PCP), 550 12th St., SW, 4th floor, Washington, DC 20202.

REVISED School Principal Questionnaire Selection Email

This online questionnaire seeks information about schools and provides important context to understanding the achievement of students taking the assessment. [School coordinator] has been assisting with PIRLS tasks at your school and can help you if you have questions. This questionnaire and the PIRLS assessment were originally intended to be administered at the end of the 2020-2021 school year, but were postponed because of the pandemic. When answering questions, please use the 2020-21 academic year as reference. Please complete this questionnaire as soon as possible, and no later than [date]. 

REVISED School Principal Questionnaire Reminder Email

We recently contacted you regarding an online questionnaire for the 2021 Progress in International Reading Literacy Study (PIRLS). Please take a few minutes to complete this important questionnaire that will provide context to the responses we receive from your students. [School coordinator] has been assisting with PIRLS tasks at your school and can assist you if you have questions. This questionnaire and the PIRLS assessment were originally intended to be administered at the end of the 2020-2021 school year but were postponed because of the pandemic. When answering questions, please use the 2020-21 academic year as reference. Please complete this questionnaire as soon as possible, and no later than [date]. 

Appendix C

Summary of Appendix C Changes:

Note for v. 16 - Per recent guidance from the International Association for the Evaluation of Education Achievement (IEA), the U.S. has decided not to administer the teacher questionnaire for the PIRLS 2021 main study and made small changes to the School Questionnaire to reflect its new strategy. Previously approved versions of the School and Teacher Questionnaires remain in this document for reference and continuity with past and future packages and are marked "ARCHIVE ONLY".

Changes to the school questionnaire are detailed in the table below.

Details of Changes to School Questionnaire:

Previous Text	Updated Text
[excerpt from Introduction] It is important that you answer each question carefully so that the information provided reflects the situation in your school as accurately as possible. Some of the questions will require that you look up school records, so you may wish to arrange for the assistance of another staff member to help provide this information.	[excerpt from Introduction] It is important that you answer each question carefully so that the information provided reflects the situation in your school as accurately as possible. Some of the questions will require that you look up school records, so you may wish to arrange for the assistance of another staff member to help provide this information. This questionnaire and the PIRLS assessment were originally intended to be administered at the end of the 2020-2021 school year, but were postponed because of the pandemic. When answering questions, please use the 2020-21 school year as reference.
1. What is the total enrollment of students in your school as of October 1, 2021?	1. What was the total enrollment of students in your school at the end of the 2020-2021 school year?
2. What is the enrollment of fifth-grade students in your school as of October 1, 2021?	2. What was the enrollment of fourth-grade students in your school at the end of the 2020-2021 school year?
3. Approximately what percentage of students in your school have the following backgrounds?	3. Approximately what percentage of students in your school had the following backgrounds at the end of the 2020-2021 school year?
4. A. Approximately what percentage of students in your school have English as their native language?	4. A. Approximately what percentage of students in your school had English as their native language at the end of the 2020-2021 school year?
4. B. Of the students currently enrolled in your school, what percentage have been identified as limited-English proficient (LEP)/English Language Learners (ELL)?	4. B. Of the students enrolled in your school at the end of the 2020-2021 school year, what percentage had been identified as limited-English proficient (LEP)/English Language Learners (ELL)?
7. Around the first of October 2021, what percentage of students at this school were eligible to receive free or reduced-price lunches through the National School Lunch Program?	7. At the end of the 2020-2021 school year, what percentage of students at this school were eligible to receive free or reduced-price lunches through the National School Lunch Program?
For the fifth-grade students in your school:	For the fourth-grade students in your school:
11. How many computers (including tablets) does your school have for use by fifth-grade students?	11. How many computers (including tablets) does your school have for use by fourth-grade students?
14. To what degree is each of the following a problem among fifth-grade students in your school?	14. To what degree is each of the following a problem among fourth-grade students in your school?
15. In your school, are any of the following used to evaluate the practice of fifth-grade teachers?	15. In your school, are any of the following used to evaluate the practice of fourth-grade teachers?
18. By the end of this school year, how many years will you have been a principal altogether?	18. By the end of the 2020-2021 school year, how many years had you been a principal altogether?
19. By the end of this school year, how many years will you have been a principal at this school?	19. By the end of the 2020-2021 school year, how many years had you been a principal at this school?
22. Please estimate the number of weeks during the current academic year where normal primary/elementary school operations have been affected by the COVID-19 pandemic.	22. Please estimate the number of weeks during the 2020-2021 school year where normal primary/elementary school operations were affected by the COVID-19 pandemic.
23. Does your school provide remote instruction or distance learning resources for primary/elementary grades when normal school operations are affected by the COVID-19 pandemic?	23. Did your school provide remote instruction or distance learning resources for primary/elementary grades when normal school operations were affected by the COVID-19 pandemic?
24. When normal primary/elementary school operations are affected by the COVID-19 pandemic, does your school support remote learning through the following?	24. When normal primary/elementary school operations were affected by the COVID-19 pandemic, did your school support remote learning through the following?

1 High poverty schools are defined as having 50% or more students eligible for participation in the National School Lunch Program (NSLP), and low poverty schools have less than 50% of students eligible for NSLP. In addition, in the main study, which includes private schools in the sample, all private schools are classified as low poverty because no NSLP information is available.

20