Attachment 5
School Coordinator Manual
Att
How do I get more information?
For questions or concerns, contact:
PISA Help Desk
Phone: 1-888- 638-2597
Fax: 240-314-2334
For more information about PISA:
Visit the PISA website: http://nces.ed/gov/surveys/pisa
Your
help with PISA is very important to its success.
The U.S.
PISA Team and the International Project Team
appreciate your
time and effort.
Thank you!
Table of Contents
1. Confirm assessmentdate and location for the assessment sessions 2
2. Prepare List of Eligible Students 2
3. About Quality Monitors 9
Chapter 2: Before the Assessment 2
4. Receive and check materials sent by PISA 2
5. Notify teachers, students and parents of the assessment 2
6. Review and update Student Tracking Form 2
7. Give School Questionnaire to appropriate person at your school 2
8. Organize and confirm assessment plans with school staff and with Test Administrator 2
Chapter 3: During/After the Assessment 2
10. Provide Test Administrator with Student Tracking Form 2
11. Be present throughout the assessment to assist Test Administrator (if possible) 2
Thank you for agreeing to be a School Coordinator for PISA. As your school has been selected to represent the United States in this large-scale international assessment program, you will play a very important role in making sure that the assessment is carried out smoothly and in a standard way. In doing so, you will contribute significantly to the PISA objectives of improving the quality of schooling for students in the United States. The purpose of this manual is to help you step by step to understand how you can assist with the successful implementation of this large scale international assessment program.
PISA stands for the Program for International Student Assessment, sponsored in the United States by the National Center for Education Statistics (NCES) of the U.S. Department of Education. PISA 2012 is the fifth PISA study that has been conducted since 2000 and involves more than 60 participating countries. The PISA Main Survey will be conducted in 2012 with the Field Test conducted in 2011.
PISA has the following characteristics:
It is the world’s largest international survey in education;
It surveys students aged about 15 years;
It assesses students’ preparedness for adult life; and
It measures students’ Mathematical, Scientific, Problem Solving and Reading Literacy.
Typically, between 4,500 and 10,000 students in each country take part in the Main Survey, and between 800 and 1,200 students do so in the Field Test.
Undertaking PISA is important because the results of the assessment can be used:
to indicate how well prepared students in the United States are for learning once they leave school;
by schools, education systems and governments to identify areas for improvement over time; and
to allow a comparison of student performance and the learning environment between different countries.
For more details about PISA, please refer to the information materials provided in your school’s packet or visit, http://www.pisa.oecd.org/.
Note 1: All information (or responses) you provide may only be used for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002), 20 U.S. Code, Section 9573]. |
Up to 42 students from your school will be sampled to complete a two-hour paper-based assessment consisting of mathematics and financial literacy questions. These students will also be asked to complete a student questionnaire. After the Paper-Based PISA and questionnaire, a sub sample of 20 students who participated will also participate in a one hour Computer-Based Assessment of mathematics, problem solving, and reading. In addition, schools will be requested to complete a School Questionnaire.
As a School Coordinator you will be responsible to act as the liaison between your school and a PISA representative. The key activities you will carry out are listed on the following page with their associated dates.
Please note that the actual assessment will be conducted by a Test Administrator who will contact you well in advance of the scheduled assessment day to discuss logistics and finalize assessment day plans.
Table 1: Key Activities of PISA School Coordinators
Chapter |
Date |
Activity |
Page number |
1. Preparation |
3-6 weeks prior to assessment |
|
6 |
|
|
|
6 |
|
|
|
9 |
2. Before the assessment |
2 weeks prior to assessment |
|
10 |
|
|
|
10 |
|
|
|
10 |
|
|
|
15 |
|
1-2 weeks prior to assessment |
|
16 16 |
|
A few days prior to the assessment |
|
16 |
3. During/ After the assessment |
Assessment Day |
|
17 |
|
|
|
17 |
|
Same week as the main session if possible |
|
17 |
|
Until the end of school year |
|
17 |
The following chapters will explain the details of each activity that you are expected to undertake. Should you have a question or concern at any time, please do not hesitate to contact the PISA Help Desk.
If you have any questions, please contact us at: PISA Help Desk Ph: 1-888- 638-2597 Fax: 1-240-314-2334 PISA Test Administrator: __________________________ Telephone: _____________________________________ Email: _________________________________________
|
PISA will take place between April 4, 2011 and May 13, 2011. It is important when confirming the scheduled assessment date with the Test Administrator to check that there are no other events taking place at your school that could interfere with the assessment session or with the attendance of the selected students.
The Paper-Based PISA will take approximately 3 hours and 15 minutes to administer as outlined in Table 2 below, and the Computer-Based Assessment (including preparation and administration time) will take about 1 hour 20 minutes, as outlined in Table 3. Please note that the Computer-Based Assessment session should take place after the Paper-Based PISA, and preferably on the same day. Because some students will participate in both the Paper-Based and Computer-Based sessions, it is best to schedule the Paper-Based PISA at the beginning of the school day and the Computer-Based Assessment in the afternoon. While administrative procedures, student questionnaire sections, and breaks are not timed, each section of the assessment is.
Table 2: Timing of Paper-Based PISA Administration
Activity |
Time |
Distributing the materials and reading the General Directions |
10-15 minutes (approximately) |
Assessment Section I |
1 hour (exactly) |
Short break |
No more than 5 minutes |
Assessment Section II |
1 hour (exactly) |
Short survey |
Up to 5 minutes |
Break |
No more than 10 minutes |
Student Questionnaire |
35 minutes (approximately) |
Collecting the materials and ending the session |
3-5 minutes (approximately) |
Total |
3 hours and 15 minutes (approximately) |
Table 3: Timing of Computer-Based Assessment Administration
Activity |
Time |
Student logons, passwords and introducing the test |
5 minutes (approximately) |
Computer-Based Assessment orientation session |
20 minutes (approximately) |
Completing the Computer-Based Assessment |
40 minutes (exactly) |
Ending the assessment session |
15 minutes (approximately) |
Total |
1 hour 20 minutes (approximately) |
Additional time before each session will be required to prepare the assessment room and materials prior to beginning the assessments.
Please also allow at least 1 hour after the Paper-Based PISA has finished before the beginning of the Computer-Based Assessment session. The Test Administrator will need to tend to some administrative tasks as well as prepare the Computer-Based Assessment area. Also, students participating in the Computer-Based Assessment will need a break to eat lunch after they complete the Paper-Based PISA and before they begin the Computer-Based Assessment.
Figure
1:
Example of List
of Eligible Students
School Name:__________________________________________________________________________ |
|
Address:______________________________________________________________________________ |
|
Telephone:____________________________ |
Fax:________________________________________ |
List prepared by:________________________ |
E-mail:______________________________________ |
Date list prepared: ___________________ |
Total number of students listed:________________ |
DIRECTIONS: Please list ALL students in your school in Grade 7 and higher born in 1995. |
(A) Student Name |
(B) Grade |
(C) Gender (Female=1; Male=2) |
(D) Birth Date (MM-YYYY) |
(F) SEN* |
|
Ted Brooks |
10 |
2 |
12 |
1995 |
|
Jeffrey Jenkins |
9 |
2 |
11 |
1995 |
|
Rose Matthews |
10 |
1 |
12 |
1995 |
|
Jennifer Trader |
10 |
1 |
05 |
1995 |
|
José Rodriguez |
10 |
2 |
04 |
1995 |
3 |
Roy Zastrow |
10 |
2 |
09 |
1995 |
3 |
Sam Walker |
10 |
2 |
07 |
1995 |
1 |
Julie Walters |
9 |
1 |
04 |
1995 |
|
Suzanne Smith |
10 |
1 |
07 |
1995 |
|
Fred Peters |
10 |
2 |
10 |
1995 |
1 |
Tammy Tran |
10 |
1 |
12 |
1995 |
|
Lisa McCoy |
10 |
1 |
08 |
1995 |
2 |
Melissa Holten |
9 |
1 |
09 |
1995 |
|
Joshua Bensen |
9 |
2 |
07 |
1995 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Criteria for PISA student eligibility are:
Enrolled in Grade 7 or higher; AND
Born in 1995.
PISA needs accurate information about all students at your school who meet the above criteria for student eligibility. A small number of these students may be unable to complete the PISA assessment. Do not include students who are known to be withdrawn on your list. All other eligible students must be added to your list of students. Later, you may need to identify students who cannot participate in PISA because of their SEN status. This will be covered in the next chapter.
Using the listing form provided by PISA (either hard copy from Appendix 1, or electronic sent to you via email) list students in your school that are eligible for PISA along with their demographic information.
Student name – last name, first name;
Grade – should be 7 or higher;
Gender – indicate 1 (female) or 2 (male); and
Birth Month –indicate as MM/YYYY. (For example, if an eligible student was born in September 1995, write 09/1995.)
Column F of the form is used to record the Special Education Needs (SEN) of eligible students. The codes are defined in Table 4 below. Please identify any student who has a special education need by placing the appropriate SEN code in column F.
Table 4: Codes for Special Education Needs
Description |
Code |
Functional disability – student has a moderate to severe permanent physical disability |
1 |
Cognitive, behavioral or emotional disability – in the opinion of qualified staff, student has a cognitive, behavioral or emotional disability |
2 |
Limited assessment language experience – student is not a native speaker of the language of the assessment in the country and has limited proficiency in this language |
3 |
Home Schooled – student is home schooled and attends school only for required or mandated testing. |
4 |
Test material not available – test material is not available in the student’s language of instruction. |
5 |
Examples of using Special Education Needs codes
Please refer to Figure 1: Example of List of Eligible Students, in which the following codes have been applied:
José Rodriguez is not a native English speaker and has less than one year of instruction in English. Therefore, he is coded as “3” in column F.
Roy Zastrow is not a native speaker of English. While he has received instruction in English for more than one year, he still has some difficulties with English expression and requires some extra help. Therefore, he is also coded as “3” in column F.
Sam Walker is blind and is coded as “1” in column F.
Fred Peters is physically disabled so he is coded as “1” in column F.
Lisa McCoy has been professionally assessed as cognitively delayed and is coded as “2” in column F.
All other students do not have any special education needs and column F is left blank. For most students column F will be left blank, meaning they have no Special Education Needs.
Hard copy lists can be faxed to the PISA Help Desk at 240-314-2334. Please include on your list, or your coversheet, your school’s name, address, and telephone number along with the name and contact information for the person who prepared the list and the date it was prepared. Also, to ensure that PISA has received all pages of the fax, please indicate the total number of students listed.
On the day of the assessment, PISA Quality Monitors will visit some randomly selected schools. The main task of the PISA Quality Monitors is to document the extent to which testing procedures have been followed and are being implemented by the Test Administrator. In order to do this, PISA Quality Monitors will visit some schools, observe the PISA assessment sessions and have a short interview (approximately 15 minutes) with you after the assessment session. Their task is strictly limited to these PISA related activities.
Should your school be selected for this visit, a Quality Monitor will contact you a few days prior to the assessment to inform you of his/her visit. On the morning of the assessment he/she will report to the school office. Please contact the PISA Help Desk if you have any questions about the visit.
About two weeks prior to the assessment, you will receive a package of the following materials:
the Student Tracking Form listing the students who have been sampled to participate in the assessment;
the School Questionnaire; and
Shortly after, the Test Administrator assigned to your school will contact you to verify that the package as been received and to confirm the date and time of the preassessment visit.
Please inform teachers, students (especially the sampled students), and parents of the sampled students according to the school’s policies.
It is very important to introduce the aims of PISA to school staff and students prior to the assessment in order to engage students and promote their participation. Therefore, please use the informational and promotional materials that have been provided to assist you with this activity.
The Test Administrator will collect a dated copy of the letter used to notify parents of the assessment prior to the assessment. A sample copy of this letter has been included with this manual. An electronic copy of the letter is available upon request from the PISA staff member who first contacted you or from the Test Administrator.
You will receive a Student Tracking Form in your packet of materials. This form lists sampled students along with their background data. You will need to review this form and complete it with missing or additional information. An example of the Student Tracking Form is provided in Figure 2 on the next page. The steps for completing the form are detailed on page 12.
Figure 2: PISA 2012 FT - Student Tracking Form (example after completion)
S
To
be checked by School
Coordinator
To
be
completed
by
School
Coordinator
Identifies
CBA students
K
For
use by School
Coordinator
if needed
angaroo
High School
Student Name |
Region |
Stratum ID |
School ID |
Student ID |
LST Line Number |
Grade |
Gender (F=1; M=2) |
Birth Date |
Study Prog. |
SEN |
N/P PISA |
COMMENTS |
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Ted Brooks |
00 |
01 |
010 |
00004 |
00003 |
10 |
2 |
12 |
1995 |
1 |
|
|
1 |
|
Jeffrey Jenkins |
00 |
01 |
010 |
00009 |
00008 |
9 |
2 |
11 |
1995 |
1 |
|
4 |
|
|
Rose Matthews |
00 |
01 |
010 |
00005 |
00012 |
10 |
1 |
12 |
1995 |
1 |
|
|
3 |
|
Jennifer Trader |
00 |
01 |
010 |
00002 |
00016 |
10 |
1 |
05 |
1995 |
1 |
|
|
9 |
|
José Rodriguez |
00 |
01 |
010 |
00014 |
00020 |
10 |
2 |
04 |
1995 |
1 |
3 |
3 |
5 |
|
Roy Zastrow |
00 |
01 |
010 |
00008 |
00024 |
10 |
2 |
09 |
1995 |
1 |
3 |
|
|
|
Sam Walker |
00 |
01 |
010 |
00011 |
00028 |
10 |
2 |
07 |
1995 |
1 |
1 |
3 |
4 |
|
Julie Walters |
00 |
01 |
010 |
00013 |
00033 |
9 |
1 |
04 |
1995 |
1 |
|
|
|
|
Suzanne Smith |
00 |
01 |
010 |
00001 |
00037 |
10 |
1 |
07 |
1995 |
1 |
|
|
7 |
|
Fred Peters |
00 |
01 |
010 |
00007 |
00041 |
10 |
2 |
10 |
1995 |
1 |
1 |
|
10 |
|
Tammy Tran |
00 |
01 |
010 |
00012 |
00045 |
10 |
1 |
12 |
1996 |
1 |
|
n |
6 |
Corrected her birth year, which is outside the eligibility criterion |
Lisa McCoy |
00 |
01 |
010 |
00010 |
00049 |
10 |
1 |
08 |
1995 |
1 |
2 |
3 |
2 |
|
Melissa Holten |
00 |
01 |
010 |
00003 |
00053 |
|
1 |
09 |
1995 |
1 |
|
n |
|
Corrected her grade, which is lower than the eligibility criterion |
Joshua Bensen |
00 |
01 |
010 |
00006 |
00057 |
9 |
2 |
07 |
1995 |
1 |
|
|
8 |
|
For columns “Student Name”, “Grade”, “Gender”, “Birth Date”, and “SEN”, identify and correct missing or incorrect information regarding student name, grade, gender, birth date, and special education needs (SEN).
Column “N/P PISA” is used to identify sampled students who cannot participate in PISA. The intention of PISA is to be as INCLUSIVE as possible. However, some students may not be able to participate for various reasons. These reasons for non-participation should be recorded in this column of the Student Tracking Form, using the following specific codes:
A small number of parents may refuse to allow their child to participate in PISA. If you have received such notification from parents, record a code “2” for this student in column “N/P PISA”.
The guidelines in Table 5 are provided to assist you in deciding whether a student with Special Education Needs can be excluded from PISA. If a student with a special education need is to be excluded from the assessment, record a code “3” for this student in column “N/P PISA”.
Note 2: Many students with a Special Education Need will be able to participate in the assessment. |
Table 5: Guidelines - Exclusion/inclusion of students with Special Education Needs
Type of SEN |
Students
who may be |
Students who should be included |
Functional disability
|
The student has a moderate to severe permanent physical disability such that she/he cannot participate in the PISA testing situation. Code 3 in column “N/P PISA”. |
The student can respond to the assessment. |
Cognitive, behavioral or emotional disability
|
The student has a cognitive, behavioral or emotional disability such that in the opinion of qualified staff, she/he cannot participate in the PISA testing situation. This includes students who are cognitively, behaviorally or emotionally unable to follow even the general instructions of the assessment. Code 3 in column “N/P PISA”. |
The student can respond to the assessment. He/she should NOT be excluded solely because of poor academic performance or disciplinary problems. |
Insufficient assessment language experience
|
The student meets ALL of the following three criteria:
|
The student meets ONLY one or two of the criteria. |
Home Schooled |
The student is home schooled and only attends school for required or mandated testing. Code 3 in column “N/P PISA” |
|
Test is not available in student’s language of instruction |
The student receives the majority of instruction in a language other than English. Code 3 in column “N/P PISA”. |
If a student has transferred to another school, record a code “4” for this student in column “N/P PISA”.
If a student has left school, but it is not known which school the student has transferred to, record a code “5” for this student in column “N/P PISA”.
If a student is enrolled in Grade 6 or lower, he/she is not eligible to participate in PISA. Record a code “n” for this student in column “N/P PISA”.
If a student was NOT born in 1995 he/she is not eligible to participate in PISA. Record a code “n” for this student in column “N/P PISA”.
All codes to be used in column “N/P PISA” for non-participant students are summarized in Table 6 below. Examples of how to use these codes are highlighted on the following page. If in doubt about which code to use, leave this column blank and instead insert a brief note in the “Comments” column on the form.
Table 6: Codes for Non-Participant
Reason for non-participation |
Code |
Parent refusal |
2 |
Excluded on SEN (Special Education Needs) basis |
3 |
Now enrolled at another school |
4 |
Not enrolled in this school, and enrollment is unknown |
5 |
Does not meet PISA criteria for eligibility |
n |
Please
refer to Figure 2, in which
the following example codes have been applied:
Jeffrey
Jenkins has recently transferred
to another school. Therefore, he is coded as “4”
in column “N/P
PISA”.
José
Rodriguez is coded as “3”
in column “SEN”. He has been taught in English
for less than one year and his
teachers believe that he will not be able to participate in the
assessment. Therefore, he is coded as “3”
in column “N/P
PISA”.
While
Roy Zastrow’s
experience with English
is limited, he has been taught in English for more than one year
and therefore does not meet the PISA criteria for exclusion on the
basis of insufficient language experience. Therefore, column “N/P
PISA” is left blank, and Roy
is considered to be able to participate in the assessment.
Sam
Walker is blind and is coded as
“1”
in column “SEN”. PISA is not offered in Braille.
Therefore, he is unable to participate in the assessment and is
coded as “3”
in column “N/P
PISA”.
Even
though Fred Peters
is functionally disabled, it does not affect his ability to
participate. So, although he is coded as “1”
in column “SEN”, column “N/P
PISA” is left blank.
The
following two students are ineligible according to the PISA
eligibility criteria. However, their
wrong demographic information was recorded in the List of Eligible
Students, and these
students were subsequently sampled.
The School Coordinator found the incorrect information on the
Student Tracking Form
and corrected it:
-
Tammy Tran
does not meet the age definition and is coded as “n”
in column “N/P
PISA”.
-
Melissa Holten
is in Grade 6 and so is coded as “n”
in column “N/P
PISA”.
Lisa
McCoy has been professionally
assessed as cognitively delayed and is coded as “2”
in column “SEN”. She does not participate in other
testing programs. Therefore, she will not participate in the
assessment and is coded as “3”
in column “N/P
PISA”.
Examples
of using Non-Participant codes
Note 3: Column “N/P PISA” of Student Tracking Form The purpose of setting a column “N/P PISA” is to record the reasons of non-participation for those students who have no possibility to participate in PISA under any circumstances even if a follow-up session is provided after the assessment. Therefore, you should NOT use a code “2” if a student is unable to attend the PISA assessment, for example, due to a competing activity in/outside the school or illness, as he/she might be able to attend a follow-up session. If a sampled student is not available for the PISA assessment for any reasons other than those described in the above steps, leave column 11 blank. The Test Administrator will record other cases of non-participation on the assessment day. |
Only
those students who have NO codes placed in column “N/P
PISA” can be assessed in PISA.
IMPORTANT! Under No
circumstances may a sampled student be replaced with any other
student.
The purpose of the School Questionnaire is to collect information about the school and its characteristics. The School Questionnaire should be completed by the school principal, but she/he may delegate the task to another person in the school. This questionnaire is to be completed in the hard copy booklet. Please distribute the questionnaire to the appropriate staff member and ask that it be completed by the date specified.
During the two weeks preceding the assessment, your PISA Test Administrator will visit your school to meet with you to discuss the following:
Confirmation of the date, time, and location of the assessment session;
Updates you have made to the Student Tracking Form regarding student demographic information, SEN codes, and exclusions;
How to handle emergencies or problem situations on assessment day; and
School procedures for the dismissal of students at the conclusion of each assessment session.
You will then discuss the notification of teachers, students, and parents of the assessment. This includes:
Verifying that Parents/Guardians have been notified,
Distributing student invitations to selected students and reminding students to bring the following to the assessment: (a) pencil; (b) eraser; (c) ruler; and (d) calculator on the day of the assessment; and
Informing teachers and anyone else concerned of any changes made to the date or time of the assessment since the original schedule was set.
The Test Administrator will again contact you just prior to the assessment date to confirm the updates of the Student Tracking Form and details of the assessment sessions. The Test Administrator will also verify that the School Questionnaire has been completed and will be ready to be collected on the day of the assessment.
Please make sure that you have updated the Student Tracking Form before the assessment. There might be some students who are eligible to participate in the assessment, but are absent from the assessment. For such cases, please leave column “N/P PISA” blank. The Test Administrator will record student attendance during the assessment, using a Session Attendance Form.
The Test Administrator will collect your Student Tracking Form to return to PISA. You should make a photocopy of the Form for yourself to keep at your school. Please store that copy until the end of the school year.
If it is not possible for you to be present throughout the assessment, you should be present at least at the beginning of the assessment session. This will help set the tone of the session and demonstrate the importance of the study.
Note 4:
|
PISA requires a high level of participation amongst the sampled students. If several students are absent from the Paper-Based Assessment session or the Computer-Based Assessment session, it may be necessary to arrange a make-up session. The Test Administrator will guide you through the process of determining whether or not you will need to arrange a make-up session.
If you and the Test Administrator decide that a make-up session should be arranged, please select a mutually agreed upon date and time that will maximize the number of students attending the session.
Note 5: There can only be ONE make-up session. Please make every attempt to ensure that absent students attend the make-up session. |
Your
help with PISA is very important to its success.
The U.S.
PISA Team and the International Project Team
appreciate your
time and effort.
Appendix 1: PISA 2012 FT- List of Eligible Students
School Name: ___________________________________________________________________________ |
|
Address: _______________________________________________________________________________ |
|
Telephone: ________________________________ |
Fax: ______________________________________ |
List Prepared By: ____________________________ |
Date List Prepared: __________________________ |
Total Number of Students Listed: _______________ |
|
DIRECTIONS: Please list ALL students in your school in Grade 7 and higher born January 1, 1995 and December 30, 1995.. |
(A) Student Name |
(B) Grade |
(C) Gender (Female=1; Male=2) |
(D) Birth Date (MM-YYYY) |
(F) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* SEN = Special Education Needs
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
File Title | PISA Report |
Author | Nora Kovarcikova |
File Modified | 0000-00-00 |
File Created | 2021-02-01 |