Justification

NATIONAL ASSESSMENT OF

EDUCATIONAL PROGRESS

Volume I

SUPPORTING STATEMENT

FOR

WAVE 2 SUBMITTAL FOR 2012

OMB# 1850–0790

Student Grade 4 Core + Writing Questions

Student Grade 4-MP3 Player Study Questions (2011 Core + Mathematics Questions)

Teacher Grade 4 Background, Education, and Training (BET) + Writing Questions

School Grade 4 School Characteristics and Policies (SCP) + Writing Questions

Student Grade 8 Core Questions

Teacher Grade 8 Background, Education, and Training + Mathematics Questions

Teacher Grade 8 Background, Education, and Training + Reading Questions

School Grade 8 School Characteristics and Policies Questions

Student Grade 12 Core + Economics Questions

Student Grade 12 Core + Reading Questions

Student Grade 12 Core + Mathematics Questions

Department Chair Economics Questions

School Grade 12 (Version A): School Characteristics and Policies Questions (operational), Economics, School Characteristics and Policies Questions (pilot), Charter School Questions

School Grade 12 (Version B): School Characteristics and Policies Questions (operational) , Economics, Mathematics Questions (pilot), Reading Questions (pilot), Charter School Questions

SD (Student with Disabilities) Instructions/Worksheets – Grades 4, 8, 12/Ages 9, 13, 17

ELL (English Language Learner) Instructions/Worksheets – Grades 4, 8, 12/Ages 9, 13, 17

June 1, 2011

Contents

1. Explanation for This Submittal 1

2. Overview of Wave 2 NAEP 2012 Assessments 3

3. Information Pertaining to the 2012 Questionnaires in This Submittal 3

4. 2012 Wave 2 Burden Information 7

5. How, By Whom, and For What Purpose the Data Will Be Used 12

6. Estimates of Costs to the Federal Government 13

Appendix A: Committee Lists 14

Appendix B: List of New Items 19

Appendix C: Sample Parent Notification Letters 20

Appendix D: Sample Description of School Coordinator Responsibilities 22

1. Explanation for This Submittal

This document contains supplemental information pertaining to the 2011–2013 National Assessment of Educational Progress (NAEP) System Clearance proposal. NAEP is administered by the National Center for Education Statistics (NCES) in the Institute for Education Sciences of the U.S. Department of Education. The National Assessment Governing Board (henceforth referred to as the Governing Board) sets policy for NAEP and determines the content framework for each assessment. The system clearance package was submitted in November 2009 and approved in April 2010 with OMB #1850-0790. The terms of clearance for OMB approvals state that each subsequent submittal activity under the system clearance is to be submitted to OMB.

NAEP consists of two assessment programs: the NAEP Long-Term Trend (LTT) assessment and the main NAEP assessment. In 2012, NAEP will administer both assessment programs. The nature of NAEP is that burden alternates from a relatively low burden in national-level administration years to a substantial burden increase in state-level administration years that include one or more assessments that support state-by-state and certain urban districts reporting. In state/district assessment years, NAEP samples approximate 1,000,000 students, while in national-only assessment years, NAEP samples approximate 100,000 students. In 2012¹, NAEP will conduct national-level assessments, so the estimated burden is much lower than in 2011 NAEP. Also, as in most years, the clearance for the NAEP 2012 collection is split into two submissions, Wave 1 and Wave 2.

This Wave 2 submittal seeks approval for background (also referred to as noncognitive) questions associated with the operational national NAEP 12th grade economics assessment; NAEP pilot tests at the 4th, 8th, and 12th grade levels; and three special studies – namely:

Grade 4 or Age 9 LTT

Student Grade 4 Core²+ Writing Questions

Teacher Grade 4 Background, Education, and Training (BET) + Writing Questions

School Grade 4 School Characteristics and Policies (SCP) + Writing Questions

Special Study to evaluate a pilot Students with Disabilities (SD) and (English Language Learners) ELL Decision Tree (at age 9 LTT)

Special Study to evaluate use of MP-3 players for administering read-aloud accommodations (will include Background Questions from 2011 Mathematics assessment)

Special Study to evaluate possible mode effects among grade 4 students taking the writing pilot

Grade 8 (Age 13 LTT)³

Student Grade 8 Core

Teacher Grade 8 Background, Education, and Training Questions

School Grade 8 School Characteristics and Policies Questions

Special Study to evaluate a pilot SD and ELL Decision Tree

Grade 12

Student Grade 12 Core + Economics Questions

Student Grade 12 Core + Reading Questions

Student Grade 12 Core + Mathematics Questions

Department Chair Economics Questions

School Grade 12 (Version A): School Characteristics and Policies Questions (operational), Economics, School Characteristics and Policies Questions (pilot), Charter School Questions

School Grade 12 (Version B): School Characteristics and Policies Questions (operational) , Economics, Mathematics Questions (pilot), Reading Questions (pilot), Charter School Questions

All Grade Levels

SD and ELL Instructions and Worksheets

These specific questions are the second group (Wave 2) of questions submitted for approval for usage in 2012. A first group of questions (Wave 1) was approved in May 2011 and contained the 2012 LTT background questionnaires. Wave 2 contains the student, teacher, school, and department chair questions to be administered as part of main NAEP and the students with disabilities (SD) and English language learner (ELL) worksheets to be used with both LTT and main NAEP.

2. Overview of Wave 2 NAEP 2012 Assessments

Much of the following broad overview of the 2012 NAEP assessments was included as part of the 2011–2013 System Clearance submittal. The Governing Board determines NAEP policy and assessment schedule, and future Governing Board decisions may result in changes to some aspects of an assessment (e.g., which subjects are assessed in which years). However, the overall methodology and assessment process will remain consistent. These assessments are currently conducted by an alliance of organizations under contract with the U.S. Department of Education. The assessments contain two kinds of questions: cognitive questions, which measure student knowledge of an academic subject; and background or noncognitive questions which gather information on demographic as well as classroom instructional procedures. The Governing Board is responsible for selecting and approving all of NAEP’s background questions, as well as the cognitive items.

The 2012 data collection activities described in this Wave 2 submission consists of the following:

• Economics (national at grade 12)

• Writing pilot (national at grade 4; computer delivered)

• Reading pilot (national at grade 12)

• Mathematics pilot (national at grade 12)

• Student Core Background pilot (at grades 4 and 12, and at age 13 LTT)

• Teacher Core Background pilot (at grade 4, age 13 LTT)

• School Core Background pilot (at grades 4 and 12, and at age 13 LTT)

• SD and ELL worksheets—completed by teachers or administrators of students identified as SD and/or as ELL (both main NAEP and LTT)

• Special Pilot Study to evaluate the new SD and ELL Decision Tree (refer to page 9 for more detail), based on the new SD and ELL policy established by the Governing Board (at ages 9 and 13 LTT)

• Special Mini-Pilot Study to evaluate use of MP-3 players for administering read-aloud accommodations (at grade 4)

• Special Study to evaluate possible mode effects among grade 4 students taking the writing pilot (paper and pencil versus computer-based delivery).

Committees that helped define the cognitive and background questions pertaining to the Wave 2 subjects are listed in Appendix A.

3. Information Pertaining to the 2012 Questionnaires in This Submittal

Note: Pilot questionnaires can include a mix of new and trend questions. A list of new or revised questions is contained in Appendix B.

Student Questionnaires

Core – Two versions of core questions will be assessed. The operational core background questions (for use with the grade 12 economics assessment) are the same ones that were administered in 2011. The pilot core background items are being pilot tested in 2012 at all three grades for potential operational use in 2013. These are newly developed questions. Since there is no grade 8 assessment administered in 2012, the new core student questionnaire items will be piloted with a subset of students in the age 13 LTT mathematics and reading assessments.

Economics – Grade 12 economics questions are trend questions from the last economics assessment in 2006.

Reading – Grade 12 pilot subject-specific reading questions are new questions being tested for potential use in 2013.

Mathematics – Grade 12 pilot subject-specific mathematics questions are new questions being tested for potential use in 2013.

Writing – New subject-specific pilot questions in writing will be administered at grade 4. The grade 4 writing assessment in 2013 will be (a) the first assessment based on a new Governing Board framework and (b) computer-delivered for the first time at grade 4. To determine if grade 4 students can appropriately respond via the computer, a mode effect special study will be conducted in which some students will respond to the writing assessment using traditional paper and pencil. Note - the same background questions will be used for the special study as for the computer-based pilot.

MP-3 Special Study - A special study to evaluate the use of MP-3 players for administering read-aloud accommodations will be conducted at grade 4. These students will be given the 2011 Mathematics assessment, which includes 15 minutes of background questions, consisting of a section of core questions and a section of mathematics-specific questions.

Teacher/Department Chair Questionnaires

Background, Education, and Training (BET; also referred to as the teacher core) - The pilot BET background items are being pilot tested at both grades 4 and 8 for potential operational use in 2013. These are newly developed questions. Since there is no grade 8 assessment in the field in 2012, the new core teacher questionnaire items will be piloted with the teachers of students in the age 13 LTT mathematics and reading assessments.

Writing - New subject-specific pilot questions in writing (Classroom Organization and Instruction) will be administered to writing teachers of the assessed grade 4 students. These are newly developed questions. The teacher questionnaire can be completed either on paper or electronically.

Department Chair Economics - Questions are trend questions from the last economics assessment in 2006.

Table 1 depicts the composition of the 2012 teacher questionnaires.

Table 1: Teacher Background Questionnaires for the 2012 NAEP Assessments
Age/Grade	Type	Section 1	Section 2
grade 4	pilot	BET	COI-Writing
grade 8 (age 13 LTT administration)	pilot	BET-Mathematics BET-Reading
grade 12	operational	Economics Department Head

BET – Background, Education and Training

COI – Classroom Organization and Instruction

School Questionnaires

School Characteristics and Policies (SCP; also referred to as the school core) – Two versions of SCP questions will be assessed. Trend SCP questions will be administered for all grade 12 school questionnaires and are the same ones that were administered in 2011. In addition, new SCP questions will be piloted at all three grades for 2012. Since there is no grade 8 assessment in the field in 2012, the new core school questionnaire items will be piloted in some of the schools in the age 13 LTT assessment.

Economics – Grade 12 economics school questions are trend questions from the last economics assessment in 2006.

Writing - Grade 4 school writing questions are new questions being piloted for potential use in 2013.

Mathematics – Grade 12 pilot mathematics-specific school questions are new questions being tested for potential use in 2013.

Reading– Grade 12 pilot reading-specific school questions are new questions being tested for potential use in 2013.

Charter School – Charter school questions are trend questions that were administered in the 2011 school questionnaires. Note that the charter school section is only completed if the school is a charter school.

Due to the large number of sections (6) and because all questionnaires need to have the SCP operational as well as Economics sections, two versions of the grade 12 school questionnaires will be administered such that each school only responds to some of the pilot sections. Each school will then complete just one of the two (shorter) versions which will lessen burden on school administrators. Table 2 depicts the composition of the 2012 school questionnaires.

Table 2: School Background Questionnaires for the 2012 NAEP Assessments
Age/ Grade	Section 1	Section 2	Section 3	Section 4	Section 5
grade 4	SCP pilot	Writing pilot
grade 8 (age 13 LTT administration)	SCP pilot
grade 12 (Versions A and B)	SCP operational	Economics operational	SCP pilot	Charter school operational
	SCP operational	Economics operational	Math pilot	Reading pilot	Charter school operational

SCP – School Characteristics and Policies

SD and ELL Worksheets

Two versions of the SD and ELL worksheets will be assessed. The operational SD and ELL worksheets are similar to the ones used in the 2011 administration, updated to reflect the subjects being assessed in 2012. The operational worksheets will be used for the grade 4 writing assessment, all grade 12 assessments, and the majority of the LTT assessments (submitted in Wave 1). The study of the decision tree pilot will include a new decision tree with language refinements to make the decision tree easier to complete and will be assessed in a portion (50 schools) of the ages 9 and 13 LTT assessments.

4. 2012 Wave 2 Burden Information

The total combined burden for the Wave 2 materials is as follows:

Table 3: Total burden for Wave 2

Table 3: Total burden for Wave 2
Respondent Category	Number of Respondents	Number of Hours
Student	29,800	7,658
Teacher/Department Chair	2,550	1,150
School (Questionnaires, Pre-Assessment Visit, and E-Filing)	1,050	3,049
School (for SD and ELL Worksheets)4	2,555	2,746
Totals	35,955	14,603

The specific per respondent and age category burden times are included in Tables 4 and 5 and include the following categories:

Students – Students in grades 4 and 12 will complete assessment booklets that contain two 25- or 30-minute cognitive blocks⁵, followed by two sections of questions (one background and one subject-specific), which require a total of 15 minutes to complete. The background section contains core questions, many of which are related to demographic information. In 2012, there will be subject-specific questions in writing, economics, reading, and mathematics. Since no grade 8 assessments will be administered in main NAEP in 2012, new student core questions will be piloted with a subset of students during the age 13 LTT assessment. These same students will also be assessed using an operational set of background questions, described in Wave 1. Therefore, the burden in terms of the number of respondents for the grade 8/age 13 LTT pilot core questions is contained in Wave 1. However, the burden for the time to complete these pilot core questions is contained in this Wave and included in Tables 3-5. Special studies to evaluate the use of MP-3 players for administering read-aloud accommodations and to evaluate possible mode effects among students taking the writing pilot will be conducted at grade 4. For the MP-3 player study, students will be given the 2011 Mathematics assessment, which includes 15 minutes of background questions.

A sample parental notification letter regarding NAEP is included as Appendix B.

Teachers/Department Chairs – The teachers of fourth- and eighth-grade students participating in NAEP will be asked to complete questions about their teaching background, education, training, and/or classroom organization and instruction. Given that no grade 8 assessments will be administered in main NAEP in 2012, the grade 8 pilot teacher questionnaire will be administered with the age 13 LTT assessment. The fourth-grade teacher burden is estimated to be 30 minutes, while the eighth-grade teacher burden is 20 minutes. The Economics Department Chair will also be asked to complete questions about economics-related course and teacher information. The burden for the Department Chairs is estimated to be 20 minutes.

Principals/Administrators – The school administrators in the sampled schools will be asked to complete a questionnaire. At grade 4, a pilot school questionnaire will be administered in schools participating in the pilot writing computer-based assessment. Given that no grade 8 assessments will be administered in main NAEP in 2012, the grade 8 pilot school questionnaire will be administered with the age 13 LTT assessment. These grade 8 questionnaires will contain only a section of new questions on School Characteristics and Policies (SCP).

Each grade-specific questionnaire will include an SCP section, followed by separate sections for each subject area, if applicable (grade 4: writing; grade 12: economics, reading, and mathematics). In the grade 12 questionnaire, the reading and math sections are pilot sections. A section with charter school items will be included as the last section in the grade 12 school questionnaires. In addition, a new pilot section of School Characteristics and Policies will be administered. As described above, due to the number of sections, two versions of the questionnaires will be administered such that each school only responds to some of the pilot sections.

The burden for school questionnaires is estimated to be 30 minutes.

Schools: E-filing – Survey sample information is collected from schools in the form of lists of potential students who may participate in NAEP. This sample information can be gathered manually or electronically (e-filing) at the school, district, or state level. If done at the school or district level, some burden will be incurred by school or district personnel. The e-filing burden is estimated at one hour per school. However, the e-filing process is only done in a subset of schools (approximately 38 percent in recent years), which is taken into account when computing the e-filing burden.

Schools: Pre-Assessment Activities – Prior to the assessment, school personnel must prepare for the assessment. Part of the preparation is the pre-assessment visit, which is the opportunity for the NCES contractor field staff to meet with the school personnel to review procedures and logistics for the upcoming assessment. In addition, miscellaneous activity related to information gathering for the pre-assessment visit and communication with the teachers and students is required. Both the pre-assessment visit and the miscellaneous activities are estimated to require one hour each of school personnel time. Thus, the total burden time for pre-assessment related activities is two hours. A sample brochure communication from the NAEP state coordinators to the participating schools describing these activities is included as Appendix C. This brochure outlines what the school coordinator will be responsible for, including a description of e-filing and the pre-assessment visit. The information in the brochure will be modified appropriately for each grade, subject area, and mode of assessment.

SD and ELL (Students with Disabilities and English Language Learners) – Worksheets are completed by teachers or administrators of students identified as SD and/or as ELL. SD and ELL burden is the number of hours that school personnel spend to complete the SD/ELL worksheets for students identified as SD and/or ELL (estimated at 10 minutes per student). Since the SD and ELL worksheets are part of this Wave 2 submittal, the burden for school personnel who complete them (whether for main NAEP or for LTT) is included with Wave 2.

SD and ELL Decision Tree pilot – This pilot study will be conducted during the age 9 and age 13 LTT assessment windows. The decision tree logic is part of the SD and ELL worksheets that are completed by school personnel and used for determining inclusion in NAEP assessments. The goals of the study are to (a) evaluate the impact of implementing a new “decision tree” for SD and ELL students on inclusion rates, and (b) to provide information from the field about language refinements needed to make the tree more user-friendly for schools. Since the decision tree will be piloted in separate schools than those using the existing decision tree, there can be a comparison from which to gauge the impact of the new decision tree on inclusion rates. The pilot will occur in 50 schools at each grade and an oversample of SD and ELL students will be taken in each school to help ensure an adequate sample size to test the new decision tree. The burden estimates in Table 5 include the SD and ELL students in the decision tree pilot.

Table 4: Wave 2 Questions - Per Respondent Burden Time

Respondent Category	Time per Respondent
Grade 4
Student Grade 4 Core (Pilot) + Writing	15 minutes
Student Grade 4 MP-3 Player Study (2011 Core + Mathematics)	15 minutes
Teacher Grade 4 Background, Education, and Training + Writing	30 minutes
School Grade 4 School Characteristics and Policies + Writing	30 minutes
Grade 8
Student Grade 8 Core	5 minutes
Teacher Grade 8 Background, Education, and Training (BET)	20 minutes
School Grade 8 School Characteristics and Policies	30 minutes
Grade 12
Student Grade 12 Core + Economics Questions	15 minutes
Student Grade 12 Core + Reading Questions	15 minutes
Student Grade 12 Core + Mathematics Questions	15 minutes
Economics Department Chair Questions	20 minutes
School Grade 12 School Characteristics and Policies (SCP) + Economics + SCP Pilot + Charter School	30 minutes
School Grade 12 School Characteristics and Policies + Economics + Reading + Mathematics + Charter School	30 minutes
All Grades
SD and ELL Worksheets	10 minutes per identified student

1. LTT burden, except for the SD/ELL component and core piloting, was included in the Wave 1 submittal.

2. These schools and students are a subset of the same schools and students who will be administered the LTT assessment; thus, the number of students and school administrators was included in Wave 1 and, thus, not included here.

3. School personnel involved in pre-assessment activities (2 hr.) and e-filing activities (1 hr.) for the subset of schools (approximately 38 percent) that perform e-filing.

4. The MP3 Player and writing mode effect studies will not have any teacher or school questionnaire components. In addition, they will be conducted in the same schools as the grade 4 assessments. Therefore, there are not any additional pre-assessment and e-filing activities. The MP-3 study also includes an oversample of SD and ELL students (30% at grade 4), while the mode effect does not (22% at grade 4).

5. At grade 12 there is an Economics Department Chair Questionnaire, included under teacher burden.

6. SD and ELL burden is the number of hours that school personnel spend to complete the SD/ELL worksheets for students identified as SD and/or ELL (estimated at 10 minutes per student). The overall SD/ELL burden is a factor of the number of students identified as SD/ELL. Based on most recent data, 22 percent of grade 4/age 9 students, 18 percent of grade 8/age 13, and 14 percent of grade 12/age 17 students were identified as SD/ELL. For example, the grade 4 SD/ELL burden is calculated as follows: 14,000 students * 22 percent estimated as SD and/or ELL * 10 minutes per student / 60 minutes in an hour = 513 hours. In addition, for 2012, there will be a 25 percent oversample of SD/ELL students for ages 9 and 13 LTT. Therefore, the estimated number of SD/ELL students is 27 percent and 22 percent of the age 9 and age 13 samples, respectively.

N/A- not applicable. SD and ELL counts for the core pilot study at age 13 LTT are already included in the age 13 LTT counts.

Italicized text- Not computed for totals. The burden for the LTT students was included in the Wave 1 burden estimates. The MP-3 and writing mode effect studies will be conducted in the same schools as the grade 4 assessments.

5. How, By Whom, and For What Purpose the Data Will Be Used

The purpose of NAEP is to collect and report assessment data on student achievement in the subject areas assessed for use in monitoring education progress. In addition to reporting overall results of student performance and achievement, NAEP also reports student performance results for various subgroups of students and on various educational factors. Guidance for what is asked in the questions is set by the Governing Board. NCES is responsible for developing the questions and for selecting the final set of questions. The questions are designed to (a) provide the information for disaggregating data according to categories specified in the legislation⁶, (b) provide contextual information that is subject specific (e.g. reading, mathematics) and has an impact and known relationship to student achievement, and (c) provide policy-relevant information specified by the Governing Board.

In the original request to OMB for system clearance, NCES requested approval of the data-gathering instruments to be used in the 2011–2013 national and state assessments. This submittal applies to the second set of questions that will be submitted as part of the 2012 assessments. An expanded description of the purposes of the data is provided in the system clearance submittal.

6. Estimates of Costs to the Federal Government

The following estimates apply to the 2012 Wave 2 administration costs:

Table 6: Administration Cost Estimates

Activity	Provider	Estimated Costs
Printing, packaging, and distribution phases of the administration (related to Wave 2) including: prepare and package the assessment and all auxiliary materials; distribute assessment booklets and materials to the test administrators for each school	Pearson (the Materials, Distribution, Processing and Scoring (MDPS) contractor)	$2.5 million
Field administration (related to LTT) including: administration of assessments; pre-assessment contact/visits to school data collection from the field	Westat (the Data Collection contractor and NAEP State Service Center contractor)	$1.2 million
Web support and maintenance related to the 2012 assessments	Fulcrum IT Services	$495,000
Totals		$4.195 million

Appendix A: Committee Lists

NAEP Background Variable Committee

Name Affiliation

Patricia Alexander University of Maryland

Arthur Applebee State University of New York at Albany

Claudia Buchman Ohio State University

Lizanne DeStefano University of Illinois

Robert Hauser University of Wisconsin-Madison

Kathleen Heid Penn State University

Henry Levin Columbia University

Peter Levine Tufts University

Linda Levstik University of Kentucky

Samuel Lucas University of California-Berkeley

Senta Raizen WestEd

Andrew Sum Northeastern University

William Walstad University of Nebraska-Lincoln

NAEP Mathematics Committee

Name Affiliation

David J. Brancamp Northwestern RPDP (Regional Professional Development Program)

Reno, NV

Sareeta Carter Benjamin Banneker Academic High School

Washington, DC

Herb Clemens Ohio State University

Marcia Cole District of Columbia Public Schools 

Carl Cowen Indiana University-Purdue University, Indianapolis

Bradford R. Findell Ohio Department of Education

Diana Rivera Garcia Departamento de Educación Puerto Rico

Anne Gonzales South Gate Middle School

South Gate, CA

Kathleen Heid Penn State University

Judy Kinley Morris Elementary School

Des Moines, IA

Carole Phillip Alice Deal Middle School

Washington, DC

Penny Roberts Greenville Elementary School

Greenville, KY

Daren Starnes The Lawrenceville School

Lawrenceville, NJ

JT Sutcliffe St. Mark’s School of Texas

Elizabeth Sweeney Boston Public Schools

Reading Committee

Name Affiliation

Patricia Alexander University of Maryland

Christine Carriere Carl Von Linne School,

Chicago, IL

Mary Beth Curtis Lesley University

Patsy Dunton Maine Department of Education

Arzie Galvez Office of Curriculum, Instruction and School Support,

Los Angeles, CA.

Evan Lefsky Leesburg High School,

Leesburg, FL

Pamela Mason Harvard Graduate School of Education

Joe McGonegal Catholic Memorial High School,

West Roxbury, MA

Margaret McKeown University of Pittsburgh

Katie Oliver Morgan Park Middle School,

Duluth, MN

Timothy Shanahan University of Illinois

Alfred Tatum University of Illinois at Chicago

Lisa White Plymouth Public Schools,

Plymouth, MA

Junko Yokota National-Louis University

NAEP Writing Committee

Name Affiliation

Arthur Applebee University at Albany, SUNY

Diane August Center for Applied Linguistics,

Washington, DC

Margretta Browne Montgomery County Public Schools,

Rockville, Maryland

Robert Crongeyer Robla School,

Sacramento, CA

Elyse Eidman-Aadahl University of California

Nikki Elliot-Schuman Office of Superintendent of Public Instruction, WA

Rayna Goldfarb Abraham Lincoln High School,

Philadelphia, PA

Carol Jago California Reading and Literature Project, UCLA

Charles MacArthur University of Delaware

Michael McCloskey Johns Hopkins University

Norma Mota-Altman San Gabriel High School,

San Gabriel, CA

Sandra Murphy University of California, Davis

Drew Sterner Tamanend Middle School,

Warrington, PA

Victoria Young Texas Education Agency

NAEP Economics Committee

Name Affiliation

Kris Bertelsen St. Charles High School,

St. Charles, MN

Stephen Buckles Vanderbilt University

Steven L. Cobb Center for Economic Education,

Denton, TX

Jaime Festa-Daigle Lake Havasu High School,

Lake Havasu City, AZ

Julie Heath University Of Memphis

Richard MacDonald St. Cloud State University

Andrea Morgan Oregon Department of Education

Kevin Smith Renaissance High School,

Detroit, MI

William Walstad University Of Nebraska-Lincoln

Appendix B: List of New or Revised Questions

2012 OMB Questionnaire Parts
Part 1-Student Pilot Background Questionnaires	New or Revised Questions
Part 1a-Student Grade 4 Core + Writing	Section 3- all new (cba-delivered)
	Section 4- all new (cba-delivered)
Part 1b-Student Grade 4 Core + Mathematics	Section 3- no new questions
	Section 4- no new questions
Part 1c-Student Grade 8	Section 3- all new except questions 1 &2.
Part 1d-Student Grade 12 + Mathematics	Section 3- all new except questions 1 &2.
	Section 4- 1h,1j,1k,1l,o,1p; 2,3,6c,9e,12,13a-f,14,18
Part 1e-Student Grade 12 + Reading	Section 3- all new except questions 1 &2.
	Section 4- 1c,4,5e,5g,6d-g,7c,7d,7h,8,9.
Part 2-Student Economics Background Questionnaire	No new questions
Part 3- Teacher Pilot Questionnaires
Part 3a-Teacher Grade 4 Core(BET-Writing)	Section BET - all new except questions 1 & 2.
Part 3b-Teacher Grade 8 Core(BET-Mathematics)	Section BET - all new except questions 1 & 2.
Part 3c-Teacher Grade 8 Core(BET-Reading)	Section BET - all new except questions 1 & 2.
Part 4- Department Chair Economics Questionnaire	No new questions
Part 5- School Pilot Questionnaires
Part 5a-School Grade 4 Core(SCP+ Writing)	SCP- all new
	Writing-1a, 1b, 1c, 2b, 4,5,6.
Part 5b-School Grade 8 Core(SCP)	SCP- all new
Part 5c-School Grade 12 Core(SCP + Mathematics + Reading)	SCP- all new
	Math-2, 3a, 3b, 3c, 3e, 3f, 3g, 3h, 3i, 3j, 3k, 3l, 4e, 4f, 4g, 4h, 4i, 6, 7a-7f,8,9a-c,10h,10i,11,12,13,14.
	Reading- 1c,1e,1g,2,3,4,5,6i,8,9,11c,11d,11e,12.
Part 6- School Economics Questionnaire	No new questions

Appendix C: Sample Parent Notification Letters

NAEP 2012 PARENT/GUARDIAN NOTIFICATION LETTER

Grade 4 Writing Computer-Based Assessment Pilot Study

(School Letterhead)

(Insert Date Here)

Dear Parent or Guardian:

We are pleased to tell you that (school name) has been selected to represent schools across the nation by participating in a pilot study for the National Assessment of Educational Progress (NAEP). NAEP is administered by the National Center for Education Statistics (NCES), within the U.S. Department of Education, and is the largest continuing and nationally representative assessment of what our nation’s students know and can do in core subjects. NAEP has been providing valid and reliable data on student performance since 1969.

The results of NAEP are released as The Nation's Report Card, which provides information about student achievement to educators, parents, policymakers, and the public. This pilot study will be used to prepare for the NAEP 2013 assessments.

In our school, the NAEP writing computer-based assessment will be given on (date). Your child (may be/has been) selected to take the assessment. In addition to subject area questions, students will be asked some questions about themselves and their educational experiences. You can access student and sample test questions at http://nces.ed.gov/nationsreportcard/parents/.

It will take about 120 minutes for most students to complete the assessment. The results are completely confidential (in accordance with the Confidential Information Protection provisions of Title V, Subtitle A, Public Law 107-347), and your child’s grades will not be affected. Your child may be excused from participation for any reason, is not required to finish the assessment, and may skip any test question. While NAEP is voluntary, we depend on student participation to provide an accurate measure of student achievement that will inform improvements in education. Your child will represent many other students, so participation is very important. However, if you do not want your child to participate, please notify me in writing by (date).

There is no need to study in preparation for NAEP. We do ask parents to encourage their children to do their best and get plenty of rest the night before the assessment.

If you would like to have additional information about NAEP, please visit http://nces.ed.gov/nationsreportcard. If you have questions or would like to review a booklet that includes sample subject area and student questions, please contact me at (telephone number) or via e-mail at (e-mail address).

We are excited that our school will be participating in NAEP, and we are pleased that your child (may be/has been) selected. We know that (school name)'s students will help us show what our nation’s students know and can do.

Sincerely,

School Principal

NAEP 2011-2012 PARENT/GUARDIAN NOTIFICATION LETTER

LTT Age 9, 13, and 17 or Grade 12 Economics, Mathematics, and Reading

(School Letterhead)

(Insert Date Here)

Dear Parent or Guardian:

We are pleased to tell you that (school name) has been selected to represent schools across the nation by participating in the National Assessment of Educational Progress (NAEP). NAEP is administered by the National Center for Education Statistics (NCES), within the U.S. Department of Education, and is the largest continuing and nationally representative assessment of what our nation’s students know and can do in core subjects. NAEP has been providing valid and reliable data on student performance since 1969.

The results of NAEP are released as The Nation's Report Card, which provides information about student achievement to educators, parents, policymakers, and the public.

In our school, the NAEP assessment will be given on (date) in (mathematics or reading) or (economics, mathematics, or reading). Your child (may be/has been) selected to take the assessment. In addition to answering questions in one subject, students will be asked some questions about themselves and their educational experience. You can access student and sample assessment questions at http://nces.ed.gov/nationsreportcard/parents/.

It will take about 90 minutes for most students to participate in the assessment. The results are completely confidential (in accordance with the Confidential Information Protection provisions of Title V, Subtitle A, Public Law 107-347), and your child’s grades will not be affected. Your child may be excused from participation for any reason, is not required to finish the assessment, and may skip any test question. While NAEP is voluntary, we depend on student participation to provide an accurate measure of student achievement that will inform improvements in education. Your child will represent many other students, so participation is very important. However, if you do not want your child to participate, please notify me in writing by (date).

There is no need to study in preparation for NAEP. We do ask parents to encourage their children to do their best and get plenty of rest the night before the assessment.

If you would like to have additional information about NAEP, please visit http://nces.ed.gov/nationsreportcard. If you have questions or would like to review a booklet that includes sample subject area and student questions, please contact me at (telephone number) or via e-mail at (e-mail address).

We are excited that our school will be participating in NAEP, and we are pleased that your child (may be/has been) selected. We know that (school name)'s students will help us show what our nation’s students know and can do.

Sincerely,

School Principal

Appendix D: Sample Description of School Coordinator Responsibilities

As the school coordinator, you are the liaison for all NAEP assessment activities in your school.

In the fall, you will be responsible for:

Registering for the MySchool website.

MySchool is used to collect information about your school and provide you with documents that you can download and customize throughout the NAEP assessment process. Multiple school staff may register to access the site. To register for MySchool, go to www.mynaep.com and complete the form using the registration ID provided by your NAEP State or Trial Urban District Assessment (TUDA) Coordinator.

Completing and submitting school information.

Click on “Provide School Information” on MySchool to enter and submit information about your school so that materials can be prepared for the assessment.

If requested, providing the NAEP State or TUDA Coordinator with a list of <selected> grade students.

NAEP requires a complete list of students in the selected grade in order to select a random sample of students to participate in the assessment. This list provides demographic information about students who will be assessed. It is usually submitted electronically and may be prepared by the school, district, or state. Your NAEP State or TUDA Coordinator will inform you if you need to provide this list. Student names will always be kept confidential and individual student responses or scores are never reported.

Before the assessment date, you will be responsible for:

Preparing for the assessment.

Numerous activities need to be completed early in January to ensure a successful assessment.

• Identify teachers who teach the assessed subjects to your <selected>-grade students.

• Arrange logistics for the assessment.

• Review the instructions for distributing and completing the worksheets for students identified as English language learners and/or students with disabilities, and distribute them to the staff person(s) most knowledgeable about how these students are tested on your state assessment.

Informing parents/guardians.

By law, parents/guardians of children selected to participate in NAEP must be informed prior to administration of the assessment that their child has been selected for the assessment, may be excused from participation for any reason, is not required to finish the assessment, and is not required to answer all test questions. Your NAEP State or TUDA Coordinator will provide a Sample Parent/Guardian Notification Letter and additional information about how this requirement should be fulfilled. These details, as well as electronic copies of the letter, will be provided through MySchool. Parent notification should be completed prior to the pre-assessment visit. Parents may also visit http://nces.ed.gov/nationsreportcard/parents/ to find out more about NAEP.

Meeting with the NAEP staff during the scheduled pre-assessment visit.

In late January, you will meet with the NAEP representative to go over information related to the NAEP administration. During the meeting, you will review the list of selected students to verify that their demographic information is accurate and complete. Details for the assessment day will be finalized, such as which students will require accommodations, how NAEP will be administered, the time and locations of the assessment, and how students and teachers will be notified. The NAEP representative will also verify that parents have been notified and will collect a copy of the parent notification letter during this visit.

Promoting the importance of NAEP with school staff.

Teachers are essential for motivating students to do their best on NAEP. The following are some suggestions on how to gain teacher support:

• Show the 5-minute NAEP video, Introducing NAEP to Teachers, at a faculty meeting. This video can be accessed at http://nces.ed.gov/nationsreportcard/videos/naep4th8th.asp.

• Inform teachers that released NAEP questions and responses, which they can use in their classroom, are accessible on the NAEP Questions Tool at http://nces.ed.gov/
  nationsreportcard/itmrlsx/.

Promoting the importance of NAEP with students.

Students who are selected for NAEP will represent students across the nation, so it is vital that these <selected grade>-graders participate and do their best. The following are some suggestions on how to encourage students to do their best:

• Speak with participating students prior to assessment day. Let them know why NAEP is important.

• Consider ways to thank students for their participation.

• Inform students that they can find past NAEP results by visiting The Nation’s Report Card website, http://nationsreportcard.gov/.

• Inform students that individual results are not released to the public and that NAEP only takes 90–120 minutes to complete.

On the assessment date, you will be responsible for:

Ensuring that students attend the session.

Prior to the assessment start time, you will need to be available to ensure that students attend. You are encouraged to remain in the room during the assessment. NAEP staff will bring all assessment materials to the school and will conduct the session(s). It is very important that attendance rates be as high as possible to avoid the need for makeup sessions. If attendance of sampled students is less than 90 percent, a makeup session will be needed, and the NAEP representative will schedule another date to administer the assessment to the students who were absent.

Thank you in advance for your help preparing for this important assessment!

1 For ease of discussion, “2012” will be used to refer to all assessments in the Fall 2011 through Spring/Summer 2012 school year.

2 NAEP collects student question data as required by law (P.L. 107-279, section 303 - i.e., race, ethnicity, socio-economic status) and that provides a context for reporting student performance. These questions are the same across subjects and are referred to as core questions.

3 Given that no grade 8 assessments will be administered in main NAEP in 2012, new core sections will be piloted with the age 13 Long-Term Trend (LTT) samples. Refer to the Wave I submittal for LTT information.

4 This includes the burden for the SD-ELL decision tree pilot.

5 The writing pilot may also contain some 20 minute blocks.

6 Education Sciences Reform Act of 2002 (ESRA), P.L. 107–279, Sec. 303, National Assessment of Educational Progress, 20 USC, Sec. 9622.

2012 Wave 2 Supporting Statement