Supporting statement Part A - wave 3, rev.

NATIONAL ASSESSMENT OF
EDUCATIONAL PROGRESS

Volume 1
SUPPORTING STATEMENT
FOR
WAVE 3 OF 2009 SUBMITTAL
(PART OF 2008–2010 SYSTEM CLEARANCE PROPOSAL
OMB# 1850-0790)
Student Grade 4 NIES (National Indian Education Study) Questions
Student Grade 4 ESBQ (Extended Student Background Questions)
Teacher Grade 4 Operational Questions (Reading, Mathematics, Science)
Teacher Grade 4 Pilot Questions (Reading, Mathematics, Social Studies)
Teacher Grade 4 NIES (National Indian Education Study) Questions
School Grade 4 Operational Questions
School Grade 4 Pilot Questions
School Grade 4 NIES (National Indian Education Study) Questions
SD (Student with Disabilities) Questions (Grade 4)
ELL (English Language Learner) Questions (Grade 4)
Student Grade 8 NIES (National Indian Education Study) Questions
Student Grade 8 ESBQ (Extended Student Background Questions)
Teacher Grade 8 Operational Reading/Language Arts Questions
Teacher Grade 8 Operational Mathematics Questions
Teacher Grade 8 Operational Science Questions
Teacher Grade 8 Pilot Reading/Language Arts Questions
Teacher Grade 8 Pilot Mathematics Questions
Teacher Grade 8 Pilot Social Studies Questions
Teacher Grade 8 NIES (National Indian Education Study) Questions
School Grade 8 Operational Questions
School Grade 8 Pilot Questions
School Grade 8 NIES (National Indian Education Study) Questions
SD (Student with Disabilities) Questions (Grade 8)
ELL (English Language Learner) Questions (Grade 8)
Student Grade 12 ESBQ (Extended Student Background Questions)
School Grade 12 Operational Questions
School Grade 12 Pilot Questions
Grade 12 HSTS (High School Transcript Study) Questions
SD (Student with Disabilities) Questions (Grade 12)
ELL (English Language Learner) Questions (Grade 12)

Rev. August 26, 2008

Explanation and Burden Information for This Submittal
This document contains supplemental information pertaining to the 2008–2010 National
Assessment of Educational Progress (NAEP) System Clearance proposal. The System Clearance
package was submitted in January 2007 and approved in May 2007 with OMB #1850-0790. The
terms of clearance for OMB approvals state that each subsequent submittal activity under the
System Clearance is to be submitted to OMB.
This submittal contains burden information and the actual background questionnaires for the
following components of the 2009 assessments:
Grade 4
Student Grade 4 NIES (National Indian Education Study) Questions
Student Grade 4 ESBQ (Extended Student Background Questions)
Teacher Grade 4 Operational Questions (Reading, Mathematics, Science)
Teacher Grade 4 Pilot Questions (Reading, Mathematics, Social Studies)
Teacher Grade 4 NIES (National Indian Education Study) Questions
School Grade 4 Operational Questions
School Grade 4 Pilot Questions
School Grade 4 NIES (National Indian Education Study) Questions
SD (Student with Disabilities) Questions
ELL (English Language Learner) Questions
Grade 8
Student Grade 8 NIES (National Indian Education Study) Questions
Student Grade 8 ESBQ (Extended Student Background Questions)
Teacher Grade 8 Operational Reading/Language Arts Questions
Teacher Grade 8 Operational Mathematics Questions
Teacher Grade 8 Operational Science Questions
Teacher Grade 8 Pilot Reading/Language Arts Questions
Teacher Grade 8 Pilot Mathematics Questions
Teacher Grade 8 Pilot Social Studies Questions
Teacher Grade 8 NIES (National Indian Education Study) Questions
School Grade 8 Operational Questions
School Grade 8 Pilot Questions
School Grade 8 NIES (National Indian Education Study) Questions
SD (Student with Disabilities) Questions
ELL (English Language Learner) Questions
Grade 12
Student Grade 12 ESBQ (Extended Student Background Questions)
School Grade 12 Operational Questions
School Grade 12 Pilot Questions
Grade 12 HSTS (High School Transcript Study) Questions
SD (Student with Disabilities) Questions
ELL (English Language Learner) Questions

These specific questionnaires are the third group (Wave 3) of questionnaires submitted for
approval for usage in 2009.
2009 Wave 3 Supporting Statement

2

The Wave 1 submittal included the core (demographic) and subject-specific student background
questions for the grades 4 and 8 operational reading and mathematics assessments.
The Wave 2 submittal contained the subject-specific student background questions for the
operational science assessments at grades 4, 8, and 12; the operational English/Language Arts
and mathematics assessments at grade 12; the pilot reading and mathematics assessments at
grades 4 and 8; the pilot civics, U.S. history, and geography assessments at grades 4, 8, and 12;
and the grade 12 core (demographic) background questions.
2009 Wave 3 Burden Information
The Wave 3 submittal contains student questionnaires (National Indian Education Study [NIES])
at grades 4 and 8 and Extended Student Background Questionnaire [ESBQ] at grades 4, 8, and
12) as well as all the operational (Reading, Mathematics, Science) and pilot (Reading,
Mathematics, Social Sciences) teacher and school questionnaires for 2009. Also included are the
Student with Disabilities (SD) and English Language Learner (ELL) questionnaires and the
information collection questions for the High School Transcript Study (HSTS) for grade 12.
The Wave 3 burden information (see chart on page 7) is listed by grade and is comprised of the
following:
Grade 4
-

Student NIES questionnaires (15 minutes) --the estimated total number of NIES students at
grade 4 is 3,800 resulting in 950 burden hours.

-

Student ESBQ questionnaires (5 minutes) --the estimated number of ESBQ students at grade
4 is 7,500 resulting in a burden of 625 hours.

-

Teacher response burden for the teacher questionnaire is .33 hours (20 minutes). (Only
teachers at grades 4 and 8 complete surveys). The estimated number of teachers completing
questionnaires at grade 4 (including the NIES surveys) is 27,351 for a response burden of
9,026 hours.

-

The response burden for personnel completing the school questionnaires is .5 hours (30
minutes). The estimated number of school personnel completing questionnaires at grade 4 is
9,117 for a response burden of 4,559 hours.

-

SD and ELL questionnaires are completed by school personnel who are most knowledgeable
about students identified as SD or ELL. SD and ELL response burden for these

2009 Wave 3 Supporting Statement

3

questionnaires is .33 hours (20 minutes). At grade 4, the estimated number of SD and ELL
questionnaires to be completed is 18,189 resulting in a burden of 6,002 hours.
-

The total burden for questionnaires at grade 4 is 21,162 hours.

Grade 8
-

Student NIES questionnaires (15 minutes) --the estimated total number of NIES students at
grade 8 is 3,500 resulting in 875 burden hours.

-

Student ESBQ questionnaires (5 minutes) --the estimated number of ESBQ students at grade
8 is 7,500 resulting in a burden of 625 hours.

-

Teacher response burden for the teacher questionnaire is .33 hours (20 minutes). (Only
teachers at grades 4 and 8 complete surveys). The estimated number of teachers completing
questionnaires at grade 8 (including the NIES surveys) is 27,388 for a response burden of
9,038 hours.

-

The response burden for personnel completing the school questionnaires is .5 hours (30
minutes). The estimated number of school personnel completing questionnaires at grade 8 is
9,129 for a response burden of 4,565 hours.

-

SD and ELL response burden for these questionnaires is .33 hours (20 minutes). At grade 8,
the estimated number of SD and ELL questionnaires to be completed is 18,213 resulting in a
burden of 6,010 hours.

-

The total burden for questionnaires at grade 8 is 21,113 hours.

Grade 12
-

Student ESBQ questionnaires (5 minutes) --the estimated number of ESBQ students at grade
12 is 7,500 resulting in a burden of 625 hours.

-

No teacher questionnaires are completed at grade 12.

-

The response burden for personnel completing the school questionnaires is .5 hours (30
minutes). The school burden at grade 12 is comprised of two components. One is the regular
school questionnaire (1808 schools for 904 burden hours), and the second component is for
HSTS (High School Transcript Study). The HSTS requires school personnel to complete
forms that relate to information contained in 12th-grade student transcripts. The HSTS burden
is 3 hours per school (the schools are a subset of schools already included in the sample and
therefore are not counted as separate schools), thus the burden is 750 schools times 3 hours =

2009 Wave 3 Supporting Statement

4

2,250 total burden hours for that component. The resulting total school burden for grade 12 is
therefore 3,154 hours (904 + 2,250).
-

SD and ELL response burden for these questionnaires is .33 hours (20 minutes). At grade 12,
the estimated number of SD and ELL questionnaires to be completed is 4,346 resulting in a
burden of 1,434 hours.

-

The total burden for questionnaires at grade 12 is 5,213 hours.

See the following tables for a summary of Wave 3 burden estimates.

2009 Wave 3 Supporting Statement

5

Wave 3 Questionnaires - Single Survey Burden Times
Grade 4
Student Grade 4 NIES (National Indian Education Study) Questions
Student Grade 4 ESBQ (Extended Student Background Questions)
Teacher Grade 4 Operational Questions (Reading, Mathematics, Science)
Teacher Grade 4 Pilot Questions (Reading, Mathematics, Social Studies)
Teacher Grade 4 NIES (National Indian Education Study) Questions
School Grade 4 Operational Questions
School Grade 4 Pilot Questions
School Grade 4 NIES (National Indian Education Study) Questions
SD (Student with Disabilities) Questions
ELL (English Language Learner) Questions

15 minutes
5 minutes
20 minutes
20 minutes
20 minutes
30 minutes
30 minutes
30 minutes
20 minutes
20 minutes

Grade 8
Student Grade 8 NIES (National Indian Education Study) Questions
Student Grade 8 ESBQ (Extended Student Background Questions)
Teacher Grade 8 Operational Reading/Language Arts Questions
Teacher Grade 8 Operational Mathematics Questions
Teacher Grade 8 Operational Science Questions
Teacher Grade 8 Pilot Reading/Language Arts Questions
Teacher Grade 8 Pilot Mathematics Questions
Teacher Grade 8 Pilot Social Studies Questions
Teacher Grade 8 NIES (National Indian Education Study) Questions
School Grade 8 Operational Questions
School Grade 8 Pilot Questions
School Grade 8 NIES (National Indian Education Study) Questions
SD (Student with Disabilities) Questions
ELL (English Language Learner) Questions

15 minutes
5 minutes
20 minutes
20 minutes
20 minutes
20 minutes
20 minutes
20 minutes
20 minutes
30 minutes
30 minutes
30 minutes
20 minutes
20 minutes

Grade 12
Student Grade 12 ESBQ (Extended Student Background Questions)
School Grade 12 Operational Questions
School Grade 12 Pilot Questions
Grade 12 HSTS (High School Transcript Study) Questions
SD (Student with Disabilities) Questions
ELL (English Language Learner) Questions

5 minutes
30 minutes
30 minutes
3 hours
20 minutes
20 minutes

2009 Wave 3 Supporting Statement

6

Wave 3
Estimated Burden for NAEP 2009 Assessments Contained in This Submittal—By Grade Level
Components

# of
Students

Student
Burden
(in hours)

# of
Teachers /

Teacher
Burden

# of Schools

(in hours)

School
Burden
(in hours)

SD/ELL (#
of school
personnel)

SD/ELL
Burden

Total
Burden

(in hours)

(in hours)

Grade 4
NIES Student

3,800

950

950

ESBQ Student

7,500

625

625

Teacher
Personnel
(Operational,
Pilot, and NIES)

27,351

9,026

School Personnel
(Operational,
Pilot, and NIES)
Grade 4 Totals

27,351

9,026

9,026

9,117

4,559

18,189

6,002

10,561

9,117

4,559

18,189

6,002

21,162

11,300

1,575

NIES Student

3,500

875

875

ESBQ Student

7,500

625

625

Grade 8

Teacher
Personnel
(Operational,
Pilot, and NIES)

27,388

9,038

27,388

9,038

School Personnel
(Operational,
Pilot, and NIES)
Grade 8 Totals

11,000

1,500

9,038

9,129

4,565

18,213

6,010

10,575

9,129

4,565

18,213

6,010

21,113

Grade 12
NIES Student

N/A

ESBQ Student

7,500

625

625

School Personnel
(Operational,
Pilot, and NIES)
School Personnel
(HSTS1)
Grade 12 Totals

Overall Totals

1,808

904

4,346

1,434

2,250

2,250

7,500

625

0

0

1,808

3,154

4,346

1,434

5,213

29,800

3,700

54,739

18,064

20,054

12,278

40,748

13,446

47,488

* Note: The burden estimates for students in Wave 3 are comprised of 15 minutes for NIES questions and 5 minutes for ESBQ questions.
1

2,338

HSTS—A subset (750 schools) of the sample of all grade 12 schools will also be sampled to participate in the HSTS.

2009 Wave 3 Supporting Statement

7

Overview of NAEP 2008–2010 Assessments
The following broad overview of the 2008–2010 NAEP assessments was included as part of the
initial System Clearance submittal. The National Assessment Governing Board (the Governing
Board) determines NAEP policy and assessment schedule, and future Board decisions may result
in changes to some aspects of an assessment (e.g., which subjects are assessed in which years).
However, overall methodology and assessment process will remain constant. In the 2009
assessment year, questionnaires will be administered to students at grades 4, 8, and 12; to
teachers at grades 4 and 8; and to school administrators at grades 4, 8, and 12.
The 2009 assessments consist of the following:

• Operational assessments in reading, mathematics, and science in grades 4, 8, and 12;
• Pilot assessments for civics, U.S. history, and geography at grades 4, 8, and 12; pilot
assessments in reading and mathematics at grades 4 and 8; and

• Special studies, including (1) American Indian/Alaska Native (AI/AN) students at grades
4 and 8 as part of the National Indian Education Study (NIES); and (2) a pilot of
extended background questions as part of the development of a new socioeconomic status
variable at grades 4, 8, and 12.
Qualitative Testing of New Questions
Prior to pilot testing new types of background variable questions, ETS employs several different
methods of qualitative testing to ensure that these questions are consistent with the framework
and directions provided by NCES and are age-appropriate for the target population. One such
method, formal cognitive laboratory procedures, involves interviewing students, teachers, or
administrators to determine whether newly developed questions are functioning as intended.
Specifically, these procedures are used to increase the comprehensibility of the questions, clarify
wording, enhance construct validity, and identify and reduce sensitive or intrusive wording.
For most of the new questions included in this package, cognitive laboratory procedures were
conducted by the previous contractor. In addition, these new questions were presented for review
to the Background Variable Standing Committee, which has met twice in 2008. Subject-specific
questions were also presented for review by the Content Area Standing Committees, including
the Reading, Mathematics, U.S. History, Civics, and Geography committees. Additionally, ETS
2009 Wave 3 Supporting Statement

8

consulted with staff members who were former teachers and researchers whose specialties are in
classroom activities for the relevant grades and subjects.
Included in this review package are background questions that have been developed specifically
for the American Indian/Alaska Native student population at grades 4 and 8 as part of the
National Indian Education Study (NIES). Due to the special nature of this study and the unique
characteristics of this population, NAEP contractors conducted extensive cognitive laboratories
with 4th- and 8th-grade students, teachers, and school administrators at 26 schools in 5 regions
across the U.S. The data collected through these interviews were used to further refine the
questions for this study. The cognitive laboratory activities were cleared on April 18, 2008 under
#1850-0803, the NIES cognitive study.
How, By Whom, and for What Purpose the Data Will Be Used
In the original request for system clearance, the National Center for Education Statistics (NCES)
asked for approval of the instruments to be used to gather data from the 2008–2010 national and
state assessments. This submittal applies to the third set of questionnaires that will be submitted
for the 2009 assessments. This set of questionnaires contains (1) operational and pilot teacher
and school questions, (2) student questions for the NIES and the ESBQ (extended student
background questions for socioeconomic status) studies, (3) SD and ELL questionnaires, and (4)
forms for collecting the HSTS data.
Given that the purpose of NAEP is to gather data on the achievement of students in the subject
areas assessed for use in monitoring education progress, and because of the program's increasing
visibility, it is incumbent on the program to develop the most reliable and valid instruments
possible. To do so, NAEP has employed four strategies in recent assessment cycles:

2009 Wave 3 Supporting Statement

9

A. Small-scale pilot testing of new materials and test administration techniques;
B. Pilot testing items to determine which items best measure the constructs under
consideration;
C. Field testing* of operational assessments to accommodate the mandated six-month
reporting; and
D. Full-scale operations.
Background questionnaire development follows the same pattern as that of cognitive item
development, although we tend to pilot fewer items with less duplication and use the resulting
data to refine the questions. Guidance for what is asked is provided by the Governing Board.
NCES develops the questionnaires and asks for input from the Governing Board prior to
submission to OMB in a two-stage process: (1) prior to pilot testing and (2) again after NCES
and its contractors make selections for the operational assessment based on pilot data. The
questions are designed to provide

• the information for disaggregating data according to categories specified in the
legislation,

• contextual information that is subject specific (e.g., reading, mathematics) and has an
impact and known relationship to achievement, and

• policy relevant information specified by NCES and the Governing Board.
In all assessment years, questionnaires are generally administered to students at grades 4, 8, and
12; teachers at grades 4 and 8; and school administrators at grades 4, 8, and 12. SD and ELL
questionnaires are completed by teachers or administrators of students identified as learning
disabled or as English Language Learners.

*

Based on the results of recent assessment cycles, field testing to accommodate six-month reporting may no longer
be needed.
2009 Wave 3 Supporting Statement
10

Estimates of cost to the federal government
The current Wave 3 submission includes only a portion of the background variable components
of the 2009 NAEP assessment. The components included in Wave 3 are:
Grades 4 and 8 teacher questionnaires;
Grades 4, 8 and 12 school questionnaires;
Grades 4 and 8 National Indian Education Study (NIES) questionnaires (for students,
teachers and schools);
Grades 4, 8 and 12 Extended Student Background Questionnaires (ESBQ);
High School Transcript Study (HSTS) at grade 12; and
Students with Disabilities (SD) and English Language Learners (ELL) questionnaires (for
use across grades and subjects).
Based on the current contracts for the 2009-2011 assessments, the total cost to the federal
government for the development, printing, distribution, scoring, analysis, and reporting for the
2009 NAEP assessment is approximately $114.5 million. Given the highly integrated nature of
the data collection and subsequent processing, analysis and reporting activities, of which the
Wave 3 components are only part, estimated costs for the overall 2009 NAEP assessment are
provided and broken down as follows:
Development Costs –$5.4 million (includes question research, development, cognitive lab
tryout of questions)
Design, Analyses, and Reporting – $14.1 million (includes creation and analysis of
design plans, item analysis, DIF analysis, scaling, summarization of data, reporting
assessment results, assembling Technical Reports, producing public use data file).
Alliance Coordination and Research -- $1.3 million (includes management of all
contractor coordination efforts, Quality Assurance and QC Plans, Develop & Maintain a
Public Communications Receipt & Tracking System).
Sampling and Data Collection -- $65.0 million (includes site visits for data collection and
assessment administrations, development of sampling design and specifications).
Materials Preparation, Printing & Distribution, Scoring –$24.6 million (includes all test
booklet preparation and printing, hiring and conducting of all training personnel for
scoring).
Web Operations and Maintenance -- $4.1 million (includes development and support of a
NAEP information management system).
2009 Wave 3 Supporting Statement

11

Administration Cost Estimates

The following estimates apply to the 2009 administration costs:
-

$1.04 million for the printing, packaging, distribution out/in, processing, and data
delivery (Wave 3 materials only)

-

The cost of administering the assessment is $19.7 million. This is not broken down by
waves here because materials for the three waves of OMB submission are used
concurrently during field administration.

-

Given that Waves 1 and 2 are in the field at the same time, the marginal cost for
administering Wave 3 is estimated to be about 15-20 percent of the total cost of
administering the assessment (or about $4 million).

2009 Wave 3 Supporting Statement

12

Information for the 2009 Components Contained in Wave 3 Submittal
Teacher and School Questionnaires
Teacher Background Questionnaires
At grades 4 and 8, teachers of students participating in the main NAEP assessments are
administered a questionnaire. No teacher questionnaire will be administered at grade 12. At
grade 4, a single teacher questionnaire will be administered across all subjects assessed based on
the structure of the schools. At grade 8, teacher questionnaires are administered for subjectspecific areas.
Both operational and pilot mathematics- and reading-specific sections of the grades 4 and 8
teacher questionnaires will be administered in 2009. The science-specific sections of the teacher
questionnaires will be operational, and the subject-specific sections for U.S. history, civics, and
geography will be piloted.
The NAEP 2009 Teacher Questionnaires are composed of multiple parts: Background,
Education, and Training (Part I) and one or more Classroom Organization and Instruction
components that are subject-specific. Given the structure of elementary school, the grade 4
teacher questionnaire contains multiple subjects in the subject specific sections of a single
questionnaire. The grade 8 teacher questionnaires focus on a single subject (or, as is the case in
the 2009 pilot, on multiple-related subjects such as U.S. history, civics, and geography), utilizing
multiple forms of questionnaires.

2009 Wave 3 Supporting Statement

13

The following table depicts the layout for the 2009 teacher questionnaires.
Teacher Background Questionnaires for the NAEP 2009 Assessment
Grade

4

Type

Section 1

Section 2

Section 3

Section 4

Operational

BET

COIMATH

COIREAD

COISCI

Pilot

BET

COISCI

COIREAD (Pilot)

COIMATH (Pilot)

Math

BET

COIMATH

BET

COIREAD

BET

COISCI

BET

COIMATH (Pilot)

BET

COIREAD (Pilot)

BET

COISS (Pilot)

Section 5

COISS (Pilot)

Operational
Reading
8

Operational
Science
Operational
Math
Pilot
Reading
Pilot
US History,
Civics, &
Geography
Pilot

BET – Background, Education and Training

COI – Classroom Organization and Instruction

School Background Questionnaires. School questionnaires will be administered at each grade.
Three forms of the school questionnaire will be administered per grade across operational and
pilot sessions alike. Each form will contain all operational school questionnaire items. The
difference between the forms is a variable (pilot) section. For most schools, the School
Background Questionnaire will not include the pilot section. However, some schools will be
administered a questionnaire which includes either an additional Reading/Mathematics or Social
Studies pilot section.

2009 Wave 3 Supporting Statement

14

The NAEP 2009 School Questionnaires are composed of up to six parts: School Characteristics
and Policies (Part I); Operational Reading (Part II); Operational Mathematics (Part III);
Operational Science (Part IV); Pilot Items for Reading/English/Language Arts, Mathematics, or
Social Studies (Part V); and Charter School (Part VI). Unlike teacher questionnaires, the same
components of school questionnaires are used across grades since school administrators will
answer the school questionnaires.
The attached table depicts the layout for the 2009 school questionnaires.
2009 School Questionnaire Components
Grade

Form
A

Section 1
SCP

Section 2
Reading
(Operational)

Section 3
Mathematics
(Operational)

Section 4
Science
(Operational)

Section 5
Reading/Math
(Pilot)

Section 6
Charter
School
(Operational)

B

SCP

Reading
(Operational)

Mathematics
(Operational)

Science
(Operational)

Social Studies
(Pilot)

Charter
School
(Operational)

C

SCP

Reading
(Operational)

Mathematics
(Operational)

Science
(Operational)

Charter
School
(Operational)

A

SCP

Reading
(Operational)

Mathematics
(Operational)

Science
(Operational)

Reading/Math
(Pilot)

Charter
School
(Operational)

B

SCP

Reading
(Operational)

Mathematics
(Operational)

Science
(Operational)

Social Studies
(Pilot)

Charter
School
(Operational)

C

SCP

Reading
(Operational)

Mathematics
(Operational)

Science
(Operational)

Charter
School
(Operational)

A

SCP

Reading
(Operational)

Mathematics
(Operational)

Science
(Operational)

Reading/Math
(Pilot)

Charter
School
(Operational)

B

SCP

Reading
(Operational)

Mathematics
(Operational)

Science
(Operational)

Social Studies
(Pilot)

Charter
School
(Operational)

C

SCP

Reading
(Operational)

Mathematics
(Operational)

Science
(Operational)

Charter
School
(Operational)

4

8

12

SCP – School Characteristics and Policies

2009 Wave 3 Supporting Statement

15

The development activities for the teacher and school questionnaires entailed reviews of (a) the
past Issue Papers for the various subjects, (b) reviews of past operational questionnaires for
teachers and schools, (c) reviews of past pilot questionnaires for the teacher and school
questionnaires that have been piloted in 2008, and (d) the item performance results based on the
operational data and the 2008 pilot testing.
SD (Students with Disabilities) and ELL (English Language Learners)
These questionnaires are completed by school personnel who are most knowledgeable about
students identified as SD or ELL. NAEP encourages the inclusion of all students who can
meaningfully participate in the assessment, including those with disabilities and those with
limited English proficiency. In order to obtain a complete picture of educational progress for all
students, it is important to collect supplemental information on students in the sample who have
been identified as having a disability or are English language learners. Students with disabilities
include those who have an individual education plan (IEP), Section 504 plan, or equivalent
documentation.
The 2008 SD and ELL questionnaires received extensive review by the NAEP State Coordinator
Inclusion Special Interest Group (SIG), the NCES Accommodations and Inclusions Coordinating
Council, and NAEP developers and field staff. The 2009 versions of these questionnaires
incorporated revisions from the 2008 versions to ensure a more efficient collection of valid
student-level data. The flowchart format reduces the number of questions required to complete
the questionnaires, thus reducing overall burden. Since schools may have several students
identified as SD and/or ELL, this format is aimed at reducing overall school burden.
High School Transcript Study (HSTS)
The 2009 High School Transcript Study (HSTS) is designed to provide information about the
course-taking behavior of students graduating from high schools in the United States in 2009.
The study is being conducted by the NCES at schools participating in the 12th-grade 2009 NAEP.
In addition to the national HSTS, NCES will collect transcripts at the state level in one state as a
pilot test of the feasibility of a state-level transcript study. The same procedures that are used for
the national study will be used in the pilot study, including the collection of textbook
information. Transcript information is linked to NAEP scores for those graduates who
2009 Wave 3 Supporting Statement

16

participated in the science or mathematics NAEP assessments, permitting analysis of the
relationships between assessment scores and course-taking behavior.
High school transcript studies have been conducted by NCES as part of NAEP in 1987, 1990,
1994, 1998, 2000, and 2005. Prior to 2005, these studies collected catalogs and transcripts from
schools, and data collection staff were instructed to obtain whatever supplemental information
was necessary to interpret these documents. Since the data collected were administrative rather
than survey data and the questions asked were not highly standardized, HSTS staff did not seek
OMB clearance. In 2005, our procedures were modified to collect non-administrative data in a
more systematic fashion. We therefore requested and received OMB approval for the 2005 HSTS.
The 2009 HSTS will, with slight modifications, use the same procedures that were used in 2005.
The HSTS provides an understanding of curricula offered in our nation's high schools,
graduation requirements, course-taking behavior, and the relationships between these factors and
achievement on NAEP. The link between the HSTS and NAEP also permit analyses of the
relationships between the variables measured in the HSTS and the school and student
characteristics collected in NAEP.
Westat (the data collection contractor) will gather the data from approximately 750 high schools.
Transcript information will be obtained for approximately 28,000 students graduating in the
spring or summer of 2009. The average number of person hours to complete the School
Information Form, locate school catalogs, pull transcripts, and complete the textbook form is
estimated as 3.0 hours per school, based on experience with prior HSTS studies. Since an
estimated 750 schools are expected to participate, total response time is therefore estimated to be
2,250 hours for the national HSTS. Students are not asked to provide information for this study.
National Indian Education Study (NIES)
Guided by Executive Order, the Office of Indian Education (OIE) enlisted the help of NCES to
study the achievement and educational circumstances of American Indian/Alaska Native
(AI/AN) students. The resultant two part study, called the National Indian Education Study
(NIES), includes both an examination of the performance of AI/AN students on NAEP, and
surveys of student attitudes, teachers’ educational practices, and other school-related factors that
might affect the educational achievement of this population.
2009 Wave 3 Supporting Statement

17

The NIES will be conducted as part of the 2009 assessment. The national sample includes
students from both public and nonpublic schools that have both large and small AI/AN student
populations. The administration of the NAEP assessment will be followed with the
administration of a questionnaire specifically designed for the NIES study. Questionnaire data
will be linked to NAEP performance data in 2009.
Extended Student Background Questionnaire (ESBQ)
These questionnaires will be administered to a pilot sample at grades 4, 8, and 12. Variables to
be measured through ESBQ are intended to assess socioeconomic status (SES). Research
literature identifies education, occupation, and income/wealth as being the three major
components of socioeconomic status. In the case of NAEP, these three components are (a)
parental education, (b) parental occupational status, and (c) family income and wealth. With the
exception of parental education, however, the other two are not asked of NAEP respondents;
thus, NAEP relies on proxy variables to best measure SES.
The 2009 ESBQ consists of 7 matrix items at grade 4 and 10 matrix items at grades 8 and 12.
The items included are the following: household composition, number of siblings, mother and
father’s employment status, ownership of a cell phone, and ownership of a variety of home
possessions (such as Internet access or clothes dryer). At grades 8 and 12, three additional items
are included: mother’s education, father’s education, and home ownership (own, rent, or other).

2009 Wave 3 Supporting Statement

18