Extant Data Collection Protocal

Mathematics Professional Development Impact Study

Extant Data Collection Protocol

Extant Data Collection Protocol

The Mathematics PD Impact study will collect demographic and student achievement data on all seventh grade students in the treatment and control schools twice in the 2007-08 school year, twice in the 2008-09 school year, and once in the 2009-10 school year. This data collection plan reflects sensitivity to issues of efficiency, accuracy, and respondent burden. Information will be gathered from existing data sources rather than imposing additional burden by collecting primary data. A key consideration in the collection of extant data via existing administrative records as opposed to administering surveys to sample members was to minimize evaluation costs and reduce respondent burden.

The purposes of the extant data to be collected are:

• Provide rosters from which to sample students for computer-adaptive testing of their achievement in the area of rational numbers in the fall and spring of each year. The use of sampling is intended to reduce the number of students that must be pulled from classroom instruction to take the test.

• Provide the outcome for the analysis of whether the focused rational numbers professional development treatment has an impact on students’ broader mathematics achievement on state accountability assessments.

• Provide covariates (control variables) that can be included in analyses to reduce unexplained variance.

• Describe the study sample in relation to the full population in participating schools and the nation.

We plan to obtain student-level data for each cohort of students. The test administration subcontractor for the study will work with a contact person in each school’s (or district’s) data processing department to obtain these data. We will submit our requests far in advance of computer-adaptive testing to give schools and districts ample lead time to respond to them; given student mobility we expect that some rosters will change and we have increased our sample size to account for these students who change classes, grades, or schools.

Each extant data collection form will consist of a written request to the school or district for an electronic file containing one record for each seventh grade student in each treatment and control school in the study. The data elements that will be requested are listed Exhibit 1, below.

If the particular data elements listed in Exhibit 1 are not available for a particular district, we will work the district to identify reasonable proxies.

Exhibit 1. Extant Student Data to Be Obtained from Districts

A. Identifiers

• School

• Teacher(s)/classroom(s)¹

• Student²

B. Student Background Variables

• Birth month and year

• Gender

• Ethnicity (current U.S. census categories)

• Days absent (up to rostering date—spring data collection only)

• National school lunch program status

• Free

• Reduced

• Neither

• English language status

• Limited English proficiency (LEP)

• Native English speaker

• Individualized Education Plan (IEP) status

• Yes

• No

• In a gifted and talented (GATE) program?

• Yes

• No

C. Student Achievement Variables

• Cohort I - Students in participating seventh grade classrooms in the 2007-08 school year

  • Student 6^th grade state accountability mathematics assessment score 2006-07

  • Student 7^th grade state accountability mathematics assessment score 2007-08

  • Student 8^th grade state accountability mathematics assessment score 2008-09

• Cohort II - Students in participating seventh grade classrooms in the 2008-09 school year

  • Student 6^th grade state accountability mathematics assessment score 2007-08

  • Cohort II Student 7^th grade state accountability mathematics assessment score 2008-09

1 It is possible that some students will be taught by more than one teacher over the course of the year. We will work with districts or schools to determine the appropriate teacher ID to list on the data file, and to document such moves if possible so that they can be taken into account in the student outcome analyses.

2 A student identifier is needed to maintain student rosters and to link students to classrooms for sampling for the student test administration; but it need not be the actual ID number used by the district. We anticipate creating a “crosswalk” document, which contains the study ID linked to local IDs, that would remain within the school or district. This “crosswalk” document would allow us to locate students to administer tests when we were in the schools, but would allow student names and local IDs to never leave the school or district office. A student identifier will also be needed to allow linking of information across files, if data are provided in more than one file. By making it possible for the project team to carry out the linking of files, an identifier would reduce the burden on the district. We plan to work with the Department and with the districts involved in the study to develop a strategy for collecting the required achievement data that meets the needs of the project and is feasible for the districts, while meeting confidentiality requirements.

Paperwork Burden Statement

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is 1850–XXXX. The time required to complete this information collection is estimated to average 15 hours per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Education, Washington, D.C. 20202–4651. If you have comments or concerns regarding the status of your individual submission of this form, write directly to: Elizabeth Warner, U.S. Department of Education, Institute of Education Sciences, Room 502a, 555 New Jersey Avenue, NW, Washington, DC 20202–0001.

American Institutes for Research • MDRC • REDA International, Inc.