Download:
pdf |
pdfImpact Evaluation of
Departmentalized Instruction in
Elementary Schools
Part A: Supporting Statement for
Paperwork Reduction Act Submission
March 8, 2018
Submitted to:
U.S. Department of Education
National Center for Education Evaluation
Institute of Education Sciences
550 12th Street, S.W.
Washington, DC 20202
Project Officer: Tom Wei
Contract Number: ED-IES-17-C-0064
Submitted by:
Mathematica Policy Research
P.O. Box 2393
Princeton, NJ 08543-2393
Telephone: (609) 799-3535
Facsimile: (609) 799-0005
Project Director: Alison Wellington
Reference Number: 50533
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
CONTENTS
PART A. SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT SUBMISSION ................. 1
Justification ................................................................................................................................ 2
A1.
Circumstances necessitating the collection of information ......................................... 2
A2.
Purpose and use of data ........................................................................................... 11
A3.
Use of technology to reduce burden ......................................................................... 13
A4.
Efforts to avoid duplication of effort ........................................................................... 14
A5.
Methods of minimizing burden on small entities ....................................................... 14
A6.
Consequences of not collecting data ........................................................................ 14
A7.
Special circumstances .............................................................................................. 15
A8.
Federal register announcement and consultation ..................................................... 15
A9.
Payments or gifts....................................................................................................... 16
A10. Assurances of confidentiality..................................................................................... 17
A11. Justification for sensitive questions ........................................................................... 18
A12. Estimates of hours burden ........................................................................................ 18
A13. Estimate of cost burden to respondents ................................................................... 20
A14. Annualized cost to the federal government ............................................................... 20
A15. Reasons for program changes or adjustments ......................................................... 20
A16. Plans for tabulation and publication of results .......................................................... 20
A17. Approval not to display the expiration date for OMB approval .................................. 21
A18. Exception to the certification statement .................................................................... 21
REFERENCES .......................................................................................................................................... 22
APPENDICES
APPENDIX A: MONITORING FORMS AND PRINCIPAL INTERVIEW PROTOCOL
APPENDIX B: TEACHER SURVEY WITH INVITATION LETTER AND NONRESPONSE
FOLLOW-UP MATERIALS
APPENDIX C: ADMINISTRATIVE RECORDS DATA REQUESTS
APPENDIX D: LETTER REQUESTING CLASS SCHEDULES AND STUDENT ROSTERS
APPENDIX E: ACTIVE AND PASSIVE PARENT PERMISSION FORMS
APPENDIX F: SCHOOL AGREEMENT FORM
APPENDIX G: DISTRICT RECRUITMENT LETTER
APPENDIX H: CONFIDENTIALITY PLEDGE FOR MATHEMATICA EMPLOYEES
ii
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
TABLES
A.1
Data collection .................................................................................................................................. 5
A.2
Research questions and data sources ........................................................................................... 12
A.3
Schedule of major study activities .................................................................................................. 13
A.4
Technical Working Group Experts ................................................................................................. 16
A.5
Estimated response time for data collection .................................................................................. 19
iii
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
PART A. SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT
SUBMISSION
This package requests clearance for data collection activities to support an evaluation of
departmentalized instruction in elementary schools. The Institute of Education Sciences (IES),
National Center for Education Evaluation and Regional Assistance, U.S. Department of
Education (ED) has contracted with Mathematica Policy Research, Inc. (Mathematica) and its
partners (Public Impact; Clowder Consulting, LLC; Social Policy Research Associates; and IRIS
Connect) to conduct this evaluation.
Departmentalized instruction, where each teacher specializes in teaching one subject to
multiple classes of students instead of teaching all subjects to a single class of students (selfcontained instruction), has recently become more popular as an improvement strategy in
elementary schools. However, virtually no evidence exists on its effectiveness relative to the
more traditional self-contained approach to instruction. This evaluation will help to fill the gap
by examining whether departmentalizing fourth and fifth grade teachers improves teacher and
student outcomes. The evaluation will focus on math and reading, with an emphasis on lowperforming schools that serve a high percentage of disadvantaged students.
The evaluation will include implementation and impact analyses. The implementation
analysis will describe schools’ approaches to departmentalization and benefits and challenges
encountered. The analysis will be based on information from schools’ study agreement forms,
meetings to design each school’s approach to departmentalization; monitoring and support calls
with schools; principal interviews; and teacher surveys. The impact analysis will draw on data
from teacher surveys, videos of classroom instruction, principal interviews, and district
administrative records to estimate the impact of departmentalized instruction on various
outcomes. These outcomes include the quality of instruction and student-teacher relationships,
teacher satisfaction and retention, and student achievement and behavior.
This package provides a detailed discussion of the procedures for these data collection
activities and copies of the forms and instruments.
1
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
Justification
A1. Circumstances necessitating the collection of information
a.
Policy context and statement of need
This evaluation is authorized by Title VIII Section 8601 of the Elementary and Secondary
Education Act (ESEA) as amended most recently in 2015 by the Every Student Succeeds Act
(ESSA). ESSA gives states considerable flexibility in designing systems to hold their schools
accountable for improving student achievement. This flexibility extends to the types of strategies
that states encourage or require their low-performing schools to adopt. However, many strategies
in use have little to no evidence of effectiveness. More research is needed to help states identify
strategies that are likely to help their low-performing schools improve.
By the upper elementary grades, low-income students’ achievement lags several years
behind that of higher-income students (Duncan and Magnuson 2011). One potential
improvement strategy is to departmentalize instruction for upper elementary grade students, an
approach in which teachers specialize in teaching specific subjects. This strategy, which
secondary schools already use almost universally, holds promise for several reasons. Many
teachers are, to some degree, more effective at teaching particular subjects (Condie et al. 2014;
Fox 2016; Goldhaber et al. 2013). Assigning teachers to those subjects could raise student
achievement. It also allows teachers to concentrate planning on fewer subjects, which may lead
to more thoughtful lessons and deeper instructional or content knowledge in those subjects (Chan
and Jarman 2004). However, some experts worry that departmentalization could harm struggling
students, particularly low-income students, by compromising student-teacher relationships
(McPartland and Braddock 1993). In particular, teaching more students may make teachers less
aware of each student’s needs; having more teachers may make students feel less connected to
each teacher. These factors could decrease student achievement, offsetting any gains from being
taught by teachers who are more effective in the subjects they teach.
Despite concerns about departmentalization in elementary grades, elementary schools are
increasingly adopting it. The percentage of elementary teachers in departmentalized settings
more than doubled over a recent 12-year period, from 6 percent in 1999–2000 to 15 percent in
2011–2012 (U.S. Department of Education [ED] 2009; Goldring et al. 2013).
Among elementary grades, the upper elementary grades may be the grades at which
departmentalization holds the greatest promise. Instruction in those grades could require more
content knowledge than in the lower grades, so there could be a greater benefit from teachers
specializing in particular subjects. Data on teachers’ effectiveness, particularly measures based
on student scores from state assessments, are also more prevalent in the upper elementary grades,
providing more information with which principals can assign teachers to the subjects they teach
best. In addition, given concerns about whether departmentalization is developmentally
appropriate for young students (Chang et al. 2008), departmentalizing the upper elementary
grades may generate fewer concerns than doing so in the lower grades.
Given the increased use of departmentalization and numerous ways it might affect students,
there is an urgent need for more evidence on its effects. The only random assignment study of
departmentalization (Fryer 2016) found that it reduced upper elementary students’ achievement
after one year but had no effect over two years. The study was limited to one district whose
2
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
schools departmentalized with little to no implementation support. A few nonexperimental
studies found associations between departmentalization and achievement ranging from positive
to negative (McGrath and Rust 2002; Taylor-Buckner 2014). However, these findings could
reflect unmeasured differences between the students or teachers in departmentalized and selfcontained classrooms. In nonexperimental studies that examined nonacademic outcomes,
students in departmentalized settings reported worse or similar feelings toward their classroom
compared with students in self-contained classrooms (Chang et al. 2008; Parker 2009).
Overall, the educational community lacks large-scale, conclusive evidence on whether
departmentalization helps or harms elementary students. This study will address the gap by
providing rigorous evidence on the effectiveness of departmentalized instruction on teacher and
student outcomes.
b.
Treatment
This study will measure the impact of switching from self-contained to departmentalized
instruction in upper elementary grades, specifically 4th and 5th grades. To help treatment schools
transition to departmentalized instruction, the study team will provide implementation support.
This support will include two design meetings before the start of the 2018–2019 school year to
help schools determine the most effective structure for departmentalization and provide
principals with advice on how to assign teachers to subjects. It will also include support calls to
treatment schools while they are implementing departmentalization during the 2018–2019 and
2019–2020 school years to help navigate any challenges that may arise.
c.
Study design and research questions
This study will use a random assignment design to estimate the impact of departmentalized
instruction in elementary schools on teacher and student outcomes. The study will recruit
approximately 200 schools from 12 districts for the study.1 The study team will randomly assign
schools to one of two groups – a treatment group that departmentalizes instruction in 4th and 5th
grades for two years (2018–2019 and 2019–2020 school years) and a control group that
continues using self-contained classrooms. In the treatment schools, principals will determine
teachers’ assignments to subjects with guidance and support from the study team.
We will estimate the impact of departmentalized instruction in two different types of
districts—those with teacher effectiveness measures based on student achievement growth and
those without these measures. Impacts of departmentalized instruction could vary across these
two sets of districts. In districts with these scores, principals can use the scores to determine
teachers’ relative effectiveness in reading and math to assign teachers to the subjects they teach
best. However, 63 percent of districts nationwide do not have teacher effectiveness measures
based on student achievement growth (Troppe et al. 2017). To examine whether impacts differ
depending on the availability of these data, we will aim to draw roughly half the school sample
from districts with teacher effectiveness data and half from districts without these data. This
study design will allow us to (1) estimate the overall impacts of departmentalized instruction
across a range of districts and (2) estimate the impact of departmentalized instruction in districts
1
Appendix G contains a copy of the letter that will be used to inform and recruit school districts.
3
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
with and without teacher effectiveness scores. It will also include implementation analyses that
will provide context for interpreting impact results and shed light on the mechanisms through
which departmentalized instruction may affect teacher and student outcomes.
The research questions for this study are:
d.
1.
What is the impact of departmentalization in grades 4 and 5 on student outcomes, such
as achievement in math and reading, attendance, and disciplinary incidents?
2.
What is the impact of departmentalization in grades 4 and 5 on teacher outcomes, such
as instructional quality, teachers’ relationships with students and parents, job
satisfaction, confidence in teaching abilities, and retention?
3.
Do the impacts of departmentalization differ based on whether principals have access to
teacher effectiveness scores when assigning teachers to subjects?
4.
How do schools structure departmentalization, including number of subjects and classes
per teacher, assignment of teachers to subjects, and time allocated to instruction and
planning?
5.
How do principals’ actual assignments of teachers to subjects compare with
assignments based solely on baseline teacher effectiveness scores?
6.
What challenges and benefits do principals and teachers perceive in switching to
departmentalization?
Data collection
This study includes multiple data collection efforts. Data for the impact analyses will be
collected from districts, schools, principals, and teachers. The study team will also collect data to
describe implementation fidelity. Since we are video-recording classrooms, we will obtain
permission from parents to include their child in video recordings. All of these data are described
below and summarized in Table A.1.
4
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
Table A.1. Data collection
Instrument/Activity
Data need
Respondent
Mode
Schedule
School agreement form
Baseline data on the number of
4th and 5th grade classrooms and
types of teacher performance data
Principals in
study schools
Paper
Spring 2018
Departmentalization design
meeting form
Structure of departmentalization
(e.g. number of teachers, subjects
and sections taught by each); how
teachers are assigned to subjects;
schedule and planning time
Study team
completes for
treatment
schools
Paper
Spring/summer
2018 at
conclusion of
design meetings
Monitoring call forms
Teacher grade and subject
assignments (all schools);
Challenges related to
departmentalized instruction
(treatment schools)
Principals in
study schools
Electronic
Fall and spring
during 2018–
2019 and 2019–
2020 school
years
Principal interview protocol
Successes and challenges related
to instructional structure; parent
communication; disciplinary
incidents (all schools); challenges
and benefits of
departmentalization; perceptions
of approach to assigning teachers
to subjects (treatment schools)
Principals in
study schools
Electronic
Spring 2019
Class schedules
Class schedules for math,
reading, or self-contained 4th
grade classes in schools selected
for video-recording
Teacher
Paper or electronic
list of each
subject/class
taught by the
teacher by day of
the week and time
Spring 2019
Student rosters
List of students in 4th grade
classes selected for classroom
video-recordings used to prepare
parent permission packets and
track returned permission forms
Teacher
Paper or electronic
list of students
enrolled in selected
teachers’
classrooms
Spring 2019
Videos of 4th grade teachers’
classroom instruction
Quality of instruction in math and
reading; quality of student-teacher
interactions
Study team
video records
one-half of 4th
grade
classrooms
Two videos of
classroom
instruction will be
conducted and
scored by the study
team for each 4th
grade teacher’s
selected class
Spring 2019
Teacher survey
Time devoted to instruction,
planning, and professional
development; teachers’
awareness of students’ learning
styles; satisfaction and confidence
in teaching; school instructional
structure; opportunities to
coordinate with other teachers;
successes and challenges during
school year
All 4th grade
teachers
Web-based survey
Spring 2019
Principals/schools
Teachers
Districts
5
CONTRACT NUMBER: ED-IES-17-C-0064
Instrument/Activity
MATHEMATICA POLICY RESEARCH
Data need
Respondent
Mode
Schedule
District administrative
records on teacher
effectiveness from the 2016–
2017 school year (in districts
with teacher effectiveness
scores)
Baseline teacher effectiveness
scores and teacher experience
Districts
Electronic records
for all 4th through
8th grade teachers
in the district who
taught math or
reading in the
2016–2017 school
year
Spring 2018
through fall
2018
District administrative
records on teacher
effectiveness from the 2016–
2017 school year (in districts
without teacher effectiveness
scores)
Data needed to estimate baseline
teacher effectiveness (students’
current and prior-year
achievement in reading and math,
student characteristics and
teacher experience)
Districts
Electronic records
for all 4th through
8th grade students
in the district in
spring 2017 (linked
to their math and
reading teachers)
Spring 2018
through fall
2018
District administrative
student and teacher records
from the 2017–2018 and
2018–2019 school years (in
all study districts)
Student achievement in reading
and math, student behavior
(attendance, disciplinary
incidents), student characteristics
(such as gender, age, special
education status, English learner
status)
Districts
Electronic records
for all 2nd–4th
graders enrolled in
study schools in
spring 2018
Summer/fall
2019
Teachers’ school assignments
and characteristics (demographic
information, educational
attainment, years of teaching
experience)
District administrative
student and teacher records
from the 2019–2020 school
year (in all study districts)
Student achievement in reading
and math, student behavior
(attendance, disciplinary
incidents), student characteristics
(such as gender, age, special
education status, English learner
status)
Electronic records
for teachers who
ever taught 4th or
5th grade in a
study school during
the 2017–2018 or
2018–2019 school
year
Districts
Teachers’ school assignments
and characteristics (demographic
information, educational
attainment, years of teaching
experience)
Electronic records
for all 2nd–4th
graders enrolled in
study schools in
spring 2018
Summer/fall
2020
Electronic records
for teachers who
ever taught 4th or
5th grade in a
study school during
the 2017–2018,
2018–2019, or
2019–2020 school
year
Parents and students
Parent permission forms
Active and passive permission
forms (depending on district
requirements and approved by
IRB) for parent or guardian to
document consent for student to
be included in videos of
classroom instruction
Parent or
guardian
Paper permission
form indicating
consent or nonconsent for
students to be
included in videos
Fall/winter
2018-2019
school year
(distributed and
collected by
study team and
teachers)
Student assent form
Student assent to be included in
classroom videos, if necessary
4th grade
students in a
video-
Paper
Spring 2019
6
CONTRACT NUMBER: ED-IES-17-C-0064
Instrument/Activity
MATHEMATICA POLICY RESEARCH
Data need
Respondent
Mode
Schedule
recorded
classroom
School agreement form. We will collect school agreement forms from all principals of
study schools in spring 2018 (Appendix F). The form will explain the study requirements and
require principals to sign the form to indicate they understand and agree to adhere to those
requirements. In addition, the form will include questions about the number of 4th and 5th grade
classrooms in the school and the types of teacher performance data available to principals. This
information will help Public Impact prepare to provide treatment schools with technical
assistance and will also be used to describe the study context.
Departmentalization design meeting form. Study staff will meet with principals of
treatment schools in the spring/summer 2018 to support treatment schools’ transition to
departmentalized instruction. At the conclusion of these design meetings, the study team will
complete forms documenting how each school will implement departmentalized instruction. For
example, these forms will describe the structure of departmentalization (including the number of
teaching positions and how subjects will be split across positions) and the daily schedule
(including the number of transitions for students, amount of individual and group planning time,
and plans for structuring subject- and grade-level planning meetings). The forms will also
indicate each teacher’s teaching assignment and the factors principals considered when making
assignments (such as the teachers’ performance in math and reading, principals’ observations of
teachers’ instruction, and teachers’ educational background). This information is necessary for
Public Impact to provide treatment schools with technical assistance and will not impose an
additional data collection burden on principals.
Monitoring call forms. The study team will speak with the study schools by phone during
the fall and spring of both implementation years (2018–2019 and 2019–2020 school years). The
study team will complete an electronic form after each call to collect information from both
treatment and control schools (Appendix A). All principals will be asked to verify teacher grade
and subject assignments and whether there have been any recent changes related to the school’s
instructional structure. Principals in treatment schools will also be asked if they are encountering
challenges implementing departmentalized instruction. Public Impact will schedule follow-up
calls with principals of treatment schools who indicate they need more support to implement
departmentalization.
Principal interview protocol. During the spring 2019 monitoring call, the study team will
also ask treatment and control principals an additional set of questions (Appendix A). The study
team will collect standardized information from all study principals on factors considered when
deciding teachers’ subject (if appropriate) and grade assignments, teachers’ communication with
parents, and the schools’ and teachers’ handling of disciplinary issues. We will also ask
principals of treatment schools about their perceptions of the challenges and benefits of
departmentalization.
Class schedules. In spring 2019, we will collect class schedules for math, reading, or selfcontained 4th grade classes in schools selected for video-recordings. The list will include all
math and reading classes, including the day of the week and time of day they are scheduled. This
7
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
information will be used to randomly select and schedule specific classes to video-record. To
limit the burden on teachers and study costs, the study team will randomly select 100 schools
(half treatment and half control) to participate in the video-recordings during the first year of
implementation. By February 2019, we will send a letter to 4th grade teachers outlining the need
to collect information on their daily/weekly class schedule and student rosters (Appendix D).
Field staff will enter the information provided by the teacher directly into an electronic
spreadsheet.
Student rosters. In spring 2019, a list of students will be obtained from 4th grade teachers
for the classes selected to be video-recorded. This list will be used to develop parent permission
packets and to accurately track returned forms. We will use this student list to obtain parent
permission for all students in classes being video-recorded during spring 2019. Field staff will
enter the information provided by the teacher directly into an electronic spreadsheet.
Videos of classroom instruction. To measure the quality of instruction and teacher-student
interactions, in spring 2019 the study team will video-record an average of two 30-minute
lessons of 4th grade classes selected to be video-recorded. Study team videographers will record
and upload the videos, and study team members will rate the videos using the Classroom
Assessment Scoring System (CLASS) observation instrument. The CLASS measures the quality
of student-teacher interactions, is valid and reliable (Kane and Staiger 2012; Pianta et al. 2012),
and has strong procedures for training raters (Pianta et al. 2012). It is also suitable for teachers in
multiple subjects. To help ensure the quality of the ratings for the study, each rater will be
assigned an even mix of treatment and control teachers and will be blind to teachers’ intervention
status. This approach will address the potential for rater bias due to expectations about the effect
of departmentalized instruction. Raters will be thoroughly trained and certified on the CLASS
with regular calibration throughout the coding period. Each lesson will be scored by a single
rater; however, to increase the reliability of the ratings, a different rater will observe each video
for a given teacher. The video recordings will occur during teachers’ normal class lessons and
will not impose any additional burden on teachers.
Teacher survey. A thirty-minute, web-based teacher survey will collect information about
teachers’ time devoted to instruction, planning, and professional development, as well as their
opportunities to coordinate with other teachers, and their perceptions of the successes and
challenges related to planning and providing instruction and building relationships with students
and parents (Appendix B). The survey will also measure teacher satisfaction and confidence in
their teaching and level of awareness of student learning styles. Teachers will also be asked to
report on their perceptions about the structure of teaching positions in their grade level and how
teachers were assigned to classes or subjects. The survey will be administered to both treatment
and control teachers; however, to limit the burden on teachers and study costs, we will only
survey 4th grade teachers during the first year of implementation (spring 2019).
District administrative records on teacher effectiveness from the 2016–2017 school
year. The study will use information on teachers’ effectiveness in math and reading from the
2016–2017 school year for two main purposes:
8
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
For teachers who taught 4th or 5th grade in a study school in the 2017–2018 school year
(the final baseline year), we will examine whether departmentalization had different
impacts on the retention of teachers with high and low baseline effectiveness scores.
For teachers who taught 4th or 5th grade in a study school in the 2018–2019 school year
(the first implementation year), we will examine whether teachers assigned to teach
math and reading in treatment schools had higher baseline effectiveness scores than
those assigned to teach math and reading in control schools. We will also compare
treatment principals’ assignment decisions with the assignments based solely on the
baseline effectiveness scores.
Although the 2016–2017 school year is not the final baseline year, it will be the most recent
school year for which principals will have teacher effectiveness information (if available) when
making decisions about teachers’ assignments for the 2018–2019 school year. Therefore, by
collecting these data, the study will be able to assess how closely principals relied on this
information when making assignments.
In districts where teacher effectiveness scores from the 2016–2017 school year are available,
we will request this information for all 4th through 8th grade teachers in the district who taught
math or reading that year (Appendix C). The requested data will not be limited only to teachers
who taught in the study schools or study grades (grades 4 and 5). As discussed above, one of the
groups whose baseline effectiveness we would like to measure consists of 4th and 5th grade
teachers in the study schools in 2018–2019. However, at the time that we request effectiveness
data (starting in spring 2018), districts will not know who those teachers are. In fact, some of
those teachers may have taught in other grades or schools in the 2016–2017 school year before
getting assigned to teach 4th or 5th grade in the study schools. Therefore, we will request data on
all teachers in the district who are expected to have effectiveness scores from the 2016–2017
school year. For teachers in the study who do not have such scores, we will impute those scores
based on their years of teaching experience.
In districts where teacher effectiveness scores are not available, the study team will estimate
teachers’ effectiveness from the math and reading test scores of their students in spring 2017
(Appendix C). To estimate teachers’ effectiveness, we will compare spring 2017 student test
scores across teachers while statistically controlling for the students’ characteristics and prior test
scores. As described previously, because we would like to measure the effectiveness of all 4th
through 8th grade teachers in the district who taught math or reading in the 2016–2017 school
year, we will request spring 2017 test scores on all 4th through 8th grade students, linked with
their teachers. Their prior test scores will come from spring 2016.
District administrative student and teacher records from the 2017–2018 and 2018–2019
school years. In the summer and fall of 2019, we will collect administrative records from the
final baseline year (2017–2018) and the first implementation year (2018–2019) on two topics
(Appendix C): (1) student outcomes and characteristics and (2) teachers’ school assignments and
characteristics. The data we will collect on these topics will be identical in districts with and
without teacher effectiveness scores. We discuss each topic next.
Student outcomes and characteristics. District administrative student records from the 2017–
2018 and 2018–2019 school years will serve two key purposes:
9
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
Compare students in treatment and control schools at baseline: We will compare their
baseline outcomes (achievement, attendance, and disciplinary incidents) from the 2017–
2018 school year. We will also compare their characteristics (for example, gender, age,
free and reduced-price lunch status, special education status, and English learner status).
These data will allow us to determine whether random assignment produced treatment
and control groups that were similar at baseline.
Estimate the impacts of departmentalized instruction on student outcomes after one year
of implementation: We will estimate impacts on student achievement, attendance, and
disciplinary incidents in the 2018–2019 school year. Students’ characteristics and
baseline outcomes from 2017–2018 will serve as covariates in the impact estimation
models to increase the precision of the estimates.
We will request these data for students enrolled in grades 2 through 4 in a study school at
the time of random assignment (spring 2018). The study sample is based on students enrolled at
the time of random assignment because those students are expected to be similar in treatment and
control schools before the study begins, whereas students who join study schools afterwards may
not be. Students enrolled in grades 3 and 4 in spring 2018 are expected to be in the study grades
(grades 4 and 5) in the first implementation year. Students enrolled in grade 2 in spring 2018 are
expected to reach a study grade (grade 4) by the second implementation year. For all students in
this sample, we will request data from both years (2017–2018 and 2018–2019) regardless of how
long they stayed in the study schools. This will allow us to have outcome data on students who
leave the study schools but stay within the district, minimizing the potential for bias from
missing outcome data.
Teachers’ school assignments and characteristics. District administrative records on
teachers’ school assignments and characteristics from the 2017–2018 and 2018–2019 school
years will serve two key purposes:
For teachers who taught 4th or 5th grade in a study school in the 2017–2018 school
year, we will use the school assignment data to estimate the impact of departmentalized
instruction on teacher retention. We will also use data on their characteristics as
covariates in the impact estimation models.
For teachers who taught 4th or 5th grade in a study school in the 2018–2019 school
year, we will compare the characteristics of teachers in treatment and control schools to
assess whether departmentalized instruction led to changes in the types of teachers who
chose to work in schools and grades with this staffing structure.
Teacher characteristics that we will collect include demographics (age, sex, race, and ethnicity);
educational attainment (certifications, degrees, and scores on licensure or certification exams);
and years of teaching experience.
District administrative student and teacher records from the 2019–2020 school year. In
the summer and fall of 2020, we will collect administrative records from the second
implementation year, 2019–2020, on (1) student outcomes and characteristics and (2) teachers’
school assignments and characteristics.
10
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
Student records from the 2019–2020 school year will allow us to examine the impacts of
departmentalized instruction on student achievement and behavior after two years of
implementation. We will collect these records for the same students who were included in the
previous request submitted in the fall of 2019—students enrolled in grades 2 through 4 in study
schools at the time of random assignment. This will allow us to examine how impacts evolve for
a constant sample of treatment and control students who were similar at the beginning of the
study.
Records on teachers’ school assignments and characteristics from the 2019–2020 school
year will serve a number of purposes. For teachers who taught 4th or 5th grade in a study school
in the 2017–2018 school year, we will examine whether departmentalized instruction affected
their likelihood of working in the same school two years later. For teachers who taught 4th or 5th
grade in a study school in the 2018–2019 school year, we will examine whether it affected their
likelihood of working in the same school the next year. For teachers who taught 4th or 5th grade
in a study school in the 2019–2020 school year, we will compare the characteristics of teachers
in treatment and control schools to assess whether departmentalized instruction led to changes in
the types of teachers who chose to work in schools and grades with this staffing structure.
Parent permission forms. We will distribute paper permission forms to parents of students
in the 4th grade classrooms selected for video-recording in spring 2019. In districts that require
active consent, we will collect permission forms from parents or guardians to document
permission for students to be included in videos of classroom instruction (Appendix E). In
districts that permit passive consent, we will collect forms from parents who indicate that they do
not give permission for their child to be included in the videos. The permission forms will be
collected in fall/winter of the 2018-2019 school year in preparation for video-recording
classroom instruction in spring 2019. Videographers will have a list of which students’ parents
provided permission to be recorded (and which did not), and they will be trained to seat children
without permission outside of the view of the camera. All consent materials will be reviewed and
approved by district research boards and the study’s IRB. The IRB approval number and contact
information will be included on the parent permission forms and accompanying letter that will
provide information about how the recordings will be used, by whom, and their destruction at the
end of the study. Study field staff will bring parent consent forms to teachers’ classrooms to be
sent home with students and returned to the teacher prior to video-recording classrooms.
Student assent form. Although student assent is typically reserved for students in 6th grade
or older, we will adhere to any IRB or district requirements to include student assent and have
included an assent form in Appendix E.
We are not requesting OMB approval for the collection of information obtained from
departmentalization design meeting forms or the video recordings of teachers’ classroom.
Neither of these data collection activities will impose an additional burden on principals or
teachers, as explained above.
A2. Purpose and use of data
Data for this evaluation will be collected and analyzed by Mathematica and its partners. This
work will be completed under contract number ED-IES-17-C-0064. The data will be used to
address the study’s research questions, as shown in Table A.2.
11
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
Table A.2. Research questions and data sources
Data sourcesa
Research question
Impacts of departmentalization on student and teacher outcomes
What is the impact of departmentalization in grades 4 and 5 on
student outcomes, such as achievement in math and reading,
attendance, and disciplinary incidents? (RQ1)
What is the impact of departmentalization in grades 4 and 5 on
teacher outcomes, such as instructional quality, teachers’
relationships with students and parents, job satisfaction, confidence
in teaching abilities, and retention? (RQ2)
District administrative records
Videos of classroom instruction, teacher
survey, district administrative records
Impacts of specific approaches to departmentalization
Do the impacts of departmentalization differ based on whether
principals have access to teacher effectiveness scores when
assigning teachers to subjects? (RQ3)
District administrative records
Implementation of departmentalization instruction
How did schools structure departmentalization, including number of
subjects and classes per teacher, assignment of teachers to
subjects, and time allocated to instruction and planning? (RQ4)
School agreement forms,
departmentalization design meeting forms,
principal interview, teacher survey
How do principals’ actual assignments of teachers to subjects
compare with assignments based solely on baseline teacher
effectiveness scores? (RQ5)
District administrative records,
departmentalization design meeting forms
What challenges and benefits do principals and teachers perceive in
switching to departmentalization? (RQ6)
Principal interview, teacher survey
a
Information from the monitoring calls will be used to identify respondents for the teacher survey; verify schools’
compliance with random assignment to implement departmentalized instruction or not; and identify teachers who are
teaching 4th and 5th grade within the study schools so that the teacher retention analysis (RQ2) can be limited to
those teachers. Information obtained from the class schedules, student rosters, and parent permission and student
assent forms will be used to identify classes to be video-recorded and students who may appear in the recordings.
The evaluation is expected to be completed in 4 years. Table A.3 shows the schedule of data
collection activities and the overall evaluation timeline.
12
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
Summer 2021
Spring 2021
X
Fall 2020
X
Summer 2020
X
Summer 2019
X
Spring 2020
X
Fall 2019
Complete departmentalization
design meeting form
Spring 2019
X
Fall 2018
Complete school agreement
form
Summer 2018
Activity
Spring 2018
Table A.3. Schedule of major study activities
X
Complete monitoring forms
Conduct principal interview
X
Obtain class schedules
X
Obtain student rosters
X
Obtain parent permission
X
Obtain student assent
X
Conduct videos of classroom
instruction
X
Conduct teacher survey
X
Collect administrative records
on teacher effectiveness
from 2016–2017 (in districts
with effectiveness scores)
X
X
X
Collect administrative records
on teacher effectiveness
from 2016–2017 (in districts
without effectiveness
scores)
X
X
X
Collect administrative student
and teacher records from
2017–2018 and 2018–2019
(in all study districts)
X
Collect administrative student
and teacher records from
2019–2020 (in all study
districts)
X
X
Prepare study report
X
X
Prepare restricted-use data
file
X
X
A3. Use of technology to reduce burden
The data collection plan is designed to obtain information in an efficient way that minimizes
respondent burden, including the use of technology when appropriate. For example, the teacher
survey will be web-based, which will enable respondents to complete the data collection
instrument at a location and time of their choice. Its built-in editing checks and programmed
skips will also reduce the level of response errors and data retrieval callbacks. However, teachers
will be able to respond to the survey by mail, phone, or in-person if they prefer. As another
example, we will ask districts to provide electronic copies of student and teacher records. While
13
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
we will specify the required data elements, we will accept any format the district wishes to use,
to reduce burden for them. To help ensure study participants’ confidentiality, districts will
upload data files directly to a secure data site.
A4. Efforts to avoid duplication of effort
No similar evaluations are being conducted, and there is no equivalent source for the
information to be collected. Moreover, the data collection plan reflects careful attention to the
potential sources of information for this study, particularly to the reliability of the information
and the efficiency in gathering it. The data collection plan avoids unnecessary collection of
information from multiple sources. For example, student achievement will be measured using
scores from state-administered student assessments, instead of administering an assessment as
part of this study.
Information obtained from the classroom observation videos, teacher survey, and principal
interview, is not available elsewhere.
A5. Methods of minimizing burden on small entities
No small businesses or entities will be involved as respondents.
A6. Consequences of not collecting data
The data collection plan described in this submission is necessary for ED to conduct an
impact evaluation of the effect of departmentalized instruction on teacher and student outcomes.
Additionally, the data collection is necessary to better understand how schools implement
departmentalization and the challenges and benefits of switching to departmentalization in
grades 4 and 5. The consequences of not collecting specific data are outlined below:
Without the district administrative student records, we would have to administer
student assessments instead of using their state math and reading test scores. Without
information on student characteristics, we would not be able to fully describe the study
sample or verify the effectiveness of school random assignment.
Without the district administrative records on teacher effectiveness, we would not
be able to assess how principals’ actual teacher assignments compare to assignments
based solely on objective measures of teacher effectiveness. We would also not be able
to examine whether the impacts of departmentalization differ when principals did or did
not have access to teacher effectiveness scores.
Without the videos of teachers’ classroom instruction that will be used to create
classroom observation rubric scores, we would not be able to measure the impact of
departmentalization on teachers’ instruction or student-teacher relationships.
Without the teacher survey, we would not have the data needed to describe teachers’
preparation experiences and background characteristics, job satisfaction, and confidence
in their teacher abilities. Without these data, we would not be able to measure the
impact of departmentalization on group or individual planning time and professional
development.
14
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
Without the monitoring calls, we would not be able to verify that study schools
remained in their assigned condition (either departmentalized instruction in grades 4
and 5, or self-contained classes). We would also not be able to identify schools that may
need additional technical assistance with departmentalizing.
Without the principal interview, we would not have information on how principals’
approaches to parent communication and disciplinary incidents differ between
departmentalized and self-contained schools. We would also not have information on
principals’ perceptions of the challenges and benefits of departmentalization and the
approach used to assign teachers to subjects.
Without the school agreement form, we would not have baseline information on how
treatment and control schools structure their grades 4 and 5.
A7. Special circumstances
There are no special circumstances associated with this data collection.
A8. Federal register announcement and consultation
a.
Federal register announcement
A 60-day notice to solicit public comments was published in the Federal Register, Volume
83 No. 5, page 795 on 1/8/2018.
To date, no public comments have been received.
The 30-day notice will be published to solicit additional public comments.
b.
Consultations outside the agency
In formulating the intervention and evaluation design for this evaluation, the study team
sought input from several individuals with expertise in departmentalized instruction, including
Lucy Steiner of Public Impact and Florence Chang of Jefferson County Public Schools.
Additionally, a technical working group (TWG) will provide input on the study design, data
collection instruments, analyses, and reports. This input will help ensure the study is of the
highest quality and that findings are relevant to policymakers, school districts, and principals.
Table A.4 lists the individuals who have agreed to serve on the TWG, their affiliation, and their
relevant expertise.
15
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
Table A.4. Technical Working Group Experts
Name
Affiliation
Expertise
Allison Atteberry
Assistant Professor, University of Colorado Boulder
Teacher assignment policies;
school reforms
Thomas Cook
Professor Emeritus of Sociology, Psychology,
Education, and Social Policy, Northwestern University
Evaluation methods
Cassie Guarino
Professor of Education and Public Policy, UC
Riverside
Methods for estimating teacher
effectiveness
James Kemple
Executive Director, The Research Alliance for New
York City Schools, New York University
School reforms; Evaluation
methods
Lisa Martin
Chief Academic and Accountability Officer, DeKalb
County School District
Departmentalized instruction;
Teacher assignment policies
Audra Parker
Associate Professor, George Mason University
Departmentalized instruction;
Teacher assignment policies
Chris Rhoads
Associate Professor, University of Connecticut-Neag
School of Education
Evaluation methods
Jonah Rockoff
Professor of Finance and Economics, Columbia
Business School
School reforms; Methods for
estimating teacher effectiveness
Brian Schultz
Chief Academic Officer, Charlotte-Mecklenburg
Schools
Departmentalized instruction;
Teacher assignment policies
c.
Unresolved issues
There are no unresolved issues.
A9. Payments or gifts
Incentives have been proposed for teachers participating in the study. The proposed amounts
are within the incentive guidelines outlined in the March 22, 2005 memo, “Guidelines for
Incentives for NCEE Evaluation Studies,” prepared for OMB. To maximize the success of our
data collection effort we will provide incentives to teachers to offset their time and effort with
completing the data collection activities. Incentives are also proposed because high response
rates are needed to make the study findings reliable. Teachers are the targets of numerous
requests for data on a wide variety of topics from state and district offices, independent
researchers, and ED. Although some districts will have solicited buy-in from teachers to
participate in the evaluation, our recent experience with numerous teacher data collection efforts
supports our view that obtaining teacher buy-in does not guarantee teachers will devote the time
it takes to complete data collection activities, and monetary incentives increase the likelihood of
their cooperation.
Teacher incentive for collecting parent permission forms. We propose to provide
teachers with an incentive for collecting permission forms from parents that will allow us to
record students during the video recordings of selected classes. Teachers will receive $25 for
distributing the parent consent forms. Because it will be critical for the study to obtain parental
permission for as many students as possible, we will offer teachers in active consent districts an
additional $25 for collecting parent permission forms for at least 85 percent of their students.
This represents a maximum of $50 for any one teacher (roughly $2 per student form and less
than the NCEE-recommended $3 per low-burden student report). We expect teachers will have
16
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
to remind students and call or email parents to obtain 85 percent returns. Our goal is to ensure
that we have as many students in the classroom as possible during the video recordings to
accurately evaluate the teacher’s performance during a typical day of instruction. Field staff from
the study team will be responsible for collecting the permission forms from the teachers. We
believe that the differential incentive proposed will further motivate teachers to collect the parent
permission forms.
Teacher respondent payment. To acknowledge the 30 minutes required to complete the
teacher survey, we propose to offer a $30 incentive to teachers who complete the survey.
A10. Assurances of confidentiality
Mathematica and its research partners will conduct all data collection activities for this study
in accordance with relevant regulations and requirements, which are:
The Privacy Act of 1974, P.L. 93-579 (5 U.S.C. 552a)
The “Buckley Amendment,” Family Educational Rights and Privacy Act (FERPA) of
1974 (20 U.S.C. 1232g; 34 CFR Part 99)
The Protection of Pupil Rights Amendment (PPRA) (20 U.S.C. 1232h; 34 CFR Part 98)
The Education Sciences Reform Act of 2002, Title I, Part E, Section 183
The research team will protect the confidentiality of all data collected for the study and will
use it for research purposes only. The Mathematica project director will ensure that all
individually identifiable information about respondents remains confidential. All data will be
kept in secured locations and identifiers will be destroyed as soon as they are no longer required.
All members of the study team having access to the data will be trained and certified on the
importance of confidentiality and data security. When reporting the results, data will be
presented only in aggregate form, such that individuals, schools, and districts are not identified.
Included in all voluntary requests for data will be the following or similar statement:
“Responses to this data collection will be used only for research purposes. The
report prepared for this study will summarize findings across the sample and will
not associate responses with a specific district, school, or individual. We will not
provide information that identifies you, your school, or your district to anyone
outside the study team, except as required by law. Additionally, no one at your
school or in your district will see your responses.”
The following safeguards are routinely used by Mathematica to maintain data
confidentiality, and they will be consistently applied to this study:
All Mathematica employees are required to sign a confidentiality pledge (Appendix H)
that emphasizes the importance of confidentiality and describes employees’ obligations
to maintain it.
Personally identifiable information (PII) is maintained on separate forms and files,
which are linked only by random, study-specific identification numbers.
17
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
Access to hard copy documents is strictly limited. Documents are stored in locked files
and cabinets. Discarded materials are shredded.
Access to computer data files is protected by secure usernames and passwords, which
are only available to specific users who have a need to access the data and who have the
appropriate security clearances.
Sensitive data is encrypted and stored on removable storage devices that are kept
physically secure when not in use.
Mathematica’s standard for maintaining confidentiality includes training staff regarding the
meaning of confidentiality, particularly as it relates to handling requests for information, and
providing assurance to respondents about the protection of their responses. It also includes builtin safeguards concerning status monitoring and receipt control systems. In addition, all study
staff who have access to confidential data must obtain security clearance from ED which requires
completing personnel security forms, providing fingerprints, and undergoing a background
check.
The program is currently preparing a system of records notice (SORN) and a privacy impact
assessment (PIA). The data are to be stored both electronically and in paper copy. The data will
be retrievable by ID, and will be maintained and disposed of in accordance with the
Department’s Records Disposition requirements. The electronic files will be kept on a password
protected server. The paper copy will be kept in a locked file cabinet, and all access to data in
both electronic and paper form will be restricted to study staff on a need to know basis. The
security protections for the content will be identified in the SORN.
A11. Justification for sensitive questions
No questions of a sensitive nature will be included in this study.
A12. Estimates of hours burden
Table A.5 provides an estimate of burden for the data collections, broken down by
instrument and respondent. These estimates are based on our prior experience collecting
administrative data from districts and obtaining parent permission, as well as pretests of the
teacher survey and principal interview protocol.
18
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
Districts
Records on teacher effectiveness
from 2016–2017 (in districts with
effectiveness scores), spring
through fall 2018
Records on teacher effectiveness
from 2016–2017 (in districts without
effectiveness scores), spring
through fall 2018
Student and teacher records from
2017–2018 and 2018–2019 (in all
study districts), summer/fall 2019
Student and teacher records from
2019–2020 (in all study districts),
summer/fall 2020
Parents
Parent permission form (100
schools’ 4th grade classes; 3
teachers per school; 22 students
per class), spring 2019
Students
Student assent form in up to 2
districts that might require assent,
spring 2019
Total (rounded)
100
200
0.25
16.7
50
200
85
170
1
56.7
170
200
85
170
0.25
14.2
300
100
300
0.5
50
150
300
100
300
0.5
50
150
300
100
300
2
200
600
600
85
510
0.5
85
255
6
100
6
16
32
96
6
100
6
24
48
144
12
100
12
20
80
240
12
100
12
16
64
192
6,600
85
5,610
0.17
317.9
953.7
85
935
0.17
53
159
1,067
3,202
1,100
a
9,836
8,531
a The
Total burden
(hours)
200
Annual Total
response
time over 3year data
collection
(hours/year)
Expected
Number of
responses
Teachers
Class schedules of 4th grade
teachers selected for videos of
classroom instruction, spring 2019
Student rosters for 4th grade
teachers selected for videos of
classroom instruction, spring 2019
Teacher assistance collecting parent
permission forms for students in
classrooms selected for videos of
classroom instruction, spring 2019
Teacher survey (all 4th grade
teachers), spring 2019
Expected
response rate
(%)
Principals
School agreement form, spring 2018
Monitoring calls in fall and spring of
the 2018-2019 and 2019-2020
school years (4 monitoring calls at
15 minutes each or 1 hour total
response time for calls)
Principal interview conducted spring
2019
Number of
targeted
respondents
Respondent/Data request
Unit response
time (hours)
Table A.5. Estimated response time for data collection
42.5
total number of targeted respondents (9,836) is the sum of targeted responses across data requests from a total of
8,512 unique respondents including 12 districts, 200 principals, 600 teachers, 6,600 parents and 1,100 students across the
three years of data collection.
19
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
The number of targeted respondents and responses are 9,836 and 8,531. The total burden is
estimated at 3,202 hours or an average of 1,067 annual burden hours calculated across 3 years of
data collection. Video recordings of classroom instruction are not calculated in the burden
estimate because the recordings will be conducted by the study team and occur during teachers’
regular class instruction. Therefore, they will not impose any additional time burden on teachers.
Time to complete the departmentalization design meeting forms is not included in the burden
estimate because the forms will be completed by study staff after they meet with principals to
provide technical assistance. It will not impose an additional burden on principals.
The total of 3,202 hours includes the time for:
200 principals to complete school agreement forms (50 hours); 170 principals (85
percent of 200 principals) to participate in four 15-minute monitoring calls (170 hours)
and one 15-minute interview (42.5 hours);
300 4th grade teachers (half of the 600 4th grade teachers that the study selects for
classroom observations) to provide class schedules (150 hours), provide class rosters
(150 hours), and collect parent permission forms (600 hours); 510 4th grade teachers
(85 percent of 600 4th grade teachers) to complete a 30-minute survey (255 hours);
6 districts to provide teacher effectiveness scores in 2018 (96 hours); 6 districts to
provide student test scores linked to teachers in 2018 (144 hours); 12 districts to provide
student and teacher records in 2019 (240 hours); and 12 districts to provide student and
teacher records in 2020 (192 hours);
5,610 parents or guardians of 4th grade students (85 percent of 6,600 4th grade parents
across the 300 classes selected for observations) to review and complete (if active
consent is required) a parent permission form (953.7 hours); and
935 students (85 percent of 1,100 from 4th grade classes in two districts) to complete a
student assent form for classroom observations (159 hours).
A13. Estimate of cost burden to respondents
There are no direct or start-up costs to respondents associated with this data collection.
A14. Annualized cost to the federal government
The total cost to the federal government for this study is $6,922,150. The estimated average
annual cost—including recruiting districts, designing and administering all collection
instruments, processing and analyzing the data, and preparing reports—is $1,730,538 (the total
cost divided by the four years of the study).
A15. Reasons for program changes or adjustments
This is a new collection.
A16. Plans for tabulation and publication of results
a.
Analysis plan
The evaluation will estimate the impact of departmentalized instruction on student and
teacher outcomes and document schools’ implementation of departmentalized instruction. The
20
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
study also includes several supplementary analyses to provide additional policy-relevant
information. Below, we describe the main impact and implementation analyses.
Impact analyses. We will use regression models to estimate the impact of departmentalized
instruction on student outcomes (standardized math and reading test scores, attendance, and
disciplinary incidents) and teacher outcomes (amount of instructional planning and professional
development, quality of student-teacher relationships, teaching practices, job satisfaction, and
retention). Because the study has a randomized controlled trial design, simply comparing the
outcomes of teachers and students in schools randomly assigned to treatment and control groups
should yield unbiased estimates of the impacts of departmentalized instruction. However, to
increase the precision of our estimates, we will also control for baseline student, teacher, and
school characteristics. We will estimate these models for the full combined sample and
separately in districts with and without teacher effectiveness scores, to see how the effects of
departmentalized instruction differ across these types of districts. Results from the impact
analyses will provide evidence on the effects of departmentalized instruction on student
achievement and other key outcomes of interest.
Implementation analyses. Our implementation analysis will describe schools’ approaches
to departmentalization and benefits and challenges encountered. We will document the structure
of departmentalization in treatment schools, including number of subjects and classes per
teacher, assignment of teachers to subjects, and time allocated to instruction and planning. We
will also describe how principals assigned teachers to subjects (in districts with and without
teacher effectiveness scores) and any implementation challenges. In both treatment and control
schools, we will document time for instruction, planning, and teacher professional development.
Understanding the implementation experiences and challenges of schools and teachers
participating in the intervention will provide important information for districts and elementary
schools considering departmentalizing instruction. The implementation analysis will also provide
important context for interpreting the impact results.
b.
Publication plan
We will present the results of these analyses in a report, projected to be released in 2021.
The report will be written in a style and format accessible to policymakers and educators and will
comply fully with the standards set by the National Center for Education Statistics.
A17. Approval not to display the expiration date for OMB approval
The Institute of Education Sciences is not requesting a waiver for the display of the OMB
approval number and expiration date. The study will display the OMB expiration date.
A18. Exception to the certification statement
No exceptions to the certification statement are requested or required.
21
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
REFERENCES
Chan, Tak Cheung, and Delbert Jarman. “Departmentalize Elementary Schools.” Principal, vol.
84, no. 1, 2004, p. 70.
Chang, Florence C., Marco A. Muñoz, and Sheila Koshewa. “Evaluating the Impact of
Departmentalization on Elementary School Students.” Planning and Changing, vol. 39,
2008, pp. 131–145.
Condie, Scott, Lars Lefgren, and David Sims. “Teacher Heterogeneity, Value-Added and
Education Policy.” Economics of Education Review, vol. 40, 2014, pp. 76–92.
Duncan, Greg J., and Katherine Magnuson. “The Nature and Impact of Early Achievement
Skills, Attention Skills, and Behavior Problems.” In Whither Opportunity? Rising
Inequality, Schools, and Children’s Life Chances, edited by Greg J. Duncan and Richard J.
Murnane. New York: Russell Sage Foundation, 2011.
Fox, Lindsay. “Playing to Teachers’ Strengths: Using Multiple Measures of Teacher
Effectiveness to Improve Teacher Assignments.” Education Finance and Policy, vol. 11, no.
1, 2016, pp. 70–96.
Fryer, Roland. “The ‘Pupil’ Factory: Specialization and the Production of Human Capital in
Schools.” NBER working paper no. 22205. Cambridge, MA: National Bureau of Economic
Research, 2016.
Goldhaber, Dan, James Cowan, and Joe Walch. “Is a Good Elementary Teacher Always Good?
Assessing Teacher Performance Estimates Across Subjects.” Economics of Education
Review, vol. 36, 2013, pp. 216–228.
Goldring, Rebecca, Lucinda Gray, and Amy Bitterman. Characteristics of Public and Private
Elementary and Secondary School Teachers in the United States: Results from the 2011–12
Schools and Staffing Survey. (NCES 2013-314). Washington, DC: U.S. Department of
Education, Institute of Education Sciences, National Center for Education Statistics, 2013.
Kane, Thomas J., and Douglas O. Staiger. “Gathering Feedback for Teaching: Combining HighQuality Observations with Student Surveys and Achievement Gains. Research Paper. MET
Project.” Seattle, WA: Bill & Melinda Gates Foundation, 2012.
McGrath, Carole J., and James O. Rust. “Academic Achievement and Between-Class Transition
Time for Self-Contained and Departmental Upper-Elementary Classes.” Journal of
Instructional Psychology, vol. 29, no. 1, 2002, pp. 40–43.
McPartland, James, and Jomills Henry Braddock II. “A Conceptual Framework on Learning
Environments and Student Motivation for Language Minority and Other Underserved
Populations.” Proceedings of the Third National Research Symposium on Limited English
Proficient Student Issues. Washington, DC: National Clearinghouse for Bilingual Education,
George Washington University, 1993.
22
CONTRACT NUMBER: ED-IES-17-C-0064
MATHEMATICA POLICY RESEARCH
Parker, Audra K. “Elementary Organizational Structures and Young Adolescents’ Self-Concept
and Classroom Environment Perceptions Across the Transition to Middle School.” Journal
of Research in Childhood Education, vol. 23, no. 3, 2009, pp. 325–339.
Pianta, Robert C., Bridget K. Hamre, and Susan Mintz. Upper Elementary and Secondary
CLASS Technical Manual. Charlottesville, VA: Teachstone, 2012.
Taylor-Buckner, Nicole C. The Effects of Elementary Departmentalization on Mathematics
Proficiency. Unpublished dissertation. New York: Columbia University, 2014.
Troppe, Patricia, Anthony T. Milanowski, Camilla Heid, Brian Gill, and Christine Ross.
Implementation of Title I and Title II-A Program Initiatives: Results from 2013–14. (NCEE
2017-4014). Washington, DC: U.S. Department of Education, Institute of Education
Sciences, National Center for Education Evaluation and Regional Assistance, 2017.
U.S. Department of Education. Schools and Staffing Survey (SASS): Number and Percentage of
Regular Full-Time Public School Teachers of Grades 1 through 4, by Classroom
Organization: Selected Years 1987–88 through 2007–08. Washington, DC: U.S. Department
of Education, 2009. Available at:
https://nces.ed.gov/surveys/sass/tables/sass0708_042_t1n.asp. Accessed June 19, 2017.
U.S. Department of Education. Schools and Staffing Survey (SASS): Number of Public and
Private School Teachers, by Years of Teaching Experience, Percentage of Students in Their
School who were Approved for Free- or Reduced-Price Lunches, and School Type: 2011–
12. Washington, DC: U.S. Department of Education, 2013. Available at:
https://nces.ed.gov/surveys/sass/tables/sass1112_504_t12n.asp. Accessed June 19, 2017.
23
www.mathematica-mpr.com
Improving public well-being by conducting high quality,
objective research and data collection
PRINCETON, NJ ■ ANN ARBOR, MI ■ CAMBRIDGE, MA ■ CHICAGO, IL ■ OAKLAND, CA ■
SEATTLE, WA ■ TUCSON, AZ ■ WASHINGTON, DC ■ WOODLAWN, MD
Mathematica ® is a registered trademark
of Mathematica Policy Research, Inc.
| File Type | application/pdf |
| Subject | OMB |
| Author | MATHEMATICA |
| File Modified | 2018-03-08 |
| File Created | 2018-03-08 |