APNCEE RtI Study OMB Part A.Jan.2012

Evaluation of Response to Intervention Practices for Elementary School Reading: School and Staff Practices

Supporting Statement for OMB Data Collection Clearance Request

Justification Part A

September 28, 2011

Revised January 19, 2012

Prepared by MDRC for

The Institute of Education Sciences

Contract ED-04-CO-0111/003

PART A: JUSTIFICATION

INTRODUCTION

The National Center for Education Evaluation (NCEE) of the Institute of Education Sciences (IES), U. S. Department of Education (ED) is conducting the National Assessment of the Individuals with Disabilities Education Improvement Act of 2004 (IDEA 2004, P.L. 108-446), part of which includes an Evaluation of Response to Intervention (RtI) practices in elementary school reading. Under certain conditions,¹ RtI may qualify as an early intervening service (EIS) that may be supported with IDEA funds to identify and serve students in general education classrooms who may be at risk for academic difficulties and eligible for special education. IES has contracted with MDRC, SRI International, and RG Research Group to conduct the Evaluation of RtI Practices in Reading project. This submission seeks clearance for the data collection instruments and analytical techniques of a study of RtI design, implementation, and impact.

This evaluation is part of the National Assessment of the Individuals with Disabilities Education Improvement Act of 2004 (IDEA 2004, P.L. 108-446) being conducted by IES. Section 664 of IDEA 2004 requires the National Assessment to evaluate “the implementation of programs assisted under this title and the impact of such programs on… improving the academic achievement of children with disabilities to enable the children to reach… challenging State academic content standards based on State academic assessments.” MDRC is undertaking the collection of information under contract with IES for this evaluation. This introduction summarizes the study objectives and the three research questions, the specifics of the analytic approach to addressing the three research questions, and data collection plans for the evaluation. This document also provides supporting statements for each of the eighteen points outlined in Part A of the OMB guidelines for the collection of information for the RtI project.

Study Objectives and Research Questions

The RtI approach has the potential to:

1. improve instruction for all struggling students by identifying learning problems early and informing instructional decisions regarding the type, intensity, and duration of interventions to address them;

2. inform the evaluation of students for specific learning disabilities by assessing their responses to research-based interventions; and

3. affect the representation of students from culturally and linguistically diverse backgrounds in some disability categories by identifying and intervening early with students who have achievement deficits.

As the study has progressed, it has become clear that there is intense interest in RtI for elementary school reading. As of 2010, 43 states have indicated that they have a state RtI framework in place (retrieved August 22, 2010, from http://state.rti4success.org/).Many districts and schools are working to put in place strong RtI models, and investigation of various types of RtI practices along with quasi-experimental analyses of their impacts can help school district, and state administrators design and implement these programs and inform Federal efforts to support RtI and related early intervening services.

Thus, this evaluation will address the following questions:

1. What is the average impact on academic achievement of providing intensive secondary reading interventions to elementary school children who have been identified as at risk for reading difficulties compared with children just above the cut point for providing intervention?

2. How do academic outcomes, including reading achievement and special education identification, vary with elementary schools’ adoption of Response to Intervention practices for early grade reading?

3. How do Response to Intervention practices for early grade reading vary across schools and how are they related to academic outcomes?

The study team will use a regression discontinuity design (RDD) to answer the first question. The RDD analysis will examine the impacts of providing more intensive reading support to children on the margin of needing such assistance. In sites where decisions about providing assistance are made based on a ranking of students’ need for assistance and a consistently applied cutoff for assistance, RDD impact estimates would be calculated by comparing student academic outcomes for children immediately above and below the cutoff point. This analysis would provide evidence on the effectiveness of providing coordinated early intervention services (CEIS) funded under IDEA to students who are at the time not identified as needing special education services but are struggling to learn how to read and are receiving more intensive instructional supports for reading in the regular education classroom (Tier 1 in RtI terminology) or in separate tiers with small student-to-teacher ratios.²

A comparative interrupted time series (CITS) design will be used to answer the second question. The CITS analysis will examine whether implementation of RtI practices is associated with greater improvements over prior academic trends in reading achievement and special education identification in schools experienced with RtI as compared to similar schools not implementing the key elements of reading RtI during the period of the analysis. This design will also examine how special education referral and placement change as RtI is implemented.

For the third question, study team will document the design and implementation of RtI in the full sample of schools (RDD and CITS) through correlational analysis of surveys of school-level staff, teachers, and reading specialists (sometimes known as interventionists). These surveys will also inform the RDD and CITS analyses by allowing us to characterize the contrast in instruction provided students identified as needing additional, intensive reading instruction and those not identified for such services. For the CITS analysis, it will also provide information on the service contrast between the RtI treatment schools and comparison schools.

Research Question #1 Addressed by a Regression Discontinuity Analysis

This approach will compare (1) reading achievement outcomes for students who, based on their benchmark reading test scores, qualified to receive additional reading support with (2) achievement outcomes for students in the same school who meet reading benchmarks initially and were not identified for extra help in reading. Experienced RtI schools typically use a benchmark test at the beginning of the fall semester to identify students for additional reading support. Students whose benchmark test scores fall below a pre-determined cutoff point are deemed at-risk and are referred to additional instructional support (treatment group), and those whose benchmark test scores are above the cutoff stay in the general education class (comparison group). The so-called “sharp” RDD assumes that the decision on receiving the added support is entirely determined by the benchmark test score. The so-called “fuzzy” RDD can accommodate a situation where other factors also influence the decision about receipt of extra support leading to a situation where some students identified for the treatment group based on the benchmark test score do not actually get the extra support and some students identified to receive regular services do get extra support. ³ Therefore, by statistically controlling for the value of the benchmark test score in a regression model, one can (under appropriate conditions) account for any unobserved differences between the treatment and comparison group and thereby obtain internally valid impact estimates for receiving more intensive, additional reading support.

The sample of schools for the RDD analysis will include schools that:

• maintain benchmark test data for each student.

• can provide information about the process of identifying students for additional instructional support, including whether identification involved a decision process based on a single benchmark score, or whether multiple benchmark test scores (and/or other factors) were used to identify students for support.

• assign students to treatment or non-treatment status (i.e., receipt or non-receipt of more intensive reading instruction under a Tier 2 intervention or other means) based on whether their value for a numeric rating (benchmark test score) is above or below a cutoff point;⁴

• maintain a record of the cutoff point(s) used to assign students to receive additional instructional support.

• maintain records tracking students’ receipt of extra reading support status throughout the year.

• are willing to allow study-administered year-end reading testing in first (and in some instances, second) grade and can provide spring reading test scores for third graders.

If the above conditions are present and if the analysis accounts correctly for the relationship between the benchmark test score and the outcome measure in a statistical model, then this approach can provide an internally valid estimate for the impact on at-risk students’ reading achievement of being identified to receive additional instructional support within an RtI system.

Research Question #2 Addressed by a Comparative Interrupted Times Series Analysis

Under a CITS design, trends in student outcomes such as reading achievement, grade promotion, and special education identification prior to the implementation of RtI practices are compared with post-implementation trends in these schools to estimate a deviation from prior trends occurring with the start of RtI (the “interruption”). This deviation in RtI treatment schools is then compared with an estimated deviation in outcomes that occurred in similar schools not implementing RtI across the same period. The estimated difference in these two deviations is the estimate of the “impact” or more properly the “association” between the adoption of RtI practices and student outcomes. The causal evidence emerging from this methodology is weaker than for either regression discontinuity or random assignment studies.

The sample of schools for the CITS analysis will have the following characteristics:

• Sufficient numbers of “treatment” schools that have experience with RtI practices to have the needed statistical power to detect relationships (as discussed elsewhere in this submission);

• Experienced RtI schools that have good historical information about the timing of RtI implementation;

• Experienced RtI schools that are implementing RtI practices with a clearly identifiable starting point;

• Appropriate, statistically-equivalent comparison schools that can be systematically identified;

• Treatment and comparison schools that have historical data on student outcomes measured using consistent metrics over three or more years⁵ prior to the first year of RtI implementation in the experienced RtI schools; and

• Treatment and comparison schools that have one or more years of follow-up data, measured using the same metrics as those used for the historical data, in the period following RtI implementation in the experienced RtI schools.

In this analysis, the study team will collect existing student records for special education referral and identification and disability category and – as available - reading achievement during the baseline period prior to RtI implementation (the interruption) and in a post-interruption follow up period. Similar data will be collected in RtI treatment schools and matched comparison schools ideally in the same districts as treatment schools. The CITS approach, details of which are described in Part B, has less methodological strength than the RDD approach in identifying causal relationships. Specifically, it does not provide causal estimates of the impact of implementing RtI practices on the student outcomes examined. Hence, this submission uses the phrase “association between RtI implementation and changes in student outcomes.”

Research Question #3 Addressed by a Comparison of Description Statistics

The descriptive analysis of RtI design and implementation will include 3 main elements:

• For all study (RtI treatment and comparison) schools: A description of the structure of RtI or other programs to assist students in reading, including universal screening or benchmark testing, offerings of reading instruction (whether offered in the general education program (often called Tier 1 in an RtI program) or in more intensive ways (often offered in a second and third tier within an RtI program); progress monitoring of students over time; use of data to make decisions about tier placements and movements; and the extent to which they have a process for determining eligibility for special education services that includes data on student’s responsiveness to the interventions.

• RDD Treatment Schools: Details of the assessment process used for benchmark testing, how these benchmark tests are used in decisions to offer or discontinue more intensive reading support and student receipt of more intensive reading support throughout the school year.

• CITS RtI Treatment and Comparison Schools: Details on the timing of the implementation of elements of any RtI program and documentation of the service contrast (presences or absence of RtI practices) over the time of the analysis.

Summary of Data Collection Plans

The purpose of the data collection is to analyze the design, implementation, and impact of Response to Intervention programs in elementary school reading. This will involve new data collection through surveys and instructional logs from principals, teachers, and reading interventionists; reading testing of students; and integration of the new data with existing records on student academic status and performance. To summarize, the data sources for these three analyses include:

1. School level survey: this survey, to be administered in both treatment and comparison schools, provides an overview of reading instruction, progress monitoring, and use of data for instructional decision making for (a) students above benchmark standards in reading, (b) students somewhat below benchmark standards in reading, and (c) students far below benchmark standards in reading. These three categories of students are also used in the teacher and intervention provider surveys. In this survey, the data is collected on the whole school’s reading practices. See appendix 1.

2. Teacher level survey: this survey, to be administered in both treatment and comparison schools, provides an overview of the reading instruction provided by grade level teachers in the school. In this survey, data is collected on the reading instruction provided by the classroom teacher to readers in the core reading block. It will signal where students in the three categories are sent for additional instruction, but will not be a source of information on reading interventions and supports that students receive when sent outside of the classroom for additional help. With this survey, the study team will be able to document (a) the content and organization of reading instruction provided by the grade level teachers, (b) data use in instructional decision making, and (c) progress monitoring within the grade level classroom. See Appendix 2.

3. Intervention provider survey: this survey, to be administered in both treatment and comparison schools, provides information on the small group reading instruction provided by reading specialists or interventionists. The survey is organized around multiple groups of students the interventionist worked with during the most recent full week. For each group, information is collected on (a) the reading skills of the students in the group (defined using the 3 categories above), (b) group size, (c) frequency and duration of instruction, and (d) instructional content/curriculum, and (e) progress monitoring. This survey will allow us to summarize how the interventionists provide and monitor reading instruction to students somewhat or far below benchmark in reading, the composition of each group they teach, how these groups are organized and the services provided to the student groups. See Appendix 3.

4. Fall reading screening/benchmark scores and instructional decision form: this form, to be used for data collection in both treatment and comparison schools, collects extant data that provides information on (a) the use of reading benchmark tests in the fall, (b) the student scores on each fall benchmark test, and (c) whether each student was assigned by the school to receive reading support following the fall benchmark test(s). It will allow us to independently determine the cut score used by the school and examine the variability in the application of school-based decision rules. See Appendix 4.

5. Rosters of students and data on reading service placement: this roster, to be used for data collection in both treatment and comparison schools, collects extant data that provides basic information on referral to different levels of intensity of reading support (including any placement in tiers of reading instruction). See Appendix 5.

6. Information on reading instruction and intervention for sample of students: this will provide more detailed information on reading instruction for a sample of children in each treatment school. The study team will draw a sample of 8 students in each school, half from students above the cutoff and not needing special reading support, and half below the cutoff and identified for special reading support. For this group, grade level teachers and intervention providers will report more detailed data on the reading instruction the sampled students receive during school year 2011-12. This will be done for 5 consecutive days of instruction, three times a year. See Appendix 6.

7. Summary of tier placement and movement: this form, adapted from the work of Dr. Ed Shapiro, will provide aggregate information on the number of children in each level of reading support (tier) at key benchmark testing points during the school year 2011-12. This provides information on the movement of students between the levels of reading supports within the school. See Appendix 7.

8. Historical student records on special education identification and reading achievement in grades 1 through 3 accessed for the CITS treatment schools and comparison schools: these extant data, collected through requests to school districts and schools, will provide information on special education identification by disability category, and reading achievement test scores (if available) for students in the CITS treatment schools for at least 3 years prior to RtI implementation and a similar period in the comparison schools and for as many years of post-RtI follow up as available. These data will be collected by accessing electronic student records kept by the school district. MDRC will make a data request from the district for student level data for the RtI treatment and comparison schools in grades 1-3 during the period of the CITS analysis. MDRC will specify the particular data items present in each district’s record system that will provide a unique student identification number, basic demographic information on the student, any available reading test score information for the student, and information on whether the student has been identified to receive special education services, including the disability category under which the student will be served. These data will be transmitted in a secure way and stored on a secure server at MDRC.

9. Follow- up spring 2012 reading test for first and second graders in the RDD treatment schools: this will involve fielding of the ECLS-K first grade reading test plus a short fluency test to first graders and second graders. The additional testing of students’ reading achievement will provide more accurate and comprehensive measures of reading achievement than the short fluency-based tests that are often used in schools in the early grades. Systematic testing of students’ broader reading achievement often is not done until third grade and key evaluation questions concern early grade reading because RtI rests on more intensive monitoring of reading progress and earlier efforts to address problems. Outcome measures of third graders will be drawn from extant student records of reading tests using the same procedures as in #8, above.

Exhibit A-1, below, shows the study components, planned school samples, and data collection proposed. Statistical methods to be employed to address research questions for the study, and the status of recruitment of study sites for each research design under OMB approved collection 1850-0872, are described in Part B of this package.

Exhibit A-1: Study Components, Planned School Samples, and Data Collection Proposed
Component of the Study	Schools	Data Collection
RDD Analysis of Research Question #1	Total of 113 RtI treatment schools. Anticipate that all CITS treatment schools will be part of this and 65 additional schools will be recruited as RDD treatment schools only.	Collection of extant fall 2011 benchmark test data for students in grades 1-3, and information on whether students are identified to receive more intensive reading instruction because their benchmark score falls below a specified cutoff point. Collection of data on reading instructional services provided students during the school year for all students, and more detailed data on the nature of reading instruction for a sample of students Fielding of the Early Childhood Longitudinal Survey-K first grade reading assessment and a short fluency test in spring 2012 for all first graders and, where feasible, a short fluency test for second graders. Collection of extant data on third grade reading test scores for spring 2012 Surveys at school level and of teachers and reading interventionists and collection of extant data on reading instruction to describe reading instruction provided students identified as needed intensive reading instruction and those not.
Component of the Study	Schools	Data Collection
CITS Analysis of Research Question #2	Total of 96 schools including 48 treatment schools experienced with RtI programs and a comparable number of comparison schools not operating key elements of RtI or beginning the program later than the treatment schools. Comparison schools ideally selected from the same districts as treatment schools, but if needed will select them from adjacent or nearby districts within the same state.	Collection of extant student records data on student characteristics and student outcomes such as reading achievement and identification for special education, including disability category. Data collected for treatment and comparison schools for baseline period prior to implementation of RtI in the treatment schools and for a follow up period after the interruption of RtI implementation. Site screening information on rollout of RtI and surveys at school level and of teachers and reading interventionists to characterize differences in reading instruction between treatment and comparison schools
Descriptive Comparison Study Analysis of Research Question #3	Total of 161 schools including all treatment schools included in the RDD analysis and the CITS comparison schools.	Surveys of all RDD treatment schools and CITS comparison schools in spring 2012 on school-level RtI design and procedures, teacher practices, and practices of those providing reading intervention services for RtI students, as well as the services offered to students not identified as RtI students. Overview of K-3 reading program and more detail on grades 1 and 3. Fielding of Early Childhood Longitudinal Survey (ECSL-K) reading assessment in spring 2012 in comparison schools if funding available and feasible in these schools. (This assessment will be fielded in all the treatment schools as part of the RDD analysis below.)

RESPONSES TO SPECIFIC QUESTIONS

A1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The focus of the Individuals with Disabilities Education Act (IDEA) has evolved from securing access to public education for students with disabilities to including them in accountability systems, providing access to the general education curriculum, and improving their academic performance. Although there is evidence of progress on some of these goals, students with disabilities continue to display a pattern of low academic achievement, despite advances in curriculum design, understanding of the components required to develop reading skills, assessments to inform instructional decision, and research-based intervention practices. Further, the diagnostic procedures for identifying and referring students with specific learning disabilities (SLD) for special education services have traditionally been based on the existence of significant discrepancies between the child’s IQ and achievement level, resulting in delays in SLD students receiving supplemental instruction until the later grades and hence causing them to fall even further behind in school. This conundrum, along with the hope of avoiding unnecessary referrals for special education services, has sparked an impetus for earlier intervening services to improve the achievement of struggling learners and to inform evaluations of whether students have SLD.

Reflecting research that suggests that low achievement may be due to inappropriate instruction and not necessarily to a disability,⁶ several model programs, assessment methodologies, and instructional advances have been developed and have come to be known as Response to Intervention (RtI).⁷ Generally, RtI practices emphasize high-quality instruction in general education classes, frequent measurement of student progress, decision rules to identify nonresponders to this instruction, and delivery of increasingly intensive interventions to nonresponders in a tiered fashion. Since the inclusion of RtI in IDEA 2004, the number of RtI initiatives across the nation has grown. As more states and local education agencies seek to adopt RtI initiatives, they need evaluation findings to make sound decisions about appropriate instructional interventions. Thus, the Evaluation of RtI Practices for Elementary School Reading is particularly timely.

The goal of this study is to identify schools operating a range of RtI models for elementary level reading, describe their design and implementation, and conduct quasi-experimental analyses of the impact of RtI on student academic outcomes and identification for special education. Our focus in this study is grades 1-3 instruction, since RtI is focused on early intervention. IES seeks approval for the data collection activities described in this request in order to support the data collection and analysis plan for the Evaluation of RtI Practices in Elementary School Reading.

Legal or Administrative Requirements that Necessitate the Collection

This evaluation is part of the National Assessment of the Individuals with Disabilities Education Improvement act of 2004 (IDEA 2004, P.L. 108-446) being conducted by IES. Section 664 of IDEA 2004 requires the National Assessment to evaluate “the implementation of programs assisted under this title and the impact of such programs on… improving the academic achievement of children with disabilities to enable the children to reach… challenging State academic content standards based on State academic assessments.” MDRC is undertaking the collection of information under contract with IES for this evaluation.

A2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information from the current collection.

The information gathered through this data collection will be analyzed statistically by the IES evaluation contractor (MDRC) and its subcontractors to study the design, implementation, and impacts of Response to Intervention programs in elementary school reading. This will involve new data collection where needed through surveys and instructional logs from principals, teachers, and reading interventionists; reading tests of early elementary students; and integration of the new data with existing records on student academic status and performance. Exhibit A-1 shows the study components, planned school samples, and collection of new and extant data proposed. The study team will use a regression discontinuity design to analyze the impacts of providing more intensive reading assistance to students identified as in need of help, a comparative interrupted time series design to analyze associations between the introduction of RtI components and trends in special education identification and reading achievement, and descriptive statistics to compare the presence of RtI components in schools in the sample. IES will review and release a report to the public describing the study’s findings.

A3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or any other technological collection techniques or forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision of adopting the means of collection. Also describe any consideration of using information technology to reduce burden.

Wherever possible the research team will use information technologies to maximize the efficiency and completeness of the information gathered for this evaluation and to minimize the burden on respondents. In particular, data will be gathered from existing electronic school administrative records. Where possible, surveys will be fielded through a web-based application allowing electronic completion of the survey. If this is not feasible, forms will be developed that can be completed and scanned electronically.

A4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use of the purposes described in Item 2 above.

The data collection effort planned for this project will produce data that are unique and targets specifically the research questions identified for this project.

Our study includes an analysis of RtI impacts on first- and second-graders’ reading achievement and other academic outcomes. However, the No Child Left Behind Act of 2001 (NCLB) does not require state-administered assessments in reading until the third grade. Even in states that do administer reading assessments in earlier grades, the state-administered assessments vary in what they measure, how closely they can measure reading proficiency, and how well they can measure student reading achievement. Thus, administering our own reading assessment for the RDD analysis adds greatly to the value of the study because it provides site-to-site consistency and assures a reliable and common reading achievement score.

This study will use data from existing sources when available. Data collection from on-site records, screeners, and surveys of the classroom teachers and interventionists will help us understand the implementation and design of the RtI programs as well as the services offered to struggling readers in a non RtI context. In all cases, the data collected through screeners, surveys, and on-site records collection do not currently exist and are needed to address the project’s research questions. The data collected through these means will also be used to inform the study team of the existence of extant data required to perform the quasi-experimental impact studies.

A5. If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize burden.

Not applicable. No small businesses are expected to be burdened by this data collection.

A6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

The systematic collection and analysis of the data described in this submission are required to accomplish the goals of improving programs and policies implemented by the U. S. Department of Education, states, districts, and schools, including reporting to Congress on the effects of practices implemented under IDEA, and providing evidence-based technical assistance to practitioners and policymakers on Response to Intervention. To date, a large-scale study of the effects of RtI practices has not occurred, so Federal technical assistance in this area has been informed primarily by smaller-scale research studies. While the study team is proposing to access extant data on student reading and other academic outcomes and on participation in reading instruction to reduce the burden on schools and students as much as possible, much of the information needed for the analysis (especially on RtI services) is not already available and requires the additional data collection described in this submission. Participation in all data collection activities is voluntary but the study team will work to assure high participation rates.

A7. Explain any special circumstances that would cause an information collection to be conducted in a manner inconsistent with Section 1320.5(d)(2) of the Federal regulations:

There are no special circumstances for the proposed data collection.

A8. Federal Register Comments and Persons Consulted Outside of the Agency

A notice about the study will be published in the Federal Register when this package is submitted in order to provide the opportunity for public comment.

In addition, throughout the course of this study, the study team will draw on the experience and expertise of a technical working group (TWG) that provides a diverse range of experience and perspectives. The TWG is made up of the following individuals:

• Carol Connor, Florida State University

• Donald Compton, Vanderbilt University

• Judy Elliott, Los Angeles Unified School District

• David Francis, University of Houston

• Paul McDermott, University of Pennsylvania

• Rollanda (Randi) O’Connor, University of California-Riverside

• Amy Sichel, Abington School District (Abington, Pennsylvania)

• Jeff Smith, University of Michigan

• Deborah Speece, University of Maryland-College Park

• Sharon Vaughn, University of Texas-Austin

A9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

Teachers who complete the teacher survey or interventionist survey will receive a $25 gift certificate for each survey completed. Teachers who complete the description of student’s reading instruction and intervention will receive a $10 gift certificate for each student per wave of data collection.

A10. Describe any assurances of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

All data collected through this study will be gathered in full compliance with applicable laws and regulations, including in particular the Education Sciences Reform Act (ESRA), Title I, Part E, Section 183 (20 USC 9573), which requires “[all] collection, maintenance, use, and wide dissemination of data by the Institute . . . to conform with the requirements of section 552 of Title 5, United States Code, the confidentiality standards of subsections (c) of this section, and sections 444 and 445 of the General Education Provisions Act (20 USC 1232 g, 1232h).” These citations refer to the Privacy Act (P.L. 93-759, 5 USC 552a); the “Buckley Amendment,” the Family Educational Rights and Privacy Act (20 USC 1232g); and the Protection of Pupil Rights Amendment (20 USC 1232h).

In addition, for student information, MDRC will ensure that all individually identifiable information about students, their academic achievements, and their identification for special education, as well as information with respect to individual schools, shall remain confidential in accordance with Section 552a of Title 5, United States Code, the confidentiality standards subsection (c) and Sections 444 and 445 of the General Education Provision Act. Subsection (c) of Section 183 of ESRA, referenced above, requires the director of IES to “develop and enforce standards designed to protect the confidentiality of persons in the collection, reporting, and publication of data.” The study will also adhere to requirements of subsection (d) of Section 183 prohibiting disclosure of individually identifiable information as well as making the publishing or inappropriate communication of individually identifiable information by employees or staff a felony.

MDRC and its subcontractors will protect the confidentiality of all information collected for the study and will use it for research purposes only. No information that identifies any study participant will be released. Further, personally identifiable data will not be entered into the analysis file; the analysis data records will contain a numeric identifier only. When reporting the results, data will be presented only in aggregate form so that individuals and schools cannot be identified. The study team will include a statement to this effect with all requests for data, and the school, teacher, and interventionist surveys will include a reminder about confidentiality protection in compliance with the legislation. All members of the study team having access to confidential data will be trained on the importance of confidentiality and data security. All data will be kept in secured locations, and identifiers will be destroyed as soon as they are no longer required.

The following safeguards are employed to carry out confidentiality assurances during the study:

• All employees at MDRC and its subcontractors sign a confidentiality pledge emphasizing the importance of confidentiality and describing their obligation to it. All persons who collect, code, or otherwise have access to raw data are trained in confidentiality issues. 

• In those situations where the data collected, used, disseminated and/or maintained for this contract include individually identifiable records covered by the Privacy Act of 1974, or personally identifiable information (PII) as defined by OMB Memorandum 07-16 (May 22, 2007), MDRC and its subcontractors comply with ED contractor security screening procedures for any staff members with access to such data, and maintain privacy safeguards as required under 5 U.S.C. 552a(m) with respect to such data.

• Access to identifying information about sample members is limited to staff members who have direct responsibility for providing and maintaining sample locating information. At the conclusion of the research, these data are destroyed.

• Identifying information is maintained in separate forms and files, which are linked only by sample identification number.

• Access to the file linking sample identification numbers with the respondents’ IDs and contact information is limited to a small number of individuals who have a need to know this information.

• Access to the hard-copy documents is strictly limited. Documents are stored in locked files and cabinets. Discarded materials are shredded.

Computer data files are protected with passwords, and access is limited to specific users. Especially sensitive data are maintained on removable storage devices that are kept physically secure when not in use.

• In safeguarding personally identifiable information (PII), MDRC and its subcontractors are also subject to ED’s requirements contained in the Department of Education’s Handbook for the Protection of Sensitive But Unclassified Information, OCIO-15, and the Department’s policy that the transmission of sensitive but unclassified information, including PII, through an e-mail requires that the contents be password protected in a ZIP file.

• If any PII is disclosed inadvertently or is at risk of disclosure due to a lost, missing, or intercepted transfer, MDRC is required to ensure that this breach is reported to ED immediately. Per OMB Memorandum M-06-19, ED is required to report suspected or confirmed breaches of PII security to a Federal incident response center within one hour of discovering the incident.

MDRC and its subcontractors will make certain that all surveys and student data are held in strict confidence, as described above, and that in no instance will responses be made available except in tabular form. Under no condition will disaggregated information be made available to school personnel. District and school staff responsible for assisting MDRC in the data collection will be fully informed of MDRC’s policies and procedures regarding confidentiality of data.

An explicit statement describing the project, the data collection, and confidentiality will be provided to study participants. These participants will include members of the RtI leadership and RtI delivery teams surveyed at participating elementary schools. Absent district mandates requiring consent, these participants’ consent will be assumed unless they explicitly opt out of a specific research activity. If districts require formal consent, adult participants will be asked to sign a statement acknowledging their willingness to participate using forms required by the district.

In addition, the following verbatim language will appear on all surveys and other applicable study-related materials:

All information gathered for this study will be kept confidential and will only be used for research purposes. The information collected about schools and students for this study will be used only for statistical purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law (Public Law 107-279, Section 183).

Because, the Privacy Act of 1974 applies to data proposed for this collection, a Notice for a New System of Records has been prepared for submission to the Federal Register.

A11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. The justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

There are no personally sensitive questions in this data collection.

A12. Provide estimates of the hour burden of the collection of information.

The estimated burden on respondents for completing the activities included in the study’s data collection activities is listed in Exhibit A-2. Respondents will include RtI reading teachers and grade level teachers, school district special education and student records staff, and elementary school principals or school-level staff identified by principals as being knowledgeable about the schools’ RtI practices.

Exhibit A-2 summarizes reporting burden on respondents to the RtI data collection instruments. The annual burden is estimated from the total number of minutes taken to complete the data collection. Thus, the total burden across all respondents is expected to be 11,886 hours or $436,751 in monetary cost.⁸

Exhibit A-2: Average Annual Burden to Respondents

Data Collection Activity	Number of Respondents Per School	Number of Schools	Total Respondents	Average Burden Hours per Response	Total Burden Hours	Hourly Rate	Estimated Monetary Cost of Burden
1. School level survey	1	161	161	1.0	161	$47	$7,567
2. Teacher level survey	12	161	1,932	.75	1,449	$36	$52,164
3. Interventionist survey	8	161	1,288	.50	644	$47	$30,268
4. Fall reading screening/ benchmark data and instructional decision form: school records collection:	1	161	161	10	1,610	$36	$57,960
5. Rosters of students in grades 1-3 showing tier placement during the year: school records data collection	1	161	161	40	6,440	$36	$231,840
6. Information on reading instruction for 8 students in each of grades 1-3 on tier movement, reading interventions, etc. school records collection:	8	113	904	1.5	1,356	$36	$48,816
7. Summary of tier placement/movement	1	113	113	2	226	$36	$8,136
Data Collection Activity	Number of Respondents Per School	Number of Schools	Total Respondents	Average Burden Hours per Response	Total Burden Hours	Hourly Rate	Estimated Monetary Cost of Burden
TOTAL			4,720		11,886	$36-47	$436,751

A13. Describe any other costs to respondents or record keepers.

Not applicable. The information collection activities do not place any capital cost or cost of maintaining capital requirements on respondents.

A14. Provide estimates of annualized cost to the Federal government. Also provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expenses that would not have been incurred without this collection information. Agencies may aggregate cost estimates from Items 12, 13, and 14 in a single table.

The proposed data collection is being supported through Department of Education contract ED-04-CO-0111/0003. This project has a total contract amount of $14,314,916, which supports the data collection and analysis described in this submission.

A15. Describe any changes in the burden from prior approvals.

This submission to OMB is a new request for approval of data collection plans.

A16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of the report, publication dates, and other actions.

The project schedule is as follows:

• Final selection of sites by winter of 2011-2012,

• Data collection for school years 2011-2012,

• Completion of data collection and creation of analysis files for final report by fall of 2012,

• Analysis of findings for final report in winter and spring of 2013, and

• Preparation of a project final report for review by the Department and release in late 2013.

A17. Describe arrangements for displaying the number provided by OMB and its expiration date.

The approval number provided by OMB and its expiration date will appear in the heading on all instruments for the RtI project.

A18. Exceptions to Certification Statement

No exceptions are necessary for this information collection.

References:

Batsche, G. (2009) The Florida Problem Solving Response to Intervention Project: Year

1 Evaluation Report. Tampa, Florida: University of South Florida and Florida

Department of Education

Bocala, C., Mello, D., Reedy, K., & Lacireno-Paquet, N. (2009). Features of state

response to intervention initiatives in Northeast and Islands Region states (Issues & Answers Report, REL 2009–No. 083). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northeast and Islands. Retrieved from http://ies.ed.gov/ncee/edlabs.

Design of the National Assessment of IDEA (July, 2006). Prepared under contract by

Westat, Rockville, MD.

Donovan, M. Suzanne and Cross, Christopher Cross editors (2002). Minority Students

in Special and Gifted Education. (National Academy Press: Washington, D.C.)

Fuchs D. & Fuchs, L. S. (2001). Responsiveness-to-Intervention: A blueprint for practitioners, policymakers, and parents. Teaching Exceptional Children, 38(1), 57-61.

Fuchs, D. & Fuchs, L. (2006). “Introduction to Response to Intervention: What, why,

and how valid is it?” Reading Research Quarterly, 41(1), 93-99. Retrieved from http://www.uoregon.edu/~cfc/classes/SPSY_607/Readings/Class%203/Fuchs.pdf

Gersten, R., Compton, D., Connor, C. M., Dimino, J., Santoro, L., Linan-Thompson, S., and Tilly, W. D. (2008). Assisting students struggling with reading: Response to Intervention and multi-tier intervention for reading in the primary grades. A practice guide. (NCEE 2009-4045). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/ publications/practiceguides/.

Grimes, J., & Kurns, S. (2003, December). An Intervention-based system for

addressing NCLB and IDEA expectations: A multiple tiered model to ensure every child learns. In Response to Intervention Symposium. Symposium held at the National Research Center on Learning Disabilities, Kansas City, Missouri. Available online at http://www.nrcld.org/ symposium2003/grimes/grimes3.html

Haager, D., Klingner, J. K., & Vaughn, S. (Eds.). (2007). Evidence-based reading practices for response to intervention. Baltimore: Paul H. Brookes.

Harr-Robins, J.J., Shambaugh, L.S., & Parrish, T. (2009). The status of state-level

response to intervention policies and procedures in the West Region states and five other states (Issues & Answers Report, REL 2009–No. 077). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory West. Retrieved from http://ies.ed.gov/ncee/edlabs.

Jankowski, E. A. (2003). Heartland Area Education Agency’s Problem Solving Model: An Outcomes Driven Special Education Paradigm. Rural Special Education Quarterly, 22(4), 29-36.

Johnson, E., Mellard, D. F., Fuchs, D., & McKnight, M. A. (2006). Responsiveness to intervention (RTI):How to do it. Lawrence, KS: National Research Center on Learning Disabilities.

Knudsen, W.E. (2008). “Coordinated Early Intervening Services Under Part B of the

Individuals with Disabilities Education Act.” Memo from the Office of Special Education Programs to Chief State School Officers and State Directors of Special Education. Retrieved November 5, 2008 from http://www.rti4success.org/images/stories/TA_amy/08-09coordinatedearlyinterveningservices-ceisunderpartbofidea.pdf.

Marston, D., Muyskens, P., Lau, M., & Canter, A. (2003). Problem-solving model for decision making with high-incidence disabilities: The Minneapolis Experience.

North Carolina Department of Public Instruction (n.d.). North Carolina Department of Public Instruction Problem-Solving Model Pilot: The Problem-Solving Process Online at http://www.dpi.state.nc.us/docs/ec/development/ learning/responsiveness/problemsolvingmodel.pdf

Rhode Island Technical Assistance Program (RITAP), online at http://www.ritap.org/rti/

RTI State Database by the National Center on Response to Intervention. (n.d).

Retrieved December 22, 2009, from http://state.rti4success.org/index.php?option=com_chart

Stecker, P., Fuchs, D., Fuchs, L. S. (2008). Progress monitoring as essential practice

within response to intervention. Rural Special Education Quarterly, 27(4), 10-17.

Sawyer, R., Holland, D., & Detgen, A. (2008). State policies and procedures and

selected local implementation practices in Response to Intervention in the six Southeast Region states (Issues & Answers Report, REL 2008–No. 063). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast. Retrieved from http://ies.ed.gov/ncee/edlabs.

Stepanek, J., & Peixotto, K. (2009). Models of response to intervention in the

Northwest Region states (Issues & Answers Report, REL 2009–No. 079). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northwest. Retrieved from www.ies. ed.gov/ncee/edlabs.

Tilly, W. D., (2007). The evolution of school psychology to science-based practice. In

A. Thomas and J. Grimes (Eds) Best Practices in School Psychology: Vol 1, (5^th ed., pp. 1-21). Bethesda, MD : National Association of School Psychologists.

Tilly, W. D., Harken, S., Robinson, W., & Kurns, S. (2008). 3 tiers of intervention. School Administrator, 65(8), 20-23.

Vaughn, S. & Fuchs, L. (2003). Redefining learning disabilities as inadequate

response to instruction: The promise and potential problems. Learning Disabilities Research & Practice, 18, 137–146.

Zirkel, P.A. (Jan/Feb, 2008). RTI After IDEA: A survey of state laws. TEACHING

Exceptional Children, Jan/Feb 2008, 71-73.

1 Knudsen, 2008.

2 The Office of Special Education Programs recently issued guidance to provide States with information regarding the use of funds provided under Part B of the Individuals with Disabilities Education Act by local educational agencies (LEAs) to develop and implement coordinated early intervening services (CEIS) for students who are currently not identified as needing special education.

3 See Van Der Klaauw (2008) and Shadish et al. (2002).

4 A fundamental RDD assumption is that students’ ratings and the cut-off point are determined independently of each other – such that assessments of individual students’ reading abilities are not influenced by considerations about whether to provide additional support to such students. The study team will verify this assumption’s validity during follow-up conversations with experienced RtI schools.

5 The literature does not provide much guidance on the minimum number of baseline years needed. MDRC tends to use three years as a minimum requirement for CITS. In general, longer baseline periods yield better estimates of trends, and, accordingly, yield better estimates of impacts based on deviations from trends.

6 National Academy of Sciences, Donovan & Cross, 2002.

7 Fuchs, D. & Fuchs, L., 2006.

8 The dollar value of school respondent burden was estimated by using information about teachers’ and school administrators’ average annual salaries, length of contract years, and average length of workdays obtained from NCES Schools and Staffing Surveys and from the National Association of Elementary School Principals. Hourly rates for the district respondents (RtI or special education coordinators) were assumed to be similar to those of school principals.

The number of anticipated respondents was estimated by using information about average student enrollments and average student/teacher ratios gathered from the NCES Common Core of Data (for estimating number of classroom teachers) and from piloting the current study’s data collection procedures with five different “experienced RtI” elementary schools (for estimating number of intervention teachers and RtI leadership team members).

The number of anticipated schools reflects the statistical power calculations presented in Part B of this justification. These calculations assume a maximum of 161 treatment and comparison schools if the CITS and RDD impact analyses are conducted and that these schools are likely to be distributed across 15 districts.

6