Ed Memo 1-3-08

Memorandum US Department of Education

Institute of Education Sciences

National Center for Education Evaluation

­­­­­­­­­­­­­

TO: Bridget Dooling

Desk Officer

Office of Management and Budget

FROM: Amy Feldman

Research Scientist

National Center for Education Evaluation and Regional Assistance

SUBJECT: Response to follow-up OMB questions for REL West Study G, Assessment Accommodations for English Language Learners, 200709-1850-005.

On December 18^th, 2007, OMB submitted 2 follow-up questions on the Assessment Accommodations study out of REL West. Both questions asked for additional detail regarding burden in the pilot versus main study to clarify the differences between them and to provide justification for different incentive amounts.

In preparing a response to your questions the contractor went through all documents submitted in the collection to provide consistency and updates across them. We have attached to the email containing this memo all of the documents that were modified consistent with our response. They are attached in "tracked" and final versions, when possible.

Also of note is that this study was approved by the contractor’s IRB recently as a certified "Exempt" study, allowing them to use an information sheet to parents passive consent) instead of an active consent process. As a result, a revised version of Appendix B is also attached.

In the revised Supporting Statement A, you will find additions to the burden table (Table 4) to distinguish between the two phases of the study and to add previously missing burden, namely school staff time to gather and organize children for assessment as well as staff time for pulling archive data.

Also in response to your email questions, I have discussed the differences between the two phases of the study below for your clarification.

The pilot phase of the study (“Item tryouts”) is for testing the instrument to see if any tweaks are necessary. Because this is an instrument-refining step, power is not a priority. Therefore, the sample for the pilot will come from only 2 schools, with approximately 50 children in each school. While the children’s assessment time will be the same (1 hour per assessment to include the survey and instrument), burden to school staff is much reduced compared to the main study by the need for only 2 staff persons, a total of 4 hours of their time for the entire pilot. These two schools will NOT be asked for student archival data.

In contrast, the main study is designed to detect impacts and therefore power is our highest priority. Our calculations indicate that we will need 50 schools and an average of 72 children per school (for a total of 3600 children). One specific implementation assumption is that schools will group 36 students in each of two classrooms to administer the exam. One teacher would be assigned the task of administering the test in each classroom for 1 hour. Again, student assessment time is only 1 hour for both the pilot study and the operational test, but now, school staff required has increased from 4 hours (2 teachers supervising 2 classrooms for 1 hour in 2 schools) to 100 hours (2 teachers supervising 2 classrooms for 1 hour in 50 schools).

The difference in school burden is really due to the additional request of the main study schools, for student archival data, increasing the main study burden by 200 hours (50 staff X 4 hours). As a result, and shown in Table 4, the total time burden in the pilot sites is 4 hours, and in the operational sites is 300 hours.

The proposed incentive differential for the two phases results from incentivizing at the school level, the difference in burden (4 hours for the pilot study vs. 300 hours for the main study –due to the larger sample and the request for student archival data), and the importance of being able to detect the MDE in the impact study.

If further explanation of the two study phases is need, please don’t hesitate to contact me.