REL SW OMB Responses

DATE: September 22, 2015

TO: Sharon Mar, OMB

FROM: Michael Vaden-Kiernan, Task 4 Lead, REL Southwest

Chris Boccanfuso, Institute of Education Sciences

SUBJECT: Response to OMB for the proposed study “Assessing the Role of Noncognitive and School Environmental Factors in Students’ Transitions to High School in New Mexico”

1. We would like to understand IES’ plan for communicating the results of this study. As this will examine the correlation of school climate and growth mindset with 9^th grade outcomes, it is important to ensure that materials are careful in framing any recommendations or next steps. We’re excited about the areas for further study this will uncover, and hope the report and materials would emphasize that, but we would want to be sure that any significant correlation is not communicated as an area to take action on without further information.

We plan to be very careful to ensure that readers of our reports understand that this the results of this study represent preliminary work in this area. We will identify any significant correlations as areas for future study and exploration. The report will further explain that, due to the nature of the study, none of the results should be interpreted as causal. The study proposal approved by IES also acknowledges that the study team will be very careful to ensure readers do not interpret the results as causal.

2. We’d suggest that these two questions also look at how the scores differ by achievement level of students, and by whether the student’s are in the racial/ethnic minority or majority of the student body.

Yes. We can include these analyses. Additional subquestions have been added to the study.

3. In looking at these relationships, will the REL control for factors like poverty, EL, and disability status?

Yes, we will control for background characteristics such as EL, SES, and disability status in our analyses. The study proposal approved by IES already included a measure of SES; EL and disability status will also be included.

4. This survey should use items from the school climate survey that NCES is developing as a model for states, districts, and schools to use in the future. Since the goal of that school climate survey is to also have it used widely, we should ensure the questionnaire items are the same as much as possible.

NCES is currently clearing its benchmarking study through OMB, and can probably provide the questions for this study to draw from. This study’s questionnaire doesn’t need to use all of the NCES ones, and there are many on the growth mindset that IES won’t find a replacement for. But we ask that the study swap out as many of its questions as possible for similar ones in the NCES survey.

We have compared the scales on our survey to those included on the NCES school climate survey. We have exchanged/added several scales from the NCES school climate survey that measure the same and/or similar constructs. A revised version of the survey has been created. The new survey items are highlighted in Attachment A-3.

Because 14 items have been added to the survey, we now estimate that the average response time per survey response will be 30 minutes. However, the burden calculations remain unchanged from the previous draft. That is because the previous version of this package already (erroneously, at the time) included 30 minutes per survey response in the burden calculations, even though our calculations at the time were 20 minutes per survey. If you refer to the previous draft of Attachment A-3, you can see that we had noted an average response burden of 20 minutes.

5. Would it be possible to use students’ GPA just on core subjects, or perhaps even just on math and English? Would that add more burden to the study?

Yes, that would add additional burden to the study. Districts would have to send us grades for all classes, and we would have to determine which courses are core subjects. This would increase time burden for the study, and we are unsure whether districts would be willing to send us students’ individual course grades for all classes. We would prefer to use students’ GPAs across all subject areas, as that presents a more complete picture of a student’s grade 9 year.