Response to OMB comments

MEMORANDUM

To: Ok-Choon Park, Ray Valdivieso, Amy Feldman

From: Catherine Walcott, REL West

CC: Gary Estes, Nikola Filby, Kenwyn Derby, Neal Finkelstein, REL West

Date: February 28, 2008

Re: Response to OMB questions: REL West Educational Needs Assessment Survey (ENAS)

Attached are our responses to written comments from OMB that we received from IES on February 26, 2008. The question numbers refer to the previous round of OMB questions and IES/REL responses (submitted by ED to OMB on February 13, 2008). We would be happy to discuss these responses with you by telephone conference if that would be helpful.

Question 2. This response implies that there will be new REL studies under the current ED contract. OMB was under the impression that all studies had been submitted for the 5 year cycle and now the RELs were in implementation mode. Please provide more information about how many additional studies are likely and when, and whether this is unique to this REL or common to all.

IES instructed us that they and OMB had a conference call Thursday, February 28^th resolving this issue. The REL was simply referring to the Fast Response projects whose topics are informed by the needs assessment information.

Question 3. This response suggests that the REL does not know exactly what the various professional associations will provide. It is OMB's understanding that many teacher association lists are biased in that new teachers are under represented. In addition, it sounds as if the REL may get a lot of names on the lists that are not teachers, and may not be able to easily identify them. If the REL requires a Needs Assessment with the level of rigor and precision it claims, then these issues should be addressed now and the plans be provided to OMB.

The concerns raised by this question are correct and we acknowledge that there may be limitations to the generalizability of the needs analysis study because of them. For example, the extent to which the lists are representative of new teachers cannot be assessed until the lists are secured. In addition, although rigorous plans to maintain high response patterns are in place, bias associated with the response patterns is possible.

Still, we will proceed with the original rigor that is described in the supporting statement, but acknowledge that all along, limitations and disclosure of threats to generalizability will be part of the research process and reporting. As we acknowledged before, the associations will provide us lists of members with a few demographic variables and contact information. The associations will provide us lists of members who meet our sample requirements. For example, we expect to get a list from the California Teachers Association that contains only FTE elementary and secondary teachers (i.e., only currently active teachers).

Fortunately, there are analytic techniques that may be of help during the analysis phase of the study; we will continue to review these strategies with our IES monitors as that stage of the project evolves. For example, if new teachers or any other key demographic groups would be underrepresented on these association lists we will have enough information from the associations in order to re-weight the sample to adjust for any major discrepancies. We will compare the characteristics of our sample with published data on the universe of teachers (or other respondent groups), so we can make weighting adjustments to ensure that our findings properly represent the population. Regardless, we will report limitations to the study as appropriate and consider them in any conclusions and interpretations of the findings.

Question 8. This response shows that contact information is only needed to initially contact the teachers. However, the requests for contact information are also on the surveys themselves. Therefore, please remove the requests for contact information from the surveys.

We have updated the four survey instruments, attached to the memo, removing the requests for contact information. Note that this version also includes explicit reference to The Education Sciences Reform Act in the confidentiality pledge in the cover letter, as advised by OMB in their comments of February 5, 2008.

2