MEMORANDUM OMB # 1850-0790 v.38
DATE: September 16, 2013
TO: Shelly Martinez
Office of Information and Regulatory Affairs, Office of Management and Budget
FROM: Patricia Etienne
National Center for Education Statistics
THROUGH: Kashka Kubzdela
National Center for Education Statistics
SUBJECT: Response to OMB passback on Wave 2 of 2014 Main NAEP Submittal
Please provide responses to the following passback from OMB:
Please update the memorandum/supporting statement where is says that the systems clearance package was submitted on November 10, 2012. According to the ROCIS, the 60 day notice was published on November 21, and the package was submitted to OMB on January 25, 2013 so we are unsure to what November 12 is in reference.
Response: The dates have been corrected in Volume I.
Please clarify how the wave 1 and wave 2 clearance packages intersect regarding sampling, full set of assessments and burden on a given school. For example, will all schools participating in the TEL assessment have an additional assessment administered as well? If so, what is the total student burden (including assessments) for this NAEP administration?
Response: The Wave 1 and Wave 2 packages represent different samples of students for the 2014 assessments. Wave 1 pertains to the Grade 8 Social studies subjects (civics, geography, and U.S. history) and the science pilot samples at grades 4, 8, and 12. Wave 2 pertains to the Grade 8 TEL assessment. The two samples are in different schools. Therefore, no school will participate in TEL and the paper-based assessments.
Below is a summary of the sample components for 2014, with each sample component being in a separate set of schools:
Grade 4 paper-and-pencil: Includes the Science pilot only and has an estimated 1,500 total number of students.
Grade 8 paper-and-pencil: Includes the U.S. History, Geography, Civics and Science pilot assessments and has an estimated 27,500 total number of students.
Grade 8 computer-based: Includes the Technology and Engineering Literacy (TEL) assessment only and has an estimated 20,000 total number of students.
Grade 12 paper-and-pencil: Includes the Science pilot only and has an estimated 1,500 total number of students.
The total individual student burden for the TEL assessment is 90 minutes (30 for survey questions, set up and directions; and 60 minutes for cognitive components). While the total individual student burden for the social studies/science assessment is 75 minutes (25 for survey questions, set up and directions; and 50 minutes for cognitive components). Therefore, the total student burden (including assessments) is 68,125 hours.
Please clarify what percentage of the school coordinators NCES proposes to debrief. If proposed as the universe, why is a sample insufficient? Please justify any sample size selected.
Response: We are planning to debrief all school coordinators in the 2014 data collection cycle. This very short debriefing is the formal ending of the assessment in the school. The purpose of the debriefing is two-fold:
Provide an opportunity for the field staff to give any materials to the school coordinator that should be kept until the end of the school year. In addition, this debriefing provides the opportunity for the field staff to raise any issues about the data collection to the school coordinator
Obtain feedback on the data collection activities. We feel it is critical to debrief all school coordinators so that any feedback from the school can be discussed with the school coordinator. This opportunity for the schools to comment on the process is important to building and maintaining the relationship NAEP has with the schools. In addition, the feedback from the schools is an important consideration when evaluating aspects of the data collection process that may need to be modified for future administrations.
Why is it necessary to ask all teachers to participate in the Teacher/School Online Completion Feedback? What is the per-teacher burden estimate?
Response: The Teacher/School Online Completion Feedback consists of one additional question at the end of the teacher and school questionnaires. The estimated burden for this question is 10 seconds and is included in the overall estimate of 20 or 30 minutes for teachers and 30 minutes for school administrators. 2014 is the first year in which the teacher and school questionnaires will be given online, without an option of submitting them in paper format. The change in administration mode was purposefully made in 2014 given the smaller NAEP sample size than in the odd-year-assessments (for example, approximately 3,000 teachers will respond to the questionnaires in 2014, compared to approximately 76,000 in 2013). As such, NCES feels it is important to understand if the respondents prefer this method prior to finalizing data collection plans for 2015, which will again include many more teachers and school administrators. If the responses from the teachers and school administrators indicate an overwhelming preference for paper administration, NCES may reconsider the administration method used in 2015. In order to include the perspectives from respondents of varying backgrounds, preferences, and computer experience, it is necessary to gather feedback on this question from all teachers and school administrators participating in 2014.
Why are some of the school coordinator debriefing questions characterized as high school student strategies when the coordinators are of elementary or middle schools? Are these questions also asked of those coordinators?
Response: The High School Students Strategies Form (question #8 on the School Coordinator Debriefing Interview Questions) will only be asked of school coordinators in schools where high school students were assessed. These questions will not be administered to elementary or middle school coordinators.
| File Type | application/msword |
| Author | joc |
| Last Modified By | Authorised User |
| File Modified | 2013-09-17 |
| File Created | 2013-09-13 |