Supporting Statement Part B

Contract No.: ED-06-CO-0029 Lab Internal Ref No.:

Mapping the Adopted Core Curriculum in the Mid-Atlantic Region

Supporting Statement B for Request for OMB Approval of Announcement Letter

May 13, 2008

Claudia Burzichelli

William Morrill

Submitted to: Institute of Education Sciences 555 New Jersey Ave., NW Room 504E Washington, DC 20208-5500 Project Officer: Ray Valdivieso	Submitted by: Regional Educational Laboratory- Mid-Atlantic Penn State University 108 Rackley Building University Park, PA 16802 Project Director: Kyle Peck

TABLE OF CONTENTS

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Procedures 4

2. Statistical Methods for Sample Selection and Degree of Accuracy Needed 4

3. Methods to Maximize Response Rates and Deal with Nonresponse 5

4. Test of Procedures and Methods to be Undertaken 5

5. Individuals Involved 6

LIST OF EXHIBITS

Exhibit A. Announcement Letter

Exhibit B. Illustrative Example of On-Line Report

Exhibit C. Affidavit of Non-Disclosure

Exhibit D. Federal Register Notices

B. Data Collection Procedures and Statistical Methods

B1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

This project is a census of all districts in each state and the District of Columbia (DC) in the Mid-Atlantic Region to collect their curriculum adoption in purposively selected subjects (math, science, and language arts literacy) and grade levels (1, 4, 8, and 11). As derived from the NCES Common Core of Data, there are 1,496 school districts in the Mid-Atlantic region. The unit of response is each district. Thus, the census resides at the district level, not the curricular or grade levels. Sampling of districts will not occur in year 1. If analysis of year 1 data indicates that sampling is feasible for New Jersey and Pennsylvania districts, we will submit an OMB modification for years 2 and 3. Data collection in DC, DE, and MD will always be a census.

We will not generalize from these grade levels to the broader population of grade levels which are K – 12. Statements about curricula will be limited to grades 1, 4, 8, and 11. Nor will we generalize to the broader population of subject areas; statements will be limited to math, science, and language arts literacy.

In sum, we acknowledge that at the grade level and subject level of the data collection effort there is a broader population from which we have purposively sampled. Eventually, the aim is to conduct a district census that would include a broader array of grade levels and curricula.

B2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedures,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Consultation with Sampling Statistician and Senior Survey Researchers

Expert opinion has confirmed our preference for a census approach. Because of strong IES interest in a sample design, the Lab sought evaluation and critique of our preferred approach from eminent experts with a wealth of experience in designing complex data collection efforts. Experts consulted included:

• Michael Yang, Sampling Statistician, ICF/Caliber;

• Robert F. Boruch, Trustee Professor of Education and Professor of Statistics, University of Pennsylvania Graduate School of Education and Wharton School , and

• Mark Dynarski, Senior Fellow, Mathematica Policy Research, Inc.

All three agreed that our concerns about using a sampling approach were legitimate—especially for the three smaller jurisdictions. All agreed that for the three small jurisdictions with limited numbers of districts, data should be collected using a census approach. All did not totally rule out a sample approach for NJ and PA, but believed that some sort of pilot or intermediate investigation with a more comprehensive data collection would be needed to collect the information foundational to a sample plan. Without such a pilot (or intermediate investigation), the experts consulted did not believe we could assert, with any level of confidence, that a sample based report on NJ and PA would be representative.

B3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

We anticipate a 95% response rate because we will use our field-based outreach staff (LES) who have existing relationships with the local education agencies developed through need sensing activities and two successful recruitment campaigns for the lab’s large-scale randomized controlled trials. We plan to provide pre-notification of the census and will conduct multiple follow-ups with all potential respondents who did not respond. We will continue with follow-up calls to all non-responders on a monthly basis until the end of the school year. Personal visits to districts would occur only if scheduled for additional REL purposes (e.g., needs assessment or educational outreach) that would require on-site visitation. If a district does not participate in the collection, the missing data will be noted (in the aggregate) on our written reports. Finally, we have made every effort to keep our census questions simple and brief to reduce the burden on the respondents.

B4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

A pre-test was conducted with eight districts across the region to determine the actual time burden for respondents when follow-up contact with districts would be necessary (i.e., the initial participation letter did not yield data submission) and includes time spent speaking to extra staff (e.g., secretaries) needed to identify which district level person(s) would be able to provide the relevant data. None of the pre-tests took longer than 15 minutes and some took as little as 8 minutes. All pre-test respondents were amenable to providing the data, none refused to answer.

B5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unity, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Individuals Involved

Name	Title	Telephone
Bill Morrill	Principal Investigator	215-860-1766
Claudia Burzichelli	Principal Investigator	732-564-9100
Jennifer Bausmith	Study Manager	732-564-0189
Chris Magarelli	Database Manager	732-564-1793

6