Att_REL West ENAS_OMB Supporting Statement Part A_08.13.07

REL West Needs Assessment Survey OMB SUPPORTING STATEMENT A

REL West Educational Needs Assessment Survey

(Task 1.1)

Request for OMB Approval

Supporting Statement for Paperwork Reduction Act Submission

Part A

Version 3

Revised Submission: August 2007

Submitted to: Submitted by:

U.S. Department of Education Regional Educational Laboratory West

Institute of Education Sciences (REL West at WestEd)

555 New Jersey Ave., NW, Rm. 308 730 Harrison Street

Washington, DC 20208 San Francisco, CA 94107

(202) 208-7078 (415) 565-3000

Project Officer: Project Director:

Rafael Valdivieso, Ph.D. Hans Bos, Ph.D.

U.S. Department of Education Berkeley Policy Associates (BPA)

(202) 208-0662 (510) 465-7884 x 217

TABLE OF CONTENTS

Supporting Statement A: Study Justification

Introduction	1
A-1. Circumstances That Make Data Collection Necessary………………………………………..	2 33
A-2. Purpose and Use of the Data……………………………………………………………………	2
A-3. Use of Information Technology To Reduce Burden…………………………………………..	4
A-4. Efforts to Identify and Avoid Duplication………………………………………………………….	4
A-5. Sensitivity to Burden on Small Entities…………………………………………………………	4 5
A-6. Consequences to Federal Program or Policies if Data are not Collected………………….	4
A-7. Special Circumstances…………………………………………………………………………..	4
A-8. Solicitation of Public Comment and Outside Consultation……………………………………	5
A-9. Payment of Gift to Participants………………………………………………………………….	5
A-10. Assurances of Confidentiality………………………………………………………………….	5
A-11. Sensitive Questions…………………………………………………………………………….	7
A-12. Estimates of Hourly Burden to Participants………………………………………………….	7
A-13. Estimate of Total Annual Cost Burden to Participants or Record-Keepers………………	8
A-14. Estimates of Annualized Cost to Federal Government……………………………………..	8
A-15. Program Changes or Adjustments…………………………………………………………….	9
A-16. Tabulation, Analysis, and Publication of Results…………………………………………….	9
A-17. Displaying the Expiration Date for OMB Approval…………………………………………..	10
A-18. Exception to the Certification Statement……………………………………………………..	10
Appendices
Appendix A. Teacher Survey
Appendix B. School Administrator Survey
Appendix C. District Administrator Survey
Appendix D. California County Superintendent Survey
Appendix E. Mail/Email Letter
Appendix F. Federal Register Notice
Appendix G. Confidentiality Forms

REL West Educational Needs Assessment Survey

Introduction

The Educational Needs Assessment Survey for the Regional Educational Laboratory West (REL West) will be conducted by Berkeley Policy Associates (BPA), under contract with WestEd. The purpose of the survey is to determine the needs of educators in the states of the western region that REL West serves—Arizona, California, Nevada, and Utah. The results of this survey will help to guide a research agenda that addresses state and regional educational needs.

The survey assesses needs as experienced by three groups of educators: teachers, school administrators, and district administrators. In addition, the survey will be administered to all 58 California County Office of Education superintendents, who serve in similar capacities to district superintendents. Slightly different versions of the survey will be administered to these four groups (See Appendices A–D).¹ This survey is designed in such a way that results may be combined for certain analyses, but the sample is designed to produce reliable survey results for each of the three main constituent groups separately as well as for each of the four states.

To obtain a representative sample of survey respondents from each of the three main constituent populations (excluding CA county administrators, as we are surveying the total population), we will use membership records from professional organizations that represent the constituent populations in each of the states. For the most part, the survey will rely on random sampling from the association membership lists to represent the diversity of the population described here. However, before drawing a survey sample, we plan to stratify the sampling frame by state and respondent type. Specifically, we will stratify the teacher and school administrator samples by school level (e.g., elementary or secondary), urbanicity, and geographic area. Some strata may be oversampled purposely to ensure sufficiently large samples to allow reliable survey results.² Association membership lists represent a vast majority of the constituent groups we are surveying in these four states and afford us access to personal email addresses and home phone numbers that we would not be able to obtain through state department of education websites. Association endorsement of the survey will also help increase the response rate. (See Part B for a more detailed discussion of the above.)

The survey will be administered primarily as an online survey. Prior to our contacting potential respondents, the professional organizations that provide us with contact information will send their sampled members a letter (or we will send the letter) announcing the survey and explaining its importance for the field and for their membership (see Appendix E. Letter). This letter will also include information on how to access and complete the survey. Using the contact information provided by the professional organizations, we will also send all sample members an email (see Appendix E. Letter) with a web-link and a phone number. Sample members will be asked to complete the survey online, but will be given the opportunity to complete it by phone by calling BPA, where staff members will be ready to take their calls and will read the same introduction, instructions, and questions as those on the online version.

A: Study Justification

A1. Circumstances that Make Collection of Data Necessary

This proposed information collection will be conducted as part of the 2005-2010 Regional Educational Laboratories Program. The current authorization for the Regional Educational Laboratories program is under the Education Sciences Reform Act of 2002 (ESRA), Public Law 107-279, Part D, Section 174, (20 U.S.C. 9564), administered by the Institute of Education Sciences’ National Center for Education Evaluation and Regional Assistance, which states that “each regional educational laboratory awarded a contract under this section shall support applied research, development, wide dissemination, and technical assistance activities by developing a plan for identifying and serving the needs of the region by conducting a continuing survey of the educational needs, strengths, and weaknesses within the region.” As required in the above legislation, the REL West will conduct a regional educational needs assessment, including systematic market research on needs from a range of stakeholders.

Further justification and detail is found in Section 1.1—Regional Educational Needs Analysis, Training and Technical Assistance Response Unit of the Institute of Education Sciences’ 2005 Statement of Work (in the Request for Proposal) for the Regional Educational Laboratory contracts (ED-05-R-0006), which indicates that “the contractor [REL West] shall assess regional needs regularly through surveys and contractor-developed outreach strategies to solicit comments from schools, teachers, and parents, and from district and state policymakers and administrators for pressing concerns that need attention.” This survey of local educators fits within a larger needs analysis effort directed at the various levels of stakeholders in the region, ascertained through a variety of means, including conducting interviews, attending meetings, and otherwise engaging in an ongoing basis in the western region.

The information to be collected in this survey is not currently available at the school and district level for these four states—as determined by internet searches and conversations with researchers, associations, and policymakers in the region—so must be ascertained in a new information collection that directly addresses specific practitioner needs.

A2. Purposes and Use of the Data

Developed for teachers and school and district administrators in the western states (Arizona, California, Nevada, and Utah), the survey is designed to yield valuable information about school practitioner needs and priorities as they relate to issues of school improvement, educating English learners, quality of teaching, teacher workforce, assessment, student readiness to learn, and secondary school reform — areas of concern and need in the region as identified by REL West staff outreach and state-level needs assessment activities over the last year, as well as reviews of recent policy and research reports. The purpose of this data collection is to supplement the REL West’s ongoing needs assessment activities with education stakeholders at the state level in the region, as required under REL West’s contract with the U.S. Department of Education, discussed in Section A1. The information collected will be used to inform the direction and content of the REL West’s research agenda in the coming years, as well as to provide broader information on needs to the audience of policymakers and other researchers in the region.

Exhibit 1 below lists the three respondent groups this needs assessment data collection activity targets, each with its own slightly different instrument to allow for more clear and accurate wording (e.g., “your school” vs. “your district”). In addition to the role groups described above, this survey will also be administered to California County Offices of Education Superintendents. This is the case only for California because of the special role County Offices of Education (COE) play in this large and populous state, with County staff having similar responsibilities in relation to the districts they serve as district staff have in relation to the schools they serve. In addition, when consulted about this project’s scope and purpose, the Association of COEs (CCSESA, or the California County Superintendents Educational Services Association) expressed interest in having their members participate in the survey.

Also included in Exhibit 1 are the content areas by which specific items are organized, which reflect the broad areas of need previously determined by REL West and listed above in this section. The table also shows the mode of administration, discussed in Section A3 below and Part B of this OMB Clearance Supporting Statement, the time burden estimate (discussed below in Section A12), and the timeline for administration, to be shifted later in the winter if OMB clearance is not secured in time to begin administration some time in January.

Exhibit 1. Data Collection Instruments (See Appendices A-D)

Instrument3	Respondent Group	Content	Mode of Administration	Time Needed	Timeline
Teacher survey	PK-12 teachers	Needs and priorities as they relate to issues of school improvement, educating English learners, quality of teaching, teacher workforce, assessment, student readiness to learn, and secondary school reform; limited experience and background information	Online or Telephone	20 minutes	January-February 2008
School administrator survey	PK-12 school administrators	Needs and priorities as they relate to issues of school improvement, educating English learners, quality of teaching, teacher workforce, assessment, student readiness to learn, and secondary school reform; limited experience and background information	Online or Telephone	20 minutes	January-February 2008
District administrator survey	PK-12 school district administrators	Needs and priorities as they relate to issues of school improvement, educating English learners, quality of teaching, teacher workforce, assessment, student readiness to learn, and secondary school reform; limited experience and background information	Online or Telephone	20 minutes	January-February 2008
California county school superintendents	California county school superintendents	Needs and priorities as they relate to issues of school improvement, educating English learners, quality of teaching, teacher workforce, assessment, student readiness to learn, and secondary school reform; limited experience and background information	Online or Telephone	20 minutes	January-February 2008

A3. Use of Improved Information Technology to Reduce Burden

The respondents for this survey should have ready access to technology; hence, most data collection is intended to be conducted online. This has the advantage of reducing costs and possible errors associated with data entry. However, all respondents will also have the option of completing the survey through a telephone interview, should they not have access to an internet connection or if they simply prefer to respond via phone. The option presented to respondents is intended to reduce respondent burden and increase response rates. The phone survey asks identical questions in the same order. A paper/mail-in survey is not going to be offered as an alternative in the interest of saving both time (for mailing and data entry) and money.

A4. Efforts to Identify and Avoid Duplication

There are no surveys of a broad spectrum of local educators (i.e., school district administrators, school administrators, and teachers) in this four-state western region asking them about their specific research-based information needs in order to improve education at their levels. This was determined through internet and literature searches and conversations with researchers, associations, and policymakers in the region. Currently no forums or other methods exist that would enable us to systematically capture information about the needs and concerns of such a large and representative sample of educators in the field (at the school and district levels).

A5. Efforts to Minimize Burden on Small Businesses or Other Entities

Small businesses and other entities (e.g., schools) will not be responsible for this survey data collection, nor will their assistance be needed in any response or information collection. Respondents for this survey, as listed in Exhibit 1, are individual employees of the public school system.

A6. Consequences if the Information is Not Collected or is Collected Less Frequently

In the absence of this survey it would be difficult for REL West to carry out its responsibility under the authorizing legislation; in particular, it would be unable to effectively determine the educational needs and concerns of such a broad spectrum of local educators. We will only collect this information in this manner once during this five-year contract (two years into it), with the possibility of conducting other needs assessment activities with these constituent groups in the future (e.g., focus groups, more focused surveys of single role groups based on results of this current data collection).

A7. Special Circumstances

This information collection will not be conducted in a manner that will require using any special circumstances.

A8. Federal Register Comments and Persons Consulted Outside the Agency

A notice about the study will be published in the Federal Register when the final OMB package is submitted. The draft of this notice is included in Appendix F.

Lead researchers for the survey have consulted on both the content and form of data collection with experts in survey design and sampling, education policy, and school and district support services within WestEd and BPA, as well as with leaders from the federally funded Comprehensive Centers that serve these states and local and regional education stakeholder organizations (e.g., representatives from state departments of education, legislators, professional associations).

A9. Payments to Respondents

No payments to respondents will be offered, nor other incentives provided, for this survey.

A10. Assurances of Confidentiality

REL West [WestEd and subcontractors including BPA] researchers and staff follow the confidentiality and data protection requirements of IES (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183), which conform to the requirements of the Privacy Act Section 552 of Title 5 of the United States Code covering the collection, maintenance, and disclosure of information from or about identifiable individuals.  We will protect the confidentiality of all information collected for the study and will use it for research purposes only.  No information that identifies any study participant will be released.  Information from participating institutions and respondents will be presented only at aggregate levels in reports. Information on respondents, if linked to their institution, will not be linked to any individually identifiable information.  No individually identifiable information will be maintained by the study team.  All institution-level identifiable information will be kept in secured locations and identifiers will be destroyed as soon as they are no longer required.

BPA obtains signed Affidavits of Nondisclosure and Confidentiality Agreements from all employees, subcontractors, and consultants that may have access to this data, and we will submit them to our NCEE COR.

All communication to survey recipients (invitations to participate and follow-up reminders) and the survey itself will include the following language assuring respondents of confidentiality:

“Responses to this data collection will be used only for statistical purposes. The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific district/school or individual. We will not provide information that identifies you or your district/school to anyone outside the study team, except as required by law.”

Copies of the confidentiality forms to be signed by BPA staff working on the study are provided in Appendix G.

BPA implements data security policies and programs. Below is an overview of BPA’s data security policy:

Policies for Class 1 Data (Confidential data, with identifying information) are:

(1) Can never leave BPA premises.

(2) Always kept in a secure place.

(3) Only authorized persons can access and use.

(4) Must be properly disposed of or transferred.

Exhibit 2 summarizes the procedures for handling Class 1 data.

Exhibit 2. Procedures for Handling Class 1 Data

	Electronic Data	Paper Data
Receipt and tracking of Class 1 materials	Notify office manager if expecting to receive confidential data Catalogue all data received	Catalogue all data received Notify office manager if expecting to receive confidential data
Can never leave BPA premises	Must work on BPA premises with these data (working from home/during business trip is not permitted)	Must work on site at BPA with these data
Create separate working analysis file	Strip individual-identifying information for analysis files, which can then be stored in access-limited folders on BPA’s LAN
Always kept in a secure place	On data server or in locked cabinet in locked server room (CD or other disk media) Must not be left unattended in public view (e.g. on desk or screen) May not be stored on laptop	Store in locked cabinet in a locked room Must not be left in public view (on desk or in common-use areas)
Only authorized persons can access and use	Limit access to the data server by use of passwords The minimum number of people who absolutely need to use the data should be given access	Key to locked cabinet to be kept securely by authorized persons The minimum number of people who absolutely need to use the data should be given access
Must be properly disposed of or transferred	Update catalogue whenever data are disposed of or transferred Mail data in a password protected and/or encrypted form on an unmarked diskette and CD Require recipient and delivery verification. If absolutely necessary to transfer via email or internet, create encrypted, password-protected files; transmit password verbally (by phone). Do not include password in email!	Update catalogue whenever data are disposed of or transferred When mailing, require recipient and delivery verification. Shred any paper with confidential data before disposing

Policies for Class 2 (Proprietary data and documents that are not Class 1) are:

(1) Only authorized persons can access and use.

(2) Must be used and stored under responsible person's oversight. Must not be left in public view (e.g., sitting out on a desk, open on computer monitor).

A11. Justifications for Questions of a Sensitive Nature

The questions on the survey do not address sensitive topics. Even so, survey recipients may choose not to participate, and they may elect to skip any question(s) they wish. In addition, the survey asks for opinions, so there are no right or wrong answers.

A12. Estimate of Information Collection Burden

As indicated earlier, the survey data collection will occur once only. Exhibit 3, below, shows that the estimated annual/total respondent burden for this data collection is 1,042 hours.

Based on a pilot test of these instruments with less than 9 people in similar roles to those that will be surveyed, the mean time to respond was 12 minutes; but as these samples are small, some surveys will be completed over the phone (taking more time), and some internet connections are slow, the estimate represents a reasonable amount of time within which respondents should be able to complete the survey.

Exhibit 3. Respondent Hour Burden Estimate

Type of Respondent	Data Collection Activity	Hour Burden per Respondent (in hours)	Annual/Total Expected Number of Respondents	Annual/Total Hour Burden (in hours)
Teachers	Teacher survey	.33	1,360	449
School administrators	School administrator survey	.33	1,360	449
District administrators	District administrator survey	.33	388	128
California county superintendents	California county superintendent survey	.33	49	16
TOTAL	------------------------------	----	3,157	1,042

The estimated annual/total hour/cost burden for all data collection is presented in Exhibit 4 below.

Exhibit 4. Respondent Cost Burden Estimate

Type of Respondent	Data Collection Activity	Annual/Total Respondents	Annual/Total Hour Burden	Hourly Rate4	Annual/Total Cost Burden
Teachers	Teacher survey	1,360	449	$19.00	$ 8,531.00
School administrators	School administrator survey	1,360	449	$36.19	$ 16,249.31
District administrators	District administrator survey	388	128	$36.19	$ 4,632.32
California county superintendents	California county superintendent survey	49	16	$36.19	$ 579.04
TOTAL	----------------------	3,157	1,042	----------	$ 29,991.67

A13. Estimate of Total Annual Cost Burden

There are no direct start-up or operation costs to respondents, other than their time to participate in the surveys, as estimated above.

A14. Estimates of Annualized Costs to the Federal Government

The total budget for the Educational Needs Assessment Survey for the REL West, to be carried out by the subcontractor BPA, is $196,200, which covers all activities including survey design, programming, implementation, data processing, analysis, and reporting. As all costs will be incurred within one year, the total cost is equal to the annual average cost. Exhibit 5 below provides a more detailed break-down of the budget.

Exhibit 5. Cost and Number of Allocated Staff Hours by Study Activity

Study Activity	Cost	Number of Allocated Staff Hours
Survey design	$15,422	168
Programming	$12,207	202
Implementation	$22,181	216
Data processing	$5,603	84
Analysis	$14,828	172
Reporting	$16,600	196
Management	$7,213	120
Subcontract for phone call follow-up	$ 102,146	2,375
TOTAL	$196,200	3,533

The proposed rates are based on current staff rates and 2,080 hours, or 260 days, per year. The standard workday is eight hours. Based on leave accrual policies as stated in the employee manual, BPA calculates an average combined sick/vacation leave of twenty days, plus eleven paid holidays per year. The cost of this leave time is included in the fringe benefit rate (38%). BPA pays the following fringe benefits: F.I.C.A., Workers’ Compensation, Health and Welfare Insurance, Retirement, Holiday/sick/vacation, and Federal and State taxes.

Other direct costs include:

Telephone: Telephone usage is estimated at $0.40 per direct labor hour, which is based on the firm’s historical usage patterns.

Supplies, Subscriptions, Books: The cost of supplies, subscriptions, books and other similar direct charge materials is estimated at $0.25 per direct labor hour, which is based on the firm’s historical usage patterns.

Copier: Copier costs are estimated at $0.60 per direct labor hour, which is based on the firm’s historical usage patterns.

Postage/Mailing: Postage and mailing costs are estimated at $0.40 per direct labor hour, which is based on the firm’s historical usage patterns.

Computer: Computer usage costs are estimated at $1.00 per direct labor hour, which is based on the firm’s historical usage patterns.

As a long-term contractor to the federal government, BPA’s indirect rates are as follows:

Labor Overhead: 49% of personnel and fringe costs

General and Administrative: 14% of all contract costs including labor overhead with the following exception: General and Administrative is charged on only the first $25,000 per year for each subcontract under $200,000, and on the first $50,000 per year for each subcontract over $200,000.

A copy of our most recent negotiated rate agreement is available upon request.

A15. Program Changes or Adjustment

This is a new study/data collection.

A16. Plans for Tabulation and Reporting of Results

Exhibit 6. Survey General Timeline

2008
January	February	March	April	May	May-September
Send mail letters to potential respondent sample Send email letters to potential respondent sample Begin phone follow-up with intensive follow-up sample after two weeks of survey being in the field	Finish phone follow-up with intensive follow-up sample by mid-October Mid-October begin data cleaning	Begin analyses Write draft report	Draft report submission due to IES	Final draft report submission due to IES	Report dissemination (to professional associations, state and local policymakers, other researchers, etc.)

As indicated earlier, the timeline proposed above may need to be shifted later in the year depending on when OMB clearance is granted.

The final report will describe the needs and priorities of educators in the western region as they relate to issues of school improvement, educating English learners, quality of teaching, teacher workforce, assessment, student readiness to learn, and secondary school reform. The report will summarize the distribution of responses to each item for each state and separately for each respondent group. Descriptive statistics, such as percentages, medians, means, ranges, and standard deviations will be used to describe the distributions. The analysis also will examine the extent to which needs expressed are consistent across different groups of respondents, including respondents in different professional roles within and across the four states, respondents in different states in the western region, and respondents in areas with different degrees of urbanicity and other key characteristics. A more detailed discussion is included in Part B of this Supporting Statement.

Once the final report is approved by IES, they will soon thereafter post the report on the public IES REL website. REL West staff will also utilize results in planning future fast-response research studies of interest to the western region, disseminate the results via email and in-person meetings to professional associations whose members participate, discuss findings with regional Comprehensive Centers that provide support to state departments of education, present findings at regional forums as invited, and share the resulting education practitioner needs with other state and local policymakers and practitioners.

A17. Display of Expiration Date for OMB Approval

No request is being made for exemption from displaying the expiration date.

A18. Exceptions

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.

1 The differences between the versions of the survey are small, to facilitate analysis across role groups. Each survey respondent group is asked to report about their particular jurisdiction (e.g., teachers are asked about their school, district superintendents are asked about their school district). Teachers are also asked more specific questions about what grade levels and subjects they teach.

2 This would mean our sample would now be a clustered random sample. However, as long as the clustering is minimal it should have very little impact on our statistical power.

3 The differences between the versions (instruments) of the survey are small. Each survey respondent group is asked to report about their particular jurisdiction (e.g., teachers are asked about their school, district administrators are asked about their school district). Teachers are also asked more specific questions about what grade levels and subjects they teach.

4 Based on averages across the four states from May 2005 State Occupational Employment and Wage estimates found at http://www.bls.gov