Title I-II OMB Survey Package Part A 12-13-21 to IES

Implementation of Title I/II-A Program Initiatives — Survey Data Collection

Supporting Statement for Paperwork Reduction Act Submission

PART A: Justification

December 2021

Contract ED-IES-11-C-0063

Submitted to:

Institute of Education Sciences

U.S. Department of Education

Submitted by:

Westat

An Employee-Owned Research Corporation^®

1600 Research Boulevard

Rockville, Maryland 20850-3129

(301) 251-1500

and

Mathematica

P.O. Box 2393
Princeton, NJ 08543-2393

Contents

Page

Part A. Justification 1

A.1. Explanation of Circumstances That Make Collection of Data Necessary 1

A.2. Purpose and Use of the Data 2

A.2.1 Data Collection Activities for Which Clearance is Requested 2

A.3. Use of Technology to Reduce Burden 3

A.4. Efforts to Identify and Avoid Duplication 4

A.5. Efforts to Minimize Burden on Small Business or Other Entities 4

A.6. Consequences of Not Collecting Data 4

A.7. Special Circumstances 4

A.8. Federal Register Announcement and Consultation 5

A.9. Payments to Respondents 5

A.10. Assurance of Confidentiality 5

A.11. Questions of a Sensitive Nature 6

A.12. Estimates of Respondent Burden 7

A.13. Estimates of the Cost Burden to Respondents 8

A.14. Estimates of Annualized Government Costs 8

A.15. Changes in Hour Burden 8

A.16. Plans for Tabulation and Publication of Results 8

A.17. Display of Expiration Date for OMB Approval 9

A.18. Exceptions to Certification Statement 9

Appendix A.State survey A-1

Appendix B.District survey B-1

Appendix C. Principal survey C-1

Appendix D. Notification and reminder materials D-1

Tables

A-1. Estimates of respondent burden 7

A-2 Estimates of annual respondent burden over a 3-year period 7

Part A. Justification

When the primary federal law governing K–12 schooling was updated in 2015 as the Every Student Succeeds Act (ESSA), it shifted many decisions to states and districts. However, through two of its core programs (Title I and Title II-A), ESSA retained federal requirements for states to set challenging content standards, assess student performance, identify and support low-performing schools, and promote the development of the educator workforce. How states and districts respond to the combination of flexibility and requirements and how policies are enacted in schools and classrooms will determine whether ESSA stimulates educational improvement as intended, which is particularly important in the wake of educational disruptions wrought by the coronavirus pandemic. The U.S. Department of Education (the Department), through its Institute of Education Sciences (IES), is requesting clearance for data collection activities to support an implementation study of Title I and Title II-A. This package requests clearance for the state, district, and principal survey instruments and to administer these surveys.

A.1. Explanation of Circumstances That Make Collection of Data Necessary

Promoting equal access to high-quality schooling is a central goal of federal education policy. The Title I and Title II-A programs authorized under the Elementary and Secondary Education Act (ESEA) as amended by ESSA further this goal by providing funds meant to help schools and districts better serve low-income students and improve teacher and principal quality. Together, these programs account for over 70 percent of ESEA funding for 2020.¹ Under Title I, ESSA offers states and districts considerable autonomy while requiring them to adopt challenging academic content standards, aligned assessments, and accountability systems that set goals and identify and support low-performing schools. Under Title II-A, ESSA provides funding for a broad array of permissible activities to improve the effectiveness of educators and achieve an equitable distribution of effective educators.

The purpose of the study of the Implementation of Title I/II-A Program Initiatives is to describe the implementation of policies and practices related to Titles I and II-A of ESEA at multiple points in time. How states, districts, and schools implement ESEA in four areas—state content standards, assessments, accountability and turning around low-performing schools, and improving educator effectiveness—is key to understanding what policies and practices are being implemented and whether they are consistent with accomplishing the Title I and Title II-A program goals of improving educational outcomes and increasing educational equity.

The 2022 data collection follows the study’s two prior data collections in 2014 and 2018, during different phases of ESEA implementation. The first data collection, in 2014, occurred during the iteration of ESEA prior to ESSA. The second data collection, in 2018, occurred when states and districts were still transitioning to fully implementing ESSA. Examining implementation four years later in 2022 will provide a look at state and district policies and related district and school practices that are more established several years after ESSA’s passage, as well as implementation at all three levels in the context of the COVID-19 pandemic.

A.2. Purpose and Use of the Data

The data will be collected by Westat and its partner, Mathematica. The purpose of the 2022 data collection is to provide information on the core policies and practices related to Title I and Title II-A being implemented at the state, district, and school levels; and the resources and supports that states and districts provide to schools and teachers. The research questions for the data collection are as follows:

1. What materials and resources are states and districts providing to support the implementation of content standards? Did districts prioritize among the state content standards in response to the coronavirus pandemic?

2. How did states, districts, and schools use assessment results for the 2021-22 school year? To what extent and how do states, districts, and schools help educators and families understand student learning needs? Do principals find assessment results useful for instruction and to identify students needing support?

3. What measures from 2020-21 did states, districts, and schools use to assess school and student needs for support, and how have states modified their school performance systems since 2018? Have states modified their accountability plans under ESSA since 2018, and if so, how? How are states identifying and supporting lowest-performing schools? How are schools supporting students’ needs in the wake of the pandemic? Do stakeholders have access to state per pupil school-level expenditure data and how are states, districts, and schools using the data?

4. To what degree and in what areas are states, districts, and schools supporting teacher effectiveness in the 2021–22 school year? What strategies are states and districts using to recruit, hire, and retain effective teachers? How are states, districts, and schools using professional development and other strategies to help teachers support educational equity and improve teacher effectiveness? How are teacher evaluation results used in districts? Are states assessing the effectiveness of teacher preparation programs, and if so, how? Do states and districts find equitable access to effective teachers for low-income and minority students?

Data from state, district, and principal surveys will be collected to address the research questions since there is no uniform source of current, detailed information on these topics for all states and a nationally representative sample of district and schools. This package requests clearance for the survey instruments and associated data collection procedures. The study’s data collection activities are listed below.

A.2.1 Data Collection Activities for Which Clearance is Requested

State survey. This web-based state survey will focus on state policies and state supports in each of the four core content areas: state content standards, assessments, accountability and support for low-performing schools, and improving educator effectiveness. The survey will be sent to the chief school officer in each of the 50 states, the District of Columbia, and Puerto Rico beginning in March 2022, with the expectation that different sections of the survey may be filled out by different State Education Agency (SEA) staff, such as the Title I Director, as determined by their areas of expertise. States receiving Title I and Title II-A funds are expected to cooperate in Department evaluations following the Education Department General Administrative Regulations (EDGAR) (34 C.F.R. § 76.591). The state survey is in Appendix A.

District survey. This web-based survey will focus on the implementation of state policies, adoption of district policies and practices, and supports provided to schools and teachers in each of the four core content areas. The survey will be administered to superintendents or their designees in 1,150 school districts or charter local education agencies beginning in March 2022. Districts receiving Title I and Title II-A are expected to cooperate in Department evaluations following the Education Department General Administrative Regulations (EDGAR) (34 C.F.R. § 76.591). The district survey is in Appendix B.

Principal survey. This web-based survey will focus on the implementation of state and district policies within schools, the usefulness of supports received from the state and district, and supports provided to teachers in each of the four core content areas. The voluntary survey will be administered to principals from a nationally representative sample of 1,725 schools (nested within the sampled districts) beginning in February 2022. The principal survey is in Appendix C.

A.3. Use of Technology to Reduce Burden

The data collection is designed to obtain information in an efficient way that minimizes respondent burden. The main method for doing so is the use of web-based surveys. Web-based surveys are preferred for several reasons:

• This survey mode enables sharing a direct link to the surveys in communications with respondents. Respondents for all surveys will receive an invitation email and reminder emails as needed. In each correspondence, a URL to the survey will be shared, which allows respondents to click directly on the link to complete the survey.

• Web-based surveys allow for the use of: (1) skip patterns, which reduce burden, and (2) built-in edit checks (e.g., ensuring percentages do not total to more than 100 percent), which reduce response errors and decrease the need for the study team to reach out to respondents to check the accuracy of responses.

• Web-based surveys facilitate the completion of the survey by multiple respondents, which we expect to occur at the state level and potentially the district level as well. For example, we expect the three sections of the state survey will be completed by three different staff members (or staff teams). As a result, the state survey URL will include embedded login information to reduce the number of communications from the study team to the state to securely provide login information separate from the survey URL. In addition, web-based surveys reduce the burden of sharing access to the survey within the state or district and allow the most appropriate individual to easily access each section and provide the data in their area of expertise. Each state and district will be assigned a single password for their agency, which can be shared with the most appropriate respondents within the state and district. More than one respondent can work on different sections of the web-based survey simultaneously.

• Using a web-based survey for all respondents decreases the cost for postage, coding, keying, and cleaning of the data. This survey mode also allows respondents to complete the survey at a location and time of their choice.

A.4. Efforts to Identify and Avoid Duplication

The study will use extant documents for state information where possible to reduce burden. For example, the state survey section on accountability and support for low-performing schools includes several questions on state accountability systems (making up approximately one-third of the survey) that the study team will try to complete using information publicly available on state education agency websites and in approved ESSA state plans. If the study team can complete these questions using current and clear information, the online state survey will skip the state respondent over those questions. However, if the study team is unsure about the clarity or timeliness of the information, the respondent will be asked the question.

A.5. Efforts to Minimize Burden on Small Business or Other Entities

No small businesses will be involved as respondents. Every effort will be made to minimize the burden on all respondents, whether they are from larger or smaller districts and schools. We will assist respondents by telephone and email.

IES has a number of related studies underway looking at other aspects of ESSA and is taking steps to maximize learning about the implementation of ESSA while minimizing the burden on individual districts and schools. In particular, the district sampling strategy for the Implementation of Title I/II-A Program Initiatives’ 2022 data collection minimizes overlap with the district samples for two other IES studies: the National Implementation Study of Student Support and Academic Enrichment Grants (Title IV, Part A) and the Study of Educational Policies, Supports and Practices for English Learners: Implementation of Title III and Social and Emotional Learning (the Title III study). IES also is coordinating the Title I/II-A school sampling strategy with the Title III study (the Title IV, Part A study does not have a school sample) and the 2022 National Assessment of Educational Progress (NAEP) to minimize burden on individual schools as much as possible. See Part B sections B.2.2.2 and B.2.2.3 for a brief overview of the other IES studies and more information on the coordinated sampling approach.

A.6. Consequences of Not Collecting Data

Not collecting the data would leave policy makers and the public poorly informed about the implementation of the core programs for federal education policy during the critical pandemic recovery period. How these states, districts, and schools implement ESSA in four areas—state content standards, assessments, accountability and turning around low-performing schools, and improving educator effectiveness—is key to understanding whether the Title I and Title II-A programs will succeed in improving educational outcomes and increasing educational equity. This information is critical to the Department’s ability to improve federal programs and support states, districts, and schools. In addition, principal perspectives are particularly important for the 2021–22 school year given that the pandemic has interrupted the usual functioning of districts and schools and heightened the need to address educational equity.

A.7. Special Circumstances

There are no special circumstances involved with this data collection. The data collection will be conducted in a manner consistent with the guidelines in 5 CFR 1320.5.

A.8. Federal Register Announcement and Consultation

The 60-day Federal Register notice (ED-2021-SCC-0129) was published on September 3, 2021, and the 30-day Federal Register notice was published on TBD.

The study engages technical working groups to advise on the study and its findings. The most recent meeting of the technical working group was on May 3, 2019. The meeting focused on findings from the 2018 data collection and included a discussion of implications for the next round of data collection, currently planned for spring 2022. Members of the technical working group included:

• Jessica Baghian, Assistant State Superintendent for Assessment, Accountability, Early Childhood, Analytics, and Data Systems, Louisiana Department of Education

• Raymond Hart, Director of Research, Council of the Great City Schools

• Frederick Hess, Resident Scholar and Director of Education Policy Studies, American Enterprise Institute

• Sara Heyburn Morrison, Executive Director, Tennessee Board of Education

• Scott Norton, Deputy Executive Director for Programs, Council of Chief State School Officers

• Morgan Polikoff, Associate Professor of Education, University of Southern California

• Martin West, Professor of Education, Harvard University

• Tonya Wolford, Chief of District Evaluation, Research, and Accountability, School District of Philadelphia

A.9. Payments to Respondents

Payments are proposed for principals since their participation is voluntary and compensating study participants for their time and effort to complete the surveys will help the study achieve high response rates. High response rates will, in turn, allow the study team to more accurately address the study’s research questions.

If allowed by district policy, principals who complete a survey will receive a $20 thank you check. Where applicable, respondents will be informed of this incentive payment with their invitation to complete the survey. The proposed amounts for these payments are within the incentive guidelines outlined in the March 22, 2005 memo, “Guidelines for Incentives for NCEE Evaluation Studies,” prepared for OMB.

A.10. Assurance of Confidentiality

The Education Sciences Reform Act of 2002, Title I, Part E, Section 183 requires, “All collection, maintenance, use, and wide dissemination of data by the Institute” to “conform with the requirements of section 552 of title 5, United States Code, the confidentiality standards of subsection (c) of this section, and sections 444 and 445 of the General Education Provision Act (20 U.S.C. 1232g, 1232h).”

The names and email addresses of potential survey respondents will be collected for the limited purpose of drawing a sample, contacting those selected to complete the survey, and following up with nonrespondents. This information is typically already available in the public domain as directory information (i.e., state, district, and school websites).

The following language will be included on the surveys under the Notice of Confidentiality:

“Information collected for this study comes under the confidentiality and data protection requirements of the Institute of Education Sciences (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183). Responses to this data collection will be used by the U.S. Department of Education, its contractors, and collaborating researchers only for statistical purposes. The reports prepared for the study will summarize findings across the sample and will not associate responses with a specific district, school, or individual. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).”

For the state and district surveys, the notice will include the following text: “States/districts receiving Title I or Title II-A funds are expected to cooperate in Department evaluations (Education Department General Administrative Regulations (EDGAR) (34 C.F.R. § 76.591)).” For the state survey, we also will note that while individual states may be identified in reporting, reports will not associate responses with a specific individual and individual respondents will not be identified.

Specific steps to guarantee confidentiality of the information collected include the following:

• Identifying information about respondents (e.g., respondent name, address, and telephone number) will not be entered into the analysis data file but will be kept separate from other data and will be password protected. A unique identification number for each respondent will be used for building raw data and analysis files.

• Confidential materials will be printed on a printer located in a limited access field room. When printing documents that contain confidential information from shared network printers, authorized study staff will be present and retrieve the documents as soon as printing is complete.

• Access to the sample files will be limited to authorized study staff only; no others will be authorized such access.

• All members of the study team will be briefed regarding confidentiality of the data.

• All data will be stored in secure areas accessible only to authorized staff members. Computer-generated output containing identifiable information will be maintained under the same conditions.

• Hard copies containing confidential information that is no longer needed will be shredded.

A.11. Questions of a Sensitive Nature

There are no questions of a sensitive nature in the surveys.

A.12. Estimates of Respondent Burden

The principal survey will be administered beginning in February 2022, and the state and district surveys will be administered beginning in March 2022. Principal survey administration begins earlier because it can be difficult to engage with those respondents after the school year ends, as principals are often out for up to several weeks over the summer.

In all, responses will be requested from 2,927 respondents, and expected from 2,646 respondents. Estimated time to complete the survey is 180 minutes for the state survey, 60 minutes for the district survey, and 30 minutes for the principal survey. Based on the study team’s experience on other similar studies and on the prior data collections under this study, we estimate a response rate of 100 percent for the state survey, 95 percent for the district survey, and 87 percent for the principal survey. Total burden for the survey data collection is 119,970 minutes or 1,999.5 hours (see Table A-1 below). The annualized burden over 3 years is presented in Table A-2.

Table A-1. Estimates of respondent burden

Informant/Data Collection Activity	Sample size	Estimated response rate	Estimated number of respondents	Minutes per completion	Number of administrations	Burden in minutes	Total Burden Hours	Total Costs
State survey	52	100%	52	180	1	9,360	156	$7,725.12
District survey	1,150	95%	1,093	60	1	65,580	1,093	$54,125.36
Principal survey	1,725	87%	1,501	30	1	45,030	750.5	$37,164.76
Total	2,927	n/a	2,646	n/a	1	119,970	1,999.5	$99,015.24

NOTE: The state survey includes the District of Columbia and Puerto Rico. Assumes an hourly rate of $49.52 per hour for state, district, and school educational administrators (derived from the Bureau of Labor Statistics’ Occupational Employment and Wages for educational administrators, May 2020. The mean hourly wage was computed assuming 2,080 hours per year). See: https://www.bls.gov/oes/current/oes119032.htm).

Table A-2. Estimates of annual respondent burden over a 3-year period

Informant/Data Collection Activity	Annual Number of Respondents	Total Annual Burden Hours	Total Annual Costs
State survey	17	52	$2,575.04
District survey	364	364.3	$18,041.79
Principal survey	500	250.2	$12,388.25
Total	881	666.5	$33,005.08

NOTE: Assumes an hourly rate of $49.52 per hour for state, district, and school educational administrators (derived from the Bureau of Labor Statistics’ Occupational Employment and Wages for educational administrators, May 2020. The mean hourly way was computed assuming 2,080 hours per year). See: https://www.bls.gov/oes/current/oes119032.htm).

A.13. Estimates of the Cost Burden to Respondents

There is no annualized capital/startup or ongoing operation and maintenance costs associated with collecting the information.

A.14. Estimates of Annualized Government Costs

The amount for the design; conduct of the state, district, and school surveys; and analysis and reporting related to the 2022 data collection is $4,382,704. The annualized cost over 3 years for these activities is $1,460,901.

A.15. Changes in Hour Burden

The 2022 data collection is a new collection. No changes apply.

A.16. Plans for Tabulation and Publication of Results

The state, district, and principal survey data will be analyzed descriptively and summarized into two reports. One report will focus on state content standards, assessments, and accountability and support for low-performing schools (including CSI schools). The second report will focus on teacher effectiveness.

Extensive descriptive analyses will be conducted using the survey data. We anticipate that straightforward descriptive statistics (e.g., means, frequencies, and percentages) and simple statistical tests (e.g., tests for differences of means and proportions) will typically be used to answer the research questions. The study is descriptive and not designed to estimate the impact of federal policies on state and local actions.

Cross-tabulations will be important to answer questions about variation across district and school characteristics. The primary characteristics of interest for the cross-tabulations are:

• District poverty level is included because Title I is specifically intended to ameliorate the effects of poverty on local funding constraints and educational opportunity.

• District size is included because it may be related to district capacity to develop and implement programs.

• District urbanicity is included because of the relationships between educational opportunity and rural isolation and the concentration of poverty in urban schools.

• School grade span (high school, middle, and elementary grades) is included because the implementation of state content standards and aligned assessments as well as accountability measures and approaches to improving student outcomes likely differ by grade level.

• School Title I status is included because the focus of Title I funds and requirements is to influence state and district policy to improve equitable access to educational opportunities.

• School CSI status is included because districts are encouraged by federal policy to focus on these lowest-performing schools to bring about improvements.

The reports may include some limited analyses of cross-level implementation by examining responses of districts, and principals by categories of state or district responses. These analyses will examine the relationship between policies and programs originating at the state or district level and implementation “on the ground” in districts and schools. Examples of such analyses include examining the actions planned by districts to turn around lowest-performing schools by the interventions required by states..

For state data, the study will report numbers of states and unweighted means and percentages. Because of the use of a statistical sample, survey data presented for districts and schools will be weighted to national totals. In addition, the descriptive tables will indicate where differences between subgroups are statistically significant. We will use Chi-Square tests to test for significant differences among distributions and t-tests for differences in means.

The reports are expected to be published in 2023 and 2024 and will be available on the IES website. Each report will be 15 pages, with a set of technical appendices and compendium of data tables. The reports will be written for an audience of policy makers and practitioners. The reports will follow the January 2020 IES Style and Report guidance and meet all 508 compliance requirements.

In addition to the reports for the Title I/II-A study, some survey data will be shared with the other IES implementation studies noted in section A.5 for their reporting. Specifically, because of the strong overlap in content, survey data related to Comprehensive Support and Improvement schools will be shared with the Study of School Improvement Plans and Their Implementation, and data related to English learners will be shared with the Title III study.

A.17. Display of Expiration Date for OMB Approval

The Institute of Education Sciences is not requesting a waiver for the display of the OMB approval number and expiration date. All data collection materials will display the expiration date for OMB approval.

A.18. Exceptions to Certification Statement

This submission does not require an exception to the Certificate for Paperwork Reduction Act
(5 CFR 1320.9).

1U.S. Department of Education. (2020). Fiscal year 2020 Congressional Action: 2020 Appropriations. Downloaded from: Department of Education Fiscal Year 2020 Congressional Action