EL Surveys OMB Part A

The U.S. Department of Education, through its Institute of Education Sciences (IES), is requesting clearance for data collection activities to support a study to evaluate and inform entrance and exit policies for English Learners (ELs). This package requests clearance for the district and school survey instruments and administration of these surveys. This data collection is a revision to an earlier data collection request submitted to OMB to obtain extant data on students from state longitudinal data systems (SLDSs) [OMB Approval # 1850-0974].

A.1. Circumstances Making Collection of Data Necessary

Classification into and reclassification out of English learner (EL) status are both high-stakes decisions with far-reaching impacts for students, educators, and education systems. To help achieve better outcomes for ELs, in 2015 the reauthorization of Title III of the Elementary and Secondary Education Act as the Every Student Succeeds Act (ESSA) required states to implement statewide standardized EL entry and exit procedures, starting in the 2017–18 school year. Standardized procedures were intended to facilitate a smoother transition of students into and out of EL status by reducing variability in EL entry and exit procedures within states. Prior to ESSA, a more decentralized decision-making process led to the inability to make fair comparisons of districts for accountability; difficulty developing coherent expectations and instructional practices for ELs (Ragan and Lesaux, 2006); inconsistent and possibly inequitable services for students who move across districts (Linquanti et al., 2016); and inconsistent decisions due to the use of subjective criteria applied by local educators (Estrada and Wang, 2018). At the same time, preserving flexibility for states to set their own entry and exit policies allows them to align the necessary level of English language proficiency with state academic standards.

Understanding the outcomes of ESSA’s shift to greater standardization in EL entrance and exit procedures is critical for federal policymakers. Policymakers need information on whether EL classification and reclassification procedures are more consistent across districts within states after ESSA, as intended. Without this, policymakers will lack an understanding of whether this policy lever has induced the conditions under which students should theoretically benefit.

Evidence is also needed on the extent to which post-ESSA procedures have helped students who exit EL status succeed in the general education curriculum. Since exited students are no longer entitled to language supports, reclassifying ELs too soon can leave these students linguistically unprepared for success in general education settings. However, maintaining EL status for too long can also lead to inefficient use of funds and lost opportunities for these students to access the general education curriculum.

Although the field has learned much about the impacts of reclassification (e.g., Hill et al., 2014; Johnson, 2019; Pope, 2016; Reyes & Hwang, 2021; Robinson, 2011; Robinson-Cimpian & Thompson, 2016; Umansky, 2016), previous work has rarely attempted to measure the degree to which local policies may moderate impacts of reclassification. In particular, although some research has examined the relative merits of dual language programs compared to English-only programs (Umansky & Dumont, 2021), policymakers lack a more general understanding of how the choice of a language instructional education program relates to whether students will experience a smooth transition out of EL status. In addition, how ELs are assigned to courses in secondary grades—starting with their experiences in middle school—may be a pivotal determinant of whether reclassification during middle school affects their future trajectory. Some research to date suggests such a mechanism for ELs (Callahan, 2005; Callahan & Shifrer, 2016; Umansky, 2016), but given the diversity of policies and procedures for course assignment nationally, policymakers lack a clear understanding of the prevalence of course assignment policies for ELs, or how these (and other local) policies moderate a smooth transition for students out of EL status.

Westat will collect survey data from districts and schools that will allow the Department to assess the prevalence of local policies and procedures that can affect outcomes for ELs and whether and how local implementation of entrance and exit procedures and other practices moderate the impacts of reclassification on student outcomes. The data that Westat has been collecting from 30 states’ SLDSs (OMB Approval # 1850-0974, summarized in section A.2.1) needs to be augmented by the data requested in this package to meet these goals.

A.2. Purpose and Use of Data

The survey data collection requested in this package (described in more detail in section A.2.2) will allow the study team to address the following research questions:

What instructional settings, programs, and services do districts and schools offer to ELs?
What is the relationship between these instructional settings, programs, and services and the impacts of reclassification on student outcomes?

The complementary research questions associated with the data being collected under the prior OMB approval #1850-0974 are:

What criteria do states have for classification and reclassification?
Are standardized statewide procedures, introduced to Title III under ESSA, associated with more consistent application of classification and reclassification procedures across districts within states?
Do standardized classification policies set by states have an impact on the instructional opportunities, experiences, and academic achievement and attainment outcomes of EL students compared to similar students not identified for EL status? How much do impacts vary across students with different characteristics, such as home language or enrollment in different language instruction educational programs?
Do standardized reclassification policies set by states have an impact on the instructional opportunities, experiences, and academic achievement and attainment outcomes of former EL students compared to students who remain ELs? How much do impacts vary across students with different characteristics, such as home language or initial proficiency in English?

A.2.1 Data Collection Activities for Which Clearance is Requested

District Survey. This web-based survey will focus on district-level policies, practices, and procedures that may affect ELs’ experiences after classification and reclassification. Specifically, the study will examine (1) procedures for initial classification of students as ELs, (2) language instruction for ELs, (3) selection or assignment processes for academic content courses in grades 6 to 12, (4) procedures for reclassifying students out of EL status, (5) monitoring of students after they leave EL status, (6) dual identification of students as an EL and as having a disability, and (7) dual language immersion programs. The survey will be administered to superintendents or their designees from a sample of 1,800 school districts in the 30 study states beginning in January 2025. Districts are expected to cooperate in Department evaluations following the Education Department General Administrative Regulations (EDGAR) (34 C.F.R. § 76.591). The district survey is in Appendix A.

School Survey. This web-based survey will be administered to principals of middle schools and focus on (1) language instruction for ELs, (2) selection or assignment processes for academic content courses in grades 6 to 8, (3) procedures for reclassifying students out of EL status, and (4) monitoring of students after they leave EL status. The voluntary survey will be administered to principals from a sample of 1,800 middle schools (nested within the sampled districts) beginning in January 2025. The school survey is in Appendix B.

A.2.2 Data Collection Approved Previously

The data collection approved under OMB #1850-0974 includes extant data from the following sources:

SLDS data. The study team is collecting student-level data on ELs, including their background characteristics (needed as covariates for regression analyses and for subgroup analyses), achievement data, program and enrollment data (to measure instructional opportunities, experiences, and attainment), and EL-specific data (to document classification and reclassification, as well as to provide covariates and subgroup analyses) from 30 states.
National Student Clearinghouse (NSC) data. For states where data on postsecondary outcomes (enrollment and persistence) are not directly available through SLDSs, the study team will obtain these data through NSC.
State policy data. The study team is collecting publicly available data from state education agencies about their classification and reclassification procedures.

A.3. Use of Technology to Reduce Burden

The data collection is designed to obtain information in an efficient way that minimizes respondent burden. The main method for doing so is the use of web-based surveys. Web-based surveys are preferred for several reasons:

This survey mode enables sharing a direct link to the surveys in communications with respondents. Respondents for all surveys will receive an invitation email and reminder emails as needed. In each correspondence, a URL to the survey will be shared, which allows respondents to click directly on the link to complete the survey.
Web-based surveys allow for the use of: (1) skip patterns, which reduce burden, and (2) built-in edit checks (e.g., ensuring percentages do not total to more than 100 percent), which reduce response errors and decrease the need for the study team to reach out to respondents to check the accuracy of responses.
Web-based surveys facilitate the completion of the survey by multiple respondents, which may occur at the district or school level. In addition, web-based surveys reduce the burden of sharing access to the survey within the district and allow the most appropriate individual to easily access each section and provide the data in their area of expertise. Each district will be assigned a single password for their agency, which can be shared with the most appropriate respondents within the district. More than one respondent can work on different sections of the web-based survey simultaneously.
Using a web-based survey for all respondents decreases the cost for postage, coding, keying, and cleaning of the data. This survey mode also allows respondents to complete the survey at a location and time of their choice.

A.4. Efforts to Identify and Avoid Duplication

The study team has examined district surveys from other studies to avoid duplication. The district and school surveys are designed to ask for information that are not readily available through extant documents, including information about classification procedures, language instruction educational programs, assignment processes to courses, reclassification procedures, monitoring policies, and dual identification of students as English learners and students with a disability.

A.5. Efforts to Minimize Burden on Small Business or Other Entities

No small businesses will be involved as respondents. Every effort will be made to minimize the burden on all respondents, whether they are from larger or smaller districts and schools. The study team will assist respondents by telephone and email.

A.6. Consequences of Not Collecting Data

Not collecting the data would leave policymakers at all levels poorly informed about (1) the consistency of classification and reclassification procedures at the district and school levels within states (a policy goal of ESSA); (2) the prevalence of policies and procedures that may affect students’ experiences while they are in EL status and after they exit; and (3) how district policies moderate the impact of reclassification on ELs’ instructional opportunities, experiences, achievement, and attainment under ESSA-era reclassification criteria.

A.7. Special Circumstances

There are no special circumstances involved with this data collection. The data collection will be conducted in a manner consistent with the guidelines in 5 CFR 1320.5.

A.8. Federal Register Announcement and Consultation

The 60-day notice to solicit public comments was published in the Federal Register on December 4, 2023 (88 FR 84136). There were no public comments received during the 60-day period. The 30-day Federal Register Notice will be published to solicit additional public comments.

The experts who formulated the study design and contributed to the content of this clearance request include Eric Isenberg (Westat), Molly Faulkner-Bond (WestEd), and Joseph Cimpian (New York University), and Eric Hedberg (Westat). In addition, the study team relies on a technical working group (TWG) of researchers and practitioners to provide input on the data collection plan for this evaluation and other methodological issues. The TWG includes researchers with expertise related to English learners, classification and reclassification policy, and evaluation methods. The study team will consult the TWG throughout the evaluation. TWG members are listed below.

Rebecca Callahan, Professor, College of Education and Social Services, The University of Vermont
John Deke, Senior Fellow, Mathematica
Anjelica Infante-Green, Commissioner, Rhode Island Department of Education
Madeline Mavrogordato, Associate Professor, College of Education, Michigan State University
Sean Reardon, Professor, Stanford Graduate School of Education
Nami Shin, Senior Research Associate, University of Kansas
Emily Tanner-Smith, Thomson Professor, College of Education, University of Oregon

A.9. Payments to Respondents

Payments are proposed for principals since their participation is voluntary and compensating study participants for their time and effort to complete the surveys will help the study achieve high response rates. High response rates will, in turn, allow the study team to address the study’s research questions more accurately.

Principals are the targets of numerous requests to complete data collection instruments on a wide variety of topics from state and district offices as well as independent researchers. The Department and several decades of survey research support the benefits of offering incentives to achieve high response rates (Dillman, 2020; American Statistical Association and American Association for Public Opinion Research, 2016; Jacob & Jacob; 2012). Accordingly, the study team proposes an incentive for the principal survey because high response rates are needed to ensure that the survey findings are reliable, and data from the surveys are essential to understand the role of local implementation in impacts of reclassification on EL students. This incentive payment acknowledges respondents’ time and effort in completing the survey. Specifically, the study team proposes to offer a $50 incentive to school leaders, if allowed by district policy, to principals who complete the 30-minute survey. Where applicable, respondents will be informed of this incentive payment with their invitation to complete the survey.

A.10. Assurance of Confidentiality

The study team is committed to maintaining the anonymity and security of the data collected for this study. All members of our study team have completed training on the use of human subjects in research. This training addresses the importance of the confidentiality assurances given to respondents and the sensitive nature of handling data. Our team also has worked with Westat’s Institutional Review Board to secure approval for this evaluation, thereby ensuring that the data collection complies with professional standards and government regulations designed to safeguard research participants.

The study team will conduct all data collection activities for this evaluation in accordance with relevant regulations and requirements. These include the Education Sciences Institute Reform Act of 2002, Title I, Part C, Section 183, which requires that the director of IES “develop and enforce standards designed to protect the confidentiality of persons in the collection, reporting, and publication of data.” The evaluation also will adhere to the requirements of Part D of Section 183, which prohibit disclosure of individually identifiable information, as well as make the publishing or inappropriate communication of individually identifiable information by employees or staff a felony. Finally, the evaluation will adhere to the requirements of Part E of Section 183, which require “[all] collection, maintenance, use, and wide dissemination of data by the Institute … to conform with the requirements of section 552 of Title 5, United States Code, the confidentiality standards of subsections (c) of this section, and sections 444 and 445 of the General Education Provisions Act (20 U.S.C. 1232 g, 1232h).”

Specifically, the names and email addresses of potential survey respondents will be collected for the limited purpose of drawing a sample, contacting those selected to complete the survey, and following up with nonrespondents. This information is typically already available in the public domain as directory information (i.e., district and school websites).

The following language will be included on the district survey under the Notice of Confidentiality:

“Information collected for this study comes under the confidentiality and data protection requirements of the Institute of Education Sciences (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183). Responses to this data collection will be used by the U.S. Department of Education, its contractors, and collaborating researchers only for statistical purposes. Reports will summarize findings across the sample and will not associate responses with specific districts or individuals. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). Districts receiving funds under Title III of the Elementary and Secondary Education Act (ESEA) are expected to cooperate with Department evaluations (Education Department General Administrative Regulations (EDGAR) (34 C.F.R. §76.591).”

The following language will be included on the school survey under the Notice of Confidentiality:

“Information collected for this study comes under the confidentiality and data protection requirements of the Institute of Education Sciences (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183). Responses to this data collection will be used by the U.S. Department of Education, its contractors, and collaborating researchers only for statistical purposes. Reports will summarize findings across the sample and will not associate responses with specific schools or individuals. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).”

After the evaluation concludes all hard copy documents, including surveys, containing confidential information that are no longer needed will be shredded. The study will produce and deliver restricted-use data files for IES containing information from all states that are able to contribute their data, excluding any direct personal identifiers. Researchers wishing to access the data for secondary analysis must apply for an NCES license and agree to the rules and procedures guiding use of restricted-use files.

A.11. Questions of a Sensitive Nature

There are no questions of a sensitive nature in the surveys.

A.12. Estimates of Respondent Burden

This request adds 3,600 respondents (responses are expected from an additional 3,150 respondents). Estimated time to complete the survey is 45 minutes for the district survey and 30 minutes for the school survey. Based on the study team’s experience on other similar studies and on the prior data collections under this study, the study team estimates a response rate of 90 percent for the district survey and 85 percent for the school survey. Total burden for the survey data collection is 118,800 minutes or 1,980 hours. Table A-1 presents details of the survey data collection as well as the data collection under the prior OMB approval #1850-0974. The annualized burden over 3 years is presented in Table A-2.

Table A-1. Estimates of respondent burden

Informant/ data collection activity	Sample size	Estimated response rate	Estimated number of respon-dents	Minutes per completion	Number of admini-strations	Burden in minutes	Total burden hours	Total costs
New Data Collection: District Superintendents and School Principals
District survey	1,800	90%	1,620	45	1	72,900	1,215	$64,747
School survey	1,800	85%	1,530	30	1	45,900	765	$40,767
Subtotal	3,600	n/a	3,150	75	1	118,800	1,980	$105,514
Previously Approved Data Collection: State Educational Agency
SLDS data extract	30	100%	30	1,200	4	144,000	2,400	$119,928
NSC data request	10	100%	10	300	1	3,000	50	$2,499
Subtotal	30*	n/a	30*	1,300	4	147,000	2,450	$122,427
Grand Total	3,630	n/a	3,180	1,375	n/a	265,800	4,430	$227,941

NOTE: The new data collection assumes a rate of $53.29 per hour for district and school educational administrators (derived from the Bureau of Labor Statistics’ Occupational Employment and Wages for educational administrators, May 2023. The mean hourly wage was computed assuming 2,080 hours per year). See: https://www.bls.gov/oes/current/oes119032.htm). The previously approved data collection assumed an hourly rate of $49.97 per hour for educational administrators (derived from the Bureau of Labor Statistics’ Occupational Employment and Wages for educational administrators, May 2021). See https://www.bls.gov/oes/current/oes119032.htm.

*From the 30 State Educational Agency respondents, there will be 40 total data collection activities.

Table A-2. Estimates of annual respondent burden over a 3-year period

Informant/data collection activity	Annual number of respondents	Total annual burden hours	Total annual costs
New Data Collection: District Superintendents and School Principals
District survey	540	405	$21,582
School survey	510	255	$13,589
Subtotal	1,050	660	$35,171
Previously Approved Data Collection: State Educational Agency
SLDS data extract	30	800	$39,976
NSC data request	10	17	$833
Subtotal	30*	817	$40,809
Grand Total	1,080	1,477	$75,980

NOTE: The new data collection assumes a rate of $53.29 per hour district and school educational administrators (derived from the Bureau of Labor Statistics’ Occupational Employment and Wages for educational administrators, May 2023. The mean hourly way was computed assuming 2,080 hours per year). See: https://www.bls.gov/oes/current/oes119032.htm). NOTE: The previously approved data collection assumes an hourly rate of $49.97 per hour for educational administrators (derived from the Bureau of Labor Statistics’ Occupational Employment and Wages for educational administrators, May 2021). See https://www.bls.gov/oes/current/oes119032.htm.

*From the 30 State Educational Agency respondents, there will be 40 total data collection activities.

A.13. Estimates of the Cost Burden to Respondents

There are no annualized capital/startup or ongoing operation and maintenance costs associated with collecting the information.

A.14. Estimates of Annualized Government Costs

The amount for the design; conduct of the district and school surveys; and analysis and reporting related to the data collection is $2,755,192. The annualized cost over 3 years for these activities is $918,397. Including the data collection under the prior OMB approval #1850-0974, the amount for all data collection activities is $7,655,192. The annualized cost over 3 years for all data collection activities is $2,551,731.

A.15. Changes in Hour Burden

This is a request to revise a previously approved collection of information. This revision results in an increase in annual number of responses of 1,050 and an increase in annual burden hours of 660.

A.16. Plans for Tabulation and Publication of Results

The study team will analyze the district and school survey data toSection A.2. The analyses will include a description of implementation of instructional settings, programs, and services, and assessments of factors that moderate the impacts of reclassification. The study team will summarize the results of these analyses in a series of reports.

Descriptive analyses. To understand the instructional settings, programs, and services districts and schools offer to students, the study team will use data from the district and school surveys along with public data on state criteria for classification and reclassification. By aligning data across three levels—states, districts, and schools—the study team will understand whether criteria are applied consistently within states, as specified in ESSA. The study team will also generate descriptive information on instructional settings, programs, and services that may affect students while they are ELs and during the monitoring period after they exit EL status. These analyses will rely on extensive descriptive analyses using the survey data. The study team anticipates using straightforward descriptive statistics (e.g., means, frequencies, and percentages) and simple statistical tests (e.g., tests for differences of means and proportions).

Moderator analyses. The study team will analyze the relationship between district- and school-level instructional settings, programs, and services and the impacts of reclassification on student outcomes through a moderator analysis that combines the survey data with SLDS data collected under OMB #1850-0974. The earlier package summarizes our analytic plans to estimate the impacts of reclassification. These methods will be used to produce the average impact of reclassification for groups of districts or schools that use one type of instructional setting, program, or service compared to groups of districts or schools that use another type of instructional setting, program, or service, with groups of districts identified based on survey responses. A statistically significant difference between the impacts of these two groups would suggest that a given instructional setting, program, or service moderates the impact of reclassification. The study team expects to publish reports in 2026 and 2027. All reporting will be available on the IES website. The reports will be short (15 pages or fewer), with a set of technical appendices and compendium of data tables. The reports will be written for an audience of policymakers and practitioners.

A.17. Display of Expiration Date for OMB Approval

The Institute of Education Sciences is not requesting a waiver for the display of the U.S. Office of Management and Budget (OMB) approval number and expiration date. The data requests to districts and schools will display the number and expiration date for OMB approval.

A.18. Exceptions to Certification Statement

This submission does not require an exception to the Certificate for Paperwork Reduction Act
(5 CFR 1320.9).

References

American Statistical Association and American Association for Public Opinion Research (2016). Joint American Statistical Association/AAPOR Statement on Use of Incentives in Survey Participation. https://www.amstat.org/asa/files/pdfs/pressreleases/2016-Joint_statement_survey_incentives.pdf.

Callahan, R. M. (2005). Tracking and high school English learners: Limiting opportunity to learn. American Educational Research Journal, 42(2), 305-328. https://doi.org/10.3102/00028312042002305

Callahan, R. M., & Shifrer, D. (2016). Equitable access for secondary English learner students: Course taking as evidence of EL program effectiveness. Educational Administration Quarterly, 52(3), 463-496. https://doi.org/10.1177/0013161X16648190

Dillman, D.A. (2020). Towards survey response rate theories that no longer pass each other like strangers in the night. In: Brenner, P.S. (eds) Understanding Survey Methodology. Frontiers in Sociology and Social Research, vol 4. Springer, Cham. https://doi.org/10.1007/978-3-030-47256-6_2.

Estrada, P., & Wang, H. (2018). Making English learner reclassification to fluent English proficient attainable or elusive: when meeting criteria is and is not enough. American Educational Research Journal, 55(2), 207-242. https://doi.org/10.3102/0002831217733543.

Hill, L.E., Betts, J.R., Chavez, B., Zau, A.C., & Bachofer, K.V. (2014). Pathways to fluency: Examining the link between language reclassification policies and student success. Public Policy Institute of California: San Francisco, CA.

Jacob, R.T., & Jacob, B. (2012). Prenotification, incentives, and survey modality: An experimental test of methods to increase survey response rates of school principals. Journal of Research on Educational Effectiveness, 5:401-418. https://www.tandfonline.com/doi/full/10.1080/19345747.2012.698375.

Johnson, A. (2019). The effects of English learner classification on high school graduation and college attendance. AERA Open, 5(2), 2332858419850801. https://doi.org/10.1177/2332858419850801.

Linquanti, R., Cook, H.G., Bailey, A.L., & MacDonald, R. (2016). Moving toward a more common definition of English learner: Collected guidance for states and multi-state assessment consortia. Washington DC: Council of Chief State School Officers.

Pope, N.G. (2016). The marginal effect of K-12 English Language Development programs: Evidence from Los Angeles schools. Economics of Education Review, 53, 311–328. https://doi.org/10.1016/j.econedurev.2016.04.009.

Ragan, A., & Lesaux, N. (2006). Federal, state, and district level English language learner program entry and exit requirements: Effects on the education of language minority learners. Education Policy Analysis Archives, 14, 20. https://doi.org/10.14507/epaa.v14n20.2006.

Reyes, M., & Hwang, N. (2021). Middle school language classification effects on high school achievement and behavioral outcomes. Educational Policy, 35(4), 590-620. https://doi.org/10.1177/0895904818823747.

Robinson, J.P. (2011). Evaluating criteria for English learner reclassification: A causal-effects approach using a binding-score regression discontinuity design with instrumental variables. Educational Evaluation and Policy Analysis, 33(3), 267-292. https://doi.org/10.3102/0162373711407912.

Robinson-Cimpian, J.P., & Thompson, K.D. (2016). The Effects of Changing Test-Based Policies for Reclassifying English Learners. Journal of Policy Analysis and Management, 35(2), 279-305. https://doi.org/10.1002/pam.21882.

Umansky, I.M. (2016). Leveled and exclusionary tracking: English learners’ access to academic content in middle school. American Educational Research Journal, 53(6), 1792-1833. https://doi.org/10.3102/0002831216675404.

Umansky, I. M., & Dumont, H. (2021). English learner labeling: How English learner classification in kindergarten shapes teacher perceptions of student skills and the moderating role of bilingual instructional settings. American Educational Research Journal, 58(5), 993-1031. https://doi.org/10.3102/0002831221997571

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Kim Standing
File Modified	0000-00-00
File Created	2024-11-16