Response to 2017-06-16 OMB TOC for EDFacts Memo

To: Office of Management and Budget (OMB)

From: National Center for Education Statistics (NCES)

Date: August 1, 2019

Subject: Response Memo to OMB’s 6/16/2017 Terms of Clearance for EDFacts Information Collection (1850-0925 v.2)

On June 16, 2017, OMB approved the EDFacts Data Collection School Years 2016-17, 2017-18, and 2018-19 request (1850-0925 v.2) stipulating the following terms of clearance:

“Congressional Review Act in early 2017, NCES did not adopt the definitions and guidance included in ESSA for the measurement of children in foster care, military-connectedness, and accountability indicators in the EDFacts collection.

The proposed collection is approved as submitted under the following terms of clearance, designed to inform OMB on key issues related to the utility of the information collected: within 24 months of the clearance date, NCES will submit to OMB a report describing:

1. differences in the measurement or definition of these concepts across reporting entities (including discussion of types of data submitted, number of levels for indicators, and variation between states);

2. a discussion of the utility of these measures given those differences, including challenges in combining data and making comparisons across states; and

3. a discussion of potential steps that ED can take to increase the utility and consistency of these measures.”

Clarification:

The opening statement above indicates that “NCES did not adopt the definitions and guidance in ESSA …”

However, in 1850-0925 v.2, NCES did adopt the language in the Every Student Succeeds Act (ESSA). What was not included was guidance proposed in documents that were withdrawn or in development. Since guidance is managed by USDOE data stewards, it would not be appropriate for NCES to include proposed definitions or guidance in the case that 1850-0925 is managed in another public input process.

Related Events:

In addition to the statute, ESSA guidance documents¹ offer information to stakeholders that range from requirements to resource (e.g., regulatory, non-regulatory, Dear Colleague letters, technical assistance etc.). In accordance with ESSA, each program office analyzed the law and determined where states have flexibility in policy decisions that impact data reporting requirements. As decisions were made, each grant program worked with their respective stakeholder groups to develop clarification and guidance. The EDFacts Data Collection School Years 2016-17, 2017-18, and 2018-19 request (1850-0925 v.2) reflected those program decisions.

Key decisions related to foster care, military connectedness, and accountability reporting requirements:

• December 2015 – Every Student Succeeds Act (ESSA) signed by President Obama; reauthorization of the Elementary and Secondary Education Act (ESEA)

• 2016 – USDOE developed ESSA regulations and non-regulatory guidance

• June 2016 - USDOE released non-regulatory guidance about students in foster care (attached with this memo in 2016-06-23 ED Foster Care Guidance.pdf)

• November 2016 – USDOE released Accountability and State Plan regulations

• March 2017 – Resolution of disapproval of ESSA Accountability and State Plan regulations

• January - May 2017 – EDFacts Data Collection School Years 2016-17, 2017-18, and 2018-19 request (1850-0925 v.2) revised to reflect current legal and policy decisions about foster care, military-connectedness, and accountability reporting

• June 16, 2017, OMB approved the EDFacts Data Collection School Years 2016-17, 2017-18, and 2018-19 request (1850-0925 v.2) stipulating the terms of clearance being addressed here

Roles:

Each EDFacts Information Collection package is developed using a data governance model for administrative data. The process includes two distinct roles:

• Technical role

  • The NCES, Elementary and Secondary Branch (ESB) is responsible for non-policy decisions related to the collection of Pre-K through 12 data. ESB uses a standardized process for identifying reporting requirements from program offices, managing the Information Technology contract used to collect data from grantees, overseeing the data security once data are collected, and ensuring USDOE stewards have access to the data once collected.

  • The technical role does not involve making, changing, or overriding policy decisions made by data stewards.

• Data steward role

  • Data stewards are data “owners.” Related to this memo, OESE is the steward of children in foster care, military-connectedness, and accountability data. Data stewards are responsible for determining reporting requirements, developing regulations and non-regulatory guidance (as needed), and using the data for monitoring grant implementation.

  • Data stewards look at data by grantee and by program. The data in EDFacts are collected with the primary purpose of meeting formula grant data reporting requirements.

Response to OMB about key issues related to utility of information collected about children in foster care, military-connectedness, and accountability indicators:

1. differences in the measurement or definition of these concepts across reporting entities (including discussion of types of data submitted, number of levels for indicators, and variation between states).

Types of data submitted

Accountability Data

There are five Data Groups collecting accountability indicator data (see Table 1).

Table 1. EDFacts Accountability Data (SY 2018-19)

Name	Definition	What schools must be reported in this file?
Graduation Rate Indicator Status	A school's performance on the graduation rate indicator.	States must report on all schools included within their accountability system. When reporting on the graduation rate indicator, states should only include high schools that have a 12th grade.
Academic Achievement Indicator Status	A school's performance on the academic achievement indicator for both mathematics and reading/language arts.	States must report on all schools included within their accountability system.
Other Academic Indicator Status	A school’s performance on the other academic indicator.	States must report on all schools included within their accountability system. When reporting on the other academic indicator, states should only include elementary schools and secondary schools that are not high schools.
School Quality or Student Success Indicator Status	A school's performance on the state-specific indicators of school quality or student success.	States must report on all schools included within their accountability system.
Progress Achieving English Language Proficiency Indicator Status	A school's performance on the progress in achieving English Language proficiency indicator.	States must report on all schools included within their accountability system that have English learners enrolled as of the reporting year.

These Data Groups collect a status for the entire education unit (school) as well as for other racial/ethnic and other populations with the school. In its first year of collection, states submitted a range of values, reflecting each state’s individual approach to collecting and determining values for accountability. The accountability indicators analysis in Table 2 is based on State submissions.

State submissions can be broken into the following types of indicators:

• Categorical/Levels – State submitted discrete levels/categories, e.g. Red/Yellow/Blue or 1/2/3, Good/Fair/Poor

• Numeric – State submitted score or aggregation of multiple indicators, usually represented as a percent value

• Composite – State submitted values within a single field for multiple indicators, e.g. a single field containing separate status values for Reading/Language Arts and Mathematics indicators

• NA – State submitted values of 'NA' (Not Applicable) or 'Not Identified'

Table 2. Types of Accountability Indicator Values Collected

Data Group	Categorical	Numeric	Composite	NA	Grand Total Reporting
Graduation rate indicator status (DG 834)	13	19	1	1	34
Academic achievement indicator status (DG 835)	14	15	4	1	34
Other academic indicator status (DG 836)	13	15	3	2	33
Progress achieving English language proficiency indicator status (DG 837)	14	17	1		32
School quality or student success indicator status (DG 838)	12	18	1	2	33

Foster Care and Military Connected Status Data

States report data about students in foster care in Adjusted-Cohort Graduation Rate (ACGR), Academic Assessment, and Assessment Participation files. States report data about military connected students in Academic Assessment and Assessment Participation files (see Table 3 below).

EDFacts provides guidance on how to report foster care students and military connected students in the file specifications²:

How are student counts reported by Homeless Enrolled Status, Foster Care Status, and Military Connected Status?

Under 34 CFR 200.2(b)(11), a State’s assessment system must enable result to be disaggregated within each State, LEA, and school by specific subgroups, including status as homeless child, status as a child in foster care, and status as a student with a parent who is a member of the armed forces on activity duty or serves on full-time National Guard duty. Consistent with these requirements, for purposes of submitting data to EDFacts, we encourage an SEA to use these same definitions, which are as follows:

• Children who are homeless. Status as a homeless child or youth is defined in accordance with section 725(2) of title VII, subtitle B of the McKinney-Vento Homeless Assistance Act, as amended;

• Children in foster care. ‘‘Foster care’’ means 24-hour substitute care for children placed away from their parents and for whom the agency under title IV–E of the Social Security Act has placement and care responsibility. This includes, but is not limited to, placements in foster family homes, foster homes of relatives, group homes, emergency shelters, residential facilities, childcare institutions, and preadoptive homes. A child is in foster care in accordance with this definition regardless of whether the foster care facility is licensed and payments are made by the State, tribal, or local agency for the care of the child, whether adoption subsidy payments are being made prior to the finalization of an adoption, or whether there is Federal matching of any payments that are made; and

• Students who are military connected. Status as a student with a parent who is a member of the armed forces on active duty or serves on full-time National Guard duty, where “armed forces,” “active duty,” and “full-time National Guard duty” have the same meanings given them in 10 U.S.C. 101(a)(4), 101(d)(1), and 101(d)(5).

Table 3. EDFacts Data with Foster Care and/or Military Connected Status (SY 2018-19)

Name	Definition	Status Reported
		Foster Care	Military Connected
Adjusted-Cohort Graduation Rate	The number of students who graduate (1) in four years or less with a regular high school diploma or (2) a State-defined alternate high school diploma for students with the most significant cognitive disabilities divided by the number of students who form the adjusted-cohort for the four-year adjusted-cohort graduation rate.	Yes	No
Adjusted-Cohort Graduation Rate	The number of students who graduate (1) in five years or less with a regular high school diploma or (2) a State-defined alternate high school diploma for students with the most significant cognitive disabilities divided by the number of students who form the adjusted-cohort for the five-year adjusted-cohort graduation rate.	Yes	No
Adjusted-Cohort Graduation Rate	The number of students who graduate (1) in six years or less with a regular high school diploma or (2) a State-defined alternate high school diploma for students with the most significant cognitive disabilities divided by the number of students who form the adjusted-cohort for the six-year adjusted-cohort graduation rate.	Yes	No
Cohorts for Adjusted-Cohort Graduation Rate	The number of students in the adjusted-cohort for the four-year adjusted-cohort graduation rate who did or did not graduate (1) in four years or less with a regular high school diploma or (2) a State-defined alternate high school diploma for students with the most significant cognitive disabilities.	Yes	No
Cohorts for Adjusted-Cohort Graduation Rate	The number of students in the adjusted cohort for the five-year adjusted-cohort graduation rate who did or did not graduate (1) in five years or less with a regular high school diploma or (2) a State-defined alternate high school diploma for students with the most significant cognitive disabilities.	Yes	No
Cohorts for Adjusted-Cohort Graduation Rate	The number of students in the adjusted cohort for the six-year adjusted-cohort graduation rate who did or did not graduate (1) in six years or less with a regular high school diploma or (2) a State-defined alternate high school diploma for students with the most significant cognitive disabilities.	Yes	No
Academic Achievement in Mathematics	The unduplicated number of students who completed the state assessment in mathematics and for whom a proficiency level was assigned.	Yes	Yes
Academic Achievement in Reading/ Language Arts	The unduplicated number of students who completed the state assessment in reading/language arts and for whom a proficiency level was assigned.	Yes	Yes
Academic Achievement in Science	The unduplicated number of students who completed the state assessment in science and for whom a proficiency level was assigned.	Yes	Yes
Assessment Participation in Mathematics	The unduplicated number of students who were enrolled during the period of the state assessment in mathematics.	Yes	Yes
Assessment Participation in Reading/ Language Arts	The unduplicated number of students who were enrolled during the period of the state assessment in reading/language arts.	Yes	Yes
Assessment Participation in Science	The unduplicated number of students who were enrolled during the period of the state assessment in science.	Yes	Yes

Variation between states

Regarding children in foster care and military-connectedness, ED has included a definition of foster care in the Department’s foster care guidance available at the following location (and attached with this memo in 2016-06-23 ED Foster Care Guidance.pdf): https://www2.ed.gov/policy/elsec/leg/essa/edhhsfostercarenonregulatorguide.pdf

Additionally, the relevant EDFacts file specifications include a definition for military-connectedness that the Department encourages States use. Despite the existence of these definitions, it should be noted that the States vary in their capacity to report these data. Especially, because these collections are in their early years. Quality and completeness of data vary but are expected to improve over time.

States had considerable flexibility in developing accountability indicators, including how they calculate the results of those indicators, consistent with statutory requirement. For example, States are using different measures for their School Quality or Student Success Indicators based on what they determined to be most meaningful for their State. States were required to report performance on each indicator for each school in accordance with their approved State plan. For more information about each State plan, see the State plan submission webpage available here.

2. a discussion of the utility of these measures given those differences, including challenges in combining data and making comparisons across states; and

Given the primary purpose of monitoring grant implementation, OESE is able to analyze data by looking within the state and reviewing data by program. While combining data can be challenging across data collection systems (i.e. EDFacts vs. Program Office collections), within EDFacts data are collected using a standardized reporting system. Therefore, EDFacts is able to create data extracts from the automated system allowing for combining data into easily interpretable data sets and reports. Furthermore, when making data comparisons across states, data limitations are known, documented, and reflected in the use of the data. Variability across states is only problematic if the data user does not understand the purpose and allowable variability in the reporting requirements. Within USDOE, comparisons across states are not problematic.

For publicly available data sets and file documentation, see The EDFacts Initiative webpage available here.³

3. a discussion of potential steps that ED can take to increase the utility and consistency of these measures

ED grant making offices are tasked with ensuring that each state grant program is aligned with statutory requirements. ESSA allows for state variation and that is what we see in the data reported. ED could mitigate poor use of the data by providing training sessions on ‘how to use ESSA data” to promote within state trend analyses and de-emphasize across state comparisons.

Furthermore, while the collection, analysis, and use of the data by ED is considered ‘primary data analysis,’ once published, the use of these data by the public and researchers are considered ‘secondary analysis of existing data.’ Therefore, data notes and documentation² that clearly support any known data anomalies and interpretations of the submitted data, within a state, and how they may impact the potential ‘secondary use’ of these data will increase the utility and consistency of these measures.

Moreover, the continued communication and transparency between ED, data submitters, and data users are a key factor in ensuring high quality reporting, analysis, and use of data. All publicly available ED documentation is reviewed by ESB and Data Stewards on an annual basis and updated to reflect changing guidance provided by ED data stewards. Ultimately, while it is the data user’s responsibility to understand the purpose of the data collection and to reflect that purpose in their analysis, ED continues to provide the necessary support and documentation, through the EDFacts Initiatives webpage (available here) and Partner Support Center (PSC), to try to ensure increased utility and consistency of data collections.

1 ESSA related resources and documents are provided at: https://www2.ed.gov/policy/elsec/leg/essa/index.html.

2 From the EDFacts SY 2017-18 FS 175 - Academic Achievement in Mathematics File Specifications, Retrieved on July 16, 2019. https://www2.ed.gov/about/inits/ed/edfacts/sy-17-18-nonxml.html.

3 Fostercare and military connected data are new for SY2017-18. These data and file documentation will be made publicly available once Data Quality review with the ED data stewards is completed.

Page 5 of 5