Part A SLDS Survey 2023-2025 v.11 30D

The National Center for Education Statistics (NCES), of the Institute of Education Sciences (IES), within the U.S. Department of Education, is requesting clearance to continue the Statewide Longitudinal Data System (SLDS) Survey collection, which is intended to provide insight on State and U.S. territory SLDS capacity for automated linking of K-12, teacher, postsecondary, workforce, career and technical education (CTE), adult education, and early childhood data. Historically, SLDS has collected information annually from State Education Agencies (SEAs) and has helped inform NCES ongoing evaluation and targeted technical assistance efforts to enhance the quality of the SLDS Program’s support to States regarding systems development, enhancement, and use. The request to conduct all activities related to SLDS 2021-2023, including materials and procedures, was approved by OMB in October 2021 (OMB#1859-0933 v.10),

This new request is to conduct all activities related to SLDS 2023-25, continuing usage of the Qualtrics information collection tool initiated in the 2023 collection. The appendices include updated communications, webinars, and Qualtrics instrument screenshots related to the SLDS 2023-25 collection. The primary change proposed in this package is a shift from an annual to a biennial collection. Minor adjustments were made to questions and language, each requested change captured in Appendix E, Section 2, with accompanying screenshots of the intended sample Qualtrics instrument (Appendix E, Section 1).

Legislative Authorization

NCES is authorized to collect this information by the Education Sciences Reform Act of 2002 (ESRA 2002; 20 U.S.C., § 9543), which establishes the SLDS Grant Program.

Prior and Related Studies

The SLDS Survey is the first formal, systematic collection of SLDS system capacity data of its kind. Aspects of these data have been obtained less systematically during regular interactions with SLDS grantees. This effort provides better information to meet NCES’ quarterly progress reporting regarding State capacity to link and use data, to inform future grant rounds, technical assistance efforts, and public knowledge of State capacity to link and use data.

Prior to the development and administration of the SLDS Survey, States provided some indication of data linkages and use to NCES program staff through annual reporting, monthly monitoring updates, and State site visits, these data do not provide a comprehensive look at data capacity. Reasons for incomplete data include, but are not limited to:

Data are collected from States that have active grants, which results in missing data for non-grantee States; and
Grantee States report primarily on their proposed – and funded – projects. For the last three collections, the SLDS Survey has allowed States to discuss the full capacity of their State data systems, which has led to more accurate information and discussion about system capacity across the field.

Prior to the development of the OMB-approved SLDS Survey, the SLDS program office attempted to collect more uniform information about data linkages (for example, asking about early learning program data linkages), but these efforts have been limited to States with active grants.

External organizations, including the Data Quality Campaign (DQC), have conducted surveys to document data linkage and use capacity at the State level. The DQC data are limited by three factors:

DQC stopped collecting information about data linkages in 2011, with no replacement data source in place.
The DQC survey relied on fairly dichotomous measures of data linking (where a yes response indicated that a State had a link in place, and a no response indicated that a State did not have a link in place). States, however, tend to implement linkages more gradually. For example, a State might conduct a pilot in which Pre-K and K-12 data are linked for one Local Education Agency (LEA), or might link data from a limited set of Pre-K sources, such as Head Start or Early Head Start. The proposed NCES measure allows for States to report on the continuum of data linkage and capacity. For example, the proposed SLDS Survey enables States to rate their own data linkage and use efforts as “Not planned,” “Planned,” “In Progress,” or “Operational.” A State that has established a pilot data linkage process would be deemed “In Progress,” while a universal roll-out would be considered “Operational.” This has proved more effective when attempting to viably gauge a State’s capacity status.
The DQC survey produced less public information about how States were matching data (for example, through a manual process or an automated process) and who was matching the data (for example, a State agency or a vendor such as NSC). Such information has proven to be quite useful when assessing States’ needs and capacity for data linkages.

2. Purposes and Uses of the SLDS Survey

Since 2005, the U.S. Department of Education has awarded approximately $826 million in 142 grants to State Education Agencies to enable them to implement and enhance their SLDS systems. Additional competitive funding was expended prior to the close of FY20 (awarded as FY19 competition and funds), and an additional grant opportunity in FY23¹. The Department continues to need to maintain a clear and formal means of summarizing and communicating the status of these systems across all States and Territories to: 1) evaluate current but everchanging needs for further systems development; 2) provide targeted technical assistance to States; and 3) accurately reflect progress on the development and use of statewide longitudinal data systems.

Survey results will continue to inform:

Future grant rounds for the SLDS grant program and technical assistance support;
Program offices in the Department of Education, Department of Labor, and Health and Human Services, in addition to external stakeholders;
State development and support efforts; and
Public knowledge of State capacity to link and use longitudinal data.

State information about State capacity for data linkages and use is vital to ensure that program dollars are targeted both for grant funding and for technical assistance development. As federal funding becomes increasingly more competitive (with SLDS infrastructure a reemerging priority), we must continue to have a clear sense of SLDS progress across the United States so that federal resources can be utilized and offered most efficiently and effectively. Prior to the development of the SLDS Survey, the SLDS grant program was responsible for providing OMB with up-to-date state capacity indicators on a quarterly basis, with the shortcoming that any changes or updates to these data primarily reflect information from active grantee states only. The report was produced based on continual communication with active grantees that allows the SLDS Program Officers to remain informed of these states’ systems’ capacity, progress, and constraints. Moreover, active grantees are responsible for providing summary reports on at least an annual basis, and this reporting validated assumptions and conversations that took place throughout the year between grantee states and SLDS Program Officers. Reporting for states without active grants had been only ad hoc. The SLDS Survey has now formalized the data collection processes to respond to these indicators and made it easier for the SLDS program to reach States without active grants.

There continues to be growing interest in SLDS capacity across the United States both internally within the Department of Education, among States and U.S. territories, and across agencies with common and shared interests (Department of Labor’s Workforce Data Quality Initiative, for example). The SLDS program regularly responds to ad-hoc questions and requests regarding State capacity for data linking and use, including, for example:

How many States can link:
- teacher preparation programs of teachers to student outcomes for students taught by those teachers (Title II);
- K12 and postsecondary data (Performance metric, OPEPD);
- K12, postsecondary, and workforce data (Performance metric, OPEPD, Department of Labor, Workforce Data Quality Campaign, White House Workforce Convening);
- K12 and early learning data (Performance metric, Early Learning Challenge Technical Assistance, Office of Special Education Programs, US Department of Health and Human Services); and
- How are States using data (Performance metric, US Department of Labor).

States and Territories themselves often seek information about which States are linking and using data, and what their processes entail. The SLDS Program facilitates States’ efforts to share promising practices with each other. This enables States to more easily collaborate, learn from each other, share resources with each other, and avoid duplicative work in the presence or absence of SLDS federal funds.

The SLDS program also receives questions about State capacity from the public, which is interested in learning which data are available at the State level and how the data might be accessed. A Statistics in Brief is released post-analysis of each SLDS Survey collection, and includes descriptive statistics, metrics and use cases showing data-linking and data-use capacity by State. Beginning with the 2018 data Statistics in Brief, interested users will be able to quickly ascertain which States have the capacity to link data across sectors (for example, which States and Territories can link K12, postsecondary, and workforce data) through the accessibility of an accompanying data file. The Statistics in Briefs also include examples of State data use capacity, for example, which States are providing feedback reports so that policymakers at the local level understand how their high school graduates are faring in postsecondary education or the workforce.

While the SLDS program has now developed and codified a recurring process for the first IES-approved Statistics in Brief for two collections, formalized IES Review greatly impacts the timelines of release. The request to move to a biennial collection moving forward allows the opportunity to place more resources into timely releases of accompanying Statistic in Brief and data files. As a result of the review delays, 2019 and 2020 Survey collections have been truncated into one Statistics in Brief detailing findings from both collections, with two data files representative of each annual collection. NCES is planning to take the same approach for the 2021 and 2022 SLDS Survey collections, as there continue to be challenges moving through the IES Review process, though data has been collected and is available. This will also allow for more in depth and longitudinal analysis of collected annual data throughout non-collection year(s).

3. Appropriate Use of Information Technology

The link to the Qualtrics information collection tool will be distributed to SEA contacts electronically, as an email, by a Program Analyst. The Program Analyst will act as the primary contact for all questions regarding the Survey, and consult NCES Program staff as needed. All states continue to have a Program Officer and Analyst contact despite their grant status (including states that do not have active grants).

A survey tool that is built with the capability and functionality to prepopulate prior year’s information and format the SLDS Survey in a clean, neat presentation has made it easier for states and territories to access, navigate, and complete the Survey. Qualtrics allows greater consistency with a more robust pre-population option and built-in data quality checks. This tool has supported more efficient analysis and reporting from the Survey, as opposed to spending such significant time on formatting, as with the 2020 Survey Google Form administration. Some additional key challenges that the Google Form presented that Qualtrics has ultimately alleviated are as follows:

Potential impacts to data quality
Limited ability to format the Survey
State Education Agencies’ inability to access the Google Form links
Additional burden on SEAs through multiple rounds of QA follow up (multiple contacts that wouldn’t have otherwise needed to occur)
Additional follow up when there were duplicate or incomplete submissions
The need for resubmissions (and QC to follow to ensure the correct submission was used)

While the SLDS Survey was originally developed using Microsoft Word, this has since evolved to a tool that intends to ease completion efforts and ultimately reduce respondent burden. The tool has a built-in skip pattern, text boxes, and formatting restrictions that cannot be manipulated.

The information collection tool has facilitated the data collection process, increased the reliability of the data, and reduced error. The Qualtrics information collection tool data can also be downloaded to a .csv or Excel file to ease the analysis process and is stored on a secure NCES server. NCES Program staff manages all data and conduct analysis in collaboration with Applied Engineering Management Corporation technical and research staff, made available via the existing Technical Assistance contract (EDTAP contract number GS00F083GA). The success of the Qualtrics tool has meant that we have received no requests for paper instruments in the last two years, and so as of this submission, the paper instrument for the SLDS has been discontinued.

Boasting a 100% response rate for the last two years, the SLDS community is quite familiar with the Survey and anticipate its dissemination at this point, but continually note the level of effort and collaboration required to complete the Survey. Shifting to a biennial collection should ultimately ease the level of burden on respondents. NCES plans to continue to host one to two webinars in 2023 to reacclimate and/or introduce the platform to States and their respective respondents, should they have experienced any turnover. Additionally, with a likely grant round in Summer 2023, it will be an ideal opportunity to notify State and Territory contacts of Survey timeline changes to an already-engaged community.

Respondents will be invited to participate in webinars via email invitation, using an abridged listserv of SLDS contacts. The listserv is used regularly to communicate with the SLDS community. Webinars will be used as a tool to continue to reiterate the purpose of the OMB-formalized survey, how to navigate the collection tool, and in what manner the data will be used by NCES. In subsequent years, NCES may host one or two webinars per year on an as- needed basis, to answer any questions states may have regarding the Survey.

4. Efforts to Identify Duplication

The information collected through the SLDS Survey does not duplicate information requested or collected by any other federal agency. Further, there is no similar current information available on a consistent national basis that could be used or modified for these purposes. Program offices within the US Department of Education often request and report on similar data. Having a single source of information will decrease redundant data collections and improve ED’s ability to provide valid and reliable data for internal and external users.

In the past, a similar but not equivalent survey has been conducted on an annual basis by DQC, a nonprofit organization participating in national effort to bring quality information to education stakeholders. Between 2005 and 2011, DQC surveyed States in an attempt to report their progress towards the building of longitudinal data systems and implementing effective data use. In 2009, DQC launched the 10 State Actions to Ensure Effective Data Use, which document States’ capacity to use the data in their systems.

While many of the questions that DQC has asked States to report on in the past parallel those set forth in the proposed SLDS Survey, the transition from data linking to data use in 2011 resulted in losing information about fundamental SLDS capacities. Because DQC has taken a new direction, States are no longer asked to report on the types of questions that can assist us in assessing SLDS progress to-date. It is crucial that these data continue to be collected at the national level to guide future efforts in SLDS development and to provide information about State capacity to link and use education data. NCES has used the DQC survey data responses in the past as a resource to help understand changes in State capacity since 2011, and evaluate State SLDS development and data use progress. More than a decade later, this resource is now outdated and the SLDS Survey has proven to be a viable solution to better understanding both capacity and use in an ever-evolving field.

5. Minimize Burden

NCES has devised several measures to minimize the response burden for States and Territories participating in the SLDS Survey. On a continual basis, questions are reviewed by the federal SLDS Program Team and the State Support Team (SST) members, a panel of experts who support the Program by offering technical assistance to States. This process led to the changes proposed within this renewal request. These proposed changes can be found in Appendix E. All SST members have held leadership positions in their respective SEAs. As a result, they are generally aware of the level of burden that the Survey is likely to impose. Each contributor took this into account when providing input, in an attempt to consider conflicting SEA responsibilities and demands and to minimize burden. As with the decision to pre-populate the instrument, SST were advocates for enhancements to the information collection tool, also to reduce the burden on States. Additionally, NCES may offer webinars to provide more information to respondents about the Survey, how to complete the instrument online, and NCES’s planned uses for the data. The SLDS Program Team is consistent in proactive efforts aimed to minimize respondent burden over the long-term.

6. Frequency of Data Collection

The SLDS Survey plans to shift to a biennial survey, scheduled to begin November 2023, and will be administered once again in November 2025 under this request. Nationwide, SLDS system capacity changes frequently (ex. Infrastructure enhancements, evolving P20W agency collaborations, State legislation impacts, etc.), but the analysis demonstrates that the COVID-19 pandemic stagnated the work to some extent. The 2019-20 Statistics in Brief and accompanying data file (anticipated May 2023 publication release) indicate very little change in results over the two-year period. Throughout the collection process, respondents were strongly urged to revisit each question response, despite pre-population. Therefore, analysis and subsequent Statistics in Brief publications indicate that shifting to an every-other-year collection would allow for more timely releases of data, with no adverse effect on the integrity of the information. Collecting data any less often than biennially would make the information too obsolete to be useful for targeted technical assistance planning.

7. Special Circumstances of Data Collection

There are no additional circumstances that will require special data collection efforts.

8. Consultants Inside and Outside the Agency

Upon development of the SLDS in 2016, OMB requested that the survey instrument be reviewed by a methodological expert within NCES. SLDS staff asked Dr. Andy Zukerberg, at NCES, to review the instrument. Dr. Zukerberg suggested revising the skip pattern, providing further (but concise) definition of key concepts mentioned, considering shortening the survey in length, and piloting it with a few SEAs. Per recommendations, the skip pattern was revised and enhancements were made to concept definitions and Survey instructions prior to piloting.

The SLDS survey was initially piloted with the Kentucky, Minnesota, and Washington State Project Teams. Each participating SEA was given approximately two weeks to complete the Survey with notification that completion might require collaboration from other SLDS stakeholders, outside of the immediate project team. Once completed, a debrief teleconference was held to discuss possible improvements, suggestions, and other feedback. In general, pilot participants indicated that they preferred the SLDS Survey over the leading, external survey designed to measure State’s progress towards SLDS development and implementation.

State pilot participants were satisfied with the length of the SLDS survey, stating that while it is somewhat extensive, it is comprehensive in assessing the current state and robustness of SLDS and P20W capacity. The length of the Survey has received recurring feedback as enhancements have been made to add data use questions and bring gradual cohesiveness across sectors, but there seems to be a clear understanding of the need and value across the field. In order to prepare for the renewal of the SLDS Survey, SST convened in many intensive sessions to review the Survey, address inconsistencies, provide clearer definitions and enhance cohesiveness in questions across sectors.

In general, as we enter the third year using the Qualtrics tool, the impact and feasibility of use has been viewed favorably.

In addition to the internal NCES review and SEA piloting, the following individuals from the SLDS State Support Team reviewed the data collection content and plans for the initial Survey release:

from Applied Engineering Management Corporation: Carla Hower (SST Lead), Kathy Gosa, Bill Hurtwitch, Charlotte Ellis, Jeff Watson, Kathi Gillaspy, Bill Huennekens, and Jeff Sellers;
from Chatis Consulting: Corey Chatis;
from Privacy Technical Assistance Center (PTAC) via Applied Engineering Management Corporation: Levette Williams.

Since the inception of the SLDS Survey, feedback and suggestions have been solicited periodically and welcomed on an ongoing basis through the following measures:

Point of contact provided on the Survey instrument,
Point of contact provided on SLDS website once,
SST,
Conferences, and
Opportunity for discussion during monthly SLDS teleconference calls.

9. Provision of payments or Gifts to Respondents

No payments or gifts will be offered to Survey respondents.

10. Assurance of Confidentiality

Data collected through the SLDS Survey are public domain data in their respective districts and States. As such, the data collection does not include a pledge of confidentiality. Confidentiality was offered the first collection year, as no data file was released with the Statistics in Brief, but NCES plans to release an accompany data file moving forward, initiated by the Statistics in Brief associated with the 2018 SLDS Survey data collection.

11. Sensitive Questions

None of the questions asked during the SLDS Survey are of a sensitive nature.

12. Estimates of Hour Burden for Information Collection

The response burden will vary by State and U.S. territory, with the expectation that on average it will take 2 hours for each SEA to complete the SLDS Survey. Although the expectation is for the Program Director or past Program Director to complete the Survey on behalf of the State or U.S. territory, staff turnover and/or level of knowledge and expertise varies by State. For example, respondents from a State that has focused on building a K12 SLDS (as opposed to a P20W) might possess limited (if any) knowledge on workforce and postsecondary system capabilities due to the fact that the workforce system might not be housed in their agency. In such cases, cross-agency communication and collaboration may be required to effectively and successfully complete the SLDS Survey. By contrast, a State or U.S. territory with a tenured respondent involved in the implementation of a P20W might be capable to complete the survey independently, with greater ease. The proposed changes found in this request are intended to bring greater consistency and definition to Survey respondents, ultimately intended to reduce burden. Proposed changes are captured in Appendix E, Section 2.

NCES may host one or two webinars associated with each collection, on an as-needed basis, to provide more information to respondents about the Survey, and to answer their questions. The webinar slides have been updated to remain compliant with IES-approved branding, and address any potential changes to screenshots as a result of the updated instrument. Screenshots can be found in Appendix E, Section 1.

The estimated hours per respondent are based on information directly provided by past state Program Directors who have completed comparable information requests in previous years, as well as field response. This remains at two hours for the purposes of this renewal request. Assuming that the respondents (state education agency administrators) earn on average $55.39^²per hour, the total annualized burden time cost to respondents for the SLDS Survey is estimated to be $5169.73. The decrease from the previous package is the result of annualizing the burden time cost of two collections (in 2023 and 2025) over the three-year period of this clearance.

Table 1. Annual burden estimates for SLDS 2023-2025

Respondent	Number of Respondents	Number of Responses	Estimated hours per respondent	Total Respondent Burden Hours
States / U.S. Territories	56	56	2	112
Webinars	56	56	0.5	28
Total for FY23	56	112	-	140
Total for FY25	56	112	-	140
Annualized Total	38	75		94

13. Estimates of Costs

SLDS Survey respondents will not incur any costs for this data collection other than their time to respond.

14. Annualized Cost to the Federal Government

Method for estimating costs: The full costs include the projected annual amount of time that Department of Education staff will spend on the Survey, separated by labor type. Contracted staff is intended to support pre-collection, collection and analysis, with federal guidance and oversight. Salary costs are based on average FTE salaries and hourly rates for contract subject matter experts and program analysts.

Table 2. Estimated Annual Cost of the SLDS Survey to Federal Government for Fiscal Year 2023

Labor Type	Annual Labor Hours	Cost
Collection and Data Entry	386	$23,259.71
Data Analysis	540	$54,307.69
Product Development and Publishing	375	$31,944.71
Total	1301	$109,512.11

Cost: Department of Education staff assigned to SLDS Survey include one-third (FTE) of one Program Officer’s (GS14) time. Contracted staff also supporting the SLDS Survey (included in SLDS Contract total) consists of 20 percent of one SST and about 10 percent of one full-time Program Analysts employed through Applied Engineering Management Corporation.

15. Reasons for Changes in Response Burden and Costs

Because the SLDS is moving to a biennial rather than annual collection, the annualized time burden will decrease.

Time Schedule for SLDS Survey

The SLDS Survey is a biennial collection. The proposed schedule is for the FY23 survey administration. The FY25 collection is scheduled as the subsequent and final cycle of this OMB approval and will follow the appropriate 2025 version of the timeline detailed in Table 3 below.

Table 3. Schedule and Timeline

2023 Timeline	SLDS Survey Collection, Processing, and Publication
November 2023	Email instructions to SEA respondents
November 2023	One or two webinars, on an as-needed basis, to provide more information about the Survey, how to complete the instrument on the new online platform (Qualtrics), NCES’s planned use of the data, and to address respondents’ questions about the Survey
November 15–30, 2023	Survey final reminder email
November 30, 2023	SEAs are urged to have finished submitting accurate and complete data
December 15, 2023	Mandatory final submission date
January 2024	Response by SEA’s to requests for clarification, reconciliation, or other inquiries from NCES. All data issues to be resolved. No files are accepted after close-out.
March 15, 2024	NCES review of files, file documentation, and brief analysis completed. Provisional responses available for internal use but not publication
July 15, 2024	Statistics in Brief and accompanying data file become public, NCES website updated. Current year collection data will be available to assess and respond to ad hoc requests
July 15 – October 2024	Respondents have the option to make update, change, or reconciliation requests to adjust state-specific data reflected in the SLDS Survey Statistics in Brief

NCES will release an IES-approved Statistics in Brief (and accompanying data file) associated with all SLDS Survey collections. Interested users and stakeholders will be enabled to quickly ascertain which States have capacity to link data across sectors, for example, which can link K12, postsecondary, and workforce data, and how they are using these data to inform policy and practice. Upon IES approval, the Statistics in Brief and data file is published to the SLDS website. The SLDS grant program is continues to be responsible for providing updated indicators to OMB on a quarterly basis, and though the data will now be collected less frequently, it continues to serve as a viable starting point to ascertaining this information for reporting. As data needs evolve and staff capacity permits, the intention is to post more data publicly. A biennial collection will allow the reappropriation of resources to ultimately make this possible. The data collected from the SLDS Survey will also be used to respond to questions from internal and external stakeholders regarding SLDS capacity in the States, and to inform future grant rounds and technical assistance planning.

17. Approval to not Display Expiration Date for OMB Approval

No approval is sought to not display the expiration date of OMB approval.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification for Paperwork Reduction Act submission.

1 https://www.federalregister.gov/documents/2023/05/05/2023-09611/applications-for-new-awards-statewide-longitudinal-data-systems

2The hourly mean wage for financial managers (SOC code 113031) working in State government (OEWS 999201) is $55.39 per the May 2022 National Occupational and Employment Wage Estimates sponsored by the Bureau of Labor Statistics (BLS). Source: BLS Occupation Employment Statistics, https://stats.bls.gov/oes/current/999201.htm Accessed on May 16, 2023.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Carrie Clarady
File Modified	0000-00-00
File Created	2023-08-31

Part A SLDS Survey 2023-2025 v.11 30D