Response to 60D Comments

Public Comments Received During the 60-day Comment Period

January 2023

Early Childhood Longitudinal Study, Kindergarten Class of 2023-24 (ECLS-K:2024)
Kindergarten & First-Grade National Data Collection and Transfer School Recruitment

ED-2022-SCC-0138 Comments on FR Doc # 2022-24003

Document: ED-2022-SCC-0138-0004

Name: Danielle Gilmore

Dear Ms. Valentine,

I am pleased to respond to the Federal Register notice of November 4, 2022, concerning the Department of Education’s (Department) request for comments on the information collection request (ICR) for the Early Childhood Longitudinal Study, Kindergarten Class of 2023-24 (ECLS-K:2024) regarding issue (4), how might the Department enhance the quality, utility, and clarity of the information to be collected. As a Ph.D. candidate at George Washington University and a graduate research assistant for the George Washington Institute of Public Policy, I focus on utilizing the lived experiences of educators to inform and enhance kindergarten through 12th grade (K12) education. From this perspective, I find that NTPS 2023-24 is highly important to the nation’s education because data inform education policy and research. Given the high importance, I utilize my expertise to critically examine the ICR so that the Department may enhance the quality, utility, and clarity of the information collected.

The ECLS program studies deliver national data on children's status at birth and at various points thereafter; children's transitions to nonparental care, early care and education programs, and school; and children's experiences and growth through the elementary grades. The ECLS program includes four longitudinal studies that examine child development, school readiness, and early school experiences. The ECLS-K:2024 will provide data about the population of children who will be kindergartners in the 2023-24 school year, focusing on children's early school experiences continuing through the fifth grade. Parents, educators, and policymakers can use information collected through the ECLS program to improve students' educational experiences. The program also provides data to analyze the relationships among a wide range of family, school, community, and individual factors with children's development, early learning, and performance in school. Education researchers and policymakers need high-quality representative data to inform future education policies that serve an equally diverse nation. Additionally, the longitudinal components provide vital information about educational experiences over time. ECLS is the only national survey that provides this kind of information. The data they provide will better inform and help to shape future education policies.

I believe it is important that the ECLS-K:2024 utilize high-validity instruments to ensure a more precise measurement of the various topics of interest included in the questionnaires. Consequently, I ask the Department to consider my comments included on the following page to ensure such validation. I present considerations for the Department to improve data quality. I appreciate the opportunity to comment on the Early Childhood Longitudinal Study, Kindergarten Class of 2023-24 (ECLS-K:2024), I hope that my suggestion is useful, and look forward to seeing the Department of Education’s survey design decisions. I am available to speak about my suggestions if needed.

Sincerely,

Danielle R. Gilmore

Ph.D. Candidate, Program Evaluation

Graduate Research Assistant

ECLS-K:2024 Survey Commentary

Parent Survey

• The ECLS-K parent survey includes constructs to measure parental income and assets. However, one asset, home ownership, is not included. It is likely income and assets vary by race, age, location, and other demographic variables. By including a question on housing arrangements, ECLS-K may be able to identify trends within subpopulations.

o Suggested Language: Which of the following best describes your housing arrangement?

o I own my home

o I rent my home

o I live with relatives

o I have other arrangements. (Please Specify)

• The ECLS-K parent survey has several constructs related to the research question P-RQ3: How do schools and teachers involve parents in their children’s education, or how do parents involve themselves in the schooling process? While ECLS-K does include questions about barriers to parental involvement, however, there is one area still missing: the use/availability of translators for parent-teacher conferences.

o Suggested Language: My child’s school offers translators when attending parent-teacher conferences.

o Yes, but I do not use the service

o Yes, and I use the service

o No, services are not available

o I do not know

• Please consider adding definitions for time-based questions. For example, the parent fall kindergarten survey has parents report on their child’s approaches to learning. The options include: never, sometimes, often, and very often. These categories are vague and highly subjected which can affect data validity. Example definitions could include: never or rarely (0-1 times per month), sometimes (2-4 times per month), often (5-7 per month), and very often (8+ times per month).

Teacher Survey

• Consider adding specificity to time-based questions in the parental survey rather than never, often, sometimes, rarely, always, etc.

• Add feeling like getting paid enough to teacher job satisfaction

• Teachers witnessing peer victimization in school climate

School Administrator Survey

• The ECLS-K school questionnaire includes a measure for “average daily attendance.” Studies have shown that average daily attendance rates can mask chronic absenteeism within schools (Attendance Works, 2016; Attendance Works & Everyone Graduates Center, 2017; Bai et al., 2018; Balfanz et al., 2016; Buckingham et al., 2013; Chang & Works, 2013). Daily attendance rates do not provide the full picture of attendance because schools can have 90% daily occupancy and still have a quarter of students chronically absent since different students miss on different days.

References:

1. Attendance Works. (2016). Whats the difference between chronic absence and truancy. In.

2. Attendance Works, & Everyone Graduates Center. (2017). Portraits of change: Aligning school and community resources to reduce chronic absence. https://www.attendanceworks.org/wp-content/uploads/2017/09/Attendance-Works-Portraits-of-Change-Main-Document-Final-Sept.-1.pdf

3. Bai, X., Ola, A., & Akkaladevi, S. (2018). Examination of the Relationship Between Class Attendance and Student Performance. Issues in Information Systems, 19(3).

4. Balfanz, R., Chang, H., & Byrnes, V. (2016). Preventing missed opportunity: Taking collective action to confront chronic absence. https://attendanceworks.org/wp-content/uploads/2017/09/PreventingMissedOpportunityFull_FINAL9.8.16_2.pdf

5. Buckingham, J., Wheldall, K., & Beaman-Wheldall, R. (2013). Why poor children are more likely to become poor readers: The school years. Australian Journal of Education, 57(3), 190-213.

6. Chang, H., & Works, A. (2013). Reducing chronic absence: What will it take. San Francisco: The.

Dear Ms. Gilmore,

Thank you for your interest in the Early Childhood Longitudinal Study (ECLS) Program and the upcoming Early Childhood Longitudinal Study, Kindergarten Class of 2023-24 (ECLS-K:2024).

To accomplish this, during item design and selection we consult with experts (both internal to the ECLS team in the government and our contractors, and also external experts brought in for the study’s Technical and Content Review Panels (TRPs and CRPs)). In addition, we select many of our items from existing tested and validated sources, as well as test items ourselves, most recently in the fall 2022 field test for the ECLS-K:2024 kindergarten and first-grade rounds. I appreciate your comments about suggested changes to the items that could help with item validity and reliability, and will share them with the item development and selection team; they will be considered in conjunction with our TRP and CRP members’ feedback and the data from our field test as item revisions are considered.

Thank you also for your item-level suggestions. Our study instruments are limited in their length and we often have to make tough decisions to meet the timing limits of the instruments. When we are unable to include particular items in a given round, we sometimes are able to include those items in a later round. For example, in the ECLS-K:2024’s sister study, the ECLS-K:2011 (nces.ed.gov/ecls/kindergarten2011.asp), we were unable to include items on home ownership in every round, but did include an item on this topic (PAQ140) in the parent interview in selected rounds. Similarly, the ECLS-K:2011 included an item in the school administrator questionnaire on translators and translation services available to parents (e.g., item E4 in the spring kindergarten instrument) in some, but not all, rounds. We will likely have to take a similar approach to these constructs in the ECLS-K:2024 given survey timing constraints.

Regarding your suggestion to add time-based references to response options, some of the ECLS items do include more specific definitions of the response options. However, specific time periods are not always used (either by the ECLS or by the original item developers), and adding time referents could in some cases inappropriately skew items, at least without testing. Also, it may not be easy for parents to answer some questions using specific time periods because they may not remember the frequency of their child’s behaviors in an exact way (e.g., how many times a month a child keeps working at something until it is finished). We appreciate the comment and will continue to consider when greater specificity in response options and their definitions is advisable.

Finally, thank you for your concern regarding the average daily attendance measure. We do plan on including other measures on student attendance in the ECLS-K:2024; for example, we expect to ask school administrators the percentage of students at their school who are chronically absent (item A12a in the spring kindergarten survey).

Sincerely,

Jill McCarroll

Study Director, Early Childhood Longitudinal Studies

https://nces.ed.gov/ecls/

National Center for Education Statistics, Longitudinal Surveys Branch
550 12th St. SW, Washington DC 20202 | 202-304-2920 | Jill.McCarroll@ed.gov

3