Part A ECLSK-2011 Spring 5th Grade

The Early Childhood Longitudinal Study, Kindergarten Class of 2010-11 (ECLS-K:2011), conducted by the National Center for Education Statistics (NCES) within the Institute of Education Sciences (IES) of the U.S. Department of Education (ED), is a survey that focuses on children’s early school experiences beginning with kindergarten and continuing through the fifth grade. It includes the collection of data from parents, teachers, school administrators, and nonparental care providers, as well as direct child assessments. Like its sister study, the Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K),¹ the ECLS-K:2011 is exceptionally broad in its scope and coverage of child development, early learning, and school progress, drawing together information from multiple sources to provide rich data about the population of children who were kindergartners in the 2010-11 school year. Data collections to date have been carried out for NCES by Westat, with the Educational Testing Service (ETS) as the subcontractor developing the child assessments. Clearances for studying the ECLS-K:2011 cohort were granted for the fall 2009 field test data collection, fall 2010 and spring 2011 kindergarten national data collections, fall 2011 and spring 2012 first-grade national data collections, fall 2012 and spring 2013 second-grade national data collections, spring 2014 third-grade national data collection, and the spring 2015 fourth-grade national data collection (OMB No. 1850-0750). Several generic clearance requests for testing various components of the study have also been approved (OMB 1850-0803).

This submission requests OMB’s approval for the spring 2016 fifth-grade national data collection. This submission also includes carry-over burden from the last approved national data collection package (OMB# 1850-0750 v.16 and 17) for the activities that will not be completed by the time this package is expected to be approved.

A.1.2 Legislative Authorization

The ECLS-K:2011 is conducted by NCES in close consultation with other offices and organizations within and outside the U.S. Department of Education. The ECLS-K:2011 is authorized by law under the Education Sciences Reform Act of 2002 (20 U.S. Code Section 9543): “The Statistics Center shall collect, report, analyze, and disseminate statistical data related to education in the United States and in other nations, including -- (7) conducting longitudinal and special data collections necessary to report on the condition and progress of education;”

A.1.3 Prior Related Studies

The ECLS-K:2011 is part of a longitudinal studies program. The two prior ECLS studies pertain to two cohorts—the kindergarten class of 1998-99 cohort and a birth cohort. Together these cohorts provide the range and breadth of data required to more fully describe and understand children’s education experiences, early learning, development, and health in the late 1990s, 2000s, and 2010s.

The birth cohort of the Early Childhood Longitudinal Study (ECLS-B) followed a national sample of children born in the year 2001, from birth through kindergarten entry. The ECLS-B focused on the characteristics of children and their families that influence children’s school readiness and first experiences with formal schooling, as well as children’s early health and in- and out-of-home experiences.

The ECLS‑K followed a nationally representative cohort of children from kindergarten through eighth grade. The base-year data were collected in the fall and spring of the 1998-99 school year, when the sampled children were in kindergarten. A total of 21,260 kindergartners throughout the nation participated by having a child assessment and/or parent interview conducted during that school year. Five more waves of data were collected: in fall and spring of the 1999-2000 school year when most, but not all, of the children who participated in the base year were in first grade; in the spring of the 2001-02 school year when most, but not all, of the children who participated in the base year were in third grade; in the spring of the 2003-04 school year when most, but not all, of the children who participated in the base year were in fifth grade; and in the spring of the 2006-07 school year when most, but not all, of the children who participated in the base year were in eighth grade.²

A.1.4 ECLS-K:2011 Study Design for the Spring Fifth-Grade National Data Collection

The sample for the ECLS-K:2011 is a representative sample of children across the country who attended kindergarten in 2010-11. The sample was selected using a multistage probability design. In the first stage, 90 primary sampling units (PSUs) that are counties or groups of counties were selected with probability proportional to size (PPS). In the second stage, public and private schools offering kindergarten or educating 5-year-olds in an ungraded setting were selected, also with PPS. The third-stage sampling units were children in kindergarten or children of kindergarten age in ungraded schools or classrooms. Children were selected within each sampled school using equal probability systematic sampling, with a higher sampling rate for Asian and Pacific Islanders (APIs) so as to achieve a minimum required sample size for APIs.

The base-year (i.e., kindergarten) data were collected in the fall and spring of the 2010-11 school year. The fall first-grade data collection was conducted in the fall of 2011 and the spring first-grade data collection was conducted in the spring of 2012. In both of these collections most, but not all, of the sampled children were in first grade. The fall second-grade data collection was conducted in the fall of 2012 and the spring second-grade data collection was conducted in the spring of 2013. In both of these collections most, but not all, of the sampled children were in second grade. The spring third-grade data collection was conducted in the spring of 2014 when most, but not all of the sampled children were in third grade, and the spring fourth-grade data collection was conducted in the spring of 2015 when most, but not all of the sampled children were in fourth grade.³

Similar to the previous years’ spring data collections, the national spring fifth-grade data collection will include direct child assessments, height and weight measurements, parent interviews, and school administrator and teacher questionnaires. As in all prior rounds of data collection, computer assisted interviewing (CAI) will be the mode of data collection for the child assessment and the parent interviews. Children will also complete an audio-CASI (computer assisted self-interview) version of a child questionnaire as they did in the third- and fourth-grade data collections. Also as in the past, school administrator and teacher data will be collected via hard-copy self-administered questionnaires. As was done in the fall second-grade and spring third-grade rounds, a subsample of children will also participate in a hearing evaluation.

Cognitive Assessments. As in the previous data collections for the ECLS-K:2011, a direct cognitive assessment will be administered in the spring 2016 fifth-grade collection. The cognitive assessment will include the domains of reading, mathematics, science, and executive functioning. It will be administered directly to the sampled children through a one-on-one assessment that includes age- and grade-appropriate items. The structure of the ECLS-K:2011 fifth-grade reading, mathematics, and science assessments will be two-stage, the same as the ECLS-K:2011 previous-round assessments.⁴ All children first will be administered a routing test. Performance on the routing test will determine which one of three second-stage tests (low, middle, or high difficulty) will be appropriate for the child’s demonstrated skill level; the child will then be administered the appropriate second-stage assessment form. The executive function measures (i.e., Numbers Reversed, Dimensional Change Card Sort, and the Flanker Inhibitory Control and Attention Test tasks) are not two-stage assessments.

Though new items were developed for inclusion in the fifth-grade ECLS-K:2011 reading, mathematics, and science assessments, a majority of items in the assessments will be the same as those included in the assessments from the earlier rounds of the ECLS-K:2011 and from the ECLS-K assessments. Items from earlier rounds of the ECLS-K:2011 are included to allow for the measurement of growth or gains in knowledge and skills as children age. Items from the ECLS-K are included in order to enable researchers to conduct cross-cohort analyses using the assessment data.

The spring fifth-grade cognitive assessment, like the spring second-, third-, and fourth-grade cognitive assessments, will use a computerized version of the Dimensional Change Card Sort (DCCS) task, which measures children’s executive functioning (specifically, cognitive flexibility). Although the DCCS was administered in a hard-copy format in the kindergarten and first-grade rounds, a switch to the computerized version was made beginning with the fall second-grade data collection because the electronic mode allows the assessment to capture response time, which is not possible using the physical card version. As children age, accuracy on the DCCS improves, so it becomes more important to capture response time as a way to measure variability in cognitive flexibility using the DCCS.

The Numbers Reversed task, the second assessment of executive function included in the ECLS-K:2011 child assessment, is identical to the Numbers Reversed task included in the previous grade data collections. This task assesses the child’s working memory. It is a backward digit span task that requires the child to repeat an orally presented sequence of numbers in the reverse order in which the numbers are presented. For example, if presented with the sequence “3…5,” the child would be expected to say “5…3.” Children are given five 2-number sequences. If the child gets three consecutive 2-number sequences incorrect, then the Numbers Reversed task ends. If the child is successful with sequences of two numbers, the child is then given five 3-number sequences. The sequences become increasingly longer, up to a maximum of eight numbers, until the child either gets three consecutive number sequences incorrect or completes all number sequences.

The third executive function task, the Flanker Inhibitory Control and Attention Test (commonly referred to as the Flanker), was introduced successfully in the spring fourth-grade data collection and will be administered again in the fifth-grade round. The task requires the child to focus on a given stimulus while inhibiting attention to stimuli flanking it. As was done for the fourth-grade assessment, arrows will be used as the stimulus and “flankers.” Children will be asked to look at the middle arrow in a row of arrows shown on the computer screen and indicate in which direction that middle arrow is pointing by pressing either the right or left arrow key on the computer’s keyboard. Sometimes the stimulus is pointing in the same direction as the flankers and sometimes it is pointing in the opposite direction. Scoring is based on a combination of accuracy and reaction time.

Child Questionnaire. Prior to the start of the direct cognitive assessment, children will be asked to complete a self-administered, computerized questionnaire. As in the third- and fourth-grade data collections, the Child Questionnaire (CQ) will be administered on a computer using audio-CASI, a software system that reads the instructions and questionnaire items to the child, while the same text is displayed on a laptop’s screen. Children will choose answers to the questions by selecting responses directly on the touch-sensitive screen of the laptop. After answering a question, the child will click on a “next” button and continue to the next question in the self-administered questionnaire.

Questions are drawn from various published and unpublished scales (appendix A includes the programmer specifications for the audio-CASI child questionnaire, which indicate the exact instructions to be provided to the child and the items that will be administered; appendix I lists the sources for the items and maps each item to a research question). The CQ consists of 48 statements and questions, which children will respond to using a variety of response scales and sets. Based on recommendations from a Socioemotional Content Review Panel (CRP) that was convened in October 2012 and a Technical Review Panel (TRP) that was convened in November 2013, the fifth-grade CQ will include items measuring children’s engagement in school, school belonging, academic stress, general life satisfaction, peer support, feelings of loneliness, social anxiety, peer victimization, and media usage. Children will be asked to indicate how often they feel certain emotions or experience certain behaviors. In addition, a few questions on student perseverance and parental monitoring of homework, grades, and free time have been added. These questions are planned for inclusion in other education studies conducted by NCES, the National Assessment of Educational Progress and the Middle Grade Longitudinal Study. Including them in the ECLS-K:2011 as well will result in the availability of information on these topics for students in several grades across elementary and middle school.

Data from the national administration of the CQ will enable researchers to compare students’ self-ratings and self-reports (e.g., of their peer relationships and school engagement) to the students’ performance on assessment items in the reading, math, and science domains.

Physical Measurements and Hearing Evaluations. In addition to the child questionnaire and the cognitive assessment, the ECLS-K:2011 direct child assessments will include measures of the children’s height and weight and, for selected children, hearing evaluations. Height and weight have been measured in all previous rounds of the ECLS-K:2011.

In the fall 2012 second-grade and spring 2014 third-grade rounds of data collection, hearing sensitivity and middle ear functioning were measured in a 30 percent subsample of the national sample. These hearing evaluations will again be conducted with the hearing subsample children in the spring fifth-grade round. The combination of three rounds of hearing data on the same subsample of children will provide important information on the prevalence and persistence of hearing problems in children during the elementary school years, as well as how these hearing issues are related to school experiences and learning.

The hearing evaluation protocol used in fall 2012 and spring 2014 will be used again in the spring fifth-grade data collection (with one addition, as discussed below). The evaluation is expected to take about 20 minutes. The protocol includes the following:

Asking the child a short set of hearing-related questions (which can be found in appendix B). The results of these questions will aid analysts in the interpretation of the collected evaluation data.
Conducting a brief visual inspection of the ears.
Obtaining measures of middle ear function in both ears.
Obtaining specific audiometric thresholds. It is expected that initially thresholds at three primary frequencies will be obtained in each ear (2000, 4000, and 8000 Hz, with a retest of 2000 Hz for test reliability). Additional frequencies (1000, 3000, and 6000 Hz) will be obtained as time and children’s attention permit.
Evaluating the child for language impairments.

The language impairment evaluation is new to the fifth-grade data collection. It is self-administered through an application on an iPod. Children will complete the language impairment screener after the completion of the other hearing evaluation tasks. The screener presents sentences to the child, some of which use correct grammar and some of which use incorrect grammar. The children are asked to indicate whether each sentence sounds okay to them.

As part of data collection, ambient noise levels in the testing rooms will be obtained because background noise can affect measurement. As in the previous rounds, the equipment specified by the cosponsoring agency (National Institute on Deafness and Other Communication Disorders; NIDCD) for evaluating hearing and measuring ambient noise will be used. Each participating child’s parent will receive a letter with information about his or her child’s hearing evaluation within 2 months after the evaluation.

Parent Interviews. A parent interview will be administered to one parent/guardian of each child in the ECLS-K:2011 study (appendix C includes the programmer specifications for the parent interview, which indicates the items that will be administered.) The interviews will be developed in English and then translated into Spanish. For parents who speak neither English nor Spanish, home and community interpreters will be used when available to administer the English-language version to parents, translating the English version to the parent’s native language during the interview. The spring fifth-grade parent interview includes the same types of questions (in terms of topics and format) that have been previously fielded in the ECLS-K, earlier rounds of the ECLS-K:2011, and other NCES studies (e.g., the ECLS-B, the National Household Education Surveys Program (NHES), the Education Longitudinal Survey of 2002 (ELS:2002), and the National Education Longitudinal Survey of 1988 (NELS:88)). More specifically, the parent instrument will ask about parent and child background characteristics, such as age and sex, if they were not collected in a previous round; the child’s immigration status if it was not collected in a previous round; parent involvement with the school; school avoidance; family structure; primary language(s) spoken in the household; the home environment; parent’s marital/partner satisfaction; the child’s friendships; child care before or after school; nonresident parents; parent’s country of origin; communication; discipline and emotional supportiveness; the parent’s psychological well-being and health; household food security; parent education; parent employment; welfare and other public transfers; and household income. Parents will also be asked to report on their children’s physical activity, dental care, routine health care, overall health, and disabilities. The fifth-grade parent interview mostly contains questions that have been asked in at least one previous round of the study. However, some questions about therapy involving animals to help children with disabilities have been added to the child’s health and well-being section. There are also some questions that have not been asked for several rounds in order to collect end-of-study information on important characteristics (e.g., parent’s education expectations for the child, marital/partner satisfaction, use of a language other than English, and outings with the child).

Classroom Teacher Questionnaires. As in previous rounds, teachers of sampled children will be asked to complete hard-copy questionnaires. The design and distribution of these questionnaires in rounds prior to fourth grade assumed that most study children were taught all school subjects by one general classroom teacher. For this reason, information on the classroom environment, teacher background, and the sampled children’s skills, abilities, and experiences at school was collected from one teacher for each child, his/her general classroom teacher.

However, as children move into the upper elementary grades, it becomes more common for children to have different teachers for at least a few subject areas, such as reading and language arts, mathematics, science, and social studies. Information about how study schools organize students for instruction for fourth grade, which was collected in the third-grade data collection, showed that there is variation in how many teachers children have for various subjects. For example, in some schools students have different teachers for all subjects; in other schools, students are pulled out of the general classroom only for science but remain in their homeroom for reading and mathematics; while in other schools students are taught all subjects by one general classroom teacher.

Thus, beginning in the fourth-grade data collection round, a new model for the design and distribution of the teacher questionnaires was introduced. The same approach will be followed for the spring fifth-grade round. In this model the reading teacher for each child is identified and that teacher will be asked to complete a questionnaire. To reduce the burden on teachers, for the fourth-grade data collection, half of the sampled children were randomly assigned to have their mathematics teacher complete questionnaires, while the other half of the sampled children were randomly assigned to have their science teacher complete questionnaires. The same teacher assignments will be made for fifth grade. That is, if in the fourth-grade data collection a child was assigned to have her mathematics teacher complete a questionnaire, her fifth-grade mathematics teacher will be asked to complete a questionnaire. Thus, every child will have a reading and either a mathematics or a science teacher identified for him/her.

All identified teachers will receive a self-administered teacher-level questionnaire (“TQ”). The TQ includes questions about the teachers such as their views on the school climate, evaluation methods used for reporting to parents, and their background and education. It also includes questions about time children spend in group activities and in lessons in general subject areas, as well as occurrences of recess.

Subject-specific child-level questionnaires will also be distributed for the identified reading, mathematics, and science teachers. Each of these questionnaires (“TQC”) includes two sections:

Part 1: Child-level questions. The questions in the child-level section ask the teacher to rate the child on academic and social skills, school engagement, and classroom behaviors. Because each child’s reading teacher will complete a child-level questionnaire, the reading TQC contains the majority of the child-level questions, while the mathematics and science TQCs contain only a few child-level questions specifically related to that subject. There are also questions in all three TQCs asking for child-specific instructional information (for example, instructional group placement and additional services the child receives).
Part 2: Classroom questions. The questions in the classroom section pertain to the reading, mathematics, or science class in which the sampled student is taught. Teachers are asked to indicate how much time is spent on specific skills and activities, as well as questions on instruction and grading practices, behavioral issues, and homework assignments.

To further reduce burden on teachers, one “key child” will be identified for each subject and class. Teachers will be asked to complete all items in both the child-level and the classroom sections of the TQC only for the designated key child; for the remainder of the sampled children in the same reading, math, or science class as the designated key child, a teacher will only need to complete the questions in the child-level section of the TQC. Teachers who teach multiple sections of a subject (for example, advanced and remedial sections of math) will have a key child identified for each of those sections, meaning that these teachers will complete the classroom questions about each section of the subject that is taught to at least one ECLS-K:2011 student. See appendix D for the TQ and appendix E for the TQC questionnaires.

The content of the various teacher questionnaires is very similar to the content of the spring fourth-grade instruments. However, several questions from earlier rounds of the study are also included in the fifth-grade instruments in order to have a final data point for various constructs introduced in earlier rounds. Specifically, items on assessment participation and on assessment accommodations for disabilities, parent participation in specific activities and parent-initiated communication with the teacher, and teacher-initiated communication with the parent, all from the spring second-grade child-level teacher questionnaire, were added to the fifth-grade reading TQC. Assessment of students with disabilities is relevant to a new focus, announced recently by the Secretary of Education, to move from a compliance model to a performance-based model for evaluation and funding special education programs.⁵ This has implications for students with disabilities because states are being asked to increase the percentage of students, particularly those with disabilities, who participate in annual state assessments. For each study child who has an IEP, the child’s regular classroom teacher will be asked about such participation in assessments and whether or not the participation included assessment accommodations. Note that the special education teacher questionnaire continues to contain a parallel item for students for whom they provide services. The other two topics-- parent involvement and teacher-parent communications--are relevant to parent involvement, and though omitted from the third- and fourth-grade questionnaires in order to reduce response burden, will be administered again in this final data collection. These were among a group of items that were not included each year because the extent of parent involvement and communication were not expected to change dramatically from year to year, though they could well change across two or three years. These constructs were examined by numerous researchers using ECLS-K data (Johnson 2011; Cooper 2010; Turney and Kao 2009; Cooper et al. 2009; Xu et al. 2009) and it is anticipated that the information from ECLS-K:2011 will continue to be important.

Additionally, new skills and activities questions that reflect the fifth-grade standards taken from the Common Core State Standards and the Next Generation Science Standards were drafted for the reading, math, and science TQCs. In all three subjects, the number of skills and activities items being asked is the same as or less than the number included in the fourth-grade questionnaires so that teacher burden will not be increased.

Data obtained from teachers in the TQ can be used to address research questions about the relationships between certain classroom and teacher characteristics and children’s academic and social development. In addition, data from the TQ can also be compared to data from prior rounds of collection in this study and in the ECLS-K.

Data obtained from teachers in the TQC can be used in several ways, for example:

to examine teacher-reported measures of cognitive and social development and compare these with information from prior rounds of data collection;
to address research questions about relationships among classroom and teacher characteristics, child-specific participation in instruction and school-based services, and children’s academic and social development; and
to compare to the results of direct assessments administered to the sampled children.

Special Education Teacher and Related Service Provider Questionnaires. Special education teachers and related service providers will be asked to complete questionnaires for ECLS-K:2011 students with an Individualized Education Program (IEP) on file at the school. Two self-administered hard-copy instruments will be used: a teacher-level questionnaire that collects information on the special education teacher’s background, education, teaching experience, teaching position, and caseload; and a child-level questionnaire that collects information on the individual study child’s disabilities, classroom placement, and services received. The information obtained through these questionnaires will be useful in examining special education curricula and the services being received by children with disabilities (see appendix F for the Special Education Teacher questionnaires).

School Administrator Questionnaires. The School Administrator Questionnaire (SAQ) will be completed by the school administrators in the schools attended by the children in the study.⁶ In the base year of data collection, one SAQ was distributed to all participating schools. However, to reduce respondent burden, in subsequent rounds two versions of the questionnaire were distributed: one for schools that completed an SAQ in a prior round of the study (referred to as “continuing schools”) and one for any school that did not previously complete the SAQ, either because the school was a new school into which an ECLS-K:2011 student had transferred or because the school did not complete the SAQ in any previous study round (referred to as “new schools”). The questionnaire for continuing schools excluded questions about characteristics that were unlikely to change from one year to the next. In the fifth-grade data collection, one version of the SAQ will again be distributed to all participating schools. This version will include questions from the new school SAQ that had been excluded from the continuing school SAQ in order to collect updated, end-of-study information from all schools.

The SAQ instrument includes a broad range of questions about the school setting, policies, and practices at both the school level and in specific grades, as well as questions about the school administrator and the teaching staff. The questionnaire remains much the same as the new school SAQ from the fourth-grade data collection, although a few items from previous data collections were added in order to collect data at a final time point. The added items are related to the percentage of students attending from the surrounding neighborhood, school capacity, hearing and vision screening, before- and after-school care and other services for parents and families, grade retention policies, and racial and ethnic composition of teaching staff.

These items will help researchers understand the school contexts for ECLS-K:2011 students. Comparisons can be made between children attending different types of schools, including public and private schools (with private schools being further identified as religious or nonreligious); rural, urban, and suburban schools; and schools of different sizes. Data from this questionnaire can be used with data from the child assessments and teacher questionnaires to investigate the degree to which education outcomes of various groups of children are associated with the differences in the schools that the children attend (see appendix G for the SAQ questionnaires).

A.2 Purposes and Uses of the Data

The ECLS-K:2011 will provide rich data sets that are generally designed to serve two purposes: descriptive and explanatory. It will provide descriptive data at a national level related to (1) children’s status at entry into kindergarten and at different points in children’s elementary school careers, (2) children’s transition into school and into the later elementary grade levels, and (3) children’s school progress through the fifth grade. Additionally, it will provide rich data that will enable researchers to test hypotheses about how a wide range of child, family, school, classroom, nonparental care, education provider, and community characteristics relate to experiences and success in school.

In addition to the descriptive objectives mentioned above, the data will describe the diversity of young children with respect to demographic characteristics such as race/ethnicity, language, and school readiness. Such information is critical for establishing policies that are sensitive to this diversity. The longitudinal nature of the study will enable researchers to study cognitive, socioemotional, and physical growth, as well as relate trajectories of growth and change to variation in home, school, and before- and after-school care setting experiences in the elementary grades. Summer learning or learning loss, which can have a considerable impact on children’s education progress, can also be examined with data collected in the fall 2011 and fall 2012 data collections. Ultimately, the ECLS-K:2011 data sets will be used by policymakers, educators, and researchers to consider the ways in which children are educated in our nation’s schools and to develop effective approaches to education. The data will be particularly valuable to policymakers, as the ECLS-K:2011 is being launched a dozen years after the inception of the ECLS-K. Analyses of the two cohorts will provide valuable information about the influences of changing policy and demographic environments on children’s early learning and development.

A.2.1 Research Issues Addressed in the ECLS-K:2011

Today’s early education environment differs from that of the past in numerous ways. Examples of the many changes that have occurred within schools and within the larger society in recent years are presented in exhibit A-1 and include changes at the policy, state, school, family, and societal levels. ECLS-K and ECLS-B data have been used by numerous researchers to examine many of these topics. The widespread use of ECLS data is a testament to the importance of the ECLS program. At the same time, both prior studies leave gaps in the research questions we may answer with the data, which is perhaps inevitable because changes in policy, research, and society are often difficult to anticipate. The ECLS-K:2011 seeks to preserve the strengths of the earlier studies by retaining much of the same content, while incorporating appropriate modifications. This allows for the use of ECLS-K:2011 data to answer some of these recently-emerging questions, while at the same time allowing for the study of a new cohort of children growing up in new circumstances and the ability to make comparisons with the earlier cohorts. Below, we discuss some of the important developments that are particularly relevant to the design of the ECLS-K:2011.

A.2.1.1 Developments in Early Education Policy

A major change in early education occurred when the Elementary and Secondary Education Act (ESEA) was reauthorized as the No Child Left Behind Act (NCLB) and signed into law in early 2002. ESEA 2002 set clear expectations for student achievement, mandated annual assessments of all children in grades 3 through 8 to measure progress toward state-defined goals, and had strong reporting requirements for schools, districts, and states. ESEA 2002 aimed to narrow or eliminate achievement gaps in education and called for accountability and higher standards for achievement. ESEA 2002 is overdue for reauthorization. In 2010, President Barack Obama released his Blueprint for Reform for the reauthorization of ESEA, which is awaiting congressional action as of the date this clearance request is being submitted. While debate over the future of ESEA continues, all but six states have received waivers in meeting some of the NCLB requirements from the U.S. Department of Education (http://www2.ed.gov/policy/elsec/guid/esea-flexibility/index.html).

Exhibit A-1. Examples of important developments relevant to the ECLS-K:2011

Policy changes

– Passage of the Elementary and Secondary Education Act (ESEA) 2002

– President Obama’s 2010 Blueprint for Reform proposal for reauthorization of ESEA

– Recent U.S. Department of Education invitation to states to apply for flexibility in meeting specific ESEA requirements in exchange for meaningful reform at the state and local levels

– Race to the Top

– The Common Core State Standards Initiative

– The New Generation Science Standards

– E-GOV Act of 2002 promoting use of the web and web-based applications to provide access to and enhance delivery of government services

Economic challenges

– Global recession and financial crisis beginning 2007/2008

– American Recovery and Reinvestment Act of 2009

– State and local budget constraints and cuts

– Sequestration of a portion of federal funds across all administrative departments in 2013

Changes in schools and challenges to schools

– Growth in school choice and increasing number of charter schools

– Increased use of technology and the Internet in schools

– Increased use of mobile devices and “bring your own device” policies

– New technologies allow different types of classroom interactions (e.g., remote personal response systems, social networking, digital textbooks)

– Blended learning where in-person instruction and technology-delivered information are combined

– Differentiated instruction

– Segmentation by subject in elementary school

– Value-added assessments

– Teacher salary and tenure reform, including incorporating measures of teacher effectiveness

– Training teachers to use technology effectively and to become online educators

– Growth of Hispanic, Asian, and multi-race child population

– Growth in English language learners (ELL) in schools, especially at young ages

– Use of data management systems to track and monitor student achievement and behavior and the use of data-driven decision making

– The increased use of “response to intervention” approaches to intervention in the general education setting and for the determination of eligibility for special education

– Increased focus on preventing problem behavior

Child health

– Epidemic of obesity and associated rise in diabetes

– Rise in incidence of:

Allergies
Asthma
Autism
Attention deficit/hyperactivity disorder

– Decline in incidence of:

Specific learning disabilities

Scientific developments

– Advances in neuroimaging techniques (e.g., fMRIs) that have led to advances in our understanding of the development of children’s learning, memory, attention, and language

– Advances in neurological research and emphasis on executive function

– Emerging research showing the trainability of cognitive processes (e.g., Rueda et al. 2005)

– Recent developments in cognitive science and learning theory

The recent adoption and state-by-state implementation of the Common Core State Standards in English Language Arts and Mathematics across the country represents another significant change in the education policy environment that can be examined given the data that are collected by the ECLS-K:2011. In 2009, a state-led effort to develop the Common Core State Standards (CCSS) was launched by state leaders, including governors and state commissioners of education from 48 states, two territories, and the District of Columbia, through their membership in the National Governors Association Center for Best Practices (NGA Center) and the Council of Chief State School Officers (CCSSO).⁷ Initially nearly all states adopted the CCSS as their state’s education standards in reading and mathematics, but districts and schools differ in the extent to which they have implemented these standards. Some states have recently decided to return to state-developed standards, and others are considering doing so. A similar effort by the National Research Council and 28 states has resulted in a framework called the “Next Generation Science Standards.” As advised by the TRP members in 2011 and 2013, the study has developed items based on these national efforts to describe the teaching of these subjects in participating students’ classrooms.

In addition to changing policies and approaches to early education and research, the United States continues to face economic challenges that affect the federal budget. Tightened state and local budgets have direct impacts on districts and schools. Reduced services and staff may affect children’s experiences in school. Beginning with the spring first-grade data collection, the school administrator questionnaire included questions asking about actions that may have occurred as a result of changes in funding, such as staff additions or contractions in the past year, changes in staff burden and salaries, adjustments in class sizes, and increases in the proportion of students eligible for free or reduced-price lunch. Researchers have studied the effect of the recession on child well-being and found adverse effects, including an increase in the number of households classified as “food insecure” (Sell et al. 2010). According to the “NSLP Fact Sheet” of the Food and Nutrition Service, USDA, the number of students enrolled in the National School Lunch Program (i.e., those receiving free or reduced-price lunch) continues to increase. For example, in 1990 over 24 million children participated in the program, while in 2011, that number was greater than 31.8 million children.

A.2.1.2 School Readiness

Education policymakers and researchers continue to investigate the most appropriate ways to promote school readiness. Most experts agree that school readiness is a multifaceted phenomenon that encompasses several domains of child development. In addition to cognitive development and pre-academic skills (e.g., letter and number recognition, emerging literacy), school readiness is conceptualized as involving the whole child, including health and physical well-being, language acquisition, social and emotional development, and interest in and enthusiasm for learning. The ECLS-K:2011, like the ECLS-K and the ECLS-B, is an important source of information related to all of these domains, and this information can be used to more fully understand how children’s early learning and development are affected by their experiences.

One effect of ESEA 2002 is a change in curricular emphasis in the early grades. ESEA 2002 emphasizes evidence-based early literacy activities that stress the development of specific literacy skills. ESEA 2002 includes two initiatives, Reading First and Early Reading First, which seek to lay the foundation for future school success by stressing the following five skills to enable children to become proficient readers:

Phonemic awareness: the ability to hear and identify sounds in spoken words;
Phonics: the relationship between the letters of written language and the sounds of spoken language;
Vocabulary: the words students must know to communicate effectively;
Fluency in reading: the capacity to read text accurately and quickly; and
Comprehension: the ability to understand and gain meaning from what is read.

ESEA 2002 and these reading programs view literacy as a learned skill that requires coherent skill-based instruction using scientifically supported curricula provided by highly qualified teachers. By ensuring that the ECLS-K:2011 assessments and teacher questionnaires measure these skills, the ECLS-K:2011 can be used to examine children’s emerging literacy and cognitive development. The focus of ESEA 2002 on early literacy skills has shifted discussions of school readiness from the range of domains mentioned above to two: (1) language development and (2) cognition and general knowledge. As a study of child growth and development generally, the ECLS-K:2011 also examines the trajectories of other important dimensions of school readiness, such as social competence, approaches to learning, and other indicators of socioemotional development.

A.2.1.3 Executive Functioning

Recent research in the cognitive and neurological sciences is providing important insights into developmental processes associated with school readiness. Of particular interest is research on the importance of executive functioning for learning and academic achievement (e.g., Blair and Razza 2007; Posner and Rothbart 2006). “Executive functioning” refers to a set of interdependent processes that work together to accomplish purposeful, goal-directed activities and include working memory, attention, inhibitory control, and other self-regulatory processes. Executive functioning processes work to regulate and orchestrate cognition, emotion, and behavior to help a child to learn in the classroom. For example, executive control, which is associated with the prefrontal cortex, involves the ability to allocate attention, to hold information in working memory, and to withhold an inappropriate response (Casey et al. 2000). Recent research has found a connection between the development of executive function skills (and accompanying brain development) and factors such as family income level (Noble et al. 2015; Blair 2013; Raver et al. 2013; Rhoades et al. 2011) and parenting characteristics (Blair 2013; Rhoades et al. 2011). For example, Blair (2013) found that parenting characteristics mediated the effect of poverty on executive function.

These cognitive and behavioral processes are predictive of reading and math achievement (Blair and Razza 2007), and there is emerging research that indicates that some of these cognitive processes are trainable (Rueda et al. 2005; Klingberg et al. 2005) and can be improved upon in regular public school classrooms without costly interventions (Diamond et al. 2007).

Many cognitive processes are necessary for learning and achievement. For example, learning, whether it involves reading comprehension, solving applied mathematics problems, or something else, involves the interaction between working memory and long-term memory and the formation of linkages between the two. The ECLS-K:2011 will be strengthened by obtaining direct and indirect measures that capture specific learning issues such as attention problems, memory problems, inability to withhold inappropriate responses, and language issues. The ECLS-K:2011 will provide information to allow for the investigation of such differences.

A.2.1.4 Demographic Changes

The United States has been experiencing demographic shifts in its population, becoming an increasingly diverse society (Frey 2011). Recent analyses of decennial census data show that from 2000 to 2010, the growth in the nation’s child population was due primarily to increases in the Hispanic, Asian, and other groups who are not White, Black, or American Indian (Frey 2011). The demographic shift is especially evident in the school-aged population. The percentage of public school students in the United States who were English language learners was higher in school year 2012-13 (9.2 percent, or an estimated 4.4 million students) than in 2002-03 (8.7 percent or an estimated 4.1 million students) (U.S. Department of Education 2015).

Language is not the only challenge for many of these children, particularly those born outside the United States. Many, especially those with parents from Mexico and Central America, have lower parental education, larger families, and lower family income than the native-born (Greico et al. 2012; Larsen 2004). However, foreign-born households are more likely than native-born ones to be headed by married couples (Greico et al. 2012). Also, families with different cultural backgrounds and from other countries may have different normative expectations for how they should interact with schools and teachers. The ECLS-K:2011 will enable researchers to examine how schools and teachers are meeting the needs of these students and their families and to measure the effectiveness of these efforts.

A.2.1.5 Hearing Impairments in the Early Elementary School Years

At least 15 percent of U.S. children aged six to nineteen have a measurable hearing loss in one or both ears (Henderson et al. 2011). Any degree of hearing loss can be educationally handicapping for children. Even children with mild to moderate hearing losses can miss up to 50 percent of classroom discussions (Flexer 1999). Unmanaged hearing loss in children can affect their speech and language development, academic capabilities and educational development, and self-image and social/emotional development (Cunningham et al. 2003). The American Academy of Audiology recommends that hearing loss be ruled out whenever a child is being considered for special education services (American Academy of Audiology 2011). Inclusion of a hearing evaluation in the ECLS-K:2011 will provide researchers with a unique ability to investigate the prevalence of hearing impairments among elementary school students and to look at associations between hearing loss and a host of education experiences and outcomes in a large-scale nationally representative study, to examine the emergence of hearing difficulties across time, and to see whether and how the timing of the emergence of hearing difficulties may be related to both environmental factors and education experiences and outcomes.

Language disorders – in which children have difficulty understanding what is said (receptive language) or communicating ideas to others (expressive language) – are also common in school-aged children (Law et al. 2000). When the language difficulty exists without any other related impairment (such as hearing loss, cognitive disability, or neurological disorder), it is called a “specific language impairment” (Moyle et al. 2013). Very few estimates of the prevalence of specific language impairment are available, as diagnostic strategies are long and complex and require specially-trained personnel; however, one study estimated that 7 percent of school-age children have specific language impairment (Tomblin et al. 1997). Language disorders can lead to reading difficulties and ultimately poor academic achievement. Without intervention, children with language disorders may develop poor social skills, emotional problems, and vocational difficulties as they grow older (Moyle et al. 2013). Screening for language impairment in the ECLS-K:2011 will provide an updated measure of prevalence of such impairments and allow researchers to examine the relationships between hearing impairment and various aspects of functioning and school outcomes.

A.3 Use of Improved Information Technology

When feasible, available technology will be used to improve data quality and reduce respondent and school burden. The ECLS-K:2011 parent interviews and child assessments will be conducted using computer-assisted interviewing (CAI). Using CAI will increase data collection efficiency by permitting preloads of available data about the sampled schools and children, online editing of information as it is entered (e.g., correcting data entry errors caught through range and logic checks or correction of information provided in a previous round of data collection), and routing of respondents through complex question branching—all of which also reduce respondent burden by producing faster interviews and reducing the need to recontact respondents to obtain missing information (which would occur, for example, if a field interviewer not using CAI does not follow a skip pattern correctly and items that should be asked are not). Parent interviews are primarily conducted by telephone; however, field interviewers will conduct interviews with parents without telephones or who are difficult to reach by making in-person visits to complete interviews. These in-person interviews will also be conducted using CAI on laptop computers. The CAI system has important features that will improve the quality of the data and reduce the burden on respondents, as follows:

Initial Contact: The CAI system will guide the ECLS-K:2011 field interviewer in making contact with the parent at the correct phone number or address and with the child at the school and will include prompts to help the interviewer identify the correct respondent.
Routing the Direct Child Assessment: The CAI system will be programmed so the initial routing tests at the beginning of the reading, mathematics, and science cognitive assessment subtests will be scored by the computer and the appropriate second-stage tests corresponding to the child’s ability level will be administered. The benefits of such a two-stage assessment are increased adaptiveness, reduced burden for the child, and increased precision of measurement because the assessors do not need to score the routing test and select the appropriate second-stage test themselves. In addition, there typically are some skip rules programmed into the CAI for reading and math that will skip children to a set of questions on a different topic or the next domain if they are struggling and have responded to several questions incorrectly. For the executive function Numbers Reversed task, the CAI system accurately determines where the task ends depending on the child’s performance. As mentioned above, the computerized version of the executive function Dimensional Change Card Sort (DCCS) and Flanker tasks allow the assessment to accurately capture response time, which becomes more important to capture in these particular assessments as children get older.
Skip Patterns: The CAI system automatically guides interviewers through the complex skip patterns in the parent interviews, thereby reducing respondent burden, reducing potential for interviewer error, and shortening the interview administration time. The respondent will not be asked inapplicable questions and the interviewers do not need to spend time determining which questions to ask.
Copying Responses: The CAI system will be programmed to copy responses from one item to another and from one round to another to prevent unnecessary repetition of questions and to aid in respondents’ recall. For example, information that is provided by the respondent early in the interview may be useful later in the interview; such information can be displayed on the screen or used as a wording fill for relevant questions to assist the respondent. Additionally, information from the previous waves of data collection can be copied to the current wave’s interview and be verified by the respondent, eliminating the need to collect the data again.
Time Intervals: The CAI system also provides automated time and date prompts that are very useful in longitudinal studies to assist respondents in remembering specific time periods. The interview can also provide the specific timeframe for the interval between the previous and the current wave of data collection, to help respondents provide information without repeating information they had given at the previous data collection period.
Receipt Control: The CAI system will provide for automatic updates to the interview status of study participants and will be used to produce status reports that allow timely and ongoing monitoring of the survey’s progress.

The use of a CAI system for the ECLS-K:2011 is critical because of the intricate and sometimes difficult skip patterns that are part of complex survey instruments and because of the longitudinal nature of the data collection in which the same respondent might be interviewed at multiple time points. Without CAI, the ECLS-K:2011 instruments would be difficult to administer over repeated measurement periods, and respondent burden would be increased.

As in the spring 2014 third-grade and spring 2015 fourth-grade data collections, the child questionnaire will be administered using audio computer-assisted self-interview (audio-CASI) technology. With this format, the items and response options are presented to the child on a touchscreen and the child enters his or her own responses by touching the screen. The responses are then saved on the laptop and will be transmitted along with the data from the assessments. There are several advantages to using an audio-CASI version of the child questionnaire. This format provides more privacy to children as they answer questions that may be sensitive for them, and administration is more standardized because all children hear the items read to them in exactly the same way with the recording. Also, electronic capture of responses reduces processing time and the potential for data entry error.

A computer-based data management system will be used to manage the sample. The sample management system uses encrypted data transmission and networking technology to maintain timely information on respondents in the sample, including contact, tracking, and case completion data. This system is particularly important as children move from one school to another over the course of the ECLS-K:2011 study. The use of technology for sample management will maximize tracking efforts, which should have a positive effect on the study’s ability to locate movers and achieve acceptable response rates.

The ECLS-K:2011 Secure Message Center, which was first used in the spring 2014 third-grade round, will be used again for the spring 2016 fifth-grade round. The message center is a secure website accessed with a username and password that has been assigned to specific users, namely field staff (field managers, school recruiters, and team leaders) and participating school coordinators. The list of children enrolled in each school who are participating in the study will be sent to the school coordinator from the data collection contractor’s home office as an attachment to a secure message. This method will make it more convenient for school coordinators to access the list of participating children, as compared to communicating via telephone. The ECLS-K:2011 Secure Message Center will also greatly enhance the security of this list, as it provides a method for sharing confidential personal identifying information between schools and field staff in a secure environment. Because of the nature of the system, the list cannot be printed or forwarded to other school staff. If the school coordinator is amenable, the message system can also be used for other types of sensitive communication between the school coordinator and the field staff (for example, when informing the field staff that a child has moved to a new school).

A.4 Efforts to Identify Duplication

The ECLS-K:2011 will not be duplicative of other studies. The ECLS-K is the only other study to collect as detailed and extensive information as the ECLS-K:2011 for a cohort of young children and to follow them throughout elementary school. The ECLS-K:2011 extends the information obtained by the ECLS-K to a new cohort, opens up possibilities to investigate new research questions, and allows important comparisons to be made between two kindergarten cohorts attending school a dozen years apart. In addition, the ECLS-K:2011 has collected data during the children’s second-grade and fourth-grade years, which the ECLS-K did not.

A literature search was conducted to identify and review research studies with the same study purpose and goals as those proposed for the ECLS-K:2011. To be included in the search the research had to be (1) a survey-based study of a population with a sample of 1,000 or more, (2) longitudinal in design, and (3) focused on children’s cognitive development in the elementary, middle, and/or secondary grades. Although similar studies were found, they were generally confined to limited geographic areas (e.g., Baltimore, Maryland; Greensboro, North Carolina) or, in the case of studies conducted on the national level (e.g., Prospects, Children of the National Longitudinal Survey of Youth [NLSY Child Supplement]), were not based on probability samples of kindergartners. For example, Prospects began with first graders and targeted Title 1 recipients. NLSY79’s Child Supplement targeted the children of female sample members of a household-based 1979 sample of 14- to 21-year-olds. The Head Start Family and Child Experiences Survey (FACES), which is similar to the ECLS-K:2011 in terms of the content and components included, has followed several cohorts of children from preschool through early elementary school. However, FACES has not followed the progress of children in school beyond kindergarten or first grade, and the samples are limited to children served by Head Start. The NICHD Study of Early Child Care and Youth Development focused on similar child development outcome areas (social, emotional, intellectual, and language development, health, and physical growth), but did not include the same depth of information about the child’s school experiences as the ECLS-K:2011 does. The NICHD sample was recruited from hospitals shortly after the birth of the children and the study’s main focus was on early child care, including maternal care and the relationship between that care and children’s developmental outcomes. Studies such as the National Education Longitudinal Study of 1988 (NELS:88) and Education Longitudinal Study of 2002 (ELS:2002) began with students in the middle and high school grades. Another major finding of the literature review was that most studies used group-administered achievement tests, which, for young children, can be less reliable than individually administered assessments. Individually administered assessments, like those used in the ECLS-K:2011, allow the assessor to establish rapport and offer motivation and supportive conditions so that each child performs to the best of his or her ability.

A.5 Method Used to Minimize Burden on Small Businesses

Private, not-for-profit, and proprietary elementary schools have been drawn into the sample. These proprietary and nonprofit schools will benefit from the study’s burden-reducing strategies (e.g., instruction packets for participants, toll-free help lines, and prepaid business return envelopes), which were designed for all types of schools.

A.6 Frequency of Data Collection

This submission describes and requests approval for the spring fifth-grade data collection, which will occur in the spring of 2016. The first data collection for the study occurred in the fall of 2010, and additional data collections have occurred in spring 2011, fall 2011, spring 2012, fall 2012, spring 2013, spring 2014, and spring 2015.

One of the main goals of the ECLS-K:2011 is to measure children’s cognitive, socioemotional, and physical growth and development, as well as changes in the contextual characteristics (i.e., family, classroom, school, and community factors) that can affect growth. The spring fifth-grade data collection is the last periodic follow-up currently planned that will collect information to be compared to baseline (kindergarten) information, thereby allowing for analyses of change for children and their environments. The frequency of data collection for the ECLS-K:2011 is linked to the rate of change that is expected for children of this age and the desire to capture information about children as critical events and transitions are occurring, rather than measuring events and transitions retrospectively. Without data collection follow-ups, the study of children’s cognitive, socioemotional, and physical development is hindered.

A.7 Special Circumstances of Data Collection

No special circumstances for this information collection are anticipated.

A.8 Consultants Outside the Agency

NCES consulted with a range of outside agencies over the life of the ECLS‑K, and such input also has informed the ECLS-K:2011 study design and instrumentation, since they draw heavily from the ECLS-K. During the early development of the ECLS-K, project staff met with representatives from a wide range of federal agencies with an interest in the care and well-being of children (see Table A-1). The goal of this activity was to identify policy and research issues and data needs. Similarly, consultation with federal agencies has occurred and continues for the ECLS-K:2011. Several of the early consultations with government agencies have resulted in interagency agreements funding questions, sections of or full study instruments, and components of the child assessments (specifically, the hearing evaluations) to the study instruments.

Project staff has also consulted several other organizations (see Table A-2) that have an interest in the care, well-being, and education of young children. The goal of this activity was to obtain additional perspectives on policy and research issues and data needs. While most of this consultation occurred during the design and conduct of the ECLS-K, there was also some outside consultation during the design of the ECLS-K:2011.

Similar to its predecessor, the ECLS-K:2011 represents a collaborative effort by education and health and human services agencies. NCES supports the development of the core design of the ECLS-K:2011. Partner agencies supporting the inclusion of the supplemental questions or sections of the study instruments that enrich the ECLS-K:2011 by providing expert input and/or funding have included: the Economic Research Service of the U.S. Department of Agriculture; the National Center for Special Education Research and the National Center for Education Evaluation and Regional Assistance in the Institute of Education Sciences of the U.S. Department of Education; the Administration for Children and Families in the U.S. Department of Health and Human Services; and the National Institute of Deafness and Other Communication Disorders, the National Eye Institute, and the Eunice Kennedy Shriver National Institute of Child Health and Human Development, all at the National Institutes of Health in the U.S. Department of Health and Human Services. The National Institute of Deafness and Other Communication Disorders is sponsoring the hearing evaluation being conducted as part of the fifth-grade data collection. Table A-1 lists the Federal agency consultants for the ECLS-K and ECLS-K:2011 and Table A-2 lists other organization consultants for the ECLS-K.

In preparation for the ECLS-K:2011 collections, the data collection contractor assembled expert panels (Technical Review Panel (TRP) and Content Review Panels (CRP)) to review and comment on issues related to the development of the study and survey instruments. The members of the panels included experts in research, policy making, and practice in the fields of early childhood education and development, elementary education, health, research methodology, special populations, and assessment.

There have been three meetings of the TRP panels. The first was a 2-day meeting held in November 2008. The meeting focused on major design and content issues, such as study periodicity, the benefits of including an assessment of science in kindergarten, the assessment of executive functioning, and the content of a Spanish language assessment for native Spanish speakers who are English language learners. The TRP members also provided suggestions for specific questionnaire items to be included in the instruments in the full-scale national data collection. Table A-3 lists the ECLS-K:2011 TRP members present at the first meeting.

The second TRP meeting was a 2-day meeting held in March 2011. The meeting focused on content for the first- and second-grade non-assessment instruments, including suggestions for specific questionnaire items to be included in the instruments in the second-grade data collection. Table A-4 lists the ECLS-K:2011 TRP members present at the second meeting.

The most recent TRP was a 2-day meeting held in November 2013. The discussion focused on the development of the fourth-grade instruments, as well as looking ahead to the fifth-grade data collection. Panel members recommended study constructs and specific items for inclusion in the parent interview, child questionnaire, and teacher and school administrator questionnaires. Table A-5 lists the ECLS-K:2011 TRP members present at the third meeting.

Table A-1. Federal agency consultants for ECLS-K and ECLS-K:2011

Diane Schilder¹

Government Accounting Office

Cindy Prince,¹ Emily Wurtz¹

National Education Goals Panel

Andy Hartman¹

National Institute for Literacy

Mary Queitzsch,¹ Larry Suter¹

National Science Foundation

Michael Ruffner,¹ Bayla White,¹

Brian Harris-Kojetin¹

Office of Management and Budget

John Endahl,¹ Jeff Wilde,¹ Joanne Guthrie,

Victor Oliviera¹

U.S. Department of Agriculture

Don Hernandez¹

U.S. Department of Commerce

Bureau of the Census

Marriage and Family Statistics

Tim D’Emillio

U.S. Department of Education, OELA

Naomi Karp,¹ Dave Malouf,¹ Ivor Pritchard,¹

Marsha Silverberg¹

U.S. Department of Education, IES

Pia Divine,¹ Esther Kresh,¹ Ivelisse Martinez-Beck, Ann Rivera

U.S. Department of Health and Human Services

Administration for Children, Youth, and Families

Gerry Hendershot,¹ John Kiley,¹ Michael Kogan,¹ Mitchell Loeb, Patricia Pastor

U.S. Dept. of Health and Human Services

National Center for Health Statistics

Howard Hoffman

U.S. Dept. of Health and Human Services

National Institute on Deafness and Other Communication Disorders, National Institutes of Health

Mary Frances Cotch

U.S. Dept. of Health and Human Services

National Eye Institute, National Institutes of Health

Christa Themann, William Murphy

Centers for Disease Control

National Institute for Occupational Safety and Health

Michael Planty, Jenna Truman

U.S. Department of Justice

Bureau of Justice Statistics

Tom Bradshaw,¹ Doug Herbert¹

National Endowment for the Arts

Jeffrey Thomas¹

National Endowment for the Humanities

Patricia McKee

U.S. Department of Education

OESE Compensatory Education Programs

Cathie L. Martin¹

U.S. Department of Education, OIE

Scott Brown,¹ Louis Danielson,¹ Glinda Hill,¹

Lisa Holden-Pitt,¹ Kristen Lauer,¹

Marlene Simon-Burroughs,¹Larry Wexler

U.S. Department of Education, OSEP

Jon Jacobson

U.S. Department of Education, NCEE

Lisa A. Gorove¹

U.S. Department of Education

OUS, Budget Service, ESVA

Elois Scott¹

U.S. Department of Education

OUS, PES, ESED

Richard Dean¹

U.S. Department of Education

OVAE, Adult Literacy

Jacquelyn Buckley

U.S. Department of Education

IES, NCSER

Jeff Evans,¹ Sarah Friedman,¹ Christine Bachrach,¹

Peggy McCardle¹

U.S. Department of Health and Human Services

NICHD, Center for Population Research

Jim Griffin and Regina Bures

U.S. Department of Health and Human Services

NICHD, National Institutes of Health

Martha Moorehouse,¹ Anne Wolf¹

U.S. Department of Health and Human Services

Office of Assistant Secretary for Planning & Evaluation, Children and Youth Policy

Katrina Baum¹

Department of Justice

Bureau of Justice Statistics

Meredith A. Miceli

U.S. Department of Education

Office of Special Education Programs

¹ Consultant for the ECLS-K only.

NOTE: Affiliation listed is the affiliation at the time input on the study was provided.

Table A-2. Other organization consultants for ECLS-K and ECLS-K:2011

Lynson Bobo

Project Associate

Resource Center on Educational Equity

Council of Chief State School Officers

Susan Bredekamp, Barbara Willer

National Association for the Education of Young Children

Jane Clarenbach

National Association for Gifted Children

Mary Jo Lynch

American Library Association

Office of Research and Statistics

Keith W. Mielkek

Children’s Television Workshop

June Million, Sally McConnell, Louanne Wheeler

National Association of Elementary School Principals

Evelyn Moore, Erica Tollett

National Black Child Development Institute

Thomas Schultz

Director, Center for Education Services for Young Learners

National Association of State Boards of Education

Larry Suter

Independent Education Consultant, Formerly of NSF and NCES

NOTE: Affiliation listed is the affiliation at the time input on the study was provided. Italicized text used for consultation that occurred for the ECLS-K:2011. All other consultations occurred for the ECLS-K.

Table A-3. ECLS-K:2011 first TRP meeting attendee list (November 2008)

Karl Alexander

Department of Sociology

Johns Hopkins University

Jim Bauman

Center for Applied Linguistics

Washington, DC

Maureen Black

Growth and Nutrition Department

University of Maryland Medical Center

Joanne Carlisle

School of Education

University of Michigan

Janet Fischel

State University of New York at Stony Brook & University Medical Center

Fred Morrison

Department of Psychology

University of Michigan

Charlotte Patterson

Department of Psychology

University of Virginia

Robert Pianta

The Center for Advanced Teaching and Learning

University of Virginia

Kit Viator

Massachusetts Department of Education

NOTE: Affiliation listed is the affiliation at the time input on the study was provided.

Table A-4. ECLS-K:2011 second TRP meeting attendee list (March 2011)

Karl Alexander

Department of Sociology

Johns Hopkins University

Jim Bauman

Center for Applied Linguistics

Washington, DC

Joanne Carlisle

School of Education

University of Michigan

Robert Crosnoe

Department of Sociology

University of Texas at Austin

David Dickinson

Department of Teaching and Learning

Vanderbilt University

Rolf Grafwallner

Maryland Public Schools

Greg Roberts

The Meadows Center for Preventing Educational Risk

University of Texas at Austin

Deborah Stipek

School of Education

Stanford University

NOTE: Affiliation listed is the affiliation at the time input on the study was provided.

Table A-5. ECLS-K:2011 third TRP meeting attendee list (November 2013)

Robert Bradley

Family & Human Dynamics Research Institute

Arizona State University

Carol Connor

Department of Psychology

Arizona State University

Robert Crosnoe

Department of Sociology and Population Research Center

University of Texas at Austin

David Dickinson

Department of Teaching and Learning

Vanderbilt University

George Farkas

School of Education

University of California, Irvine

Gary Ladd

Sanford School of Social and Family Dynamics

Arizona State University

Megan McClelland

Hallie E. Ford Center for Healthy Children & Families

Oregon State University

Greg Roberts

The Meadows Center for Preventing Educational Risk

University of Texas at Austin

Judy Snow

State Assessment Director

Montana Office of Public Instruction

NOTE: Affiliation listed is the affiliation at the time input on the study was provided.

Ten meetings of the CRP panels were also held: reading (May 2009), mathematics (May 2009), science (May 2009), English language learners (August 2009), executive function (November 2009; March 2011, December 2012), socioemotional development (March 2011; October 2012), and teacher practices (March 2011). For each of these specific content areas, panel members provided critical review of the instruments for inclusion in the national data collections. The meetings focused on the appropriateness and adequacy of specific instruments by considering features such as domain coverage, age appropriateness, and technical quality. Table A-6 lists the ECLS-K:2011 CRP members.

Table A-6. ECLS-K:2011 CRP member list, by panel

Reading Panel

Susan Conrad

Independent consultant, assessment development

Gloria Johnston

Education National University

Alba Ortiz

University of Texas at Austin

Barbara Wasik

Temple University

Mathematics Panel

Doug Clements

State University of New York, Buffalo

Donna Compano

Independent consultant, assessment development, math facilitator, elementary teacher

Lizanne DeStefano

University of Illinois at Urbana-Champaign

Leah Parker

Journeys Academy, Gifted Education Specialist

Science Panel

Christie Bean

JJ Ciavarra Elementary School

Kathy DiRanna

University of California - Irvine

Angela Eckhoff

Clemson University

Christine Y. O’Sullivan

Science Consultant

Michael Padilla

Clemson University

Table A-6. ECLS-K:2011 CRP member list, by panel (continued)

English Language Learners Panel

Jamal Abedi

University of California at Davis

Catherine Crowley

Teachers College

Eugene E. García

Arizona State University

Vera Gutierrez-Clellen

San Diego State University

Executive Function Panel

Clancy Blair

New York University

Adele Diamond (March 2011 meeting only)

University of British Columbia

Lisa Jacobson (December 2012 meeting only)

Kennedy Krieger Institute

Megan McClelland

Oregon State University

Philip Zelazo

University of Minnesota

Socioemotional Development Panel

Pamela Cole (March 2011 meeting only)

The Pennsylvania State University

Rick Fabes

Arizona State University

Karen Bierman (October 2012 meeting only)

The Pennsylvania State University

Allan Wigfield (October 2012 meeting only)

University of Maryland

Ross Thompson (March 2011 meeting only)

University of California, Davis

Carlos Valiente (March 2011 meeting only)

Arizona State University

Dorothy Espelage (October 2012 meeting only)

University of Illinois

Teacher Practices Panel

Stephanie Al Otaiba

Florida State University

Hilda Borko

Stanford University

Carol Connor

Florida State University

Barbara Wasik

University of North Carolina

NOTE: Affiliation listed is the affiliation at the time input on the study was provided.

A.9 Provision of Payments or Gifts to Respondents

Obtaining high response rates is critical for all longitudinal studies. At the start of the study, it was essential to establish the good will of respondents and to demonstrate that we value their participation in the study. Good will can be established by using well-designed respondent materials that inform respondents about the goals of the study and their role in it, the field staff establishing a rapport with the respondents, professionalism among the field staff, and a small token incentive. The same general incentive plan that was approved by OMB for the spring 2015 fourth-grade ECLS-K:2011 data collection is proposed for the spring 2016 fifth-grade data collection. The plan is designed to help respondents to recognize the merits of the study and thereby encourage high response rates.

As described below, we propose to provide monetary incentives to school staff, as has been done in prior rounds of data collection for the ECLS-K:2011. Parents and children will not receive any significant incentive, monetary or otherwise. As in the past, children will be given ECLS-K:2011 pencils with the sun logo that they use for the math portion of the assessment. In the fourth-grade round, we also gave children a multi-ink pen with the study logo printed on it. Since the fifth-grade round is the last planned data collection point and these study children have participated in assessments for many years, some of them twice a year, we propose providing children with a larger token as a thank you for their continued cooperation throughout the study. While we are still deciding on the token, it would be something that costs between $5 and $10, such as a duffel bag or educational game. The token would be mailed to the sampled children prior to the assessments. Distributing the gift outside of school will avoid potential resentment from non-study children who would not receive a gift. Also as in the fourth-grade round, we plan to send a set of ECLS-K:2011 post-it notes with the sun logo (included in appendix H) with the parent letter discussing the new round of data collection. The study is now entering its sixth year, and both parents and children have been asked to participate many times, some as many as nine times. These tokens of appreciation are being sent as a small gesture in an effort to maintain enthusiasm for and a positive attitude about the study. The parent response rates have consistently been lower than desired (between 67 percent and about 80 percent), so another goal of providing these small tokens of appreciation is to maintain the participation of parents who have partaken consistently in the past and encourage the participation of those who have not. Additionally, it can leave the children and families with a positive impression of their overall experiences with the study, which in general benefits the field of survey research.

A.9.1 School Incentives

High levels of school participation are integral to the success of the study. Without a school’s cooperation, there can be no school, teacher, or child data collection activity at that facility. NCES recognizes that administrators will assess the study’s burden level before agreeing to participate. To offset the perceived burden, NCES intends to continue its use of strategies that have worked successfully in the past for the ECLS-K:2011, the ECLS-K, and other major NCES studies (High School and Beyond, the National Education Longitudinal Study of 1988, and the Education Longitudinal Study of 2002). It is important to provide schools with an incentive because the study asks a lot of them, including allowing field staff to be in their schools for several days; providing a contact person and space for the children to be assessed and to have their hearing evaluated; removing children from their classes while they are assessed; and obtaining information about the school, the teachers, and the children.⁸ Given the many demands and outside pressures that schools face, it is essential that they see that we understand the burden we are placing on them and that we value their participation. As was done for the other ECLS-K:2011 data collections, we propose to remunerate schools $200 per school. An honorarium check in the amount of $200 will be mailed to each school with sampled children enrolled at the end of the spring fifth-grade data collection along with a thank you note thanking the school for its participation.⁹

A.9.2 School Administrator Incentives

To build response rates for the school administrator questionnaire, we propose to remunerate school administrators. In the ECLS-K, when no incentive was provided for administrators until the third-grade round of data collection, the field period had to be extended (for both kindergarten and first grade) to obtain response rates for the school administrator questionnaire that were closer to the desired rate of 85 percent or higher. Providing school administrators with an incentive will reduce the potential for needing to extend the field period and help avoid delays in data delivery. We will offer school administrators a $25 incentive in the spring fifth-grade collection, the same amount that was given to school administrators in the previous rounds of the ECLS-K:2011; the incentive will be attached to the questionnaire given to the school administrator to complete. In the spring third-grade round of the ECLS-K:2011, we offered school administrators a $25 incentive and a completion rate of 93 percent was achieved for the school administrator questionnaire.¹⁰

A.9.3 Teacher Incentives

In the base-year, first-, and second-grade collections of the ECLS-K:2011, teachers received $7 per child-level questionnaire because they were asked to provide a significant amount of information about each study child based on their observations of these students. A check for the incentive was attached to the package of instruments the teacher received each fall and spring. For the spring third-grade collection of the ECLS‑K:2011, OMB approved a change in the incentive structure to the model that was used in later rounds of the ECLS-K. This incentive plan was also approved by OMB and used successfully in the spring fourth-grade round. We propose to continue this incentive structure for the fifth-grade data collection as well because teachers are again being asked to provide a significant amount of key information about the study children’s school experiences and outcomes. General classroom teachers will still be given $7 per subject-specific TQC, along with an additional $20 associated with the teacher-level TQ. Also consistent with the fourth-grade collection, special education teachers will receive $7 for each child-level questionnaire and $20 for the teacher-level questionnaire. We are proposing to use the same incentive structure for all teachers, regardless of the specific questionnaires they are being asked to complete, to protect against any perception of unfairness that might result if teachers within a school talk to one another about the amount they have received for a specific questionnaire.

Based on what occurred in the ECLS-K, we expect that teachers will have on average two sampled children linked to them, resulting in a total remuneration of $34 ($7 each for each subject-specific TQC and $20 for the teacher-level TQ). The estimate for special education teachers is the same. Incentive checks will be attached to the package of instruments each teacher receives.

A.9.4 School Coordinator Incentives

School coordinators act as the study liaison between study staff and their school and, as such, they play a very important role in the ECLS-K:2011.¹¹ They communicate necessary information to parents, notify teachers of their role in the study and encourage their participation, arrange the assessment logistics (e.g., space to conduct the assessments), and collect hard-copy teacher and school administrator questionnaires. For this reason, school coordinators will be offered a $25 incentive for providing assistance to the study in the spring fifth-grade data collection.¹² The $25 checks will be attached to the packets mailed to the coordinators at the start of data collection. The study offered the same incentive to the school coordinators during the previous rounds of ECLS-K:2011.

A.10 Assurance of Confidentiality

The ECLS-K:2011 plan for protecting confidentiality of the project participants conforms with the following federal regulations and policies: the Privacy Act of 1974 (5 U.S.C. 552a), Privacy Act Regulations (34 CFR Part 5b), the Education Sciences Reform Act of 2002 (20 U.S. Code Section 9573), the Computer Security Act of 1987, the NCES Restricted-Use Data Procedures Manual, and the NCES Standards and Policies.

All adult respondents who are participating in research under this clearance are informed that the information they provide will be protected from disclosure except as required by law (20 U.S. Code Section 9573) and that their participation is voluntary. All adult respondents receive an introductory letter that explains NCES’s and the contractor’s adherence to policies on disclosure.¹³ Also, this information appears on the cover of each of the study self-administered questionnaires. This information was provided to parents as the guardians for their children when their cooperation was sought during the base year of the study.

Since early spring 2010 (when recruitment for the kindergarten data collections began), information about the protection of data from disclosure has been conveyed to state, district, and other school officials at the time their cooperation for the study was sought. As sampled children move to new schools, this information is provided to the districts in which those schools are located, if necessary (i.e., if there are no participating schools in those states and districts already). New schools in the study will receive the letter developed for schools to which sampled children transfer that can be found in appendix H of this clearance request, as well as the study brochure that was approved in May 2010 (see appendix H of OMB No. 1850-0750 v. 8).

Respondent letters to parents summarize the data protection assurances; namely, that data will be combined to produce statistical reports, that no data will be published that link the respondent to his/her responses; that participation is voluntary; and that there is federal statute that protects the data from disclosure except as required by law (20 U.S. Code Section 9573).

All contractor staff members working on the ECLS-K:2011 project or having access to the data (including monitoring of interviews and assessments) are required to sign an NCES Affidavit of Nondisclosure and a Confidentiality Pledge. They also are required to complete mandatory training on data confidentiality and the safe handling of data. The contractor will keep the original notarized affidavits on file and submit PDF copies of all affidavits to NCES quarterly. In addition, contractor staff will complete background screening in compliance with ACS Directive (OM:5-101).

During the course of data collection, interviewers will be equipped with laptop computers, which store any necessary preloaded data, as well as the information collected on a given day during the data collection round. The interviewers will be instructed to keep the computers and any hard-copy case materials in a secure place in their homes when they are not being used. When the interviewer is in the field collecting interview or assessment data, he or she is instructed to keep all materials and the computer in his/her possession at all times. When driving a car to or from his/her appointments, the computer and all materials will be locked out of sight, so as not to provide an inviting opportunity for burglary. The interviewers will be instructed to transmit the electronic data for a case to a central database on the same day the case is completed. Data transmitted electronically will be encrypted during transmission. The laptop configuration is designed with security and confidentiality considerations in mind. In order to access any of the applications, the interviewer must enter a project-specific password and an interviewer identification code, both of which are checked against encrypted versions of the same data; if the password or interviewer identification code is entered incorrectly repeatedly, the interviewer is “locked out” of the application. All data files will be encrypted on the computer hard disk. In the event of a hardware failure in the field, the home office will swap the interviewer’s laptop for a new one. The contractor will maintain a supply of “hot spares,” i.e., laptop computers loaded with all necessary ECLS-K:2011 software, which require only the specific interviewer’s identification code and assignment before being sent out.

All mailing of respondent materials, laptops, and hard-copy case materials used by assessors to manage their workload will be done using Federal Express, which has a sophisticated tracking system designed to locate any misdirected packages. All packages will require the recipient’s signature for delivery. To the extent practical, the study name and logo will not be included on hard copy materials used by field staff to record school or respondent information. In the event of a loss of hard copy materials, this procedure would make it more difficult for someone who finds the materials to associate a school or respondent with the study.

In addition, as in the fourth-grade data collection round, the ECLS-K:2011 Secure Message System will be used to share materials containing sensitive information (e.g., children’s names) between the field staff and school staff. In earlier rounds of the ECLS-K:2011 the list of participating children was sent separately from all other study materials via Federal Express and contained no study identifying information. With the ECLS-K:2011 Secure Message System, this list of participating children is shared electronically, rather than in hard copy. The system does not allow for the list to be printed or forwarded to other staff, enhancing the confidentiality of the materials.

Finally, all computer assisted interviewing (CAI) applications will have an audit trail of the case data on the hard disk, so that if the main data files are corrupted, the data can be reconstructed from the audit trails.

After data collection, all personally identifiable information will be stored on a secure server and password protected with access limited to authorized project staff. Personally identifiable data will also be protected through the coding of responses so that no one individual respondent can be identified (specifically or by deduction) through reported variables in the public access data files. NCES will monitor the conduct of the contractor to ensure that the confidentiality of the data is not breached.

NCES understands the legal and ethical need to protect the privacy of the ECLS-K:2011 survey respondents and, with the contractor, has extensive experience in developing data files for release that meet the Government’s requirements to protect individually identifiable data from disclosure. The contractor will conduct a thorough disclosure analysis of the ECLS-K:2011 data when preparing the data files for researchers’ use. This analysis will ensure that NCES has fully complied with the confidentiality provisions contained in 20 U.S. Code, Section 9573. To protect the privacy of respondents as required by 20 U.S. Code, Section 9573, respondents with high disclosure risk will be identified, and a variety of masking strategies will be used to ensure that individuals may not be identified from the data files. These masking strategies include:

Swapping data on both the public- and restricted-use files;
Omitting key identification variables such as name, address, telephone number, and school name and address from both the public- and restricted-use files (though the restricted-use file will include NCES school ID that can be linked to other NCES databases to identify a school);
Omitting key geographic identification variables such as state or ZIP Code from the public-use file;
Collapsing categories or developing categories for continuous variables to retain information for analytic purposes while preserving confidentiality in public-use files; and
“Topcoding” and “bottomcoding”¹⁴ continuous variables in public-use files.

Exhibit A-2. Confidentiality pledge

EMPLOYEE OR CONTRACTOR’S ASSURANCE OF CONFIDENTIALITY OF SURVEY DATA

Statement of Policy

{Contractor} is firmly committed to the principle that the confidentiality of individual data obtained through {Contractor} surveys must be protected. This principle holds whether or not any specific guarantee of confidentiality was given at time of interview (or self-response), or whether or not there are specific contractual obligations to the client. When guarantees have been given or contractual obligations regarding confidentiality have been entered into, they may impose additional requirements which are to be adhered to strictly.

Procedures for Maintaining Confidentiality

1. All {Contractor} employees and field workers shall sign this assurance of confidentiality. This assurance may be superseded by another assurance for a particular project.

2. Field workers shall keep completely confidential the names of respondents, all information or opinions collected in the course of interviews, and any information about respondents learned incidentally during field work. Field workers shall exercise reasonable caution to prevent access by others to survey data in their possession.

3. Unless specifically instructed otherwise for a particular project, an employee or field worker, upon encountering a respondent or information pertaining to a respondent that s/he knows personally, shall immediately terminate the activity and contact her/his supervisor for instructions.

4. Survey data containing personal identifiers in {Contractor} offices shall be kept in a locked container or a locked room when not being used each working day in routine survey activities. Reasonable caution shall be exercised in limiting access to survey data to only those persons who are working on the specific project and who have been instructed in the applicable confidentiality requirements for that project.

Where survey data have been determined to be particularly sensitive by the Corporate Officer in charge of the project or the President of {Contractor}, such survey data shall be kept in locked containers or in a locked room except when actually being used and attended by a staff member who has signed this pledge.

5. Ordinarily, serial numbers shall be assigned to respondents prior to creating a machine-processible record and identifiers such as name, address, and Social Security number shall not, ordinarily, be a part of the machine record. When identifiers are part of the machine data record, {Contractor’s Manager of Data Processing} shall be responsible for determining adequate confidentiality measures in consultation with the project director. When a separate file is set up containing identifiers or linkage information which could be used to identify data records, this separate file shall be kept locked up when not actually being used each day in routine survey activities.

6. When records with identifiers are to be transmitted to another party, such as for keypunching or key taping, the other party shall be informed of these procedures and shall sign an Assurance of Confidentiality form.

7. Each project director shall be responsible for ensuring that all personnel and contractors involved in handling survey data on a project are instructed in these procedures throughout the period of survey performance. When there are specific contractual obligations to the client regarding confidentiality, the project director shall develop additional procedures to comply with these obligations and shall instruct field staff, clerical staff, consultants, and any other persons who work on the project in these additional procedures. At the end of the period of survey performance, the project director shall arrange for proper storage or disposition of survey data including any particular contractual requirements for storage or disposition. When required to turn over survey data to our clients, we must provide proper safeguards to ensure confidentiality up to the time of delivery.

8. Project directors shall ensure that survey practices adhere to the provisions of the U.S. Privacy Act of 1974, and any additional relevant laws that are specified in the contract, with regard to surveys of individuals for the Federal Government. Project directors must ensure that procedures are established in each survey to inform each respondent of the authority for the survey, the purpose and use of the survey, the voluntary nature of the survey (where applicable), and the effects on the respondents, if any, of not responding.

PLEDGE

I hereby certify that I have carefully read and will cooperate fully with the above procedures. I will keep completely confidential all information arising from surveys concerning individual respondents to which I gain access. I will not discuss, disclose, disseminate, or provide access to survey data and identifiers except as authorized by {Contractor}. In addition, I will comply with any additional procedures established by {Contractor} for a particular contract. I will devote my best efforts to ensure that there is compliance with the required procedures by personnel whom I supervise. I understand that violation of this pledge is sufficient grounds for disciplinary action, including dismissal. I also understand that violation of the privacy rights of individuals through such unauthorized discussion, disclosure, dissemination, or access may make me subject to criminal or civil penalties. I give my personal pledge that I shall abide by this assurance of confidentiality.

Signature

A.11 Sensitive Questions

The ECLS-K:2011 is a voluntary study, and no persons are required to respond to the interviews and questionnaires or to participate in the assessments. In addition, respondents may decline to answer any question they are asked. This voluntary aspect of the survey is clearly stated in the advance letter mailed to adult respondents, the study brochure,¹⁵ and the instructions of hard-copy questionnaires, and it is stressed in interviewer training to ensure that interviewers are both communicating this to participants and following these guidelines. Additionally, assessors are trained that children may refuse to participate at the time they are visited for an assessment and staff are to respect the children’s wishes. Also, prior to the start of the child questionnaire, children are instructed that if they wish to skip a particular question, they should touch the “Next” button displayed on the touchscreen laptop’s screen without choosing a response. The following describes the general nature of the national data collection instruments that will be used during the spring fifth-grade data collection, as well as topics that may be sensitive for some respondents.

School Administrator Questionnaires. The items in the School Administrator Questionnaire are not of a sensitive nature and should not pose sensitivity concerns to respondents.

Teacher Questionnaires. The information collected in the subject-specific questionnaires could be regarded as sensitive, because the teacher is asked to provide information about children’s social skills (including ability to exercise self-control, interact with others, resolve conflict, and participate in group activities); problem behaviors (e.g., fighting, arguing, anger, depression, low self-esteem, impulsiveness); learning dispositions (e.g., curiosity, self-direction, inventiveness); liking or avoiding school; relationships with peers; and experiences with peer victimization, both as a victim and as the aggressor. A study of bullying, a construct closely related to peer victimization, by the National Institute for Child Health and Human Development (NICHD) found that 16 percent of middle school students reported being bullied (Nansel et al. 2001). Fewer studies have been done with younger children, but those that have been published suggest that bullying is experienced by many children and is related to negative outcomes. Glew et al.’s (2005) study of third through fifth graders found that 22 percent of children were classified as victims, bullies, or both. Victims, and children who were both bullies and victims, had lower achievement scores and were more likely to feel like they did not belong at school compared to bystanders (Glew et al. 2005). Kochenderfer and Ladd (1996) found a relation between victimization and school adjustment outcomes, with victimization related to children’s loneliness and desire to avoid school. Given these findings and the White House anti-bullying initiative, having the ECLS-K:2011 collect information about peer victimization for this national sample of elementary school children would be useful. All of the items in these questionnaires have been administered in one or more prior rounds of data collection for the study.

Within the set of questions about the teacher’s views on school climate and the school environment, there are some questions that could be deemed sensitive by some teachers. Teachers may feel that rating statements regarding their satisfaction with their work (e.g., I really enjoy my present teaching job) are sensitive in nature. These items are included because prior research (e.g., Perrachione, Rosser, and Peterson 2008; Luekens, Lyter, and Fox 2004; Rhodes, Nevill, and Allen 2004) indicates that teacher satisfaction may be associated with relevant constructs such as staff retention and stability. Prior to their participation, teachers will be informed and assured that their information will be protected from disclosure except as required by law and that their responses will not be shared with their employers or the parents of their students. Also, teachers and school administrators will be given an envelope in which they can place their completed questionnaire and seal it before returning their questionnaire to the school coordinator. All of the items in these questionnaires have been administered in one or more prior rounds of data collection for the study.

Direct Cognitive Assessments. The direct cognitive assessments are essential in determining children’s performance levels as they progress through school. Because schools often use different standards in their own assessments of children and a uniform set of assessment instruments and procedures is needed for the ECLS-K:2011, school-developed assessments cannot be used in the ECLS-K:2011. The items to be included in the ECLS-K:2011 reading, math, and science assessments undergo a sensitivity review and are not themselves sensitive in nature. Similarly, the executive function assessment is not sensitive in nature. However, direct assessments of children do raise certain concerns about the assessment procedures to be used. Of primary concern is the length of the assessments. The cognitive assessments, while untimed, are designed to be administered on average within a 60-minute time period. The child questionnaire is designed to be administered in 10 minutes and measurement of height and weight adds another 5 minutes to the total child assessment time. Children who participate in the hearing evaluations will have an additional 20 minutes of assessment time. NCES has developed instruments appropriate to the ages of the participating children, and every effort will be made to staff the study with field assessors who have prior experience in working with children. Issues specific to working with children also figure prominently in assessor training so that the field staff can respond appropriately to children who may become upset or frustrated by the assessment. Also, this is the last round of data collection currently planned for the study, and it is important to collect a final data point for outcomes measured in previous rounds.

Child Questionnaire (CQ). Some of the questions contained in the child questionnaire may be deemed sensitive, particularly those related to fear of negative evaluation, social distress, peer victimization, and worry or stress about school. These types of items were added to the child questionnaire at the recommendation of the October 2012 CRP and November 2013 TRP. Some of the items added to the fifth-grade child questionnaire were also used in the ECLS-K.

Based on CRP and TRP recommendations, items measuring fear of negative evaluation that came from a longer social anxiety scale were included in the third- and fourth-grade child questionnaires and will be used again in the fifth-grade round. In one study, children reporting a high level of fear of negative evaluation using items from this social anxiety scale self-reported lower perceived social acceptance and lower global self-worth (La Greca and Stone 1993). The TRP also recommended adding social distress items from a scale measuring loneliness (Asher, Hymel, and Renshaw 1984) because the scale taps feelings of peer rejection and connectedness to the school social environment. These social distress items were also used in the fourth-grade child questionnaire.

The child-reported items measuring peer victimization were used in the third- and fourth-grade questionnaire and are proposed again for the fifth-grade child questionnaire. These items mirror the items that were fielded in the second-, third-, and fourth-grade parent and teacher instruments (and are included in the fifth-grade teacher instruments), thus allowing researchers to analyze the relationship between children’s own report of peer victimization and the perceptions of peer victimization as reported by adults. Members of the March 2011 CRP panel recommended this approach and suggested the items, which are adapted from a scale developed for this age group (Espelage and Holt 2001).

Child-reported items measuring worry or stress about school were added to the fifth-grade child questionnaire. These items were used in the third-, fifth-, and eighth-grade child questionnaires for the ECLS‑K. Administering these items in the ECLS-K:2011 will allow researchers to examine differences between the cohorts in students’ perception of worry or stress about school work.

As with other respondents, children will be told that they can skip any question(s) they do not wish to answer as part of the instructions for completing the questionnaire.

Parent Interviews. Several topics that will be addressed in the spring fifth-grade parent interview could be sensitive in nature for some respondents. Questions about family income, marital/partner satisfaction, parental depression, child-rearing and disciplinary practices, parent-child conflict, children’s friends, whether the child avoids school, children’s disabilities, children’s receipt of tutoring, children’s country of origin, and contact with a child’s nonresidential parent will be included in the parent interview. All of these questions have been asked in earlier rounds of the ECLS-K:2011 and will provide another time point in the study for information on these topics. Some of these questions were also included in cognitive interviews for the spring fourth-grade parent interview and no parents indicated that they felt the questions were too sensitive to answer. Results from the ECLS-K:2011 showed that there were low levels of missing data in the parent interviews for these items. For example, in the spring kindergarten round of the ECLS-K:2011, response rates for sensitive items such as family income and marital/partner satisfaction were in the 90’s (90.5 percent and 93.6 percent, respectively).

Prior research indicates that the topics in the parent interview are correlated with children’s achievement and help to predict children’s preparedness for and success in school. Collecting data on these topics will allow researchers to go beyond descriptive analyses of variation in children’s performance by basic background characteristics such as race/ethnicity and sex. Researchers will be able to test hypotheses about how a wide range of family characteristics relate to early success in school. Therefore, it is important to include questions in the parent interviews on the potentially sensitive topics listed above. Like other study participants, parents will be told that they can refuse to answer any question they wish.

A.12 Estimated Response Burden

Table A-7 outlines the estimated respondent burden for data collection activities for which this submission is requesting approval (the national spring fifth-grade data collection). Included in these estimates, where appropriate, is the time that a respondent would need to gather and compile the data and the clerical time needed to fill out the form. The spring fifth-grade national data collection includes direct cognitive assessments and self-administered questionnaires with children; measurements of children’s height and weight; evaluations of children’s hearing; parent interviews; self-administered teacher-level questionnaires for teachers; reading, math, and science teacher child-level self-administered questionnaires for teachers; special education teacher teacher-level self-administered questionnaires; special education teacher child-level self-administered questionnaires; and school administrator self-administered questionnaires.

Table A-7 also outlines respondent burden for recruitment for spring fifth-grade, which was approved in a previously cleared package (1850-0750 v.16 and 17). The table includes 5 minutes per parent respondent to read the birthday cards the study sends to children to keep in touch with them and obtain updated address information (see appendix H for a the birthday cards). Recruitment burden time includes the time necessary to read study materials sent to parents, teachers, and school administrators; time during which teachers would discuss the study with a data collection staff member; and time the school administrator will take discussing the study with a school recruiter attempting to secure the school’s participation.

The total number of respondents across all of the data collection activities listed in table A-7, i.e., school administrators, teachers, school coordinators, and parents, is estimated to be 38,220.¹⁶ Children are also included in the total number of respondents, as they complete the Child Questionnaire. The estimated respondent time burden across all these activities for which clearance is currently being sought translates into a cost amount of $540,599 for 21,140 hours.^¹⁷The cost for estimated carryover respondent time burden is $450,045 for 14,968 hours. Thus, total estimated cost for all activities in table A-7 is $990,644, The time children will spend completing the Child Questionnaire and participating in the hearing evaluations has been included in the estimated burden, although the time children will spend completing the cognitive assessments has not been included in the estimated burden.

Table A-7. Estimated respondent burden for the national spring fifth-grade data collection and previously cleared fifth-grade tracking and recruitment activities

Respondent type	Sample n	Response rate/ selection rate	Number of respondents	Hours per instrument	Instruments per respondent	Number of responses	Total burden hours¹
Spring-Fifth National Data Collection
Direct Assessment	11,429	.90	10,286	1.33	1	10,286	13,680
Child Questionnaire	11,429	.90	10,286	0.17	1	10,286	1,749
Hearing Evaluations	3,225	.90	2,903	0.33	1	2,903	958
Parent Interview	11,429	.90	10,286	0.75	1	10,286	7,715
School Administrator Questionnaires (SAQ)	3,490	.90	3,141	1.00	1	3,141	3,141
Teacher-level Questionnaire (TQ)	8,355	.90	7,520	0.21	1	7,520	1,579
Teacher Child-level Reading Questionnaire (TQR)- key child	5,715	.90	5,144	0.43²	1	5,144	2,212
Teacher Child-level Reading Questionnaire (TQR)- additional child	5,715	.90	5,144	0.20	1	5,144	1,029
Teacher Child-level Mathematics Questionnaire (TQM)- key child	2,857	.90	2,571	0.22²	1	2,571	566
Teacher Child-level Mathematics Questionnaire (TQM)- additional child	2,857	.90	2,571	0.03	1	2,571	77
Teacher Child-level Science Questionnaire (TQS)-key child	2,857	.90	2,571	0.20²	1	2,571	514
Teacher Child-level Science Questionnaire (TQS)-additional child	2,857	.90	2,571	0.03	1	2,571	77
Special Education Teacher-level Questionnaire (SPA)	1,326	.90	1,193	0.25	1	1,193	298
Special Education Teacher Child-level Questionnaire (SPB)	2,652	.90	2,387	0.25	1	2,387	597
School Coordinator Assistance³	3,490	.90	3,141	0.20	NA⁵	3,141	628
Tracking for Spring Fifth-Grade
Parent	11,406	1.0	11,406	.084	1	11,406	958
School Coordinator	3,490	1.0	3,490	1.00	1	3,490	3490
Recruitment for Spring Fifth-Grade
Parent	11,406	1.0	11,406	0.25	1	11,406	2,852
Teacher	8,355	1.0	8,355	0.50	1	8,355	4,178
School Administrator	3,490	1.0	3,490	1.00	1	3,490	3,490
Total Burden Requested in this Submission	-	-	35,567	-	-	61,429⁶	21,140⁷
Total Carry-over Burden for 5^th-grade Tracking and Recruitment	-	-	26.741	-	-	38,147	14,968
Total	-	-	38,220⁴	-	-	99,576⁶	36,108⁷

NA Not applicable

¹ Calculations are based on rounded numbers.

² The teacher burden for the subject-specific questionnaires is reported at the child level to accurately reflect the teacher burden. Based on the results of the spring 2014 teacher timings tests, the assumption of the burden for reading teachers is that it will take them an average of 30 minutes to complete the questionnaire for the key child and 12 minutes to complete the questionnaire for an additional child (for a total of 42 minutes to complete two reading questionnaires). The assumption for mathematics teachers is that it will take them an average of 13 minutes for the key child’s questionnaire and 2 minutes for an additional child’s questionnaire (for a total of 15 minutes for two mathematics questionnaires). The assumption for science teachers is that it will take them an average of 12 minutes for the key child’s questionnaire and 2 minutes for an additional child’s questionnaire (for a total of 14 minutes for two science questionnaires).

³ School coordinators are school staff members who help organize the logistics for the assessment visit. They do not complete a study instrument.

⁴ The total number of respondents contains no duplication of the number of listed instruments each respective respondent is asked to complete. Shaded numbers do not contribute to the calculation of the total. Included in the total are children (for the Child Questionnaire only), parents, school administrators, teachers (both regular and special education), and school coordinators. The number of students participating in the hearing evaluations is not included in the total number of respondents because these students are also participating in the CQ and, therefore, are already counted as part of the total count. Teachers will complete the TQ and subject-specific TQCs; reading teachers of all sampled children will complete questionnaires, half the sampled children’s mathematics teachers will complete questionnaires, and half the children’s science teachers will complete questionnaires. One special education teacher completes both SPA and SPB. The sample of students taking the direct assessment is not included in this count because it is not subject to the Paperwork Reduction Act reporting.

⁵ NA (Not Applicable) is used in this cell to indicate that the school coordinator does not complete questionnaires in the study.

⁶ The sample of students taking the direct assessment is not included in the total number of responses count because it is not subject to the Paperwork Reduction Act reporting.

⁷ The sample of students taking the direct assessment is not included in the total burden hours count because it is not subject to the Paperwork Reduction Act reporting.

NOTE: Information in the table that appears in gray font (i.e., tracking and recruitment for the spring-fifth grade data collection) pertains to activities and burden that were approved in a previously cleared package. It is included here because these activities will continue after this current submission is cleared. Shaded numbers do not contribute to calculation of the totals.

A.13 Estimates of Cost to Respondents

There are no costs to the respondents to participate beyond the time needed for school coordinators to act as a liaison with the school, for parents to answer the interview questions, for teachers and school administrators to complete the questionnaires, and for the children to participate in the assessments and hearing evaluation. No equipment, printing, or postage charges will be incurred by the participants.

A.14 Cost to the Federal Government

The data collection activities for the fifth-grade data collections are being carried out under NCES contract ED-IES-14-C-0119 with Westat. The period of performance for this ECLS-K:2011 contract, which includes the spring fifth-grade national data collection, runs from September 2014 through August 2018. The total cost to the Government for contractor and subcontractor costs for this contract is $16,720,340. This cost estimate includes all spring-fifth grade data collection activities, design enhancements, and data file delivery and documentation. Table A-8 provides the study costs by year of the contract for the fifth-grade data collection.

Table A-8. Study costs per year of the contract for the fifth-grade data collections

Year	Amount
Sept. 2014-August 2015	$1,160,059
Sept. 2015-August 2016	$12,259,398
Sept. 2016-August 2017	$2,834,544
Sept. 2017-August 2018	$467,039
Total	$16,721,040

A.15 Reasons for Changes in Response Burden and Costs

The apparent decrease in the burden requested for this collection as compared to the burden last approved (OMB# 1850-0750 v. 16 and 17) is due to the fact that the last approval was for the fourth grade data collection, tracking, and recruitment, plus fifth grade tracking and recruitment, while this request is for the fifth grade data collection, tracking, and recruitment only. Also, the number of school coordinators estimated for inclusion in the fifth-grade tracking effort has been adjusted to a slightly lower number due to a revised estimate of the number of schools that will be included in the fifth-grade sample.

A.16 Publication Plans and Time Schedule

Information relevant to the data collection will be part of the reports resulting from the spring fifth-grade data collection. A data file with data from the fifth-grade collection will be produced and made available to researchers in a public-use format. Also produced from the fifth-grade collection will be a restricted-use data file. Researchers who are approved by NCES’s data confidentiality office for a restricted-use license can access restricted-use data files, which include more sensitive items and items that pertain to smaller numbers of children (e.g., information about the presence of specific disabilities). To be approved for a restricted-use license, researchers must demonstrate that they have a research question that cannot be answered with the public-use data and that they have the infrastructure to keep the data secure to prevent loss or unauthorized use. Codebooks and user’s manuals will be produced for use with the public- and restricted-use data files. All data will be merged at the child level. Data files will include all instrument variables (except for those that gather directly identifying information, such as the names of household members) and relevant associated variables, such as derived variables and assessment scores. Data will be released through Electronic Codebook (ECB) software that allows users to create customized data files in standard statistical software packages (SPSS, SAS, and Stata) and to view codebook information. A file record layout will also be provided so that analysis packages other than SPSS/PC, SAS/PC, and Stata/PC (e.g., analysis packages for Apple computers) can be used to analyze the ECLS-K:2011 data.

The ECLS-K:2011 reports and publications will include detailed methodological reports describing all aspects of the data collection effort and psychometric reports outlining properties of the study instruments, as well as reports that describe the population of children who were kindergartners in the 2010-11 school year as they progress through school.

The operational schedule for the ECLS-K:2011 spring fifth-grade data collection is shown in table A-9. Table A-9 also shows the operational schedule for the tracking and recruitment activities in the spring of fifth grade, which were approved in previous clearance requests (1850-0750 v.15-16).

Table A-9. Operational schedule for ECLS-K:2011 data collection activities

Activity	Start date	End date
Sample Tracking and Recruitment for Spring Fifth-Grade Data Collection
Mail birthday cards	6/1/2015	6/1/2016
Pre-assessment call	8/19/2015	12/11/2015
Tracking movers and updating field management system	8/19/2015	12/11/2015
Parent, teacher, school administrator, school coordinator mailings	2/15/2016	4/15/2016
ECLS-K:2011 Spring Fifth-Grade Data Collection
Identify and subsample movers¹	8/31/2015	12/16/2015
Print/program assessment	7/27/2015	12/2/2015
Print/program questionnaires	10/23/2015	1/25/2016
Train data collectors	3/8/2016	3/13/2016
National data collection	3/16/2016	7/15/2016
Process data	3/17/2016	8/12/2016
Construct data files, develop user’s manual	8/15/2016	7/14/2017
Methodology/psychometric reports	10/19/2016	12/14/2018

¹ Activities for identifying and subsampling movers were approved in a previous OMB package.

NOTE: Information in the table that appears in gray text (i.e., sample tracking and recruitment for fifth-grade data collection) pertains to activities and burden that were approved by OMB in a previous package. It is included here because burden for these activities is being carried over since the activities have not yet been completed.

A.17 Approval for Not Displaying the Expiration Date for OMB Approval

No exemption from the requirement to display the expiration date for OMB approval of the information collection is being requested for the ECLS-K:2011.

A.18 Exceptions to the Certification Statement

No exceptions to the certification statement apply to the ECLS-K:2011.

1Throughout this package, reference is made to the Early Childhood Longitudinal Study, Kindergarten Class of 1998-99. For ease of presentation, it will be referred to as the ECLS-K. The new study for which this submission requests approval is referred to as the ECLS-K:2011.

2At each follow-up stage, a small percentage of children had been retained in a grade at some point prior to the wave of interest and therefore were in a grade lower than the target grade of that follow-up stage. In addition, a small number of children were found to be advanced to a higher grade. These off-grade students were not excluded from the study.

3Since the study began, some children have been retained in a grade lower and some children have been advanced to a grade higher than the modal grade of the study’s children. While the study refers to the data collection rounds by the modal grade for most children in the cohort at the time of data collection, children are still included in the study even if they are in grades other than the modal grade due to retention or advancement.

4In kindergarten, the science assessment had just one stage.

5 U.S. Department of Education press release, 6/24/2014. New accountability framework raises the bar for state special education programs. Retrieved 5/25/2015 from http://www.ed.gov/news/press-releases/new-accountability-framework-raises-bar-state-special-education-programs.

6The questions about school characteristics may be completed by a designee, but the study requests that the administrator complete the section about his/her own characteristics and background.

7 As described in http://www.corestandards.org/about-the-standards/development-process/ as of May 22, 2014.

8 In a few cases, a field staff member may need to visit the school on an additional day or two, for example if a child is absent on the date(s) of the assessment and a make-up day is scheduled.

9Remuneration will not be provided to schools into which study children have transferred if those schools are not attended by at least four ECLS-K:2011 study children. Because only a few children will be assessed in most of these transfer schools, the burden on the school is minimal. For example, fewer field staff will visit the school, a smaller assessment space can be used, and likely only one classroom and fewer teachers are asked for assistance. School administrators, teachers, and (if applicable) special education teachers will still be remunerated for the completion of the hard-copy questionnaires in transfer schools as they are in other study schools.

10Because final reconciliation of the spring-fourth grade data collection has not yet been completed, the response rate for the spring-third grade round has been provided here.

11The school coordinator will often be the same school staff member who served in this role during a previous round of data collection. If that person is not available, then a new staff member will be identified by the school administrator to act as a liaison to the study.

12Remuneration will not be provided to school coordinators in schools into which study children have transferred since kindergarten if those schools are not attended by at least four ECLS-K:2011 study children. This is because of the lower burden on these school coordinators, as opposed to schools in which more sampled children attend.

13Spring fifth-grade recruitment materials were approved in a previous clearance package – OMB# 1850-0750 v.16.

14Topcoding and bottomcoding refer to the process of recoding outlier values to some acceptable end value. For instance, everyone with a personal income higher than $200,000 may be recoded to $200,001 or more to eliminate the outliers.

15The study brochure was approved in a previous OMB clearance package (OMB No. 1850-0750 v. 8).

16Schools are asked to assign a staff member to help coordinate the assessment activities at the school; these school coordinators are counted in the total number of respondents and their burden hours are counted. However, school coordinators do not complete any study instruments as part of their role as coordinator.

17Total estimated cost was calculated using average hourly earnings from the 2014 National Compensation Survey sponsored by the Bureau of Labor Statistics (BLS) for all respondents except the study child. The average hourly earnings are as follows: $22.71 for parents, $27.32 for elementary school teachers, $27.47 for elementary school special education teachers, and $44.13 for education administrators. The school staff member who functions as the school coordinator may be a teacher, an administrator, or a member of the school support staff. Given the variability in wages for these groups of school staff, teacher wage, which falls between administrator wage and support staff wage, was used as the wage for school coordinator. If mean hourly wage was not provided, it was computed assuming 2,080 hours per year. Study child/student wage is based on the federal minimum wage. Source: BLS Occupation Employment Statistics, http://data.bls.gov/oes/ datatype: Occupation codes: All employees (00-0000); Elementary school teachers (25-2021); Special education teachers, Kindergarten and Elementary School (25-2052); and Education Administrators, Elementary and Secondary School (11-9032); accessed on July 14, 2015.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	2021-01-24

Part A ECLSK-2011 Spring 5th Grade