International Early Learning Study (IELS) 2018
Field Test Recruitment
Supporting Statement Part A
Submitted by:
National Center for Education Statistics (NCES)
U.S. Department of Education
Institute of Education Sciences
Washington, DC
December 2016
Revised February 2017
TABLE OF CONTENTS
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 1
B.2 Procedures for the Collection of Information 1
B.3 Maximizing Response Rates 3
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
Part B of this submission presents information on the collection of information employing statistical methods for recruiting for and conducting the International Early Learning Study (IELS) field test in 2017. A request to administer the field test will begin clearance in March 2017, including the overarching plan for the 2018 main study sample. Please note that the sampling methodology provided in this document may be subject to change upon award of the IELS 2018 national data collection contract. Any revisions will be submitted to OMB for approval as a change request as soon as they become available.
The IELS assesses children just prior to entering primary school. For international comparability, this is defined as students who are five to five-and-a-half years old enrolled in school. For the United States, the expectation is that children enrolled in kindergarten, in public and private schools, will be assessed near the beginning of the kindergarten school year. This is an important transition point for U.S. children, and it is also the point at which it is feasible to reach at least 95 percent of U.S. children through a school-based sampling approach. The universe for the selection of schools for IELS is all types of schools in all states of the 50 United States and the District of Columbia. Within sampled schools, students will be selected for participation by drawing a random sample among kindergarten children that are the target age.
Schools
International standards do not require a formal probability sample of schools for the IELS field test. It is sufficient that the samples of schools be representative of a broad range of schools from across the U.S. Schools must be officially registered institutions that provide services for children of the target age for at least 2 hours per day and 100 days a year. The U.S. expects that 30 schools will participate in the field test.
For the IELS field test, schools will be selected for recruitment from the most recent available universe files, the 2012-13 Common Core of Data (CCD), to develop the public school list, and from the 2012-13 Private School Universe Survey (PSS) to develop the private school list. The less restrictive sampling requirements of the IELS field test mean that schools that require extensive research applications may not be included.
To obtain a school sample that is broadly representative of schools across the U.S., we will target a convenience sample of schools that offer kindergarten. We will use the sample stratification characteristics used in other studies of this type (e.g., PISA) including census region, locality (city/urban fringe/town/rural MSA), school type (public/private), grade span, and minority enrollment.
Students
The OECD IELS field test requirements call for a minimum student sample of 450 students, and we estimate that approximately 476 will participate. In each school in the field test, 19 students in the target population will be randomly sampled from a list of all target-age students provided by the school in the field test, with the expectation that about 16 will participate. Depending on each school’s policy, either active (explicit) or passive (implicit) parental consent materials will be distributed to the sampled students within each school.
This section presents information on the recruitment procedures and basic information on the planned instrumentation.
School Recruitment
Gaining schools’ and students’ cooperation in voluntary research is increasingly challenging and employing effective strategies for gaining the cooperation of schools are especially of paramount importance. Attempts are being made to solicit a letter of endorsement from the respective state education agencies to include in recruitment materials sent to districts and schools. Schools will then be recruited both directly and at the district level.
State Endorsement. To encourage district and school participation in the study, their respective state education agencies will be contacted to inform them about the study and to request a letter of endorsement (appendix A). Within 3 days of sending the letter to the state, senior IELS recruitment staff will contact the state to secure support for the study. Endorsement letters received by the state will be included in all mailings to districts and schools within the state. Where possible, NCES will work with NAEP State Coordinators to coordinate recruitment within states. This has been successful in other assessments.
Special Handling Districts. Contacting special handling districts begins with updating district information based on what can be gleaned from online sources, followed by calls to verify the information about where to send the completed required research application forms, and, if necessary, to collect contact information for this process. During the call, inquiry will also be made about the amount of time the districts spend reviewing similar research applications. To allow sufficient time for special districts’ review processes, this operation will begin upon receiving OMB’s approval, and continue until we receive final approval or denial of our request from each contacted district, up until October 31, 2017.
School Recruitment. After state contacts have been completed, whether or not an endorsement letter is received, school districts will receive a mailing about the study. The district introductory information packet will include a cover letter, brochure, and Frequently Asked Questions (FAQs) (appendix A). Three days after mail delivery of the packet, an IELS recruiter will make a call to secure the district’s cooperation, answer any questions the superintendent or other district staff may have, review the list of schools sampled in the district, and confirm key information about the schools (e.g., kindergarten present, size of enrollment, etc.). Information collected during this call will be used to confirm which schools in the district are eligible for participation in the study, and to obtain contact and other information helpful in school recruitment.
The study staff will be prepared to respond to requirements such as research applications or meetings to provide more information about the study. If a district chooses not to participate, the recruiter will document all concerns listed by the district so that a strategy can be formulated for refusal conversion attempts.
During the recruitment effort, once a school has agreed to participate, the IELS study team will work with the appropriate school staff member to discuss the IELS activities at the school, including gathering student rosters, distributing consent materials to parents of sample students, and understanding what arrangements need to be made for the onsite assessments. In early communications, the recruiter will also gather information about what type of parental consent procedures need to be followed at the school; hours of operation, including early dismissal days, school closures/vacations, and dates for standardized testing; and any other considerations that may impact scheduling student assessments (e.g., planned construction periods, school reconfiguration, or planned changes in leadership).
Parent Recruitment
Information about schools’ procedures for obtaining consent for students to participate in the study will have been gathered during school recruitment. Schools generally require one of two types of consent: implicit or explicit. Both types of consent require that parents be notified that their children have been selected for the study. With implicit consent, the school does not require verbal or written consent for a student to participate in the study – parents are asked only to notify the appropriate person if they do not want their child to participate (appendix B). With explicit consent, children may participate only if their parents provide written or oral consent for their children to do so (appendix B). Proactive parent recruitment will be focused on maximizing the number of parents who: (1) return signed explicit consent forms and (2) complete the parent questionnaire.
After the student sample is drawn within a school, the initial communication with parents consisting of introductory and consent materials (appendix B) will be distributed to parents in the way each school believes to be most appropriate and effective (e.g., sending the materials home with students; the school or district sending the materials directly to parents; and/or trained IELS study team recruitment staff contacting parents directly by mail, email, and/or phone). The initial materials will introduce the study, explain the study’s purpose and the importance of student and parent participation, describe what is involved in participation, and specify the consent procedure that is being used by their school. The materials will include a consent seeking letter to all parents plus a consent form where explicit consent is required (appendix B), a colorful recruitment-oriented brochure (appendix A) and a sheet of FAQs about the study (appendix B). Additionally, in schools using explicit consent, the parental consent form for student’s participation, which will be included in the initial communication materials, will ask parents to provide their contact information (appendix B).
Field Test Instrumentation and Design
Child Assessment
The direct child assessment will be a one-on-one untimed assessment comprised of tasks that assess academic and social emotional skills, with electronic data capture (about 60 minutes per child). Direct assessment items to be administered in the IELS field test will assess the following domains: language/emergent literacy, mathematics/numeracy, executive function/self-regulation, and social emotional skills (e.g., empathy and trust).
Questionnaires
Contextual information will be collected to inform the relation between children’s learning and development and important demographic, social, economic, and education variables.
Although the specific contextual data to be collected and methods to be used are still to be determined, it is expected at this time that data on the following categories will be collected from teachers and parents: early childhood education and care participation, home learning environment, and individual characteristics. The parent questionnaire will ask for information about the child and the home, and the teacher questionnaire will ask teachers to provide information on each study child’s knowledge and skills. Both questionnaires will be available electronically and in hard copy.
Studies have increasingly experienced challenges in obtaining the cooperation of districts and schools. Loss of instructional time, competing demands (such as district and state testing requirements), lack of teacher and parent support, and increased demands on principals impede gaining permission to conduct research in schools. The IELS recruitment teams will be trained to communicate clearly to districts, dioceses, private school organizations, schools, teachers, and parents the benefits of participating in the IELS field test, and what participation will require in terms of student and school personnel time. Recruiters will be trained to address concerns that districts and schools may have about participation, while simultaneously communicating the value of the study and the school’s key role in developing instruments that ensure high-quality data focusing on young kindergarten students.
Our approach to maximizing school and student response rates in the main study includes the following:
Assigning personal recruiters for specific schools;
Using experienced recruiters;
Developing persuasive written materials;
Avoiding refusals by focusing on strategies to solve problems or meet obstacles to participation;
Incentives for schools, school coordinators, and teachers (see Section A9); and
Contact with schools and school coordinators at set intervals throughout the year preceding the assessment.
These approaches are based on recommendations from an NCES panel and experience with previous administrations of international assessments such as PISA and TIMSS.
Participation in the field test is an international requirement for participating in the IELS main study. The main focus of the field test is to collect enough assessment data to perform reliable tests of the items. However, during the field test, procedures for conducting the main study, including recruitment methods for obtaining school and student participation also will be evaluated. This information will be used to: (a) determine the final main study design, and (b) improve our recruiting strategies and materials for the main study.
Overall direction for the IELS is provided by Dr. Dana Kelly, Branch Chief for International Assessments, and Ms. Mary Coleman, Project Officer, at the National Center for Education Statistics (NCES), U.S. Department of Education.
Key international colleagues include the following:
Overall project direction: Dr. Sacha DeVelle, Australian Council for Educational Research;
Survey design: Dr. Wolfram Schultz, Australian Council for Educational Research; and
Study Operations: Ms. Julianne Henke, IEA Data Processing Center.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |