Supporting Statement Part A revised 2-16-2007

U.S. Department of Education

Institute of Education Sciences

National Center for Education Evaluation

Follow Up to the Even Start

Classroom Literacy Interventions

and Outcomes Study

Office of Management and Budget

Clearance Package Supporting Statement

and Data Collection Instruments

Part A: Justification

February 16, 2007

TABLE OF CONTENTS

INTRODUCTION 1

STUDY OVERVIEW 5

A. JUSTIFICATION 5

A.1 Circumstances Making the Collection of Information Necessary 5

A.2 Purposes and Uses of the Data 6

A.3 Use of Technology to Reduce Burden 12

A.4 Efforts to Identify Duplication 12

A.5 Methods to Minimize Burden on Small Entities 13

A.6 Consequences of not Collecting Data 13

A.7 Special Circumstances 13

A.8 Federal Register Comments and Persons Consulted Outside the Agency 13

A.9 Payments or Gifts 14

A.10 Assurances of Confidentiality 14

A.11 Justification of Sensitive Questions 15

A.12 Estimates of Hour Burden 15

A.13 Estimates of Cost Burden to Respondents 16

A.14 Estimate of Annual Cost to the Federal Government 16

A.15 Program Changes or Adjustments 17

A.16 Plans for Tabulation and Publication of Results 17

A.17 Approval to Not Display the OMB Expiration Date 19

A.18 Explanation of Exceptions 19

APPENDIX A: DATA COLLECTION INSTRUMENTS

LIST OF EXHIBITS

Exhibit 1 Conceptual Model of Even Start 2

Exhibit 2 Interventions Tested in the CLIO Study 4

Exhibit 3 Data Collection Instruments and Schedule for the CLIO Follow Up Study 6

Exhibit 4 CLIO Child Assessment for Kindergarten……………… 7

Exhibit 5 CLIO Child Assessment for First Grade……………… 8

Exhibit 6 Estimated Response Burden for Respondents-Spring 2004 ..16

Exhibit 7 Draft Outline for Final Report on the CLIO Follow Up Study 18

FOLLOW UP TO THE EVEN START CLASSROOM LITERACY INTERVENTIONS AND OUTCOMES (CLIO) STUDY

SUPPORTING STATEMENT

FOR PAPERWORK REDUCTION ACT SUBMISSION

INTRODUCTION

The importance of high-quality early childhood education for all children is widely recognized in the education community. Young children who have strong vocabularies and exposure to and experience with the sounds that make up language (phonological awareness) are more likely to become successful readers and experience academic success throughout their school careers. Furthermore, parents play a critical role in the development of their children’s language, social, and cognitive skills. Children who have parents who play, talk, and read with them have an important advantage. Parents who understand the ways in which they can positively contribute to their children’s early learning are better equipped to support their children’s academic success. And, parents who themselves are competent readers are more likely to have jobs that adequately support their family’s needs.

Even Start is a family literacy program, established in 1989, whose primary goal is to improve the academic achievement of low-income young children and their parents, especially in the area of reading. Even Start projects offer four integrated instructional activities for parents and children: (1) interactive literacy activities between parents and their children (PC services), (2) training for parents regarding how to be the primary teacher for their children (PE services), (3) parent literacy training that leads to economic self-sufficiency (AE services), and (4) age-appropriate early childhood education to prepare children for success in school and life experiences (ECE services). The underlying premise of Even Start, and of the family literacy model generally, is that all of these four instructional components are necessary and are maximally effective when integrated into a unified program. Exhibit 1 illustrates a conceptual model for Even Start with hypothesized outcomes.

Two experimental evaluations¹ have shown that Even Start projects, as implemented prior to 2001, were not effective at enhancing the literacy skills of participating children and their parents. Of particular importance, the most recent national evaluation showed that the early childhood classroom experiences provided by Even Start did not have sufficient emphasis on language acquisition and reasoning.

Exhibit 1. Conceptual Model of Even Start

The results from these evaluations prompted a re-examination of the Even Start model to determine how it could be improved. The evaluations documented that Even Start projects generally implemented each of the four instructional components required at a sufficient level of intensity, comparable to mainstream programs offering the components of family literacy programs, and also implemented the operational requirements imposed by Congress and the U.S. Department of Education (ED). The principal issue for the CLIO study is how to identify specific interventions or instructional strategies that are effective and could be used to strengthen Even Start services. This is consistent with the mission of the evaluation center at the Institute of Education Sciences as well as Even Start’s second legislative evaluation requirement (Section 1239 (2)), which is to identify effective programs that can be duplicated and used in providing technical assistance. This is also consistent with the requirement for research that examines the components of successful family literacy services (Section 1241(a)). (see Section A.1 of this document). This approach also supported the strengthened mandate of Even Start, based on the Literacy Involves Families Together Act (LIFT, 2000) and the No Child Left Behind Act (NCLB, 2001), which call for Even Start projects to provide:

• High-quality, intensive instructional programs,

• Instructional programs based on scientifically-based reading research, and

• Reading readiness activities based on scientifically-based reading research.

The CLIO study, then, was designed to address the following two major research questions related to program effectiveness:

• Are the enhanced family literacy interventions being tested in CLIO more effective than existing Even Start services?

• What is the contribution of an enhanced parenting education curriculum to the effectiveness of the CLIO interventions?

The CLIO study also examined the following secondary questions:

• It is possible that some CLIO projects were more faithful in implementing the curricula than others. Are child and parenting outcomes better in projects with higher fidelity to their assigned curriculum?

• Can we identify superior instructional practices? Which ECE instructional practices are related to better child outcomes? Which PE instructional practices are related to better parenting outcomes?

• What is the role of participation in determining outcomes? Is more child participation related to better child outcomes? Is more parent participation related to better parenting outcomes?

• Which parenting practices are related to better child outcomes? Do interactive reading skills and responsiveness relate positively to child literacy and social competency?

CLIO tested four enhanced curricula (see Exhibit 2) as interventions in Even Start projects and compared them to a control group of projects. All four of the curricula had an early childhood education component that provided enhanced instruction in early language and literacy. As required by the Even Start legislation, all of the study sites provided parenting education and parent-child literacy activities. However, two of the CLIO interventions included enhanced parenting education and parent-child literacy activities that were integrally linked with the early childhood education instruction (i.e., linked both conceptually and in instructional approach), while two interventions asked projects to continue to provide their typical parenting education and parent-child literacy activities. In keeping with the family literacy approach of Even Start, all of the projects in the study provided adult education to parents, but the adult education programs were not systematically changed as part of the study.

Exhibit 2. Interventions Tested in the CLIO Study

Group	Components that were enhanced	Name of enhanced curriculum
Treatment A1	Early childhood education only	Let’s Begin with the Letter People
Treatment A2	Early childhood education and parenting education/parent-child literacy activities	Let’s Begin with the Letter People and Play and Learning Strategies (PALS)
Treatment B1	Early childhood education only	ECE Partners for Literacy
Treatment B2	Early childhood education and parenting education/parent-child literacy activities	ECE/PE Partners for Literacy
Control	None	“As is” Even Start services

The CLIO sample of participating Even Start projects was chosen in the following manner. In spring 2003, CLIO surveyed all Even Start projects to identify (a) the number of children by age being served in the project, (b) where and how early childhood education services were being delivered to preschool children, and (c) the distribution of Even Start preschool children across early childhood education. To be eligible for CLIO, projects had to meet the following criteria:

• Serve a minimum of either: (a) five 3- and 4-year olds in one center-based classroom, or (b) eight 3- and 4-year olds in two center-based classrooms;

• Provide at least 12 hours per week of center-based early childhood instruction;

• Serve a majority of families who speak either English or Spanish, and;

• Be willing to participate in the study, including being randomly assigned to one of the five study conditions.

Due to the voluntary nature of participation and the specific eligibility criteria for the study, the CLIO projects were not intended to be nationally representative. However, the 120 recruited projects were spread over 33 states in all regions of the country, and varied a great deal on characteristics such as urbanicity, number of families served, percentage of families who were English Language Learners, and experience implementing Even Start.

The CLIO study design included random assignment of projects into one of the four treatment groups or the control group as described above. The study design also included within-site controls, as a year of baseline data were collected on each project before the enhanced interventions were implemented and tested. Lastly, CLIO was designed to measure impacts on preschoolers and their parents in preschool through first grade. To this end the study conducted child assessments, parent assessments, parent interviews, and videotaped parent-child interactions. The study surveyed project directors and collected participation data. The study also collected information on the staff and instructional processes in Even Start classrooms, so that participants’ performance could be linked to their education experiences in Even Start, and asked teachers to rate their students using the Teacher Child Report (TCR) form.

STUDY OVERVIEW

While the original CLIO study examined enhanced family literacy interventions in Even Start and parent and child outcomes over the course of the intervention, the CLIO Follow Up study will explore whether effects from preschool are sustained through the early school years. To gauge this longer-term impact, the CLIO Follow Up study adds a major research question: Do the enhanced family literacy interventions tested in the original CLIO study produce longer-term effects at the end of kindergarten and at the end of first grade?

The CLIO Follow Up study will follow the original CLIO study families as their children enter school through the end of first grade to determine the longer-term impacts, if any, of the CLIO interventions. The Follow Up study will collect data in spring 2007 on kindergartners who participated in CLIO as preschoolers in spring 2006. In spring 2008, data will be collected on first graders from this same group. Each data collection cycle will include child assessments, parent interviews, parent-child videos, teacher surveys, and teacher rating forms.

A. JUSTIFICATION

A.1 Circumstances Making the Collection of Information Necessary

There are requirements for evaluation at all three levels of the Even Start program – Federal, State, and local. As shown below, section 1239 of ESEA requires ED to conduct an independent evaluation of the program:

From funds reserved under section 1232(b)(1), the Secretary shall provide for an independent evaluation of programs assisted under this part—

1. to determine the performance and effectiveness of programs assisted under this subpart;

2. to identify effective Even Start programs assisted under this subpart that can be duplicated and used in providing technical assistance to Federal, State, and local programs; and

3. to provide State educational agencies and eligible entities receiving a subgrant under this subpart, directly or through a grant or contract with an organization with experience in the development and operation of successful family literacy services, technical assistance to ensure local evaluations undertaken under section 1235(15) provide accurate information on the effectiveness of programs assisted under this subpart.

Also, as shown below, Section 1241 requires ED to carry out family literacy research:

SEC. 1241. RESEARCH.

(a) IN GENERAL.—The Secretary shall carry out, through grant or contract, research into the components of successful family literacy services, in order to—

(1) improve the quality of existing programs assisted under this subpart or other family literacy programs carried out under this Act or the Adult Education and Family Literacy Act; and

(2) develop models for new programs to be carried out under this Act or the Adult Education and Family Literacy Act.

A.2 Purposes and Uses of the Data

The U.S. Department of Education (ED) will use the information gathered by the CLIO Follow Up study to answer the overarching research question regarding the longer-term effects of the CLIO preschool family literacy interventions at the end of kindergarten and first grade.

Each data collection period will include child assessments, parent interviews, parent-child videos, teacher surveys, and teacher rating forms.

Exhibit 3 provides a summary of the data collection schedule and instruments. The following sections briefly describe the measures proposed for the CLIO Follow Up study. The proposed instruments for the Parent Interview, Teacher Survey, and Parent-Child Video are provided in Appendix A².

Exhibit 3. Data Collection Instruments and Schedule for the CLIO Follow Up Study

Instrument	Spring 2007	Spring 2008
Child Assessment	X	X
Parent Interview	X	X
Parent-Child Video	X	X
Teacher Survey	X	X
Teacher Rating Form	X	X

Child Assessment

The CLIO Follow Up child assessment is designed to measure skills related to school achievement, especially in the area of early reading and literacy. The instrument is drawn from several published assessments. In selecting appropriate tests we considered the age of children in our sample (5- and 6-year olds), language and literacy outcomes of interest, continuity with the CLIO preschool instrument, and coverage of the major reading components as identified by Reading First, including phonemic awareness, phonics, fluency, vocabulary, and comprehension. An additional consideration in selecting tests for the child assessment was keeping the instrument to a reasonable length in order to avoid losing the child’s attention, which can negatively affect their performance on the assessment. The proposed instrument will take approximately 40 minutes to administer (slightly longer for children whose home language is Spanish).

Exhibit 4 shows the components of the CLIO child assessment to be used with kindergarten students. The assessment consists of seven subtests in English and one subtest in Spanish. The subtests are described in detail below.

Exhibit 4. CLIO Child Assessment for Kindergarten

Subtest	Reading domain
Peabody Picture Vocabulary Test (PPVT)	Receptive vocabulary
TVIP (Spanish version of PPVT for children whose home language is Spanish)	Receptive vocabulary in Spanish
Expressive One-Word Picture Vocabulary Test	Expressive vocabulary
Grammatic Understanding	Grammar, syntax
Elision	Phonemic awareness
Letter-Word Identification	Alphabetic knowledge, basic reading
Word Attack	Phonics
Passage Comprehension	Reading comprehension

Note: Grammatic Understanding is from the Test of Language Development (TOLD); Elision is from the Preschool Comprehensive Test of Phonological and Print Processing (Pre-CTOPPP); Letter-Word Identification, Word Attack, and Passage Comprehension are from Woodcock-Johnson III.

The Child Assessment for first grade will closely resemble the kindergarten instrument, with a few exceptions. For first grade, we will administer the Elision subtest from the Comprehensive Test of Phonological Processing (CTOPP), using the version developed for children ages 5 and 6 years old. This subtest replaces the Elision subtest used in the kindergarten instrument (from the Preschool Comprehensive Test of Phonological and Print Processing, or Pre-CTOPPP). The first grade version will have one additional subtest to measure reading fluency, the Reading Fluency subtest from the Woodcock-Johnson-III battery. Exhibit 5 shows the components of the CLIO child assessment to be used with first grade students.

Exhibit 5. CLIO Child Assessment for First Grade

Subtest	Reading domain
Peabody Picture Vocabulary Test (PPVT)	Receptive vocabulary
TVIP (Spanish version of PPVT for children whose home language is Spanish)	Receptive vocabulary in Spanish
Expressive One-Word Picture Vocabulary Test	Expressive vocabulary
Grammatic Understanding	Grammar, syntax
Elision	Phonemic awareness
Letter-Word Identification	Alphabetic knowledge, basic reading
Word Attack	Phonics
Passage Comprehension	Reading comprehension
Reading Fluency	Reading fluency

Note: Grammatic Understanding is from the Test of Language Development (TOLD); Elision is from the Comprehensive Test of Phonological Processing (CTOPP); Letter-Word Identification, Word Attack, Passage Comprehension, and Reading Fluency are from Woodcock-Johnson III.

The subtests of the child assessment are described as follows:

• The Peabody Picture Vocabulary Test (PPVT) measures receptive vocabulary and has been widely used in other early childhood studies. The child is not required to define words but to show he/she understands the meaning of the word by pointing to a picture that best represents the meaning of the word. The adaptive version includes a core, basal, and ceiling set, and administration is halted if or when the child makes a specified number of errors. The PPVT is highly correlated with other measures of cognitive ability and has demonstrated predictive validity in the Head Start Family and Child Experiences Survey (FACES) longitudinal study. The internal reliability statistic (Cronbach’s alpha) for this subtest was 0.89 in the spring 2004 CLIO data collection.

• The TVIP is the Spanish version of the PPVT vocabulary test and measures listening comprehension for spoken words in Spanish. This test is adapted from the PPVT-R, which correlates highly with other measures of vocabulary.

• The Expressive One-Word Picture Vocabulary Test measures children’s verbal expression of language and is appropriate for ages 2-0 through 18-11. The test has been recommended by external research panels (Temple University forum, 2003; NICHD/HHS advisory group, 2002) to assess language in young children.

• The TOLD-3 Grammatic Understanding subtest measures the ability to comprehend the meaning of a sentence, with an emphasis on syntax and morphology. Knowledge of syntax is important in constructing and understanding sentences. The internal reliability statistic (Cronbach’s alpha) for this subtest was 0.78 in the fall 2004 CLIO data collection.

• The Preschool CTOPPP Elision subtest (for kindergartners) measures phonological awareness in young children, ages 3 to 5. The subtest measures the child’s ability to recognize English word parts, such as components of compound words, syllables and phonemes. The internal reliability statistic (Cronbach’s alpha) for this subtest was 0.82 in the spring 2004 CLIO data collection.
  
  The CTOPP Elision subtest (for first graders) measures the extent to which an individual can say a word, then say what is left after dropping out designated sounds. Early items test syllable segmentation and later (more difficult) items test phoneme deletion. Phoneme deletion is a higher order skill in the development of phonological awareness, appropriate for testing in the spring of first grade. The test publisher cites a reliability of .92 for 6-year-old children.

• The Woodcock-Johnson (W-J) subtests of Letter-Word Identification, Word Attack, and Passage Comprehension provide measures of basic reading comprehension and phonics. These tests are appropriate for use with individuals ages 2-6 through adult. National norms are available for these tests. In addition, the W-J publisher has established clusters as another way of interpreting scores, for example, the basic reading cluster includes the Letter-Word Identification and Word Attack subtests. The test publisher cites a reliability for the reading cluster of .98 for 6-year-old children. Passage Comprehension has a reliability of .96 for 6-year-old children.

The W-J Reading Fluency subtest measures a child’s ability to quickly read simple sentences and indicate whether the statement is true or not. The child must complete as many items as possible within a 3-minute time limit. Reading Fluency has a reliability of .89 for children ages 6 and 7 years old, as cited by the publisher.

Parent Interview

The CLIO parent interview is designed to collect information about family characteristics that are most likely to influence the child’s language and literacy development. Parent interviews will be administered in English or Spanish based on the parent’s preference. The kindergarten and first grade parent interviews are similar to the parent interview used in the original CLIO study. They have each been revised to include age-appropriate questions regarding parent-child activities and parent ratings of child accomplishments and behavior. Interview components include the following:

• Parent-child activities, with specific items asking about the frequency of language and literacy activities;

• Home literacy environment, including books in the home, frequency of parent reading, and parent’s self-rating of English literacy skills;

• Family household and demographic information, including parent-child relationships and the quality of the child’s home life;

• Parent ratings of their child’s learning accomplishments and concerns about the child’s development or behavior;

• Parent’s educational attainment, income, and workforce participation; and

• Communication between the parent and child’s school, including contact with the child’s teacher.

The parent ratings of their child’s accomplishments include language and literacy items that are used to create an emergent literacy scale developed by the Head Start FACES. This scale has shown significant increases from fall to spring in FACES and correlates with parent reports about the frequency of home-based learning activities. The parent interview is approximately 30 minutes in length.

Parent-Child Video

The parenting curricula implemented in the original CLIO study focused on teaching parents two main skills—how to nurture the social and emotional growth of their child and how to be their child’s first teacher. Measuring the value added of the enhanced parenting component was built into the design of the original CLIO study and was one of the study’s main research questions.

To evaluate parenting behaviors, it is necessary to observe parent-child interactions under somewhat natural conditions. For the original CLIO study, parents were asked to read a book to their child “as they would normally at home” and then were asked to spend 5 minutes playing together with a toy. The CLIO assessors videotaped these tasks, and then trained CLIO coders used a series of coding instruments to score these behaviors. These coding instruments included the following:

• Reading Aloud Profile – Together (RAPT). The Reading Aloud Profile – Together (RAPT) is an observation measure that focuses specifically on reading behaviors of both the parent and the child. It has been shown that the practice of specific behaviors during joint book reading can promote children’s engagement in reading, and help them better comprehend the story and understand the conventions of print. Mutual questioning and responding, making stories relevant to the child’s life, giving praise and feedback, explaining, physically sharing the book, monitoring a child’s understanding, and adjusting language are all behaviors that enhance children’s literacy skills and comprehension.

• Quality Indicators. The Quality Indicators for RAPT included three five-point Likert scales. These quality items focus on three aspects of the reading interaction: (1) the degree to which the parent introduced and contextualized new vocabulary to support child’s learning; (2) the extent to which the parent used open-ended questions that invite the child to engage in prediction, imagination, and/or rich description; and (3) the depth of child’s engagement with the reading activity.

• Contingency Scoring Sheet. The “Contingency Scoring Sheet” instrument included eight 7-point Likert scales, five of which characterize the parent’s behavior and three of which characterize the child’s behavior.

While each of the instruments described above was used to score the reading task, only the Contingency Scoring Sheet was used to score the toy task. In addition, a Quality Control Checklist was completed on all received video tapes. The Quality Control Checklist provided information on the extent to which the video administrators followed standard protocol.

Parents of the kindergarten CLIO Follow Up children will be asked to read the book “Click, Clack, Moo: Cows That Type” by Doreen Cronin for the book task. This book is ranked by the publisher as being appropriate for ages 3 to 7. Parents will be given the option of reading either the English or Spanish version. The toy will be a set of wooden blocks with letters, numbers, and pictures on them, allowing the parents and children to interact in a number of different ways: stacking the blocks, counting them, spelling out words, and identifying and talking about the letters, pictures, and colors.

While first-graders will use the same toy, they will use a different book, “If You Give a Mouse a Cookie” by Laura Joffe Numeroff. This book is appropriate for ages 4 to 8 and not too difficult for low-literate parents to navigate. The book is of a desirable length to allow the parent to interact with the child and engage in some of the higher-end cognitive items, like asking open-ended questions or vocabulary development.

Teacher Survey

The kindergarten teacher survey will collect information about the school and classroom environment as well as the teacher’s educational background, certifications, experience, and professional development. Teachers will be asked to report on the frequency of 17 specific reading and language activities used in the classroom.

In the first grade teacher survey the sections asking about reading and language activities and professional development will be revised to incorporate questions appropriate for first grade. Both versions of the teacher survey contain questions based on the teacher surveys for the Reading First Implementation Study and the Reading First Impact Study.

Teacher Rating Form

Children’s development of social skills and positive behaviors is an important objective for early childhood education programs and is associated with success in school. A Teacher’s Child Report (TCR) form has been developed to allow kindergarten teachers to rate children’s social skills and classroom conduct and also provide information about any health or developmental concerns. The teacher ratings of children’s behavior, along with information from the parent interview, will supplement the direct assessments to allow for fairer and more robust appraisals of children’s skills and competence. While teacher and parent ratings are not as objective as direct assessments, teachers and parents provide critical information because they see children over extended periods of time and in a variety of settings.

The TCR, originally developed for the Head Start FACES, uses two scales to measure social skills and behavior problems in preschool and kindergarten. The Cooperative Classroom Behavior scale includes 12 items adapted from the Personal Maturity Scale (Alexander and Entwisle, 1988) and the Social Skills Rating System (Elliott, Gresham, Freeman, and McCloskey, 1988). These items ask teachers to report the frequency of positive behaviors, including cooperation with adults, friendly play, and sharing with other children. The Behavior Problems scale consists of 14 items adapted from the Personal Maturity Scale, Child Behavior Checklist (Achenbach, Edelbrock and Howell, 1987), and the Behavior Problems Index (Zill, 1990). Problem behaviors include disruptive or overly aggressive behavior, hyperactivity, excessive shyness, and social withdrawal. The two scales were combined into a single social competency scale to measure social emotional outcomes for CLIO preschool children. The internal reliability statistic (Cronbach’s alpha) for the combined scale was 0.92 in the spring 2004 CLIO data collection. The same TCR will be used for kindergartners and first-graders in the CLIO Follow Up Study.

A.3 Use of Technology to Reduce Burden

We will use a variety of information technologies to maximize the efficiency and completeness of the information gathered for the CLIO Follow-Up study and to minimize the burden the study places on respondents at all levels. For example, during the data collection period, a toll-free number will be available to permit respondents to contact the contractor with questions or requests for assistance. Also, a computer-based field management system will be used by the contractor to monitor the flow of data collection activities – from data collection to processing and coding to entry into the database. This monitoring will help to ensure the efficiency and completeness of the data collection process.

A.4 Efforts to Identify Duplication

The Follow Up study is designed to build on and coordinate with ongoing and recently completed research efforts including:

• The Even Start Classroom Literacy Interventions and Outcomes Study (CLIO),

• The Head Start Family and Child Experiences Study (FACES),

• The National Head Start Impact Study (HSIS),

• The National Evaluation of the Early Reading First program,

• The Early Childhood Longitudinal Study (ECLS), kindergarten cohort and birth cohort, and

• Previous national Even Start evaluations.

We have reviewed the existing literature as well as recent GAO reports and are satisfied that, although the data from these studies will serve to complement the results of the CLIO Follow Up Study, none of the data elements proposed for this study are available through another source.

A.5 Methods to Minimize Burden on Small Entities

No small businesses or entities are involved in this project. The burden on schools, vis-à-vis teachers who will complete the Teacher Survey and TCRs, has been minimized wherever possible.

A.6 Consequences of Not Collecting Data

The Even Start program, as authorized by Part B, Subpart 3 of Title I of ESEA, includes a mandate that the program be evaluated to determine the performance and effectiveness of the Even Start program as well as to identify effective Even Start programs that might be duplicated (section 1239 of ESEA). Further, section 1241 of ESEA calls for ED to carry out research into the components of successful family literacy services. The original CLIO study and the CLIO Follow Up study were both designed to provide the Department of Education with the information necessary to respond to these legislative requirements.

A.7 Special Circumstances

No special circumstances apply to this data collection.

A.8 Federal Register Comments and Persons Consulted Outside the Agency

A 60-day notice was published in the Federal Register on October 6, 2006 with an end date of December 15, 2006 to provide the opportunity for public comment. In addition, throughout the course of this study, we will draw on the experience and expertise of the study’s technical working group (TWG) members who provide a diverse range of experience and perspectives:

• Gene Brody, University of Georgia

• Thomas Cook, Northwestern University

• David Francis, University of Houston

• Barbara Goodson, Abt Associates

• Larry Hedges, Northwestern University

• Chris Lonigan, Florida State University

• Robin Morris, Georgia State University

• Lynne Vernon-Feagans, University of North Carolina at Chapel Hill

• Barbara Wasik, Johns Hopkins University

A.9 Payments or Gifts

Participation in the CLIO Follow Up study will place demands on participants. Parents will be asked to participate in an interview and a video, both of which will provide the study with necessary outcome data. Furthermore, the two years of Follow Up study data collection will constitute years 2 and 3 of data collection from the preschoolers and their parents from the original CLIO study, and retention is a critical issue to keeping response rates high. We will offer a monetary incentive to parents for the time spent being interviewed and videotaped. For parents, the incentive will be handed to them after they have completed the in-person parent interview and the videotaping. Kindergarten and first-grade teachers will be asked to fill out a rating form for each child and a survey. These teachers have no stake in the Even Start program in general or the CLIO study specifically, so there is no reason for the teachers to respond to this data collection. The offer of the incentive will encourage participation. For teachers, the incentive will be mailed to them upon receipt of a completed survey and child rating form(s). The incentive amounts will be as follows:

• Cash ($15) for each parent who is interviewed;

• Cash ($10) for each parent who participates in the videotaping;

• Small gift (stickers) for each child who is assessed; and

• Cash ($20) for each teacher who completes both the teacher survey and the child rating form(s).

A.10 Assurances of Confidentiality

Since we are collecting personally-identifiable information, we note the applicable IES statute, as follows:

"The Education Sciences Reform Act of 2002, Title I, Part E, Section 183 requires "All collection, maintenance, use, and wide dissemination of data by the Institute" to "conform with the requirements of section 552 of title 5, United States Code, the confidentiality standards of subsection (c) of this section, and sections 444 and 445 of the General Education Provision Act (20 U.S.C. 1232g, 1232h)." These citations refer to the Privacy Act, the Family Educational Rights and Privacy Act, and the Protection of Pupil Rights Amendment.

In addition for student information, "The Director shall ensure that all individually identifiable information about students, their academic achievements, their families, and information with respect to individual schools, shall remain confidential in accordance with section 552a of title 5, United States Code, the confidentiality standards of subsection (c) of this section, and sections 444 and 445 of the General Education Provision Act.

Subsection (c) of section 183 referenced above requires the Director of IES to "develop and enforce standards designed to protect the confidentiality of persons in the collection, reporting, and publication of data".

Subsection (d) of section 183 prohibits disclosure of individually identifiable information as well as making any the publishing or communicating of individually identifiable information by employees or staff a felony."

IES is also working on the requirements for a systems of records for this study. The notice was published in the Federal Register on December 6, 2006.

Regarding confidentiality, all Westat staff members, including field staff, will be informed of confidentiality issues and related Westat procedures and will sign a Westat confidentiality pledge. Participants will be reassured in person and in writing that their participation in the study is completely voluntary. A decision not to participate will not affect involvement in their school or classroom. Further, if they choose to participate, they may refuse to answer any question they find intrusive. All interview responses will be held strictly confidential and no answers will be reported to any outside program or agency, except as required by law. For reporting, all responses will be combined so individuals cannot be identified. All interviews and assessments will take place in the respondent’s home or in a setting where the respondent’s privacy can be assured. Westat’s data collection and data processing procedures will be set up to protect the anonymity of respondents. Westat staff will be responsible for maintenance of security.

A.11 Justification of Sensitive Questions

Several questions of a potentially sensitive nature are included in the parent interview to enable us to understand the social context of children and their families who are or were enrolled in Even Start. These include questions about reading habits, use of social services, and income, and the information they provide is crucial to understanding family needs, identifying risk factors for the child’s development and fully describing the contextual factors in families that impede or facilitate family well-being. In all cases, questions on these topics are either part of a standardized measure, or have been carefully pretested, or have been used extensively in prior studies with no evidence of harm.

A.12 Estimates of Hour Burden

The estimated burden for Even Start respondents is provided in Exhibit 6. The total annual responses are 4,015. Parents provide 2 responses each year (one for the parent interview and one for the videotaping), children one response each year, and teachers one response each year for the teacher survey and approximately 1.38 responses for the teacher rating form. (The 1.38 figure is based on our estimate that the average kindergarten teacher will have 1.38 Even Start students in his or her class.) (There is no burden associated with obtaining teachers’ contact information. With the school name from the previous parent interview, the school mailing information will be obtained from the National Center for Education Statistics’ Common Core of Data file. The survey and child rating forms will be mailed to the teachers in each school by labeling the package as “Teacher of (name of CLIO child).”) The total annual hours requested is 967. The estimate of burden cost assumes teacher time at $20.00 per hour.

Exhibit 6. Estimated Response Burden for the CLIO Follow Up Study.

Instrument	Number of Respondents	Responses per Respondent	Total Responses	Average Burden Hours per Response	Total Burden Hours	Total Burden Cost
Spring 2007
Parent Interview-kindergarten	850	1	850	0.50	425.00	$0
Parent-Child Video	1,700	1	1,700	0.25	425.00	$0
Teacher Rating Form	615	1.38	850	0.08	68.00	$1,360
Teacher Survey-kindergarten	615	1	615	0.08	49.20	$984
Subtotal for Spring 2007	3,780	--	4,015	--	967.20	$2,344
Spring 2008
Parent Interview-1st grade	850	1	850	0.50	425.00	$0
Parent-Child Video	1,700	1	1,700	0.25	425.00	$0
Teacher Rating Form	615	1.38	850	0.08	68.00	$1,360
Teacher Survey-1st grade	615	1	615	0.08	49.20	$984
Subtotal for Spring 2008	3,780	--	4,015	--	967.20	$2,344
Grand Total	7,560	--	8,030	--	1,934.40	$4,688
ANNUAL TOTAL	3,780	--	4,015	--	967.20	$2,344

A.13 Estimate of Cost Burden to Respondents

There are no additional respondent costs associated with this data collection other than the hour burden estimated in item A.12.

A.14 Estimate of Annual Cost to the Federal Government

The estimated cost for the CLIO Follow Up study is $3.25 million for three years, or $1.08 million per year.

A.15 Program Changes or Adjustments

The 967 annual hours of burden is because this is a new data collection.

A.16 Plans for Tabulation and Publication of Results

From the key research question of the CLIO Follow Up study (Do the enhanced family literacy interventions tested in the original CLIO study produce longer-term effects at the end of kindergarten and at the end of first grade?), we developed ten hypotheses that we will address with Follow Up data. The primary hypotheses will be addressed through experimental methods, and the secondary hypotheses must be addressed through non-experimental methods. Both primary and secondary hypotheses will be closely linked to the parallel sets of hypotheses in the original CLIO study.

The three primary hypotheses of the CLIO Follow Up study are as follows:

1. The experimental CLIO preschool curricula improve child English literacy and child social competency at the end of both kindergarten and first grade;

2. The combinations of experimental ECE and PE curricula yield larger improvements than do the ECE curricula by themselves; and

3. The experimental PE curricula lead to improved parenting practices at the end of both kindergarten and first grade.

The seven secondary hypotheses are:

4. Particular ECE instructional practices, such as support for oral language, are associated with higher levels of English literacy at the end of both kindergarten and first grade;

5. Particular ECE instructional practices are associated with higher levels of child social competency at the end of both kindergarten and first grade;

6. Parent interactive reading skill and responsiveness at the preschool stage of child development are associated with improved child English literacy and child social competency at the end of both kindergarten and first grade;

7. Particular PE instructional practices, such as PE time spent on parenting skills when the child is at the preschool stage, are associated with parenting practices when the child is in kindergarten and first grade;

8. Particular PE instructional practices, such as PE time spent on parenting skills when the child is at the preschool stage, are associated with higher levels of child English literacy and child social competency at the end of both kindergarten and first grade;

9. Any effects of preschool curricula, preschool instructional practices, or preschool parenting practices are moderated by the teaching practices of kindergarten and first grade teachers; and

10. Any effects of preschool curricula, preschool instructional practices, or preschool parenting practices are moderated by parenting practices when the children are in kindergarten and first grade.

Additional hypotheses to be explored in the CLIO Follow Up study may arise out of the original CLIO study report. Also, findings of the original CLIO study report may cause us to reorder the Follow Up study hypotheses. For example, if the intention to treat (ITT) analysis of child literacy, child social competency, and parenting practices all fail to find evidence that the experimental curricula had any effects on these outcomes, but the non-experimental analysis shows that particular practices did appear to be helpful, then we may want to place more emphasis on the secondary hypotheses about the durability of those effects. As another example, if curriculum effects are found, we may wish to focus more on the dynamics of these effects, such as determining whether preschool effects tend to attenuate with subsequent child maturation, stay constant, or even increase.

A single report will include the analyses and tabulations from the spring 2007 and 2008 CLIO Follow Up data collections. A draft of the annotated final report outline is presented in Exhibit 7. Data collection will take place during the months of March through June 2007 and March through June 2008. The CLIO Follow Up Study report is expected to be released in September 2009.

Exhibit 7. Draft outline for final report on the CLIO Follow Up Study

Executive Summary

Chapter 1: Background

Rationale for and purpose of the study

Review of findings from the original CLIO study

Hypotheses for Follow Up Study

Chapter 2: Data Collection

Follow Up eligibility rules

Response rates

Instrument descriptions

Videotape coding procedures

Chapter 3: Analysis Methods

Assessment scoring

Multiple comparisons

Scale formation

Covariate selection

Missing data compensation

Selection bias correction

Software

Chapter 4: Results for Primary Hypotheses

Experimental effects on child literacy

Marginal contribution of parenting education

Experimental effects on parenting practices

Chapter 5: Results for Secondary Hypotheses

Superior ECE instructional practices for literacy

Superior ECE instructional practices for child social competency

Superior parenting practices for literacy and social competency

Superior PE instructional practices for parenting

Superior PE instructional practices for child literacy and social competency

Moderating effects of kindergarten and first-grade teacher practices

Moderating effects of parenting practices at kindergarten and first grade ages

Chapter 6: Summary and Discussion

Summary

Implications for Even Start

A.17 Approval to not Display OMB Expiration Date

All data collection instruments will include the OMB expiration date and number.

A.18 Explanation of Exceptions

No exceptions are requested.

1 St.Pierre, R., J. Swartz, B. Gamse, S. Murray, D. Deck & P. Nickel (1995). National evaluation of the Even Start family literacy program: Final report. Cambridge, MA: Abt Associates Inc.; St.Pierre, R., A. Ricciuti, F. Tao, C. Creps, J. Swartz, W. Lee, A. Parsad & T. Rimdzius (2003). Third National Even Start Evaluation: Program Impacts and Implications for Improvement. Cambridge, MA: Abt Associates Inc.

2 IES staff have been instructed by the Office of the Director to exclude examinations and proprietary materials from information collection clearance packages. Thus, the child assessment instruments and the teacher rating form (which was adapted from several copyrighted instruments) are not attached to the package.