Adult ESL Explicit Literacy Impact Study (KA)

July 21, 2006

Dear XXXXXXXXX
Thank you for the information you have shared with us in regards to the Adult ESL Explicit
Literacy Impact Study, a groundbreaking study designed to test the effect of a high quality,
innovative curriculum on the English literacy and language development of low-literate adult ESL
learners. The study is sponsored by the U.S. Department of Education’s Institute of Education
Sciences, and is scheduled to begin in fall, 2007. This letter is to apprise you of the status of our
program selection process, and to invite you to participate in the next phase of this process.
The selection of programs that can implement the study is key to the study’s success and involves a
multi-phase process to ensure that programs that are being considered meet study criteria. As you may
recall, a project staff member interviewed you requesting information on whether your program has:
a. Site enrollments of low-literate adult ESL literacy learners large enough to support the
evaluation (i.e., enrollment of at least 90 low-literate adult ESL students per semester or
instructional period).
b. At least 4 classes of about 20–25 low-literate adult ESL learners that provide 120 hours of
instruction per semester or other term, and meet at least two hours at a time.
c. At least 5 trained adult ESL literacy instructors who teach the target classes.
d. Two pairs (or all 4) of the classes meet at the same time and location.
e. Managed enrollment or an enrollment process where a majority of learners enter during
the beginning of a course.
Based on our prior discussions, we believe that your program meets these criteria. We now would
like you to participate in a conference call, which will be conducted by project staff, including one
of our research partners. During this call, we will explain the study’s objectives and the benefits of
your participation to your program. We will also review each criterion for inclusion in the study with
you to confirm the information we collected earlier and to fill information gaps related to enrollment
policies, class schedules, teacher qualifications, and the nature of your current curriculum. In
addition, we will explain the study’s methodology and request from you names of any other people
that we need to contact to secure your participation.
As we proceed to phase two, please remember that the study is specifically about low literacy adult
learners who span a range of low-level literacy abilities. In phase two of the selection process, we
want to focus our discussion more on the specific sites and classes which will participate in the study.
We will want to learn more about the students, including their enrollment patterns, attendance and
retention in these specific classes and sites. We also want to know more about the teachers of these
classes who will potentially participate in the study, including their willingness to participate. To
prepare for the conference call, we would like you to gather more information about these topics,

including any data you may have. Please also consider which staff members are appropriate to join
in the conference with us (i.e. ESL specialists, other site or program directors, teachers who might
want to be involved to help us determine the site’s eligibility for the study).
I am including more information about the study and the curriculum, including the benefits of
participation, the site selection criteria, and the random assignment process. One of the project staff
members will call you in a few days to confirm your interest in the study and to schedule the date
and time of the conference call. If you have any questions, please don’t hesitate to contact me or
email Anestine Hector-Mason at ahector-mason@air.org; 202-403-6913.
Thanks for your kind consideration and we look forward to speaking with you.
Sincerely,

Larry Condelli, PhD
Project Director
American Institutes for Research
1000 Thomas Jefferson Street
Washington, DC 20007
Tel: 202-403-5331
Email: lcondelli@air.org

Site Selection Criteria for
Explicit Literacy Impact Study
Selecting sites that meet the study’s criteria is very important to the study’s proper
implementation. Sites in different geographic regions in the US that meet the following
criteria will be considered for the study:
• Operate large ESL programs, serving lowliterate adult ESL students. To meet the

sample size requirements, we need programs in the
study that offer at least four beginning-level ESL
classes per semester with a minimum of 20–25 lowliterate students in each class. Over two semesters,
this will result in about 180 low-literacy students
receiving instruction in either the explicit literacy
or the existing ESL group. By low-literacy we
mean students who have only limited literacy skills
in their native language. We verify literacy level
through a brief native language literacy screener.

• Spanish speakers. While we believe that
speakers of all languages will benefit from the
curriculum, some instructional activities build
on students’ Spanish literacy. Therefore, we want
a large majority of students in the classes to be
native Spanish speakers to allow us to conduct a
full evaluation of all activities in the curriculum.

• Managed enrollment policy. To implement

the explicit literacy curriculum effectively, it is
necessary that most students receive the curriculum
from start to finish. This means that the students
are only allowed to enroll at a limited time at the
beginning of the class term. Programs with openenrollment policies can only be considered if the
large majority of learners enroll during a limited
time period at the beginning of the session.

• Sufficient class time. The curriculum is
designed in 60 one-hour modules of instruction
that are to be followed by one hour of instruction
normally provided (e.g., conversational, life
skills-type instruction). Therefore, in a 2 hour

class, 1 hour would be devoted to the explicit
literacy instruction. This suggests that, ideally,
a class would need to meet at least four times
per week for 2 hours each day for 15 weeks, for
a total of 120+ instructional hours. Other class
arrangements are possible as long as they can
implement the full curriculum within 15 weeks.

• Student retention. Student retention is key

to evaluating the curriculum because students
must receive a sufficient amount of instruction.
Therefore, it is important that the number of
students who fail to complete the entire scheduled
instruction is minimal. We need classes in the study
that have a record of high student retention.

• Do not already offer explicit literacy
instruction. Explicit instruction is the systematic,

sustained, and deliberate teaching of language and
literacy skills and of strategies for comprehension.
In ESL literacy, three essential features of this
type of instruction are (a) to help students see
language and literacy connections, (b) to make the
learning objectives transparent, and (c) to provide
a cumulative and sequential approach to literacy
learning. This is a new approach in adult ESL and
we believe that no programs currently provide this
type of instruction.

•Teachers. Selected sites must have at least

5 teachers who are willing to participate in the study.
Two of the teachers will be assigned randomly
to teach the explicit literacy curriculum and one
teacher will be a back-up who will only teach if either
of the two assigned teachers cannot participate.
All teachers should have some experience teaching
low-literate adult ESL students.

Curriculum for
Explicit Literacy Impact Study
The goal of the curriculum is to increase the English literacy skills of students who speak no
or little English and have limited literacy skills in their native language.
Explicit Literacy Principles

The curriculum uses an approach to teaching
known as “explicit teaching” and will focus on
the development of literacy skills, including skills
associated with learning to read in another language,
vocabulary development, listening/reading and
comprehension. The following are some key
principles of explicit literacy instruction:
• Curriculum, instruction, learning experiences
and assessment must be aligned.
• Teaching goals and curriculum must be clearly
defined.
• Lessons must involve instructional modeling
or demonstration, guided practice, corrective
feedback, and practice.
• Instruction must promote the development of
skills and strategies for making meaning.
• Teaching of oral language and literacy must
be systematic and purposeful.
• Curriculum and instruction must entail
transparent connection between language and
literacy.

Materials

The curriculum reflects research in second
language acquisition, reading, and adult literacy
development. A fully developed set of teaching
materials includes:
• illustrated and detailed lesson plans,
• demonstration materials on PowerPoint slides
or overheads,
• student handouts and manipulatives,
• a student workbook,
• a student reader with picture dictionaries, and
• a cd-rom of recorded reading in two speeds.

Lessons

Lessons are organized by themes of interest to
adult learners and learning activities are integrated
and contextualized. The lessons are structured in a
way that allows teachers to move through a series
of activities that include providing comprehensible
input and guided practice for pairs or small groups;
demonstrating the skill or strategy to be learned,
and conducting a short teacher-led review. Each
lesson will include teacher observations and
strategies for dealing with a multi-level classroom.
Lessons reflect an explicit approach to teaching that
emphasizes the deliberate teaching of skills and
strategies associated with literacy development, a
systematic effort to help students understand how
English works, ample opportunities for students
to engage in the content and alignment of the key
skills through demonstration, guided practice, and
ongoing informal assessment.
The curriculum is rich in illustrations and
photographs and uses multiple modes of learning
(visual, text-based, auditory). It seeks to promote
transfer of knowledge and builds on both the
linguistic knowledge and the real world knowledge
that adults bring to the learning process.

Teacher Training and Ongoing Support

The curriculum developers will conduct a week-long
training session for teachers that will familiarize
them with the instructional strategies that are part of
explicit teaching and to offer them opportunities to
engage in, and reflect on, teaching demonstrations.
Teachers will be supported during implementation
through on-line and on-call coaching, listserv
conversations with fellow teachers, web resources,
and a classroom visit.

Benefits of Participating in the
Explicit Literacy Impact Study
Selected programs will play an important part in a large-scale study that has the potential
to make very important contributions to the Adult English as a Second Language (ESL)
field specifically, and Adult Education in general. In addition, the programs will receive the
following direct benefits:
• The Explicit Literacy curriculum. Each

teacher assigned to deliver the explicit literacy
curriculum will receive a free copy of the
curriculum, including a copy of all corresponding
supplemental materials such as detailed lesson
plans, PowerPoint slides and other demonstration
materials, handouts and manipulatives, student
workbooks and reader with picture dictionary, and
a CD-ROM of recorded readings. This curriculum
is based upon the best research available. Programs
will also receive one additional copy of the
curriculum at the conclusion of the study, at no
charge.

• Training. Each teacher assigned to deliver

the explicit literacy curriculum will receive free
training in a one-week course offered prior to the
start of class. These teachers will receive training
in using the new explicit literacy curriculum and
materials designed to improve the literacy skills of
low-literate adult ESL learners. AIR teamed with
a curriculum developer, Literacy Work, to create
a training program built around principles of
adult learning, effective ESL teaching, and explicit
literacy instruction. In addition to the free training,
all corresponding training materials, travel, and
accommodations during the training are free of
charge to teachers assigned to deliver the explicit
literacy curriculum.

• Technical assistance. The curriculum developer

will be available to provide ongoing technical
assistance to the explicit literacy teachers through
a project web site and regular contact by telephone
and e-mail. The web site will provide useful
instructional modeling video clips and a listserv
forum, as well as other useful implementation tips
and materials. The developer will also be available
for questions by phone, and will hold weekly
phone calls with teachers. In addition, Literacy
Work and AIR staff will conduct at least one
class observation during the instructional period
to assess whether teachers are implementing the
curriculum faithfully. Literacy Work staff will
also provide feedback and additional training to
teachers as needed.

• Cost-free program evaluation. The Explicit

Literacy Impact study will use a random
assignment design to measure whether the
explicit literacy curriculum makes a difference in
improving the English literacy skills of low-literate
adults compared to the instruction usually offered.
We will provide your program with the results of
this evaluation for the students in your program.

In addition to the results of the impact study, we
will provide you with descriptive analyses on the
characteristics of the students in the program,
their test performance, patterns of attendance and
other data. There will be no burden on program
staff to obtain this information because research
staff will collect all post-test assessment data,
conduct interviews and track students.

Random Assignment in the
Explicit Literacy Impact Study
The random assignment of students and teachers is a key component of the study because
it enables us to assure that changes over time in student performance are due to the new
curriculum rather than to differences in teacher and student characteristics and/or naturally
occurring changes in the students and their social and educational environments.
What is random assignment?

Researchers use random assignment in impact
studies to form two statistically equivalent groups
of participants in the most objective way possible.
For this study, the term “participants” refers to
teachers and students who are eligible for and agree
to participate in the study. A list of participating
students will be created and those students on the
list will be randomly assigned to classrooms. Some
of the students will be assigned a classroom in which
the explicit literacy curriculum will be delivered
and some will be assigned to classrooms in which
the program’s existing curriculum is delivered.
Participating teachers will also be randomly assigned
to teach either the explicit literacy curriculum or
the existing curriculum.

How does random assignment work?

The random assignment process works much the
same as a lottery, so each participant—whether
a student or a teacher—has the same chance of
ending up in the class using the explicit literacy
instruction or to the one using the program’s
existing ESL curriculum. In this study, a computer
will be used to randomly assign both the teachers
and the students.

• Teachers. We will work with the programs to

identify teachers who agree to be in the study.
Among this group, we will randomly assign
the teachers to the explicit literacy group or
the existing ESL curriculum group. Only those
assigned to the explicit literacy group, along
with one back-up instructor, will be trained

in the curriculum. The back-up instructor
will only be used for this study if he/she is
needed as a substitute for a teacher that has
been randomly assigned to the explicit literacy
class but is unable to participate for unforeseen
reasons..
• Students. Students who register for ESL
instruction will be screened at intake to
determine their literacy level. Those who agree
to participate in the study and who are at a
low level of literacy (as determined through
a brief native language literacy screener) will
be randomly assigned, using a web-based
computer program, to either the explicit
literacy group or the existing ESL group. The
attached chart illustrates our planned process.

Why randomly assign teachers?

Teachers will be randomly assigned to either group
to ensure we are specifically testing the approach
to teaching ESL and not other factors that are tied
to their willingness to teach the new curriculum
or other teacher characteristics. Teachers who
volunteer to teach this new curriculum may differ,
in key ways that affect the outcome of the study,
from teachers who are not interested in teaching
the new curriculum. To control for this selection
bias we will select from a pool of teachers who are
willing to teach the explicit literacy curriculum and
randomly assign them to one group or the other.

Why randomly assign students?

This study will be measuring the effect of the
explicit literacy instruction relative to the existing

ESL instruction. To do so, we will need to compare
the outcomes of the students assigned to each
group after program completion. Similar to our
reason for randomly assigning teachers, if we were
to allow students to self-select into each group, we
may find that students who seek out explicit literacy
instruction are more motivated or differ in other
important ways to those who choose the existing
program. By randomly assigning students, the two
groups are virtually identical in all ways except that
one group receives the explicit literacy instruction,
while the other receives instruction as it is currently
being offered. This way, any differences in outcomes
among students in both groups can be attributed to
the explicit literacy curriculum.

Is random assignment a fair way to select
students who participate in the explicit
literacy curriculum?
Random assignment is fair because it gives all
participating students an equal chance of being
selected for the explicit literacy curriculum.
Personal factors play no role in whether a student
is selected.

No students will be denied services as a result of this
study. If students are not assigned to the explicit
literacy instruction, they are assigned to the class
that they would have participated in had this study
not been implemented.

What is required of the sites to do random
assignment?

The evaluation team will work with each site
to develop a random assignment process that
is not unduly burdensome. We are developing

a simple web-based system that will record
information regarding the students at intake and
will automatically assign students to classes using
a random process. We will work with the sites to
determine the most appropriate time to collect
the intake information, conduct the screening,
and determine the random assignment group. It is
likely that the screening process will occur during
the intake process and the random assignment will
occur when students show up for the first few days
of class. We also will work closely with sites to
ensure that teachers assigned to the explicit literacy
classes avoid disclosing information about features
and techniques in the explicit literacy curriculum.

Will participants assigned to the existing ESL
curriculum group ever have an opportunity to
receive instruction from the explicit literacy
curriculum?
Assuming the new curriculum proves effective,
which we do not know prior to an evaluation of
the curriculum’s impact, a copy of such curriculum
will be provided to all programs after the study’s
completion, and programs are free to use it with
students.

What happens if a participant “drops out” of
the program after being randomly assigned to
either group?

Students who drop out of the class assigned to them
will be viewed as if they were still part of the study.
Researchers will continue to collect follow-up data.
Students who return to the school during the study
period will be assigned to the group to which they
were originally assigned.

Random Assignment Flowchart

Student Application

Program Intake
Initial Screening of Students (program
administers usual screening and
assessment to identify beginning ESL
students who would normally attend
study classes)

Student Literacy Screening
(Native language assessment)

Students with Limited Native
Language Literacy
(Informed consent obtained)

Included in study classes

Students with Low or Advanced
Native Language Literacy Levels
Excluded from study, but students will
receive normal instruction provided by the
program in classes not in the research study

Random Assignment (Students)
First Two Days of Class
u Students check in at site
u Random assignment occurs
u Initial assessment and interview conducted
u Students informed of class assignment
u Students begin class

Existing ESL Instruction

Explicit Literacy Instruction

Assessment

Assessment

Analysis of Impact