Classroom Observation Protocol and Interview

Classroom Observation Protocol & Interview¹

Teacher Observed: Date of Observation: School: Grade:

Number of students observed: Observation start time: Observation end time:

Observation Setting:  Pull-out, small ELL group  Immersion classroom  Dual-language/Bilingual classroom  Sheltered classroom  Other ________

Domain Code

Domain/ Indicator

Mark "X" if observed "NA" if not applicable

Evidence Notes
(brief description of the nature and quality of this domain, including examples/quotes)

Dimension: Instructional Setting

A

ELL/Reading & Writing Materials

"X" if observed

 

Independent reading materials are easily accessible to all students.

 

 

There are sufficient copies of reading materials for all students in the class.

 

 

A leveled, well-organized collection of books exists.

 

 

The book collection includes a variety of genres and topics.

 

 

Support materials for writing are available for students (e.g., visible alphabet poster, word cards, Word Wall)

B

Space and Room Arrangement

"X" if observed

 

Room arrangement allows for whole group instruction

 

 

Room arrangement allows for small group instruction

 

 

There are designated learning centers set up for independent work.

 

 

There are comfortable places for students to read.

 

 

All students can see and hear during whole class instruction.

 

 

Linguistic and cultural diversity are valued in the classroom (e.g., there are visual displays and examples of traditions from multiple regions and cultures)

C

Displays

"X" if observed

 

There are display materials related to reading on the walls.

 

 

There are display materials on the walls produced by students or collaboratively by the teacher and students.

 

 

There are display materials that are useful in instructions; that is, students can learn from them, they aren't merely for decoration

 

 

Dimension: General Instructional Practices

D

Assessment Practices

"X" if observed

 

Teacher monitors student progress throughout the lesson with on-the-run assessments, anecdotal records, checklists, etc.

 

 

Teacher responds to students’ needs on the spot by shifting instructional focus, pace, or emphasis while teaching

 

 

E

General Instruction

"X" if observed

Teacher models skills and strategies during lesson.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher makes relationships between concepts overt

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher emphasizes distinctive features of new concepts (broad range of examples and non-examples; examples used to show relevant and irrelevant features).

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher provides prompts and cues in how to use strategies, skills, and concepts (e.g., guided practice, scaffolds, steps, and procedures).

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher teaches difficult vocabulary prior to lesson, or during lesson as needed.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher receives a high level of response accuracy in context of lesson objectives (e.g., spelling accuracy on spelling test vs. spelling accuracy on a written assignment).

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher engages in ongoing monitoring of student understanding and performance during lesson (elicits responses from all students, including students having difficulty with task at hand; calls on range of students; poses questions that students can answer).

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

F

Differentiated Instruction

"X" if observed

 

Teacher scaffolds learning for students appropriately (provides the right amount of support to move students from one level of understanding to a higher level).

 

 

Teacher provides comprehensible input (e.g., gestures, real-life objects, chanting, photos, etc.) to help student understanding during lessons.

 

 

Teacher speaks appropriately given students’ proficiency level (e.g., slower rate, enunciation, and simple sentence structures).

 

 

Teacher modifies instruction for students as needed during the lesson (breaks down task into smaller/simpler components; modifies assignments to promote success; provides specialized instruction).

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher provides extra instruction, practice, or review for students having difficulty with task at hand).

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

G

English Language Development

"X" if observed

Teacher adjusts own use of English to make concepts comprehensible

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher uses visuals or manipulatives to teach contents.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher gives oral directions that are clear and appropriate for level of students’ English language development.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher structures opportunities for students to speak.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher selects and incorporates students’ responses, ideas, examples, and experiences into the lesson.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher provides explicit instruction in English language use, and includes the use of cues and prompts.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher gives students wait time to respond to questions.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher encourages students to give elaborate responses (prompts students to expand on one-word or short answers; prompts student to provide more information; prompts student to give more complete responses).

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher and/or students strategically use students’ native language to help students understand content.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher uses gestures and facial expressions in teaching vocabulary and clarifying meaning of content.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Rate the quality of opportunities for independent practice (i.e. those provided for students to practice reading, writing, and speaking English).

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

H

Teacher-Student Interactions

"X" if observed

 

Teacher creates an anxiety-free setting for students. Teacher language is calming and encouraging.

Talk is centered on what students are learning rather than on controlling behavior.

 

 

Teacher provides frequent opportunities for interaction and dialogue between teachers and students as well as among students about lesson concepts; teacher encourages elaboration.

 

 

The timing for teaching points is appropriate (e.g., teacher does not interfere in a matter that interferes with children's reading & writing, teacher allows sufficient wait time after asking a question, etc.)

 

 

Students are actively engaged throughout the lesson.

Expectations for student behavior are clearly communicated in multiple ways and students appear to follow routines (resulting in a “smooth” classroom flow).

Teacher secures and maintains student attention during lesson, as needed.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher gives feedback on academic performance (reiterates, clarifies, reinforces; communicates clearly what students did correctly or how they can improve; focuses on lesson objective).

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher focuses on performance specifics (i.e., not just “Good”, or “Wrong”).

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Dimension: Lesson Implementation

I

Thematic Unit (Whole Group) Instruction

"X" if observed

 

Setting the Scene: The teacher…

- Builds background knowledge.

- Introduces strategies, skills, and the purpose of the lesson.

- Makes explicit connections to the text (e.g., “learn by doing” activities, give student opportunities to practice using relevant language, etc.).

 

 

The teacher provides instruction in oral language development connected to a content area theme.

 

 

The teacher conducts shared reading to develop students’ literacy skills in the context of the theme.

 

 

The teacher conducts shared writing to develop students’ literacy skills in the context of the theme.

 

 

J

Small Group Instruction

"X" if observed

 

Setting the Scene: The teacher…

- Builds background knowledge.

- Introduces strategies, skills, and the purpose of the lesson.

- Makes explicit connections to the text (e.g., “learn by doing” activities, give student opportunities to practice using relevant language, etc.).

 

 

Reading the Text: The teacher…

- Introduces the book, genre, characters, etc. (“book talk”).

- Provides skill instruction in phonics and vocabulary.

- Introduces and models reading strategies for students.

- Provides students opportunities to read independently.

- Observes students reading and provides support.

 

 

Returning to the Text: The teacher…

- Provides explicit instruction on reading strategy use, language development, literacy skills, phonics, and word study.

- Gives students the opportunity to revisit the featured reading skill/strategy (e.g., by participating in an activity or responding to teacher’s questions).

- Encourages students to discuss what they have read.

 

 

Responding to the Text: The teacher…

- Engages students in discussion about the text.

- Provides students an opportunity to write about the text.

- Creates opportunities for academic language use.

- Develops language via corrective feedback.

- Extends language via responsive feedback.

 

 

Teacher checks student comprehension of text by asking questions.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher engages students in meaningful interactions about text.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

K

Phonics/Word Study Instruction

"X" if observed

 

Instructional configuration:

- Instruction was conducted whole group.

- Students were grouped by stage of language acquisition.

- Students were grouped by primary language.

 

 

- Students engage in singing or chanting to learn phonics/word study skills.

 

 

- Students are exposed to manipulatives such as word wall cards, word sort cards, letter tiles, dry erase boards, etc. to learn phonics/word study skills.

 

 

- Students have opportunities to engage in interactive writing.

Teacher provides systematic, explicit instruction in decoding skills (primary) or on sentence and word structure (intermediate).

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher provides systematic, explicit instruction in phonemic awareness.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher provides systematic, explicit instruction in letter-sound correspondence.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Teacher provides systematic, explicit instruction in vocabulary and vocabulary development.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

L

General Instructional Environment

Rate the extent to which students are “on-task” during literacy activities.²

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

The length of the literacy activities appears to be the right length for most students.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

Transitions between instructional activities are short and efficient.

1	2	3	4
Not Effective	Partially Effective	Moderately Effective	Very Effective

If teacher is in Treatment School, skip to Materials 1 below. If teacher is in Control School, skip to Materials 2 below.

Materials 1.

MATERIALS USED DURING OBSERVATION – For TREATMENT CLASSROOMS ONLY
Whole Group	Small Group	Phonics/Word Study	Other
 Thematic Unit Teacher’s Guide	 Leveled guided reading books	 Audio CD	
 Chant Poster	 Newcomer book	 Phonics Song Charts (1-3 only)	
 Concept Poster	 Benchmark books	 Take-Home Books (1-3 only)	
 Newcomer Book	 Reading Strategy Cards	 Word Wall Starters (1-3 only)
 Oral Language Audio CD	 Other ___________________	 Enhanced Phonics Song Charts (1-3 only)
 Shared Writing Card	 Other _________________	 Enhanced Phonics Take-Home Books (1-3 only)
 Writing Resource Guide		 Enhanced Phonics Audio CD (1-3 only)
 Picture Cards		 Word Study Song Charts
 Language Learning Masters		 Word Study Activity Masters
 Home-School Connection Masters		 Word Sort Card Masters
 Standardized Test Practice Masters		 Other ___________________
 Big Book/Class Selection/Small Book		 Other ___________________
 Big Book/Class Selection Audio CD
 Total Physical Response (TPR) Cards
 Manipulative Chart (1-3 only)
 Transparencies (4-5 only)
 Academic Language Builder (4-5 only)
 Other ___________________
 Other ___________________

Materials 2.

MATERIALS USED DURING OBSERVATION – FOR CONTROL CLASSROOMS ONLY
Whole Group	Small Group	Phonics/Word Study	Other
 Teacher’s Guide	 Leveled guided reading books	 Song Charts	
 Charts	 Benchmark books	 Take-Home Books	
 Posters	 Reading Strategy Cards	 Word Wall Starters	
 Audio CDs	 Other ___________________	 Word Sorts
 Picture Cards	 Other ___________________	 Audio CD
 Transparencies		 Other ___________________
 Big Book/Class Selection/Small Book		 Other ___________________
 Other ___________________
 Other ___________________

The U.S. Department of Education wants to protect the privacy of individuals who participate in surveys.  Your answers will be combined with other surveys, and no one will know how you answered the questions.  This survey is authorized by law (1) Sections 171(b) and 173 of the Education Sciences Reform Act of 2002, Pub. L. 107-279 (2002); and (2) Section 9601 of the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind (NCLB) Act of 2001 (Pub. L. 107-110). Responses to this data collection will be used only for statistical purposes.  The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific district or individual.  We will not provide information that identifies you or your district to anyone outside the study team, except as required by law. 

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is xxxx-xxxx. The time required to complete this information collection is estimated to average 5 minutes per respondent, including the time to review instructions, gather the data needed, and complete and review the information collected. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202. If you have comments or concerns regarding the status of your individual submission of this form, write directly to: U.S. Department of Education, Institute of Education Sciences, 555 New Jersey Avenue, NW, Washington, DC 20208.

CLASSROOM OBSERVATION TRANSCRIPT

Date:

School:

Teacher observed:

Grade:

Number of students:

Observation start time: Observation end time:

Observation purpose:

Insert time during transition points.

Use Methodological Notes [MN], Observational Comments [OC], Observational Notes [ON] or Theoretical Notes [TN] to demarcate your notes in [brackets].

ELEMENTS OF OBSERVATION TRANSCRIPTS

MN: Methodological Note – Use this to note anything related to the observation process and data collection. Examples include your battery running out, inability to hear, parents leaning over your computer, etc..

Example.

[MN: My battery ran out and I had to switch to my notepad. I missed some of the student teacher dialogue as a result.]

OC: Observation Comment – Use this to note actions (emotional and physical) and behaviors observed in the classroom. Examples include teacher gestures, volume, and tone, student behaviors, classroom dynamics – things that may not be reflected in the mere dialogue you are transcribing.

Example.

[OC: A disengaged student got up from his desk to whisper to his neighbor. The teacher stopped instruction and gave

him a disapproving stare with her hands on her hips.]

ON: Observation Note – Use this to note what you think an activity or observation might mean with low or high level of inference. Examples may include what teachers don’t do in instruction, how a teacher chooses to work with/respond to a student and what that might mean, or how students do/don’t work together and what that might say about the learning environment and dynamic in the classroom. These notes all relate to what it might mean.

Example.

[ON: The teacher continues to ask an ESL student the same comprehension question, despite the fact that the

student did not seem able to read the text. She doesn’t direct him to glean meaning from the pictures and she doesn’t try to make connections between the story content and the student’s familiar environment or experiences. She offers no comprehensible input. It seems like she has not spent any time building his background or assessing his ability level. She expects the same from him as her English speaking students.]

TN: Theoretical Note – Use this to note your interpretations or theoretical ideas of what you observed. These should link back to the research questions and study purpose. It connects what you observed to the greater purpose of the study. Examples may include interpretation related to level of implementation, teacher capacity, material use, or student engagement and learning.

Example.

[TN: This teacher has a solid grasp of small group instruction. She differentiates instruction as needed and assesses students as she progresses through the lesson.]

INTERVIEW PROTOCOL FOR TEACHERS

instructions: For each classroom observation performed the following pre-observation question and post-observation interview needs to be performed. Please note that all instructions for observers/interviewers, who are administering the interview questions, are italicized. If possible, researchers should review the teacher’s implementation log from the prior week to identify any areas of low implementation or concerns. These areas should be addressed with the teacher during the interview time. The interview should last about 20-30 minutes. If there is insufficient interview time, the researcher may want to consider emailing interview questions in advance and prioritizing key questions that must be addressed during the interview time. Prior to the site visit, note the names of the school principal – you or the teacher may refer to him/her by name during the interview.

Date: ____________________ Time:____________________

Interviewee(s):_________________________________

School: ______________________________________

Interviewer: ___________________________________

Pre-observation questions

If possible, email teachers in advance to remind them of the observation day and time as well as to collect information about the materials they will be using during the observation. Also request the lesson and lesson purpose for the day of observation. If you can’t collect this information in advance, try to touch base with the teacher in the classroom prior to your observation. In this case, upon entering the classroom say “hi” to the teacher and express appreciation for allowing you to observe the lesson. Ask the teacher to just do what she/he would normally do and that you are just going to unobtrusively observe. Ask the following brief questions:

1. What ESL materials will you be using today? Is this the first day students will be exposed to these materials?

2. What is your intent or purpose for the lesson today? What do you want to accomplish?

Post-Observation Interview

Thank the teacher for allowing you to observe the class and try to validate or compliment something that happened in the class during your observation. Remind the teacher of the purpose of this observation in conjunction with the research study as well as your role in protecting the anonymity of the teacher’s responses. Please note that the post-observation interview should be completed the same day as the actual observation if at all possible. Please ask the following:

about today’s lesson

1. Did the instruction I observed go as you intended?  Yes  No

2. a. Did you accomplish what you wanted to?  Yes  No

b. Why or why not?

3. a. Was today’s instruction pretty representative of how you do things?  Yes  No

b. If not, why?

about your class/GROUP OF STUDENTS

4. a. Do you have a wide range of levels of literacy development across the students in

your class?  Yes  No

If yes, please describe the range.

b. Do you have a wide range of stages of language acquisition across the students in

your class?  Yes  No

If yes, please describe the range.

5. a. Do you have any special education students who participate in your ELL instruction?

 Yes  No

b. Do they receive additional services? If yes, what type?

about your ELL instruction

6. Describe your approach to ESL instruction and the materials you use. [Probe on nature/frequency of instruction with ELLs]

7. a. What assessments do you use with your ELL students?

b. If teacher is in CONTROL school, skip to 7c. What assessments, other than OWE/RISE, do you used with your students (if any)?

c. How often do you assess your ELL students?

d. How does assessment guide your teaching? Can you give me a recent example?

8. a. How do you group students for small-group instruction (e.g., by language proficiency, literacy level, primary language, etc)?

1. Do you use flexible grouping?  Yes  No

2. Have you changed these groups often?  Yes  No

3. If yes, on what did you base regrouping students?

9. How are you differentiating your instruction for your ELL students?

10. If teacher is in CONTROL school, skip to 11. Have you used any supplemental ELL materials to the OWE curriculum?  Yes  No

a. If yes, what materials?

b. If no, have you found a need to supplement your curriculum?  Yes  No

11. What kind of professional development and support do you receive in ELL instruction?

[Probe: How often have you received support? What is the nature of support you’ve received?]

[Probe: How would you describe the quality of support you’ve received from your RISE facilitator?]

12. a. Are you receiving adequate support at the school level for your ELL instruction?

 Yes  No

b. Are you receiving adequate support at the district level for your ELL instruction?

 Yes  No

c. Do you have opportunities to confer with other ELL teachers?

 Yes  No

d. In what ways/areas do you need additional support?

e. What are the most significant ways in which the OWE/RISE programs have influenced your ELL

instruction?

13. How well do your current ELL materials align with your district’s ESL/ELD standards?

14. How well do your current ELL materials meet the content, oral language, literacy, and academic language objectives necessary for optimal student learning?

15. What aspects of your current ELL program/materials have been most valuable to your ELL instruction?

16. What kind of barriers, if any, have you experienced in your ELL instruction? For teachers in TREATMENT schools, what kind of barriers have you experienced in your implementation of the OWE and RISE programs?

17. What, if any, have been the shortcomings of your current ELL materials? For teachers in TREATMENT schools, what have been the shortcomings of the OWE/RISE programs? How could these issues be addressed?

ABOUT STUDENT IMPACTS

18. a. Have your current ELL materials had an impact on students’ interest in reading?

 Yes  No

b. If yes, what have you observed that leads you to believe this?

19. a. Based on your observations and assessments, what impact has your implementation of your current ELL program had on student language acquisition and literacy development?

b. Is this level and rate of learning typical compared to other students?

The U.S. Department of Education wants to protect the privacy of individuals who participate in surveys.  Your answers will be combined with other surveys, and no one will know how you answered the questions.  This survey is authorized by law (1) Sections 171(b) and 173 of the Education Sciences Reform Act of 2002, Pub. L. 107-279 (2002); and (2) Section 9601 of the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind (NCLB) Act of 2001 (Pub. L. 107-110). Responses to this data collection will be used only for statistical purposes.  The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific district or individual.  We will not provide information that identifies you or your district to anyone outside the study team, except as required by law. 

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is xxxx-xxxx. The time required to complete this information collection is estimated to average 5 minutes per respondent, including the time to review instructions, gather the data needed, and complete and review the information collected. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202. If you have comments or concerns regarding the status of your individual submission of this form, write directly to: U.S. Department of Education, Institute of Education Sciences, 555 New Jersey Avenue, NW, Washington, DC 20208.

1 This protocol includes items adapted from the English Language Learner Classroom Observation Instrument (ELLCOI). Psychometric properties of the original instrument are reported in Baker et al. (2004). Adaptations have been made with the understanding that the psychometric properties of the ELLCOI pertain solely to the original scale. Researchers will pilot test the adapted instruments with a small sample of teachers prior to use to ensure that the instrument retains important psychometric properties, in particular reliability and validity of use.

2 When rating this item, consider that the more “on task” a teacher is, the “more effective” s/he is in practice.

A Study of the Differential Effects of ELL Training and Materials

Exhibit B: Data Collection Instruments