Assessing Early Childhood Teachers' Use of Child Progress Monitoring to Individualize Teaching Practices

ATTACHMENT l

EDIT TEACHER INTERVIEW PROTOCOL

EDIT TEACHER INTERVIEW PROTOCOL

• During the interview, I will be taking some notes about our discussion. To help me keep track of your responses to the questions, with your permission, I will audio-record our conversation. Again, this information will not be shared with anyone from the program; it is meant simply to serve as a record of what you and I discussed. Is that okay? [INTERVIEWER: Start recording.]

• An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this collection is 0970-0355 and it expires 03/31/2018.

• Our project team is developing a tool called the Examining Data Informing Teaching (the EDIT) to understand how teachers use ongoing assessments. I’d like to ask about your experience using ongoing assessment—in other words, how you collect information about children’s progress on a regular basis. I’m also interested in hearing about how you use the information you collect.

• This interview will last about an hour. At the very end of our discussion, I will ask you to fill out a brief questionnaire. It takes about 5 minutes. As a reminder, your participation in this project is completely voluntary. All the information you share with me will remain private; no one from your program will see or hear your responses. You and others in your classroom, your students and your center will not be identified by name in any published reports. The information we obtain will be used for research and educational purposes to make the EDIT tool better and to teach researchers how to use the tool. Project documents will not refer to individual people, classrooms, or programs.

• Do you have any questions before we begin the interview?

• To start, I have a few quick questions about your classroom and the focal children that you selected:

How many teachers are in your classroom?

And how many children?

What is the age range of the children in your class?

IF CANNOT INFER FROM CLASS AGE RANGE: And how old is [FOCAL CHILD 1] and [FOCAL CHILD 2]

How did you choose FOCAL CHILDREN as the children to focus on?

INTERVIEWER: Probe to understand whether the teacher used evidence to select the focal children or based the selection on her own impressions.

What do you see as each focal child’s strengths and challenges in language and literacy?

How about social skills?

INTERVIEWER: If the teacher’s documentation did not include documentation of the video recorded activities, ask if the teacher has any documentation available. If so, review the documentation with the teacher.

I. CONDUCTING ASSESSMENTS

• First, I’d like to ask you about how you conduct ongoing assessments.

How do you use the information you collect through observations or other ongoing assessments to inform what you do with children to help them learn? Using [ONE OF THE TWO FOCAL CHILDREN] as an example, please walk me through your process for collecting information, reviewing the information, and then using the information to inform instruction.

INTERVIEWER: See if the teacher will give an overview of the whole process in response to this initial question. Then use the remaining questions to probe as needed.

What types of information do you collect about [ONE OF THE TWO FOCAL CHILDREN]?

Do you do activities like the ones you did in the videos? [INTERVIEWER: You may list the video activities.]

Observations/anecdotal records? [INTERVIEWER: Use the program’s terminology.]

Samples of the child’s work or play?

Others (maybe checklists, photos or videos?)?

How do you decide which learning objectives to collect information about for [ONE OF THE TWO FOCAL CHILDREN]?

In other words, how do you choose the knowledge, behavior, or skill that you will assess for that child on a given day?

[INTERVIEWER: Tailor probes based on documentation.]

• Do you plan what information you will collect or do you look for opportunities to asses as they present themselves in the day or both?

• IF YES: Tell me about the planning process. Is it part of lesson planning or a separate process?

How often do you collect information about [ONE OF THE TWO FOCAL CHILDREN]?

How do you decide how often to collect information on a specific learning objective?

• IF SOME LEARNING OBJECTIVES HAVE 3 OR MORE PIECES OF EVIDENCE: I noticed that you had several [name types of documentation] that provide information about [learning objective]. Can you tell me more about that? How do you decide when to collect more frequent assessment information and what do you do with that information?

• IF ONLY 1-2 PIECES OF EVIDENCE FOR EACH OBJECTIVE, ASK: Are there some learning objectives that you collect information about more frequently? How do you decide when to do this?

• Do you collect information with the same frequency for all children?

Do you assess a skill or learning objective in the same way each time, or do you use different activities?

• IF DIFFERENT ACTIVITIES: Can you provide an example and explain why you used different activities?

• IF SAME WAY: Can you provide an example and talk about how you decided to use that activity.

How do you know if the child you are working with understands what you are asking them to do during an assessment activity?

Does anyone help you collect information about [ONE OF THE TWO FOCAL CHILDREN], such as an assistant teacher?

Do parents or others at home help you collect information about [ONE OF THE TWO FOCAL CHILDREN]?

Do you take the same approach to collecting information about other children in the classroom?

Do you collect information with the same frequency for each child?

Do you collect information on the same learning objectives for each child?

• Do you do the same types of activities with each child?

Are there children in the classroom who are dual language learners or who have special needs?

IF YES: How do you approach collecting information for those children?

• IF YES: For a given activity, like [EXAMPLE FROM VIDEOS], would you do the activity in the same way with each child?

IF NO: Can you give me an example of how you might change an activity?

IF ANECDOTAL RECORDS PRESENT: When do you usually observe and take notes about a child? Do you just observe and take notes or are you interacting with the child at that time (e.g., asking questions, commenting on what they are doing or making suggestions)?

When do you collect most information about what a child knows and can do? Is it during a whole-class learning activity, small-group activities, free choice activities, or do you pull the child aside and work him or her separately?

Do you feel like gathering information about a child with the [ASSESSMENT SYSTEM/METHOD] takes time away from instruction? About how much time do you spend documenting and taking notes each day [if collect information less often, ask “each week”]?

IF USING A SYSTEM WITH COMPUTER ENTRY: Tell me about how you typically enter observations/anecdotal records?

How often do you enter data [photos, anecdotal records, information from checklists] into the system?

When do you enter data into the system?

Does anyone help you enter data into the system, such as assistant teacher?

• Do you record observation data in any other ways, such as taking notes by hand at first or using a checklist and then entering them into a computer?

• IF NOT A COMPUTERIZED SYSTEM: How do you store and organize the observations/anecdotal records, checklists, and other information that you collect?

INTERVIEWER: Add probes as needed based on documents and videos.

II. ORGANIZATION AND INTERPRETATION

In this next section, I’d like to hear about how you review the information that you collect.

How do you use work samples (in other words, examples of children work)?

How do you use information from activities like the ones you did in the videos?

INTERVIEWER: Probe about one of the focal children.

How do decide if [ONE OF THE TWO FOCAL CHILDREN] is making enough progress in a given learning objective or domain?

How do you decide if [ONE OF THE TWO FOCAL CHILDREN] has met a performance goal or developmental expectation?

IF USING A SYSTEM WITH RATINGS: Does anyone help you evaluate or rate [ONE OF THE TWO FOCAL CHILDREN]?

For example, do you rate [ONE OF THE TWO FOCAL CHILDREN] with your assistant?

Do you ask for parent input on ratings for [ONE OF THE TWO FOCAL CHILDREN]?

IF USING A SYSTEM WITH RATINGS: Does anyone assist you in reviewing and interpreting your ratings for [ONE OF THE TWO FOCAL CHILDREN]? Please tell me about how that works.

Do you receive any type of assistance to help you gauge whether your ratings are accurate, such as assistance from leadership staff, a mentor, or a peer teacher?

Does anyone help you think about the results and what they mean (for example, whether [ONE OF THE TWO FOCAL CHILDREN] is making adequate progress)?

How do you know if [ONE OF THE TWO FOCAL CHILDREN] needs additional support or a new approach?

Do you use benchmarks or guidelines?

If so, where do they come from (for example, are they provided by the assessment system or another source)?

How do you decide if [ONE OF THE TWO FOCAL CHILDREN] is making the progress that he or she should be making in a given learning objective or domain?

Do you look at progress in a different way when children are doing well like [FOCAL CHILD 1] than when there are some struggles like [FOCAL CHILD 2], or do you look at progress in the same way for all children?

How do you decide if the progress that they are making between reporting periods is enough for them to be where they need to be in those skills by the end of the year?

How often do you review the information that you collect to see [ONE OF THE TWO FOCAL CHILDREN]’s progress over time?

Do you ever look at the change in [ONE OF THE TWO FOCAL CHILDREN]’s progress within a reporting period, in addition to looking at the change between reporting periods?

How do you review the information that you collect?

Do you look at how [ONE OF THE TWO FOCAL CHILDREN] is doing in each learning domain and objective?

• For example, do you compare how [ONE OF THE TWO FOCAL CHILDREN] is doing in different domains, such as language, literacy, or social-emotional development?

• If so, do you look at the change in [ONE OF THE TWO FOCAL CHILDREN]’s progress over time?

• Do you compare [ONE OF THE TWO FOCAL CHILDREN]’s progress to the progress of other children in the class? If so, how?

• Do you look at how the entire class is doing in each learning domain? Do you look by objective? For example, do you look at a particular domain, like literacy, and see how all children are doing?

• If so, do you look at the change in progress for the entire class over time?

What do you do if you see inconsistency in [ONE OF THE TWO FOCAL CHILDREN]’s performance on the same task over time?

• For example, if [ONE OF THE TWO FOCAL CHILDREN] does well on a task one day and poorly on that task the next day, how do you explain that?

We’ve been talking about what you do with [FOCAL CHILDREN]. Now let’s think about the other children in the class. Is there anything else you do, or anything you do differently, when you review the information you collect for the other children in the class?

Assessment information can be used for a lot of different purposes. How do you use the information from the [ASSESSMENT] (e.g., report to managers, communicate child progress to families, use for planning lessons or other instructional decisions)?

Do you share information from the assessments with families?

• IF YES: How do you decide what you share?

• How do you share it with families?

INTERVIEWER: Add probes as needed based on documents and videos.

III. LESSON PLANNING AND INSTRUCTIONAL DECISIONS

Now I would like to talk about planning for instruction. We use the word ‘instruction’ to include all that you do to help children learn and grow, including center activities that you provide, small group interactions, books, songs, questions, and other interactions that you have with children. Some instruction may be an “in the moment,” but most teachers plan at least some of their instruction.

How do you plan your instruction? [INTERVIEWER: Listen for daily, weekly, monthly.]

What information do you use when planning? [INTERVIEWER: Many teachers may select a unit aligned with a season and planning may be divorced from assessment; or teachers may depend only on child interest in a topic. They are trusting the curriculum to ‘cover’ all the skills needed.]

When you are planning what you will do for the next [FILL IN TEACHER TIME PERIOD FOR PLANNING] do you also plan to collect specific information about children? How do you decide what you will collect?

Do you use any of the information that you collect about children to inform your planning, such as decisions about what to teach, which activities or units to use, or how to support children’s learning?

IF YES: Please give me an example of how you do this. Walk me through how you use the information in planning instruction or making instructional decisions.

• Do you consider just current performance or do you also consider how much progress children are making in a particular area?

• You mentioned that you draw on [INSERT EVIDENCE TEACHER USES FOR ASSESSMENT] to make decisions about child progress. How does it affect your decisions about instruction and about what to assess/observe?

• Please give an example of an instructional decision that you made that was based on information that you collected about a child. [INTERVIEWER: Prompt from documentation if available.]

INTERVIEWER: Listen for at the individual, small group, and/or whole class level.

What ideas or sources do you use in selecting instructional strategies or approaches?

• Instructional approaches could be additional practice, how you group children, prompting or questioning or using pictures or objects to help children understand something, etc.

INTERVIEWER: If not mentioned, probe about use of adaptations or modifications designed to meet specific child needs; peer supports; adult supports; and environmental supports.

How do you communicate with other team members, like your assistant, on how to work with specific children?

Does anyone give you input or guidance on how to use the information you collect to guide lesson planning, such as a mentor?

FOR EACH TYPE OF PLANNING DOCUMENT INCLUDED IN THE DOCUMENTATION: How do you use the [DOCUMENT NAME]?

Did you personally create the [DOCUMENT NAME]?

• If so, how did you create it?

• What information did you draw on to create it?

• How often do you create or revise the [DOCUMENT NAME]?

For which children do you create the [DOCUMENT NAME]?

• How do you use the [DOCUMENT NAME]?

Do all children experience all the activities?

IF NO: How do you decide which activities or lessons children will get?

How do you group children?

IF THE TEACHER USES SMALL GROUPS: How do you determine which children are in which groups?

Do you plan additional activities for individual children?

IF YES: What types of strategies do you use in an individual activity?

Do you ever decide to have a child do something differently from others in the class, for example, use different materials or only do parts of an activity or do more of something? Do you individualize instruction within a group by using any of the following?

INTERVIEWER: If the teacher mentions any of the following, ask for a description/example.

• Plan different types of questions for different children?

• Use cues or different prompts for some children?

• More opportunities for practice?

• Change something in the environment? (e.g. changing seating or positioning, going to a quieter area, providing visual prompts, etc.)

• Use peers to help a child or model for a child (Peer strategies)?

• [Other adaptations and modifications] Are there any other changes that you make to an activity to support an individual child’s learning?

• [If teacher mentions child interests or strengths] How do you incorporate a child’s interests or strengths into instruction?

Are you ever in a situation where you want to make one of these modifications for a child but are unable to? [INTERVIEWER: Note that we are trying to get at if the teacher faced any logistical constraints.]

Do you have a method for recording whether your instructional plan was implemented?

How do you evaluate the success of your instructional plans?

• How do you know whether your small groups or individualized learning plans work?

• Do you have a method to document and keep track of whether individual interventions were successful for specific children?

• How do you decide to continue, change, or stop the individual intervention? 

• [IF NOT DISCUSSED ALREADY:] Do you collect additional information (re-assess or conduct additional observations) during or after the individual learning plan? What do you do with that information? Please give an example.

• Can you share an example of an individual intervention that you have used with particular children and what were the results?

INTERVIEWER: Add probes as needed based on documents and videos.

IV. SUCCESSES AND CHALLENGES

Finally, I would like you to reflect on your experience using assessments to individualize instruction in your classroom.

What would you consider successes? In other words, what are the most useful aspects of your use of ongoing assessment from your perspective?

What factors or circumstances contributed to these successes?

What challenges have you experienced with using ongoing assessment information?

• Are there any parts of the assessment process in your program that you find are not particularly useful? If so, tell me more.

What did you do to address these challenges, and how successful has that been?

V. WRAP-UP

Is there anything else we haven’t covered that you’d like us to know about using assessments in your classroom?

REMIND TEACHER TO COMPLETE SHORT SAQ

PROVIDE GIFT CARD AND FILL OUT RECEIPT

• LET TEACHER KNOW THAT DEBRIEF IS COMING SOON—ASK FOR TIMES THAT ARE GENERALLY GOOD

• Thanks again for your time and your valuable insights!