Instructional Leader Interview Protocol

Appendix F

Instructional Leader Interview Protocol

Instructional Leader Interview Protocol

Toolkit to Support Evidence-Based Algebra Instruction in Middle and High School

1. Introduction [5 minutes]

Thank you for agreeing to meet with us and discuss your experience supporting teachers using the Algebra Toolkit and implementing the recommended practices in their classrooms during this school year. My name is [NAME OF FACILITATOR] and this is my colleague, [NAME OF NOTE TAKER]. We are from the Regional Educational Laboratory Central, also known as REL Central, which is funded by the U.S. Department of Education and operated by Mathematica and its partners. REL Central is conducting this study.

[IF INSTRUCTIONAL LEADER IS ALSO A TEACHER (SUCH AS A LEAD TEACHER) WHO USED THE TOOLKIT IN A TEACHING CAPACITY: We understand that you had a dual role during this study as both an instructional leader who guided other teachers through using the Toolkit and facilitated the PLC sessions, as well as a teacher who used the Toolkit in your own classroom. We’d like you to answer the questions in this interview from only the perspective of an instructional leader and focusing on your experiences as facilitator of the PLC sessions. We have a separate focus group in which we will gather insights from the teacher perspective.]

This conversation with you today will help us understand what has gone well with using the Toolkit resources and supporting teachers using the Toolkit to implement the recommendations in their classrooms. We’d also like to learn what challenges you encountered and how we can further improve the Toolkit and its resources. Your insights are extremely valuable. Our conversation today will take about 60 minutes, and afterwards we will send you a $50 gift card to thank you for your participation. We will record our conversation and take notes; this will help ensure we correctly document what you say. The recording will be erased after we have completed writing up our notes. If you’d like to say something that you prefer not to be recorded, please let us know and we will pause the recorder. [START THE RECORDER].

Now I want to review the consent statement we sent with the email discussion for this interview and then obtain your verbal consent to participate [READ THROUGH CONSENT FORM FROM THIRD PARAGRAPH ONWARDS]. Do you have any questions? [IF NO QUESTIONS, OBTAIN THE INSTRUCTIONAL LEADER’S VERBAL CONSENT ON THE RECORDING BY ASKING THEM TO STATE THEIR NAME AND IF THEY AGREE TO PARTICIPATE IN THE INTERVIEW].

We will keep the information you share with us private. We will only use your first name as we talk today. If we use quotes from this interview, we will not attribute them to you. Nobody except the study team’s researchers will see any of the notes from our discussion.

We are also speaking with other instructional leaders who supported teachers using the Toolkit during this school year. We will take our notes from this discussion today and combine them with the information from the interviews with other instructional leaders. This will help us understand the instructional leaders’ experiences with using the Toolkit and supporting teachers as they used the Toolkit and implemented the recommended instructional practices.

Please keep in mind:

• There are no right or wrong answers to our questions and how you respond to questions will not negatively affect you. We are just here to learn about your experiences with using the Toolkit and supporting teachers using the Toolkit to implement the recommended instructional practices.

• You do not need to answer any questions you do not wish to answer and refusing to participate or answer a question will not negatively affect you in any way.

Do you have any questions before we get started?

Our conversation today will focus on 5 main topics. We’d like to hear about how leading and facilitating PLC sessions went; your understanding and use of the Toolkit resources; how supporting teachers’ use of the Toolkit went; how supporting teachers’ understanding and implementation of the three recommendations went; and your thoughts on sustaining the three recommendations in the classroom.

2. PLC leadership and facilitation [10 minutes]

As the instructional leader, you were responsible for coordinating, planning for, and facilitating the PLC sessions within each of the Toolkit’s four modules. I’d like to start by asking you a few questions about those responsibilities.

1. In what ways were the Toolkit resources useful as you prepared for and facilitated the PLC sessions for teachers, and do you have suggested improvements to make them more useful?

   1. Probe, if necessary, on the resources: (1) facilitator’s guides; (2) calendars; (3) agenda

   2. Were there any additional resources that you think could’ve helped with PLC session preparation or facilitation?

2. What went well with facilitating the PLC sessions for teachers?

   1. Probe if necessary: Were teachers engaged; did you set norms and did teachers follow them; were you able to cover the intended content during the PLC sessions; were teachers prepared for their PLC sessions?

3. What challenges did you experience facilitating the PLC sessions for teachers?

   1. What strategies did you use to attempt to address, bypass, or overcome these challenges?

4. How easy or difficult did you find it to schedule the PLC sessions around teacher availability? Please describe what went well and what was challenging.

   1. If you encountered challenges, please describe how you attempted to overcome them.

3. Instructional Leader Understanding and Use of the Toolkit Resources [10 minutes]

Now, I’d like to ask a few questions about your perceptions of various Toolkit resources and activities. As you recall, the main resource for instructional leaders like yourself was a facilitator’s guide. There were additional resources, which were mainly geared towards teachers, but you may have utilized them as well. These include online interactive activities as well as diagnostic and monitoring tools.

[NOTE FOR FACILITATOR IF NEEDED: the available diagnostic and monitoring tools include the Plan-Do-Study-Act (PDSA) tool, the teacher self-reflection tool, the visitation tool, the student survey, and the student knowledge assessment tool].

1. To what extent do you feel that you understood what the Toolkit was, including its various resources and its purpose?

2. How easy was it to use the Toolkit’s resources? Were any of the Toolkit’s resources easier to use than others? What made these resources easier to use?

   1. Instruction to facilitator: Probe on each resource if necessary: (1) facilitator’s guide, (2) online interactive activities, (3) diagnostic and monitoring tools, and (4) administrator guide.

3. What challenges did you experience using the Toolkit’s resources? Were any resources particularly challenging to use?

   1. Instruction to facilitator: Probe on each resource if necessary: (1) facilitator’s guide, (2) online interactive activities, (3) diagnostic and monitoring tools, and (4) administrator guide.

   2. What strategies did you use to attempt to address, bypass, or overcome these challenges?

   3. What suggestions do you have for improving the Toolkit resources?

4. Did you try to integrate the Toolkit resources and activities into existing teacher supports?

   1. IF YES: What worked well and what was challenging?

   2. IF NO: Were there barriers to integrating them? Please describe.

5. What advice would you give to someone who is planning to facilitate a similar PLC using this toolkit next semester? [possible probes for the time involved to prepare, how to keep participants engaged, how to convince administrators of the value]

4. Supporting Teacher Use of the Toolkit’s Resources and Recommendations [25 minutes]

Next, I’d like to ask you about your experience supporting teachers as they used the Toolkit resources and as they learned about and used the three recommendations for improving their algebra instruction through Plan-Do-Study-Act (or PDSA) cycles. As a reminder, the Toolkit’s resources include the online interactive activities and the diagnostic and monitoring tools. We’d like to hear what went well, what was challenging, and how you and the teachers overcame these challenges.

As a reminder, the three recommendations for improving algebra instruction are:

(1) Use solved problems to engage students in analyzing algebraic reasoning and strategy,

(2) Teach students to utilize the structure of algebraic representations, and

(3) Teach students to intentionally choose from alternative algebraic strategies when solving problems.

1. The first stage of the PDSA cycle is “Plan,” where teachers create individual plans to implement the recommendation into their instruction. In the “Plan” phase, teachers define what successful implementation of the recommendation will look like for themselves and their students. They also choose data tools that can measure how well the recommendation is being implemented and whether it is contributing to intended outcomes. Please describe what your support typically looked like with teachers as they used the Toolkit resources to plan for using the recommended practices in their classroom and collect accompanying data.

   1. What went well with supporting teachers as they used the Toolkit resources to plan for using the recommended practices and collect accompanying data?

   2. What was challenging with supporting teachers as they used the Toolkit resources to plan, and what strategies did you use to attempt to address, bypass, or overcome those challenges?

2. After the “Plan” phase is the “Do” phase, where teachers implement the recommendation according to the plan they developed in the “Plan” phase. As a part of the “Do” phase, they are also expected to collect data from their students to get a sense for which topics need additional attention. Please describe the process you followed for providing support and guidance to teachers as they began using the recommendations in their classrooms and collecting data to (1) capture how implementation was going and (2) record their results.

   1. What went well with the data collection support?

   2. What was challenging, and how did you address, bypass, or overcome those challenges?

   3. In your opinion, were any of the Toolkit resources especially useful for helping teachers learn about the three recommendations or how to implement the recommendations in their classroom?

   4. Were there any Toolkit resources that did not appear to help teachers learn about the three recommendations or how to implement the recommendations in their classroom?

      1. Could you describe why the resource(s) were not useful?

      2. Please describe any suggestions for how those resources could be adjusted to make them more useful for helping teachers learn about and implement the recommendations.

3. Following the “Do” phase is the “Study” phase, where teachers review the results of their data and reflect on how the results compare with the definition of successful implementation of the recommendation defined during the “Plan” phase. Please describe the process you followed for providing support and guidance to teachers as they reviewed and analyzed data to assess the effects of the practices in their classrooms.

   1. What went well with the data analysis support?

   2. What was challenging and how did you address, bypass, or overcome those challenges?

   3. How did you see teachers use that data to determine strategies for improving their teaching practices?

4. The final phase of the PDSA cycle is the “Act” phase, where teachers use insights from the “Study” phase to determine how they will revise and strengthen their implementation of the recommendation. Please describe how you supported teachers as they sought to strengthen their implementation of the recommendation as the final stage of the PDSA cycle.

   1. To what extent did you see teachers use insights from the “Study” phase to revise and strengthen the way they used the recommendation in their classrooms?

   2. What went well with supporting the use of insights to revise and strengthen the use of the recommendations in teachers’ classrooms?

   3. What was challenging, and how did you address, bypass, or overcome those challenges?

5. What suggestions do you have for how the PDSA cycle could be adjusted to make it more useful for instructional leaders as they guide teachers through using the Toolkit resources?

6. Overall, which of the three recommendations were teachers most successful with implementing in their classrooms?

1. Why do you think teachers were most successful with using that recommendation in their classrooms?

7. Overall, which of the three recommendations was most challenging for teachers to implement in their classrooms?

2. Why do you think this recommendation was challenging for the teachers to use in their classrooms?

3. What strategies did you and the teachers use to attempt to address, bypass, or overcome these challenges?

8. To what extent are the three recommendations aligned with local or state mathematics standards and priorities?

   1. Probe if not well aligned: Can you envision any additional resources or framing for the three recommendations that could help facilitators tie these into local standards, goals, or priorities?

9. Do you feel your ability to support teachers in using the three recommendations in the classroom has changed after using the Toolkit? Please describe.

5. Sustaining the Three Recommendations in the Classroom [5 minutes)

This last set of questions focuses on the continued support of teachers using the three recommendations in their classroom in the future.

1. Which, if any, of the Toolkit’s resources do you think you will continue to use beyond this school year? Please describe.

2. Are there any aspects of the PLC structure or PDSA cycle format that you think you will continue to use with teachers you support? Why or why not?

3. How likely is it that you will continue to support teachers in implementing any of the recommendations in their algebra classrooms beyond this school year?

   1. IF LIKELY: Which recommendation(s) are you likely to continue supporting? Could you describe what you think that support might look like?

   2. IF UNLIKELY: Could you describe the reasons why you don’t see yourself continuing to support teachers in implementing the recommendations?

4. While the Toolkit is meant to be used in a group of teachers, there may be situations where this isn’t feasible. With that in mind, do you think a teacher who is new to teaching algebra would be able to use the Toolkit resources to improve their instruction either on their own or with a modest amount of guidance, outside of the context of a PLC?

   1. Do you have any suggestions for how to introduce teachers to the Toolkit and support them as they use it in this type of situation?

5. Is there anything else you would like to share about your experiences facilitating a PLC using the Toolkit that you haven’t had a chance to say yet?

That is all the questions that I have today. Thank you!