Phone Interview

APPENDIX D

District Staff Interview Protocol

District Staff Interview Protocol

Prior to conducting district staff interviews, we will request documents related to all of the policies described below. We will use this information to inform our interviews with district staff. We will conduct interviews with district staff who are knowledgeable in each of these policy types covered in the protocol. For each section of the protocol, we describe the type of district respondent we expect to interview.

A. Overview of Equitable Distribution Policies

The purpose of this section is to obtain an initial list of the equitable distribution policies that districts have implemented. The subsequent sections of the protocol will gather more detailed information on these and other policies. These questions are for all district respondents.

1. To what extent is an inequitable distribution of teacher quality an issue for your district? Provide a rating on a scale of one to five, with five representing the highest priority issue for the district and one representing the lowest priority issue for the district. Briefly describe why you chose this rating for your district.

2. In the past three years, has the district implemented policies or strategies designed to promote the equitable distribution of teacher quality? At this point, we are interested in identifying relevant policies but do not yet need the details of the policies. I will obtain more detailed information about these policies at a later point. Include policies that began more than three years ago and were in place over the last three years. For each strategy:

1. What is the policy/strategy? How is it designed to address the distribution of teacher quality?

2. Which schools are targeted by the policy/strategy? Make sure to clarify whether the policy/strategy targets elementary and middle schools.

3. When did the district begin implementing the policy/strategy? Is the policy/strategy still being implemented? If not, when did it end?

Additional questions for any policy described here that is not covered in the sections below.

The [policy not covered below] you mentioned is not something we had considered before, so I have a few follow up questions about that policy before moving on.

4. Has the policy/strategy changed over time?

5. Which teachers are affected by the policy in terms of grades, subjects, and experience (i.e. new/existing)?

6. If the policy involves the use of teacher performance measures, what performance measures are used? Refer to Section H as needed.

7. What challenges has the district faced in implementing the policy?

3. In the past three years, has the district measured the distribution of teacher quality in the district?

1. If so, in what school years?

2. What measure of teacher quality did the district use?

3. How did the district measure the distribution?

4. What did the district learn about the distribution? Was this information used to inform any of the policies described above?

B. Teacher Compensation

These are questions for the district staff person knowledgeable about teacher compensation and evaluation policies.

1. Over the last three years, could teachers earn additional pay for teaching in a high-need school or for assuming additional roles or responsibilities in a high-need school? For each opportunity to earn additional pay:

1. In what school year did the district first offer the bonus? Is the bonus still being offered? If not, in what school year did the district stop offering it?

2. What criteria are used to identify schools eligible for the bonus? How many schools are targeted in each school year? Make sure to clarify whether elementary and middle schools are eligible.

3. Which teachers are eligible to receive the bonus, in terms of (i) grade levels, (ii) subject areas; (iii) teacher type (i.e., new teachers hired by the district; teachers in the district who transfer to the school; and/or existing teachers in the school); and (iv) any other criteria?

4. What is the bonus amount that teachers can earn? Does the amount differ by grade. subject area, or other criteria? If so, how?

5. Is the award provided as (i) a bonus (i.e. pay that does not affect salary),
   (ii) permanent salary increase, (iii) higher starting salary; or (iv) other (ex. student loan forgiveness, relocation assistance)? Can teachers earn the award once or multiple times?

6. What are the requirements for earning the bonus?

   • Does a teacher have to commit to teaching or to continue teaching in the targeted school? If so, for how many years?

   • Does a teacher have to take on additional roles or responsibilities to earn the award? If so, describe the roles/responsibilities.

   • Is some or all of the award based on performance? If yes,

     • Is performance based on a measure of student achievement? [If yes, ask questions in Section H.1].

     • Is performance based on a classroom observation measure? [If yes, ask questions in Section H.2].

     • Is performance based on other measures of teacher performance (ex. principal rating of teacher performance, student or parent survey, teacher portfolio)? If so, what other measures of performance are used for the award? [If yes, ask questions in Section H.3]

   • How are these performance measures combined? What weights are assigned to each measure?

7. Approximately how many teachers have earned a bonus in each year it was implemented?

8. Has the design or implementation of the award changed in the last three years? If so, why?

2. Over the last three years, could teachers in any district school earn additional pay based on their performance? We are interested in policies that provide additional pay for teachers based solely on performance. For each opportunity to earn additional pay:

When and Where Pay for Performance is Offered

1. In what school year did the district first offer the bonus? Is the bonus still being offered? If not, in what school year did the district stop offering it?

2. Is the opportunity to earn the performance-based bonus offered to teachers in all schools or a subset of schools? If a subset of schools, what criteria are used to identify schools eligible for the bonus? How many schools were selected? Has the number or type of eligible schools changed over the last three years? Make sure to clarify whether elementary and middle schools are included.

3. Which teachers have the opportunity to earn the performance-based bonus (i.e. subjects, grades)? Can teachers choose not to participate in the performance-based bonus program? Is participation voluntary? Make sure to clarify whether elementary and middle school grades are included.

Amount and Type of Bonus Pay

4. What is the maximum amount teachers can earn for the performance-based bonus? Does this amount differ by grade or subject area? If so, how?

5. What percentage of eligible teachers earned the maximum bonus for each year?

6. For each year, approximately what percentage of eligible teachers earned a bonus of each size indicated below based on their performance? If the bonuses are determined by percentage of teacher salary, please use your best estimate of the dollar amounts.

   • $0 (no bonus)

   • $1 - $999

   • $1,000 - $1,999

   • $2,000 - $3,999

   • $4,000 - $5,999

   • $6,000 - $7,999

   • $8,000 - $9,999

   • $10,000 - $11,999

   • $12,000 or more

7. Is the bonus provided as (i) pay in addition to salary, (ii) permanent salary increase, or (iii) other (ex. student loan forgiveness, relocation assistance)?

How Performance is Measured

8. How is performance measured for the bonus? At what level is performance measured, the teacher level, grade level, team level, school level, or other level?

9. How are these performance measures combined? What weights are assigned to each measure? Do the weights vary by teacher type?

• Is performance based on a measure of student achievement? If so, [If yes, ask questions in Section H.1].

• Is performance based on a classroom observation measure? If so, [If yes, ask questions in Section H.2].

• Is performance based on other measures of teacher performance (ex. principal rating of teacher performance, student or parent survey, teacher portfolio)? If so, what other measures of performance are used for the award? [If yes, ask questions in Section H.3]

10. What are the criteria for earning a bonus based on these performance measures? Do teachers set their own performance goals or does the district set the performance criteria?

11. Has the design or implementation of the bonus changed in the last three years? If so, why?

3. Did the district obtain the support of the teachers’ union for these compensation policies?

4. Did the district face challenges in implementing these compensation policies? If so, describe the challenges and how the district addressed them.

5. Do not ask these questions if the district provided a teacher salary schedule. According to the salary schedule, what is the base year salary for a teacher with a bachelor’s degree and no teaching experience? A bachelor’s degree and ten years of experience? A master’s degree and ten years of experience? A master’s degree and twenty years of experience?

C. Teacher Recruitment

These are questions for the district staff person knowledgeable about teacher staffing policies.

1. When a school in the district has a vacancy, who has primary responsibility for identifying and recruiting applicants—the school, the district, or another entity? Is this true for different types of teachers? Has this changed in the last three years?

2. Do new applicants submit an application to the district before interviewing for a vacancy? Does the district screen new applicants before they interview for a vacancy? If so, how? If a school wants to fill a vacancy with a new applicant, who has primary responsibility for selecting a new applicant, the district or school? Have these policies changed in the last three years? If so, why?

3. Over the last three years, has the district recruited teachers specifically for schools that the district identifies as high-need? Include policies that began more than three years ago and were in place over the last three years. If so, what was the start date of the initiative? Is it still being implemented? If not, what is the end date?, and…

1. What were the criteria for identifying the high-need schools that were targeted each year? How many schools were targeted each year? Make sure to clarify whether elementary and middle schools are included.

2. What types of teachers or teaching candidates were targeted by the recruitment efforts? Approximately how many teachers did the district aim to recruit into high need schools?

   • What type of recruitment activities were used to recruit teachers for teaching in high-need schools? Are these recruitment activities targeted to teachers in the local area, across the state, and/or in other states?

3. Has the district ever recruited existing teachers within the district to transfer into high-need schools? If so, how are district teachers targeted for transfer and what recruitment activities are used?

4. Outside of the general resources provided to all principals in the district, in the last three years have principals in high-need schools received (i) additional resources for teacher recruitment and hiring or (ii) training on how to recruit and hire teachers? In what school years were these resources provided?

1. What were the criteria for identifying the high-need schools that were targeted? Did the criteria change over time? How many schools were targeted each year? Make sure to clarify whether elementary and middle schools are included.

2. For resources: What type of resources do principals in high-need schools receive? How much was invested in high-need schools?

3. For trainings: How often is the training conducted? How much total time would a typical principal spend in training (minutes) in a school year? Are principals required to participate? What topics are covered? Who provides the training?

5. Over the last three years, has the district recruited teachers for high-need schools from the following sources? If so, for what grade levels and subject areas? Approximately how many candidates are hired each year from each source?

1. Teaching Fellows or other programs that recruit mid-career professionals to teach in high-need schools, Teach for America, Teacher Residency Programs, Urban Academies, programs that recruit retired teachers, or other programs that specifically recruit or prepare teachers for teaching in high-need schools?

2. Has the district developed partnerships with Institutes of Higher Education (IHEs) to prepare teachers specifically for teaching in high need schools? or for teaching in certain subject areas? If so, which IHEs and approximately how many teachers are hired from each?

6. What was the role of the teacher’s union in designing and implementing these recruitment strategies?

7. Did the district face any challenges in implementing these recruitment strategies? If so, what were the challenges and how were they overcome?

D. Teacher Transfer/Hiring Policies

These are questions for the district staff person knowledgeable about teacher staffing policies.

1. Provide the start and end dates for when principals can fill positions through the following means: (i) voluntary transfers; (ii) involuntary transfers; and (iii) new hires. Has this timing changed over the last three years? If so, how and why?

1. Are there limitations on when new applicant hiring can occur relative to the period when voluntary and/or involuntary transfers occur?

2. Must all or a certain proportion of transfers be hired before new teachers can be hired? If yes, what proportion?

3. Do these timelines differ if the district plans to make or is making layoffs? If so, how?

4. Do high need schools receive any priority or flexibility in the timeline for hiring new applicants? If so, what are the criteria for identifying high need schools? Make sure to clarify whether elementary and middle schools are included.

5. In each of the last three school years, approximately how many vacancies has the district had available for voluntary transfers, involuntary transfers, and new applicants? If you don’t have exact numbers, your best guess is fine.

6. Over the last three years, for a typical vacancy, how many weeks before the beginning of the school year is the position filled? Typically, when has the district filled about 90 percent of its vacancies for the next school year, and what proportion of vacancies are still unfilled at the start of the school year? Does this differ for different types of schools? If so, how?

2. Over the last three years, has the district implemented any policies that promote an early timeline for identifying and filling vacancies? Include policies that began more than three years ago and were in place over the last three years. If so, what policies and what are the start and end dates for implementation?

1. Does the district offer incentives or impose penalties that encourage teachers to provide early notification about planned retirements? If so, describe the incentive or penalty and the timeline for when a teacher is expected to notify the district about retirement.

2. Has the district revised the timing of the budget planning or approval process to promote an early timeline for identifying and filling vacancies? If so, how was the timeline revised and how did it affect the timing of when vacancies are identified and filled?

3. Does the district allow certain schools to identify and fill vacancies earlier than others? For example, is the district required to notify high-need schools about their vacancies before other schools?

3. We would like to understand the process by which teachers in the district voluntarily transfer between schools in the district. For each of the questions below, please describe how and why the policy changed in the last three years.

1. Which teachers are eligible to voluntarily transfer? How many years of experience (in the district) does a teacher need before they can voluntarily transfer?

2. Do certain types of voluntary transfers receive priority over others in filling vacancies (ex. seniority)? If so, which teachers receive priority and how?

   • Can voluntary transfers fill positions that are currently held by novice teachers or other types of teachers?

3. What role do schools play in deciding whether to hire a voluntary transfer applicant? How much choice do schools have in this process?

   • Can principals reject a voluntary transfer applicant for a vacancy in their school?

   • Are there circumstances in which a principal must accept a voluntary transfer applicant?

   • Are there circumstances in which a principal must choose from a list of voluntary transfer applicants?

4. We would also like to understand the process by which teachers are involuntarily transferred between schools in the district. Has this process changed in the last three years? If so, please describe how and why the policy changed in answering the questions below.

1. What are the reasons a teacher would be involuntarily transferred?

   • Can a teacher be involuntary transferred due to (i) a position being cut in a school, (ii) a decision by the principal (if so, for what reasons?), (iii) lacking the necessary certification, (iv) redistributing staff due to enrollment shifts?

2. When a teaching position is cut in a school, what is the process for determining which teacher is involuntarily transferred? Do principals have a say in this process? Can principals select/propose the teacher who will be involuntary transferred or can principals reject an involuntary transfer?

3. Do certain types of involuntary transfers receive priority over others in applying, interviewing, or filling vacancies (ex. seniority)? If so, which teachers receive priority and how?

4. What role do schools play in deciding whether to hire an involuntary transfer? How much choice do schools have?

   • Can principals reject an involuntary transfer?

   • Are there circumstances in which a principal must accept an involuntary transfer?

   • Are there circumstances in which a principal must choose from a list of involuntary transfer applicants?

5. Has the district experienced any challenges in implementing these transfer/hiring policies? If so, what are they and how were they addressed?

E. Teacher Evaluation

These are questions for the district staff person knowledgeable about teacher compensation and evaluation policies.

1. How often are teachers evaluated? How does the frequency of evaluations differ for different types of teachers (for example, novice vs. tenured teachers; teachers rated satisfactory vs. unsatisfactory)? Has this policy changed in the last three years? If so, describe when the change occurred, what was changed, and why.

2. What measures of teacher performance does the district use to evaluate teachers? Has this changed in the last three years?

1. Does the district include measures based on student achievement? [If yes, ask questions in Section H.1]

2. Does the district include classroom observation measures? [If yes, ask questions in Section H.2]

3. Does the district include other measures of teacher performance? [If yes, ask questions in Section H.3]

3. How are these measures combined to evaluate teachers’ performance? What weights are assigned to each measure?

4. What type of rating or score do teachers receive based on their evaluation?

1. In each of the last three school years, approximately what proportion of teachers receive each rating? Approximately what proportion of teachers receive a satisfactory rating or higher and what proportion receive an unsatisfactory rating?

5. What information do teachers receive regarding their evaluation results? Does the information explicitly identify a teacher’s strengths, weaknesses, and areas for improvement?

6. Are certain teachers required to receive support or participate in professional development based on their evaluation rating/score? If so, which teachers and how much support or professional development?

7. Does a teacher’s evaluation rating affect whether they are (i) granted tenure; (ii) voluntarily or involuntarily transferred; (iii) laid off; (iv) awarded a step or lane increase in the salary schedule; (v) provided additional compensation; or (vi) required to attend professional development? If so, how?

8. What was the role of the teacher’s union in designing and implementing these evaluation policies?

9. Has the district experienced any challenges in implementing these teacher evaluation policies? If so, describe the challenges and how the district addressed them.

F. Teacher Tenure

These are questions for the district staff person knowledgeable about teacher staffing policies.

1. What are the criteria for earning tenure? Have these criteria changed in the last three years? If so, provide the start and end dates for each change.

1. How many years of teaching experience are required to earn tenure? Does this differ for teachers with prior teaching experience in other districts?

2. Is tenure based on the teacher evaluation measure described in Section E? If not, what teacher performance measures based on student achievement [If so, complete section H.1.]? classroom observations [If so, complete section H.1.]? or other measures of teacher performance [If so, complete section H.3]

3. Are there other criteria or requirements for earning tenure? If so, briefly describe.

2. In each of the last three school years, about what proportion of teachers who apply for tenure are granted tenure?

3. Has the district experienced any challenges in implementing these teacher tenure policies? If so, what are they and how are they addressed?

G. Teacher Layoffs

These are questions for the district staff person knowledgeable about teacher staffing policies.

1. Has the district laid off any teachers for the last three school years? If so, approximately how many teachers and in which school years (as needed, distinguish between when layoff occurred and when teachers lost their positions)? What grade levels and subject areas were affected by the layoffs? Does the district expect to lay off teachers in the upcoming school year?

2. When the district implements layoffs (i.e. reduction in force), what is the basis for deciding which teachers will be laid off? Is there any flexibility in how these criteria are applied? Has this policy changed in the last three years? If so, what are the start and end dates of the change?

3. If layoffs are rescinded, do recalled teachers have a right to fill vacancies in the district? Has this policy changed in the last three years? If so, what are the start and end dates of the change?

1. What priority do recalled teachers receive in filling vacancies (i.e. relative to transfers and new applicants)?

2. What role do schools play in selecting recalled teachers? Can principals reject a laid off teacher who is recalled? If yes, are there circumstances in which a principal must accept a laid off teacher? If no, do principals have any choice in selecting a candidate?

3. Do certain types of recalled teachers receive priority in receiving teaching placements? If so, which ones and how?

4. What challenges did the district experience in implementing these teacher layoff policies?

H. Teacher Performance Measures

This section of the protocol is not covered by itself during interviews. Rather, it is used to gather information on the teacher performance measures used for all other policies in the protocol. The questions in this section are designed to gather consistent information about the teacher performance measures used by a district across different policies. During each interview and across interviews for a district, interviewers should document when the same performance measure is used for multiple policies. If the same performance measure is used for different policies, this section does not need to be repeated each time.

1. For each teacher performance measure based on student achievement, address the following:

1. Describe the teacher performance measure.

2. Which teachers are subject to this measure (i.e. grades, subjects)?

3. Is the measure based on student achievement from a standardized test, such as a state assessment?

4. Is the measure based on student achievement at the school, team, grade, classroom, or other level, or a combination of levels?

5. Is the measure based on student achievement level (i.e. student performance level in a single year) or based on a measure that accounts for student achievement in the prior year (ex. student growth or value-added model)?

6. Does the district use a value-added model to measure teacher performance? If so, what is the name of the model and/or who designed it?

7. What are the start and end dates for when the district used this measure?

2. For each classroom observation measure, address the following:

1. Which teachers are subject to this measure (i.e. grades, subjects)?

2. How many classroom observations does each teacher receive? How long is the typical observation?

3. Who conducts classroom observations of teachers—school principal, grade or team leader in the school, content specialist, teacher mentor, or other? Does the district train observers on how to conduct classroom observations?

4. Are teachers rated for whether they demonstrate evidence of effective teaching according to a rubric of effective classroom practices that is enforced consistently across the district, or are the observation criteria left to the discretion of the observer? Who developed the tool/rubric? What are the start and end dates for when the district used this tool/rubric?

3. For each of the other teacher performance measures, address the following:

1. Describe the performance measure and how it is defined.

2. Which teachers are subject to this measure (i.e. grades, subjects)?

I. School and Principal Policies

These are questions for the district staff person knowledgeable about school improvement policies.

1. Over the last three school years have any schools implemented a school turnaround approach? If yes, how many schools, what grade spans (i.e. elementary and middle), and in what school years? In addition:

1. Did these schools implement a turnaround model due to (i) restructuring status under NCLB; (ii) a School Improvement Grant; or (iii) another reason?

2. Indicate how many schools implemented each turnaround strategy for each year.

   • Replace principal and at least 50 percent of staff and revise instructional program.

   • Restart the school under the management of private entity (i.e., charter operator, charter management organization, or education management organization).

   • Transform the school by developing educator effectiveness, implementing a comprehensive instructional program, extending learning time, and providing support and operating flexibility.

   • Restart the school under the management of the state.

   • Close/consolidate the school and enroll the students in other schools.

   • Other approaches?

2. Besides the turnaround activities described above, has the district closed high-need schools in the last three years? If so, how many each year? How were schools targeted? Make sure to clarify whether elementary and middle schools were included.

1. How were students in the closed schools reassigned to other schools in the district? How were teachers in the close schools reassigned to other schools in the district?

2. Which of the following aspects where considered when redistributing (i) parental preference; (ii) geographic location of student; (iii) academic performance of receiving schools; (iv) other factors?

3. Besides the school turnaround activities described above, has the district implemented other initiatives in the last three years to improve working conditions in high-need schools (ex. school safety, instructional resources)? If so, beginning in which school year? Which schools are targeted? Make sure to clarify whether elementary and middle schools were included. How is it designed to improve working conditions? How and why has the policy changed in the last three years?

4. Besides the school turnaround activities described above, has the district implemented any of the following initiatives in the past three years? If so, what is the start date for implementation of each initiative? Is the initiative still being implemented? If not, what is the end date?

1. Remove or transfer principals in high-need schools on the basis of school performance?

2. Target principals of high-need schools for additional training or professional development?

3. Provide incentives for serving as a principal in a high-need school? Provide incentives based on performance for principals in high-need schools?

4. Recruit from principal preparation programs that prepare principals for high-need schools? If so which programs?

5. Established additional requirements for principals hired to lead high-need schools (for example, experience)?

6. Other policies or initiatives designed to address principal quality?

5. What challenges did the district experience in implementing these school improvement policies?

J. Teacher Development

These are questions for the district staff person knowledgeable about professional development and teacher induction policies and programs.

1. In the last three years, has the district provided professional development, coaching, or other training targeted to teachers in high-need schools? If so,

1. In what school years?

2. Which schools were targeted? What were the criteria for identifying high-need schools? Make sure to clarify whether elementary and middle schools are included.

3. Which types of teachers were targeted in terms of (i) grade level; (ii) subject area; (iii) new or existing teachers; or (iv) other types?

4. Describe the professional development or coaching in terms of the (i) content; (ii) format (i.e. in-person training, online course, etc.); and (iii) provider of the training?

5. Approximately how many hours of professional development was each teacher expected to receive?

2. In the last five years, has the district had a structured induction program for teachers newly hired in the district? If so, in what school years?

1. Is the induction program required by the state?

2. Is the induction provided by an existing program or organization?

3. Which teachers participate in the district’s induction program? During which years of a teacher’s career do they participate in an induction program? First? Second?

4. Are mentors assigned to new teachers as part of the induction program? Are the mentors based at the same school as the teacher? Do the mentors teach the same grade and/or subject area as the teacher? What are the eligibility requirements to serve as a mentor? Are mentors trained? If so, how and what is the approximate length of the training? Do mentors serve in that role full-time or part-time? If part-time, do they continue teaching or what are their other responsibilities?

5. What is the total amount of time a mentor is expected to meet with a new teacher in their first year? [Interviewers can ask the number of times a mentor is expected to meet with a new teacher and the expected length of each meeting.] Are mentors expected to observe beginning teachers in the classroom and offer constructive feedback on their teaching?

6. Is the mentoring instructionally focused? Is the content of the induction mentoring guided by a curriculum? If so, is the curriculum based on teaching standards? If so, which ones?

7. Is the induction program aligned with other district initiatives? If so, how?

K. District Context

These are questions for all district respondents.

1. In the past three years have there been any demographic shifts in the district that might have affected the school assignment of teachers or students in the district (ex. public housing policy, district boundary changes).

2. Has the district implemented any policies that redistributed students within the district? Has the district implemented any policies designed to ensure integration of students by race and/or socioeconomic status? If so, please describe briefly, including the start and end dates of the policy.

3. Does the district have any policies or practices for the tracking of students in certain grades or subjects? If so, do these policies differ across schools?

4. Have there been any changes in the superintendent or the director of human resources over the last three years?

5. Did the district have the support of the teachers’ union for the equitable distribution policies discussed above? Why or why not? Did the support or lack of support affect implementation? If so, how?

6. In the last three years, has your district received funding or resources from external sources (ex. federal government, foundations) to develop or implement policies that could influence the distribution of teachers? If so, what are those policies?