Impact Evaluation of Math Professional Development

Appendix A

Recruitment Materials

Appendix A-1

IES Letter of Support

U NITED STATES DEPARTMENT OF EDUCATION

INSTITUTE OF EDUCATION SCIENCES

National Center for Education Evaluation and Regional Assistance

December XX, 2012

Dear District/Principal/Teacher,

I am writing to introduce you to a new project funded by the U.S. Department of Education (ED) called the Math Professional Development (MPD) study. This project is aimed at improving teachers’ math content knowledge, instructional practices and student achievement. Through a contract with the American Institutes for Research (AIR) and its partner organizations, ED is seeking up to six school districts from across the country to participate in the project. In addition to being part of a high-profile, innovative study, participating districts will receive free access to high-quality professional development in math for a portion of its teachers.

The purpose of this project is to build upon the growing body of information about the types of in-service teacher training that may improve student math achievement. This project fits within ED’s priorities to boost teacher quality among the Science, Technology, Engineering, and Mathematics (STEM) workforce. The study will examine the impact of professional development on teacher knowledge, classroom practices, and student achievement that is intensive, sustained, interactive, and focused on comprehensively enhancing teacher content knowledge and integrating this knowledge into the classroom.

The study seeks volunteer 4^th grade teachers in 10-12 elementary schools per district to participate. Using a lottery, about half of the teachers within each school will be invited to participate in the professional development, which includes the widely used Intel Math course and a math-focused professional learning community. Up to 10 teachers per district, roughly equally divided between grades K-3 and 5-8 will also have the opportunity to participate in portions of the professional development. We will be in touch with you soon to provide details on how you can be involved with this project.

On behalf of the U.S. Department of Education, I encourage your participation, as many schools and teachers are needed to generate clear and usable results from this evaluation. Thank you in advance for your consideration. If you have any questions about the study, please feel free to contact AIR’s project director, Jessica Heppen, at any time at 202-403-5488 or jheppen@air.org. In addition, if I can provide any assistance to you in this matter, please feel free to contact me at 202-208-0452 or Thomas.Wei@ed.gov. Thank you for your time, and we look forward to speaking with you.

Sincerely,

Thomas Wei, Ph.D.

Project Officer, Math Professional Development Study

Appendix A-2

District Screening Protocol

Math Professional Development Study

District-Level Screening Protocol

Paperwork Burden Statement

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is XXXX–XXXX. This information collection is voluntary. The time required to complete this information collection is estimated to average 30 minutes per response. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202–4651. If you have comments or concerns regarding the status of this review of your district’s eligibility for study participation, write directly to: Thomas E. Wei, U.S. Department of Education, Institute of Education Sciences, Room 500H, 555 New Jersey Avenue, NW, Washington, DC 20208-5500.

Protocol

The telephone call should be made to the district person responsible for elementary education, math curriculum, or professional development. If the district is large enough to have an elementary mathematics division, the person overseeing this program is the best person to contact.

These individuals may be identified on the district’s website under Elementary Education or math instruction/professional development. In the rare case in which a contact is no longer current, you may need to call the district’s general number and ask for the best person to talk to about math instruction and related professional development. If the person you reach does not agree that he or she is the right person to provide the necessary information, ask that person to recommend someone else.

Before the call, please send to the district contact(s) a scheduling e-mail with the materials that describe the study. When you’re ready for the call, start with the following conversation:

Good morning {or afternoon}, my name is _____, I am with American Institutes for Research (AIR), an independent research firm in Washington, D.C. AIR is launching the initial phase of a U.S. Department of Education study of mathematics professional development (PD) focusing on 4th grade teachers. This study will examine the effects of an intensive, content-focused math professional development program that we will provide for a number of teachers over the summer and next school year. In funding and implementing this professional development program, our goals are for your teachers to directly benefit from the PD and to learn more generally about designing effective professional development. If you would like to learn more about our study, additional details are provided in the materials I sent you a few days ago.

As part of the initial phase of this study, we are reaching out to a number of districts to assess their eligibility and interest in participating. In today’s call, which will take about 20–30 minutes, I would like to learn about math instruction in your district, particularly for teachers in elementary grades. The information you provide will enable us to determine whether your district is eligible to participate in the study. If your district is eligible, we will get in touch with the appropriate individuals in your district at a later date to further gauge interest in participating.

Thank you in advance for taking the time to speak with me today. Before we begin, do you have any general questions?

PART I: Current Math Instruction

First, I would like to ask a few general questions about 4th grade math instruction provided in your district.

1. In your district, is 4th grade math instruction typically provided by general elementary teachers in self-contained classrooms (i.e., each general elementary teacher teaches both math and other core subjects)? Or is 4th grade math instruction typically departmentalized (i.e., some teachers specialize in teaching mathematics to more than one class of students)?

 Typically self-contained classrooms  Typically departmentalized

If typically departmentalized, then ask:

2. Is this the case in all schools?

 Yes  No

If YES – terminate the interview and explain why schools with a departmentalized instructional model are excluded from the study.

If NO – continue to question 3.

3. Are 4th grade students typically sorted based on their ability level into different classes for math instruction?

 Yes  No

If typically sorted by ability, then ask:

4. Is this the case in all schools?

 Yes  No

If YES – terminate the interview and explain why schools that sort students into classes based on ability are excluded from the study.

If NO – continue to question 5.

5. Can you please describe your current elementary math curriculum? (open-ended description)

Prompts: What are the names of the textbooks, curricular programs, or Web-based materials most often used in schools in your district for 4thgrade math instruction? What year were these adopted? Is the 4th grade math curriculum the same in other elementary grades (e.g., Grades 3 and 5)?

6. Do all elementary schools use the same textbook or curriculum materials for 4th grade, or is this locally determined?

  Same textbook/curriculum used by most schools

  Different textbook/curriculum, locally determined

7. To what extent do you believe the content of the 4th grade math classes in your district aligns with the major domains and clusters of the Common Core State Standards in Math (CCSSM)¹ or the 4th grade math standards adopted by your state? (open-ended response)

Prompts: What content areas or domains are emphasized in your district curriculum for 4th grade math? Would you say that the content coverage in 4th grade math classes in your district is fairly similar, or does it vary a lot by school?

PART II: Planned Changes for the School Year 2013–14

Next, I would like to ask a few general questions about any changes planned for the school year 2013–14 for 4th grade instruction and teachers in your district.

8. Is your district planning to make any changes regarding 4th grade math instruction, such as rolling out new standards, adopting a new textbook or curriculum, or launching a new PD initiative?

 Yes  No

If Yes, probe:

Could you tell me about the planned changes? (open-ended description)

Do you consider this change to be  Major  Moderate  Minor

9. Do you know if your district is planning to make any changes regarding 4th grade English/language arts or science instruction, such as adopting a new textbook or curriculum or launching a new PD initiative, or any other changes that you think could affect 4th grade teachers or 4th grade math instruction?

 Yes  No

If Yes, probe:

Could you tell me more about the planned changes? (open-ended description)

Do you consider this change to be  Major  Moderate  Minor

PART III: Student Testing

Next, I would like to ask questions about your district’s current practices and plans for student testing in math.

10. Can you tell me a bit about the state math assessment and any changes planned to take place next year? For example, some states are adapting their math assessments to include more items aligned to the Common Core standards. (open-ended description)

If respondent indicates there have been changes to the state math assessment, or that changes are planned for next year, probe:

Would you say that these changes have caused your district to revise its math instruction in some way? (open-ended description)

11. What current math assessments are used for progress monitoring in your district?

12. Is the district planning to adopt new formative or progress monitoring math assessments for 4th grade math in 2013–14?

 Yes  No

If Yes, probe:

What is the new math assessment, and what is the timeline for its adoption?

PART IV: Professional Development for 4th Grade Math Teachers

Next, I would like to ask a few questions about different types of professional development (PD) in which your 4th grade teachers have participated and are likely to participate over this coming year.

13. What kinds of PD is your district planning for 4th grade teachers during the summer of 2013 or during the school year 2013–14? If you do not know at this point what the offerings will be for the next school year, could you give us your best estimate based on your current knowledge? (open-ended response)

As the person is describing the PD in the district, gather the following information for each type of PD:

Type of PD:

 Multi-day seminars about math content, curriculum, or CCSSM for math during the summer?

 Day-long seminars about 4th grade math content, curriculum, or CCSSM during the summer or school year?

 One-on-one or small-group math coaching for 4th grade teachers?

Focus of PD:

 Math content, specify ______________________________________________

 Curriculum

 Instructional methods

 CCSSM

Provider of the PD:

 District personnel

 External facilitator, such as facilitator from the curriculum provider

 Other (university partnership, etc.) ________________________________________

PD program materials:

 Intel^® Math

 Other math program __________________________

 Math curriculum provider materials _______________________________

 District-developed materials _____________________________________

Proportion of 4th grade math teachers expected to participate: ___________________________________

If respondent cannot provide specific proportion, prompt:

 Majority of 4th grade teachers in majority of schools

 Majority of 4th grade teachers in minority of schools

 Minority of 4th grade teachers in majority of schools

 Minority of 4th grade teachers in minority of schools

14. Does your district currently have or plan to form professional learning communities (PLCs) that include 4th grade math teachers?

 Yes  No

If Yes, probe:

• What is or will be the focus of the PLCs? (Multiple boxes may be checked.)

 Math content, specify______________________________________________

 Curriculum

 CCSSM

 Other, specify_____________________________________________________

• Are/will the PLCs be school-based, or do they include teachers from multiple schools? If they include teachers from multiple schools, do you know if they take place online? If online, is the PLC part of a national program or network of educators?

 Single school-based

 Span across schools in district

 Online program spanning across schools in the district

 Online program spanning across schools and districts (a national program/network of educators)

• Are/will they span across grade levels and subject areas?

 Yes (please explain: _________________________________)

 No

• Who facilitates/will facilitate the PLCs?

 District/school personnel

 External facilitator, such as facilitator from the curriculum provider

 Other (university partnership, etc.) ________________________________________

• Is/will a specific program or approach be used? If Yes, probe:

 Mathematics Learning Community (MLC)

 Other program ___________________________________________

 District-developed approach ________________________________

• What proportion of 4th grade teachers in the district are participating/will participate in the PLCs? ___________

If respondent cannot provide specific proportion, prompt:

 Majority of 4th grade teachers in majority of schools

 Majority of 4th grade teachers in minority of schools

 Minority of 4th grade teachers in majority of schools

 Minority of 4th grade teachers in minority of schools

15. What kinds of PD has the district provided in the last two years related to math curriculum, content, or the Common Core State Standards for 4th grade math teachers? (open-ended response)

Probe:

• Content of the training (write down): __________________________________________

• Length and timing of the training (how many hours, during summer, fall, etc. of YEAR):

_________________________________________________________________________

• What proportion of 4th and 5th grade teachers in the district would you say participated in this PD?

_________________________________________________________________________

If respondent cannot provide specific proportion, prompt:

 Majority of 4th grade teachers in majority of schools

 Majority of 4th grade teachers in minority of schools

 Minority of 4th grade teachers in majority of schools

 Minority of 4th grade teachers in minority of schools

16. During the last two years, has the district participated or is the district currently participating in a state-sponsored Math Science Partnership (MSP) grant?

 Yes  No

If Yes, then ask question 17; otherwise, continue to end of the interview section.

17. Did/does the MSP grant focus on improving elementary teachers’ knowledge of math?

 Yes  No

If Yes, then ask questions 18 and 19; otherwise, continue to termination of the interview section.

18. Can you please describe the MSP grant activities? (open-ended response)

19. What proportion of 4th grade teachers have recently (or are currently) participating in the MSP with this focus?

____________________________________________________________

If respondent cannot provide specific proportion, prompt:

 Majority of 4th grade teachers in majority of schools

 Majority of 4th grade teachers in minority of schools

 Minority of 4th grade teachers in majority of schools

 Minority of 4th grade teachers in minority of schools

Thank you for your time. We will carefully review the information you gave us today. Whom should we contact about your district’s potential participation in the study? For example, can we call you for further information, or should we contact a specific district employee? (open-ended response)

If you have any questions for me, you can reach me at [contact information]. Thank you again for your time.

Appendix A-3

School Screening Protocol

Math Professional Development Study

School Screening Protocol

Paperwork Burden Statement

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is XXXX–XXXX. This information collection is voluntary. The time required to complete this information collection is estimated to average 5 minutes per response. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202–4651. If you have comments or concerns regarding the status of this review of your district’s eligibility for study participation, write directly to: Thomas E. Wei, U.S. Department of Education, Institute of Education Sciences, Room 500H, 555 New Jersey Avenue, NW, Washington, DC 20208-5500.

Protocol

Thank you for the opportunity to meet with you today and discuss the Math Professional Development (MPD) Study funded by the U.S. Department of Education. We are very excited about your school’s interest to participate in the study and are looking forward to working with you. Participation in this study is voluntary for schools and teachers.

Please complete the form so we can learn more about the math instruction taking place in your school.

If you have any questions or concerns, please contact Jessica Heppen at jheppen@air.org or 202-403-5488 or Anja Kurki at akurki@air.org or 202-403-5153.

School Name: ______________________________ Principal Name: ____________________________

E-mail: ______________________________ Phone: ____________________________

1. How many 4th grade teachers and classrooms will your school have in fall 2013?­­­

(Please mark the number for each; if the number of teachers and the number of classes is not the same, please let us know the reason for the difference.)

_______ 4th grade teachers

_______ 4th grade classes

2. Is 4th grade instruction in your school non-departmentalized?

(That is, do 4th grade teachers teach multiple subject areas, including math?)

 Yes  No  Combination of departmentalized and non-departmentalized teachers

If you answered “No” or “Combination,” please explain briefly:

3. Do 4^th grade teachers in your school typically team-teach?

(That is, do two 4^th grade teachers share responsibility for providing math instruction to the same group of students?)

 Yes  No

If yes, does team teaching occur all of the classes? Please explain briefly:

4. Do you anticipate any major changes taking place in your school during the 2013–14 school year that might affect
   4th grade math instruction (e.g., new curriculum, new assessments, new professional development initiative)?

 Yes  No

If yes, please specify:

5. Which of the following math professional development opportunities were offered to your 4th grade teachers
   during summer 2012 and/or school year 2012–13? Please check all that apply.

• Multi-day seminars about math content, math curriculum, and/or the Math Common Core State Standards. Please specify the number of days: _________________

• Daylong seminars about 4th grade math content, math curriculum, and/or the Math Common Core State Standards. Please specify the number of days: __________________

• Individual or small-group math coaching for 4th grade teachers. Please specify the number of hours per month: ______________________

6. Which of the following math professional development opportunities will be offered to your 4th grade teachers during summer 2013 and/or school year 2013–14? Please check all that apply.

• Multi-day seminars about math content, math curriculum, and/or the Math Common Core State Standards. Please specify the number of days: _________________

• Daylong seminars about 4th grade math content, math curriculum, and/or the Math Common Core State Standards. Please specify the number of days: _________________

• Individual or small-group math coaching for 4th grade teachers. Please specify the number of hours per month: _________________

7. Do your 4^th grade teachers participate in any math-related Professional Learning Communities (PLCs)?

 Yes  No

If yes,

1. How often does the PLC meet and for how long?

2. When do PLC meetings occur?

 Before school hours  During school hours  After school hours

3. What do they typically do or discuss during PLC meetings?

4. Who participates in the PLC? Are they within or across grade levels? Within or across schools?

5. Is there a facilitator/leader for the PLC meetings?

 Yes  No

If yes, who is the facilitator (a 4th grade teacher, a mathematics coach, etc.)?

8. Do your 4^th grade teachers have common planning time?

 Yes  No

If yes,

1. How often do they have common planning time and for how long?

2. When do the common planning sessions occur?

 Before school hours  During school hours  After school hours

3. What do they typically do or discuss during common planning time?

4. Is there a facilitator/leader for the common planning sessions?

 Yes  No

If yes, who is the facilitator (a 4th grade teacher, a mathematics coach, etc.)?

9. How much time per week would you say 4^th grade teachers spend collaborating on an informal basis?

10. What math textbooks or materials do you currently use for 4th grade instruction, and when were these textbooks and/or materials adopted?

Textbook:______________________________________ Year Adopted:______________________

Textbook:______________________________________ Year Adopted:______________________

Materials:______________________________________ Year Adopted:______________________

Materials from Web:_____________________________ Year Adopted:______________________

11. Do all 4th grade teachers use the same textbooks and/or materials in their math instruction?

 Yes  No

12. Does your school use any supplemental math programs for 4th grade math instruction?

 Yes  No

If yes, please specify:

Thank you! We look forward to further discussing potential participation in the Math Professional Development Study with you!

Appendix A-4

Teacher Interest Form

Math Professional Development Study

Teacher Interest Form

Paperwork Burden Statement

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is XXXX–XXXX. This information collection is voluntary. The time required to complete this information collection is estimated to average 10 minutes per response. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202–4651. If you have comments or concerns regarding the status of this review of your district’s eligibility for study participation, write directly to: Thomas E. Wei, U.S. Department of Education, Institute of Education Sciences, Room 500H, 555 New Jersey Avenue, NW, Washington, DC 20208-5500.

Math Professional Development Study

Fourth Grade Teacher Interest Form

Thank you for the opportunity to meet with you today and discuss the Math Professional Development (MPD) study funded by the U.S. Department of Education. We are very excited about your interest to participate in the study and are looking forward to working with you. Participation in this study is voluntary, and we hope you see it as a great opportunity to focus on math teaching and learning. If you decide to change your mind and are no longer interested in participating, please let us know as soon as possible.

We would greatly appreciate it if you would complete this form so we can learn more about you and your plans for this summer.

If you have any questions or concerns, please contact Jessica Heppen at jheppen@air.org or 202-403-5488 or Anja Kurki at akurki@air.org or 202-403-5153.

School Name: ______________________________ Teacher Name: _________________________________

E-mail: ______________________________ Phone: _________________________________

Please circle the appropriate answer.

I expect to teach Grade 4 math at this school during the 2013-14 school year (at the 4th grade level; not above or below)	Yes	No	Do not know
In my 4th grade class next school year, I expect to teach multiple subjects in addition to math. (That is, instruction in my school is non-departmentalized.)	Yes	No	Do not know
In my 4th grade class next school year, I expect to have students with mixed-ability levels. (That is, 4th grade students in my school are not sorted into ability groups for math instruction.)	Yes	No	Do not know

4. Do you participate in any math-related Professional Learning Communities (PLCs)?

 Yes  No

If yes,

1. How often does the PLC meet and for how long?

2. When do PLC meetings occur?

 Before school hours  During school hours  After school hours

3. What do you typically do or discuss during PLC meetings?

4. Who participates in the PLC? Are they within or across grade levels? Within or across schools?

5. Is there a facilitator/leader for the PLC meetings?

 Yes  No

If yes, who is the facilitator (a 4th grade teacher, a mathematics coach, etc.)?

5. Do you have common planning time with the other 4th grade teacher(s) in your school?

 Yes  No

If yes,

1. How often do you have common planning time and for how long?

2. When do the common planning sessions occur?

 Before school hours  During school hours  After school hours

3. What do you typically do or discuss during common planning time?

4. Is there a facilitator/leader for the common planning session?

 Yes  No

If yes, who is the facilitator (a 4th grade teacher, a mathematics coach, etc.)?

As you know, a total of 200 4th grade teachers in six districts across the country will participate in this project. All participating teachers will be asked to complete several data collection activities. In addition, based on a lottery, about half of the teachers will participate in the content-focused math professional development program that we are studying in this project.

• Teachers assigned to the PD Program Group will participate in the study’s professional development program during summer 2013 and during school year 2013–14, in addition to any professional development required by the district.

• Teachers assigned to the Business-as-Usual Group will continue to participate in any professional development offered by the district in summer 2013 and during school year 2013–14, but not the study’s professional development program.

The lottery will assign volunteer teachers to either the treatment or control group within school, meaning that schools with two volunteer teachers would have one participating in the study’s professional development program and the other participating as a member of the control group. (Schools with three volunteers would have either two in the treatment group and one in the control group, or vice versa. Schools with four volunteers would have two in each group, and so forth.)

Most of the professional development hours for the study—potentially two full weeks—will occur in summer 2013. Detailed scheduling will occur after the lottery takes place, but for now, please provide some initial information about your summer 2013 availability on the next page. Please note that your availability to participate will not affect whether you will be selected to participate in the study’s professional development. The selection of teachers will be solely based on the lottery described above.

6. Please X out dates on which you know you will NOT be available to attend a summer professional development session.

June 2013
Sun	Mon	Tues	Wed	Thurs	Fri	Sat
						1
2	3	4	5	6	7	8
9	10	11	12	13	14	15
16	17	18	19	20	21	22
23	24	25	26	27	28	29
30

JULY 2013								AUGUST 2013
Sun	Mon	Tues	Wed	Thurs	Fri	Sat		Sun	Mon	Tues	Wed	Thurs	Fri	Sat
	1	2	3	4	5	6						1	2	3
7	8	9	10	11	12	13		4	5	6	7	8	9	10
14	15	16	17	18	19	20		11	12	13	14	15	16	17
21	22	23	24	25	26	27		18	19	20	21	22	23	24
27	29	30	31					25	26	27	28	29	30	31

7. Thank you for providing your availability. Please read and sign the statements below. If you would like more information about the project before signing, or have any questions, please don’t hesitate to contact us.

I understand the purpose of this study and the requirements of participation.

I understand that participation is voluntary.

I am interested in being part of this project.

________________________________________ _________ /________ /_________

Signature Date

1 The five CCSSM domains in Grade 4 are Operations and Algebraic Thinking, Number and Operations in Base Ten, Number and Operations: Fractions, Geometry, and Measurement/Data.

3