Grantee Screening Interview (Module A)

Appendix B

Grantee Screening Protocol—
Module A: MSAP Project Director Interview

Conversion Magnet Schools Evaluation

Grantee Screening Protocol—
Module A: MSAP Project Director Interview

(2004 Grantee Cohort)

April 2007

Introduction

Interviewer: Begin each module with this introduction. If the official has responded to an earlier module, just indicate the goal and expected duration of the module now being administered.

Good morning [or afternoon], I am _____, with [the American Institutes for Research/Berkeley Policy Associates], a research firm in [Palo Alto/Oakland, California]. We are doing preliminary work for a Department of Education research study on the relationship between magnet programs and the academic achievement of the students who attend them. The study will focus on elementary schools that used funds from Magnet Schools Assistance Program (MSAP) grants to introduce new magnet programs beginning in 2004 or 2007. As a first step, we are contacting all of the MSAP grantees with new magnet elementary schools to determine whether they meet certain criteria required for the study. If these interviews show that a scientifically rigorous study is possible, we will begin a major evaluation study late this year.

Today, we need to verify information we have gathered about your MSAP-funded schools and some non-magnet schools.

You have been identified to us by [the MSAP Office/ the magnet project director in this district] as the person who can answer our questions about [topic]. A few days ago, we scheduled this appointment with you and sent you a brochure about the study as well as a copy of the questions we’d like to discuss with you. As indicated in the brochure, the answers you give us are confidential, and responses to the study’s data collections will be used only for statistical purposes. The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific district, school, or individual. We will not provide information that identifies respondents, students, or districts to anyone outside the study team, except as required by law. [Interviewer: Verify that the official has received these documents and has them available to read during the interview.]

Your participation in this interview is voluntary, though of course we hope you will be willing to help us in this important evaluation for the Department of Education.

It will take about 30 minutes to go through the questions I have for you. Are you able to talk with me now? Or should we set another appointment?

Module A: MSAP Project Director Interview—Questions About MSAP Magnet and Comparison Schools

The Conversion Magnet Schools Evaluation will compare student outcomes for a set of MSAP-funded elementary schools that were converted into magnets in 2004 and a set of otherwise comparable non-magnet schools. Both sets of schools will have been in operation for 3 years prior to 2004-2005 and have clearly defined attendance areas from which most of their students are drawn. Answers to the following questions will allow us to identify magnet and comparison schools that meet the requirements of the study. First I will ask you some basic questions about the elementary schools in your district that have been funded by your 2004 MSAP grant. If one or more of these schools appears to meet study requirements, I will continue with some additional questions about these and other schools in your district.

1. MSAP-funded Magnet Schools—Basic Eligibility Questions

Interviewer: Present the respondent with a list of MSAP-funded elementary magnet schools and ask for verification of pre-coded information and completion if information is missing. If responses to the items in question 1 indicate that any of the listed schools is a conversion magnet with a whole-school program and a clearly defined attendance area, continue with questions 2 and 3 for these schools. If none of the schools appears to meet these criteria, thank the respondent and end the interview.

	MSAP School 1	MSAP School 2	MSAP School 3	MSAP School 4
a. School Name
	Verification section (write yes/no)
b. School has a whole school magnet program (not a program-within-a-school).
c. Grades served (2006-2007)
d. The school had no magnet program (federal or local) between 1998 and 2004.
e. The school has been open continuously since at least 2001-2002.
f. The school has a clearly defined neighborhood attendance area (i.e., it does not overlap with another school’s attendance area).
g. Attendance area boundaries have not changed since 2001-2002.

(continued on next page)

1. MSAP-funded Magnet Schools—Basic Eligibility Questions (continued)

	MSAP School 1	MSAP School 2	MSAP School 3	MSAP School 4
School Name
	Verification section (write response)
h. If boundaries have changed: 1. Approximately what percent of students who were eligible to attend the school before the change are no longer eligible? 2. Approximately what percent of students currently eligible to attend the school were not eligible before the change?
i. Are all neighborhood students admitted to the school, or do they apply for admission for a limited number of seats?

Interviewer: If none of the listed schools appears to meet the criteria for inclusion in the study, ask the respondent for confirmation of this conclusion, thank him or her for talking with you, and terminate the interview. Otherwise, transfer the names of the apparently eligible schools into the Question 2 table and proceed with questions 2 and 3.

2. MSAP-funded Magnet Schools—Follow-up Questions for Schools Meeting Basic Requirements for Inclusion in the Study

	MSAP School 1	MSAP School 2	MSAP School 3	MSAP School 4
a. School Name
Implementation Questions
b. What is the approximate proportion of the school’s enrollment that is from the school’s attendance area?
c. In what grades can non-resident students apply for admission? (Circle all grades that apply.)	PK K 1 2 3 4 5 6	PK K 1 2 3 4 5 6	PK K 1 2 3 4 5 6	PK K 1 2 3 4 5 6
d. In what grades are students typically accepted into the magnet? (Circle all grades that apply.).	PK K 1 2 3 4 5 6	PK K 1 2 3 4 5 6	PK K 1 2 3 4 5 6	PK K 1 2 3 4 5 6
e. Was the magnet program implemented in all grades in 2004-2005? If not, how were the grades phased in?
f. Has this school been identified for Program Improvement under NCLB? If so, in what year of the process was the school in 2006-2007?	1 2 3 4 5	1 2 3 4 5	1 2 3 4 5	1 2 3 4 5
g. Will this school continue as a magnet beyond 2006-2007? If so, how will the magnet program be funded?

3. Non-magnet Comparison Schools

Interviewer: Present district informant with the pre-determined list of potentially best comparison schools (with roughly the same ethnic/poverty enrollment composition and baseline test performance as one or more MSAP magnets) and ask the following verification/completion questions:

	Comparison School 1	Comparison School 2	Comparison School 3	Comparison School 4
a. School Name
b. Grades served (2006-2007)
c. The school has had no magnet program (federal or local) since 1998.
d. The school has been open since at least 2001-2002.
e. The school has a clearly defined neighborhood attendance area.
f. Attendance area boundaries have not changed since 2001-2002.
g. If boundaries have changed: 1. Approximately what percent of students who were eligible to attend the school before the change is no longer eligible? 2. Approximately what percent of students currently eligible to attend the school was not eligible before the change?

(continued on next page)

3. Non-magnet Comparison Schools (continued)

	Comparison School 1	Comparison School 2	Comparison School 3	Comparison School 4
School Name
h. What is the approximate proportion of the school’s enrollment that is from the attendance area?
i. Has this school been identified for Program Improvement under NCLB? If so, in what year of the process was the school in 2006-2007?	1 2 3 4 5	1 2 3 4 5	1 2 3 4 5	1 2 3 4 5
j. Does this school have any programs designed to attract non-resident students to enroll? (Please specify.)
Please give any reasons why this school would not be a good comparison school for these MSAP magnets (e.g., school will be closed, relocated, or become a magnet soon; has a low proportion of English learners while the magnet has a high proportion, etc.) Interviewer: Present list of potentially matching MSAP conversion magnets.