Evaluation of the Early Warning and Intervention Monitoring System (EWIMS)

Draft Protocol—EWIMS Team Member Interview
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a
collection of information unless such collection displays a valid Office of Management and Budget
(OMB) control number. The valid OMB control number for this information collection is XXXXXXXX. The time required to complete this information collection is estimated to average 30
minutes per response. This information collection is voluntary. If you have any comments
concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write
to: U.S. Department of Education, Washington, DC 20202–4651. If you have comments or
concerns regarding the status of your individual submission of this form, write directly to:
Christopher Boccanfuso, U.S. Department of Education, Institute of Education Sciences, Room
506D, 555 New Jersey Ave. NW, Washington, DC 20208-5500.
** This survey will be administered in treatment schools only

Opening
1. Why don’t we start by having you tell me a bit about your background?
Probe as necessary:
o What is your title and role at [insert school/district name]?
o How long have you been [insert title] at [insert school name]?
2. Did you volunteer for the EWIMS Team or were you recruited? Why did you agree to be part
of this team?
3. What are your responsibilities on the EWIMS team?
4. About what percentage of your overall work time is spent on EWIMS related activities?
5. How long have you served on the team?

EWIMS Implementation
Next, I’m going to read you the seven steps of the seven-step EWIMS Implementation Process;
could you tell me whether or not these steps have been implemented so far at your school, or
whether they are in progress? [Interviewer: Read each step and ask if has been done at least
once.]
Step 1:
Step 2:
Step 3:
Step 4:

Establish roles and responsibilities
Use the EWS Tool
Review the EWS data
Interpret the EWS data

Step 5: Assign and provide interventions
Step 6: Monitor students
Step 7: Evaluate and refine the EWIMS process
Now let’s talk more about each step. I am going to ask you questions about each step, even if
you don’t think it has been implemented yet.
Step 1: Establish Roles and Responsibilities
Now let’s talk about roles and responsibilities of being a member of the EWIMS team.
6. What do you see as your main responsibilities, both as an individual and as part of the team?
Probe as necessary:
o How often and for how long does the EWIMS Team meet? Does everyone usually make
it to these meetings?
o What is a typical agenda for these meetings? What topics do you normally discuss?
o How do you get the data and reports to inform your discussions about students?
o What would you say have been the key achievements of the team? What are the key
challenges that the team faces?
Step 2: Use the EWS Tool
It is my understanding that you use the Early Warning System of EWS Tool to support
implementation of EWIMS at your school.
7. Can you describe how you use the EWS Tool? Who is responsible for loading data into the
EWS Tool? How frequently is the tool updated (e.g., annually, per grading period, monthly)?
8. How frequently does the team examine reports or data from the EWS Tool? Are there
particular reports that you typically use? Which reports are they?
Probe as necessary:
o Which reports from the tool are most useful in informing school policy?
9. How is information shared with individuals beyond this group? Is access to student-level
information an issue?
10. What challenges do you face in using the EWS Tool?
Step 3: Review the EWS data
I’m interested in how the EWIMS Team reviews data from the EWS Tool as you seek to help
those students who are at risk of not graduating.

11. Can you describe the process you use for reviewing the student data and the at-risk
indicators?
12. How do you review data at the beginning of the school year? What data to you look at? How
do you identify students who are in need of support? Is this different from how you review
these student data later in the school year? If yes, how?
13. How do you review the data from the EWS Tool?
Probe as necessary:
o How is the EWIMS information sorted, organized, and prioritized so that the team can
take action?
o Does the team discuss all students flagged as at risk or only certain cases? What are the
decision rules behind this?
Step 4: Interpret the EWS data
Once you have identified students who are flagged as at risk upon review of the EWS data, how
do you diagnose students’ needs?
14. Can you describe this process?
Probe as necessary:
o What additional data do you use to decide what a student needs? Is access to these data a
challenge?
o Do you gather information by discussing at-risk students’ teachers? To students? To
parents?
15. Does the EWS Team collect and consider additional data to help interpret why certain
students are being flagged as at risk?
Probe as necessary:
o What sort of additional data are collected—that is, information from student’s teachers or
other adults who interact with the student, data from one-on-one meetings with the
student or parents?
o What types of information are difficult or challenging to collect?
Step 5: Assign and provide interventions
Let’s talk about what happens after a student is identified or flagged as at risk.
16. Could you tell me what kind of interventions are available at your school to support student’s
needs?
Probe as necessary:
Describe the interventions available:

o Academic: content recovery courses, tutoring as an academic support, team teaching,
personalization, etc.
o Engagement and attendance: attendance and behavior monitors, family engagement,
personalization, counseling and mentoring, career and college readiness, etc.
17. How do you match students to these supports/interventions? Do you have a way of
determining the students’ level or type of need?
Probe as necessary:
o The level of student support—individualized, targeted, or universal.
o Does your school have an inventory of interventions?
o What gaps exist in the current list of available interventions for students?
18. Has the use of the EWS Tool and data contributed to the consideration of adding a new
support or intervention for students who are flagged as at risk? Has it made you consider
discontinuing the use of an intervention/support? Please describe.
19. What do you see as the greatest challenge to assigning students who are flagged as at risk to
supports/interventions?
Step 6: Monitor Students
20. Do you monitor students who are assigned to supports/interventions? Can you describe this?
How frequently do you monitor students’ progress? What data do you use to monitor
students’ progress in supports/interventions?
21. How do you make decisions about whether to change a student’s assigned support/
intervention when he or she continues to be flagged as at risk? How do you make decisions
about whether a student no longer needs support/interventions because he or she is no longer
flagged as at risk? Do you use data beyond those available in the EWS Tool? If yes, what
data do you use?
22. Do you use early warning system data to make decisions about which interventions/supports
seem to work for students in your school? Have you eliminated or added any
interventions/supports on the basis of this information?
23. Has your ability to match students to supports/interventions has improved? If yes, to what do
you attribute this change?
24. What are the challenges for monitoring students and interventions?

Step 7: Evaluate and Refine the EWIMS Process
25. Are there aspects of EWIMS implementation that you have refined or plan to refine on the
basis of your experience or the experience of others? If yes, please describe. Note: Schools
that have implemented this for the first time during the current year may have less
information about this.
26. What do you see as your school’s next steps in implementing the EWIMS over the next few
months?
EWIMS Goals, Challenges/Successes, and Advantages/Disadvantages
27. Overall, how would you say EWIMS implementation is going at your school?
28. What do you see as your school’s next steps in implementing the EWIMS over the next few
months?
29. What do you hope to achieve in your implementation of EWIMS at your school by the end of
the year?
30. At the end of the year, what would you expect to see in place or to have resulted from your
implementation to deem it a success?
31. What challenges in implementing EWIMS has your school experienced so far this year, if
any?
Probe as necessary:
o Technology challenges? (e.g., accessing student-level EWS data)
o Resource challenges? (e.g., identifying a group of individuals to support EWIMS
implementation, allotting time to meet regularly)
o Implementation challenges? (e.g., reviewing EWS student-level risk indicator data,
collecting additional student data (beyond early warning system data) to determine
students’ needs, diagnosing the needs of students who are flagged as at-risk, developing a
list of supports and interventions for at-risk students, identifying effective supports and
interventions for at-risk students, developing a systemic strategy to assign students to
interventions/supports, matching at-risk students to supports and interventions,
monitoring students in the supports and interventions to which they are assigned,
determining which interventions/supports are working for which at-risk students,
identifying schoolwide strategies to improve the likelihood students will graduate)
o Leadership challenges? (e.g., empowering EWS team members to make decisions about
students’ needs, interventions and supports)
o Communication challenges? (e.g., communicating the necessity for an EWIMS,
communicating the findings of the EWIMS team, soliciting input to the EWIMS team)

32. What aspect of the EWIMS implementation do you feel has been the most successful or most
useful (if any)? (e.g., identifying students who might be at risk, discussing interventions for
students, collaborating with staff around data)
33. What do you see as the advantages and disadvantages to EWIMS?
34. If an administrator at another school were to ask you whether they should try to implement a
similar system, what would you tell them?
Supports
35. What pilot events have been most and least helpful to you in implementing EWIS at your
school/district?
Probe as necessary:
o Technical and implementation trainings?
o Community of practice?
o On-site support?
o Responsive technical assistance?
36. What do you believe could improve EWS implementation in your high school?
Probe as necessary:
o Training on the implementation of EWS
o Training on the use of student data
o Ongoing coaching
o Financial resources
o Additional tools or reports
o Sharing of practices related to implementation among other districts or high schools
o Other
Close
37. Is there anything else that you would like to tell us about your participation in the EWIMS
implementation or participation in this study?