ED School Climate Surveys (EDSCLS) Additional Item Cognitive Interviews

Case ID#

Interviewer Initials ______

Date:

Time:

School Climate Surveys

STUDENT SURVEY
2015

Conducted by:
U.S. Department of Education
National Center for Education Statistics

Administered by:
Research Support Services, Inc

MATERIALS NEEDED FOR INTERVIEW

• INTERVIEWER PROTOCOL BOOKLET (THIS BOOKLET)

• RESPONDENT QUESTIONNAIRE

• DO YOU HAVE THE CORRECT VERSION OF THE BOOKLET AND QUESTIONNAIRE?

• CONSENT FORM (TWO COPIES)

• INCENTIVE

• PAYMENT RECEIPT

• DIGITAL RECORDER AND EXTRA BATTERIES

• PENS AND PENCILS

STEP 1: WELCOME AND INTRODUCTION SCRIPT

Interviewer Instructions: The following scripts should NOT be read verbatim. Text in italics is suggested content with which the interviewer should be thoroughly familiar in advance. Text in brackets [ ] presents instructions for the interviewer.

Hello, my name is <your name > and I work for Research Support Services. It’s nice to meet you, thanks for coming to help us out today.

[Create small talk to build rapport with the student]:

You are here today to help us out with a new survey that we are working on with the U.S. Department of Education. This survey asks about your opinions of the school you currently attend. A survey is just a set of questions about your experiences. This is not a test. I will ask you to take the survey as if you were taking a real survey. There are no right or wrong answers, and it’s ok if you do not know some of the answers. I will not be grading your work. Once you are done I will ask you questions about the survey. This will help us make the survey better and clearer for other students just like you.

You can choose whether or not you want to participate. You may also skip any question you do not want to answer. [We have discussed this research with your parent(s)/guardian and they know that we are also asking you for your agreement. If you are going to participate in the research, your parent(s)/guardian also have to agree. But if you do not wish to take part in the research, you do not have to do so, even if your parents have agreed.] Because the information you provide is so important, I am going to be taking notes while you are working and while we are talking [INSERT IF THERE IS AN OBSERVER: and my colleague will also be observing our session today].

Also, this interview will be recorded so researchers can review the tapes later. Your responses and any information that refers to you or the name of your school will not be included in our report. Your responses will be used for research purposes only and will not be disclosed, or used for any other purpose except as required by law. [if ask: Education Sciences Reform Act of 2002, 20 U.S. Code, Section 9573]

Remember, you are not being tested or graded; we want you to tell us what you think about the survey. Would you like to be a part of this study? [If respondent agrees, continue, if not, end the interview.]

If at any time you want to stop, just let me know.

[If, for any reason, the participant is no longer interested in participating, thank the participant for his/her time and end the interview.]

Before we continue, do you have any questions about what I just said? [Answer any questions the student may ask.]

PROVIDE RESPONDENT WITH A COPY OF THE ASSENT FORM. ASK THE RESPONDENT TO READ THE FORM (OR READ IT TO HIM/HER, DEPENDING ON RESPONDENT PREFERENCE), ANSWER ANY QUESTIONS, AND HAVE THE RESPONDENT SIGN THE FORM. LEAVE A SEPARATE COPY OF THE FORM WITH THE RESPONDENT.

• SIGNED ASSENT FORM COLLECTED

• COPY OF ASSENT FORM GIVEN TO RESPONDENT

• IF THE RESPONDENT HAS ASSENTED TO RECORDING, START THE RECORDER.

STEP 2: THINK-ALOUD PRACTICE

While you fill out the survey I want you to think out loud about how you are answering some of the questions. Hearing you talk about how you figure out your answers to the questions will help me understand how to make the questions better. I am more interested in how you arrive at your answers than in the answers you give me. To help you get started on thinking aloud, we are going to do a practice activity. I’m going to ask you a question and ask you to think aloud as you decide on your answer.

How many windows are there in the house or apartment where you live?

Please tell me what you are thinking while you are figuring out your answer.

Probe as needed for detail: How did you arrive at that answer? What were you visualizing when you were deciding on that answer? Can you tell me more about what you are including in your count?

When complete: Thank you; that’s the kind of detail I’m looking for when you talk to me about your answers to the questions in the survey.

HAND RESPONDENT A COPY OF THE QUESTIONNAIRE AND A PEN.

STEP 3: COMPLETION OF THE QUESTIONNAIRE

Now I would like you to complete the questionnaire. While you are answering the questions, please think about the school you currently attend. I’d like you to think aloud while you decide on your answer. Please read aloud anything that you would normally read to yourself if I were not here.

Remember that I am very interested in hearing you talk about what you are thinking about as you figure out your answers to the questions.

Probe bank (from Willis)

• Can you tell me in your own words what that question was asking?

• What does the word [term] in this question mean to you?

• How easy or difficult is it to remember [topic]?

• How easy or hard was it to choose an answer?

• Tell me what you are thinking?

• How did you come up with that answer?

• You said [answer]. Can you tell me more about that?

• [In response to R actions or comments]: Tell me what happened? Tell me what you were doing?

• [If you pick up on a visual cue like a furrowed brow or a puzzled look]: “Tell me what you are thinking.”

• [If you pick up on a visual cue that indicates confusion like paper flipping or rereading an old question before answering this question]: “Tell me what just happened. Tell me about what you were just doing.”

Say to respondent: Please answer these questions thinking about the school you are attending right now.

National Center for Education Statistics ED School Climate Surveys (EDSCLS) – Cognitive Testing Items

Throughout the survey, "This school" means activities happening in school buildings, on school grounds, on school buses, and at places that hold school-sponsored events or activities. Unless otherwise specified, this refers to normal school hours or to times when school activities/events were in session.

How strongly do you agree or disagree with the following statements about this school? Mark One Response.

13. It is easy to talk with teachers at this school. Sengrel12

Strongly Agree

Agree

Disagree

Strongly Disagree

PROBES

1. What is this question asking?

2. What does ‘talk with teachers’ mean in this question?

14. My teachers care about me. Sengrel14

Strongly Agree

Agree

Disagree

Strongly Disagree

PROBES:

1. What does this question mean to you?

15. At this school, there is a teacher or some other adult who students can go to if they need help because of sexual assault or dating violence. Sengrel153

Strongly Agree

Agree

Disagree

Strongly Disagree

PROBES

1) Tell me about your answer.

2) What is this question asking you?

3) What does ‘if they need help’ mean in this question

4) What does sexual assault mean in this question?

5) What does dating violence mean in this question?

16. My teachers make me feel good about myself. Sengrel17

Strongly Agree

Agree

Disagree

Strongly Disagree

30. I feel like I am part of this school. Ssafemo56

Strongly Agree

Agree

Disagree

Strongly Disagree

31. I feel socially accepted. Ssafemo57

Strongly Agree

Agree

Disagree

Strongly Disagree

32. I feel safe at this school. Ssafpsaf59

Strongly Agree

Agree

Disagree

Strongly Disagree

PROBES:

1. What is this question asking?

2. What does ‘safe’ mean in this question?

33. I feel safe going to and from this school. Ssafpsaf60

Strongly Agree

Agree

Disagree

Strongly Disagree

PROBE:

1. What does to and from school mean to you?

34. Students at this school feel unsafe because of sexual assault or dating violence. Ssafpsaf154

Strongly Agree

Agree

Disagree

Strongly Disagree

PROBES

1. Tell me more about your answer?

2. What is this question asking?

3. You said students (do/do not) feel unsafe in your school. Tell me more about that. What does unsafe mean in this question?

4. You answered XX tell me why you chose that answer instead of [one below]

5. And tell me about how you chose that answer instead of [one above]

35. I sometimes stay home because I don’t feel safe at this school. Ssafpsaf63

Strongly Agree

Agree

Disagree

Strongly Disagree

PROBE

1. What is this question asking?

STEP 4. THANK YOU FOR PARTICIPATING

We are finished. Thank you for helping us. The information you’ve provided will be very helpful for making this survey better.

Interviewer Instructions: Collect materials and escort student out of the interviewing room and back to his or her parent. Please give the student an incentive [$25], the parent an incentive [$25] if they brought their child to and from the interview, and thank them for time and effort.

Case ID#

Interviewer Initials ______

Date:

Time:

School Climate Surveys

INSTRUCTIONAL/Non-Instructional STAFF SURVEY
2015

Conducted by:
U.S. Department of Education
National Center for Education Statistics

Administered by:
Research Support Services, Inc.

Years Working in Current Title	 2 or less  3-6  7 or more
Staff Title _______________	 Core  Non-Core  Instructional Staff  Non-Instructional Staff

MATERIALS NEEDED FOR INTERVIEW

• INTERVIEWER PROTOCOL BOOKLET (THIS BOOKLET)

• RESPONDENT QUESTIONNAIRE

• DO YOU HAVE THE CORRECT VERSION OF THE BOOKLET AND QUESTIONNAIRE?

• CONSENT FORM (TWO COPIES)

• INCENTIVE

• PAYMENT RECEIPT

• DIGITAL RECORDER AND EXTRA BATTERIES

• PENS AND PENCILS

STEP 1: WELCOME AND INTRODUCTION SCRIPT

Interviewer Instructions: The following scripts should NOT be read verbatim. Text in italics is suggested content with which the interviewer should be thoroughly familiar in advance. Text in brackets [ ] presents instructions for the interviewer.

Hello, my name is <your name > and I work for Research Support Services, Inc. It’s nice to meet you, thank you for coming to help us out today.

[Create small talk to build rapport with the teacher/Admin]:

You are here today to help us out with a new survey that we are working on with the U.S. Department of Education. This survey asks about your opinions of the school where you work: [school name]. I will ask you to take the survey as if you were taking a real survey. There are no right or wrong answers, and it’s ok if you do not know some of the answers. As you are answering the survey questions I will ask you about them. I will also stop you once in a while to ask you questions. This will help us make the survey better and clearer for other teachers and school instructional staff just like you.

Because the information you provide is so important, I am going to be taking notes while you are working and while we are talking [INSERT IF THERE IS AN OBSERVER: and my colleague will also be observing our session today].

Also, this interview will be recorded so researchers can review the tapes later. Your responses and any information that refers to you or the name of your school will not be included in our report. Your responses will be used for research purposes only and will not be disclosed, or used for any other purpose except as required by law. [if ask: Education Sciences Reform Act of 2002, 20 U.S. Code, Section 9573]

Remember, in this study we want you to tell us what you think about the survey. Would you like to be a part of this study? [If respondent agrees, continue, if not, end the interview.]

If at any time you want to stop, just let me know.

[If, for any reason, the participant is no longer interested in participating, thank the participant for his/her time and end the interview.]

Before we continue, do you have any questions about what I just said? [Answer any questions the participant may ask.]

PROVIDE RESPONDENT WITH A COPY OF THE INFORMED CONSENT FORM. ASK THE RESPONDENT TO READ THE FORM (OR READ IT TO HIM/HER, DEPENDING ON RESPONDENT PREFERENCE), ANSWER ANY QUESTIONS, AND HAVE THE RESPONDENT SIGN THE FORM. LEAVE A SEPARATE COPY OF THE FORM WITH THE RESPONDENT.

• SIGNED CONSENT FORM COLLECTED

• COPY OF CONSENT FORM GIVEN TO RESPONDENT

• IF THE RESPONDENT HAS CONSENTED TO RECORDING, START THE RECORDER.

STEP 2: THINK-ALOUD PRACTICE

While you fill out the survey I want you to think out loud about how you are answering some of the questions. Hearing you talk about how you figure out your answers to the questions will help me understand how to make the questions better. I am more interested in how you arrive at your answers than in the answers you give me. To help you get started on thinking aloud, we are going to do a practice activity. I’m going to ask you a question and ask you to think aloud as you decide on your answer.

How many windows are there in the house or apartment where you live?

Please tell me what you are thinking while you are figuring out your answer.

Probe as needed for detail: How did you arrive at that answer? What were you visualizing when you were deciding on that answer? Can you tell me more about what you are including in your count?

When complete: Thank you; that’s the kind of detail I’m looking for when you talk to me about your answers to the questions in the survey.

HAND RESPONDENT A COPY OF THE QUESTIONNAIRE AND A PEN.

STEP 3: COMPLETION OF THE QUESTIONNAIRE

Now I would like you to complete the questionnaire. While you are answering the questions, please think about the school you currently work at: [school name]. I’d like you to think aloud while you decide on your answer. Please read aloud anything you would have read to yourself if I were not here.

Remember that I am very interested in hearing you talk about what you are thinking about as you figure out your answers to the questions.

Probe bank (from Willis)

• Can you tell me in your own words what that question was asking?

• What does the word [term] in this question mean to you?

• How easy or difficult is it to remember [topic]?

• How easy or hard was it to choose an answer?

• Tell me what you are thinking?

• How did you come up with that answer?

• You said [answer]. Can you tell me more about that?

• [In response to R actions or comments]: Tell me what happened? Tell me what you were doing?

• [If you pick up on a visual cue like a furrowed brow or a puzzled look]: “Tell me what you are thinking.”

• [If you pick up on a visual cue that indicates confusion like paper flipping or rereading an old question before answering this question]: “Tell me what just happened. Tell me about what you were just doing.”

How strongly do you agree or disagree with the following statements about this school? Mark One Response

28. I can manage almost any student behavior problem. Isafemo58

Strongly Agree

Agree

Disagree

Strongly Disagree

PROBES:

1. What is this question asking?

2. When you answered this question, what kinds of 'student behavior problem' did you think of?

29. I feel safe at this school. Isafpsaf59

Strongly Agree

Agree

Disagree

Strongly Disagree

30. The following types of problems occur at this school often: physical conflicts among students. Isafpsaf60

Strongly Agree

Agree

Disagree

Strongly Disagree

PROBES:

1. What is this question asking?

2. IF AGREE OR STRONGLY AGREE: Tell me about the kinds of physical conflicts that occur.

IF DISAGREE OR STRONGLY DISAGREE: What do you think is meant in this question by physical conflicts?

31. The following types of problems occur at this school often: robbery or theft. Isafpsaf61

Strongly Agree

Agree

Disagree

Strongly Disagree

32. The following types of problems occur at this school often: vandalism. Isafpsaf62

Strongly Agree

Agree

Disagree

Strongly Disagree

33. The following types of problems occur at this school often: student possession of weapons. Isafpsaf64

Strongly Agree

Agree

Disagree

Strongly Disagree

PROBE:

1. Tell me about your answer

2. What do you think they mean here by 'often'?

34. The following types of problems occur at this school often: sexual assault or violence. Isafpsaf143

Strongly Agree

Agree

Disagree

Strongly Disagree

PROBES:

1) Tell me about your answer.

2) What is this question asking you?

3) What does sexual assault mean in this question?

4) What does sexual violence mean in this question?

35. The following types of problems occur at this school often: physical abuse of teachers. Isafpsaf66

Strongly Agree

Agree

Disagree

Strongly Disagree

PROBES:

1) Tell me about your answer.

2) What does physical abuse mean in this question?

36. The following types of problems occur at this school often: student verbal abuse of teachers. Isafpsaf67

Strongly Agree

Agree

Disagree

Strongly Disagree

PROBES:

1) Tell me about your answer.

2. What would you consider often for this type of occurrence?

STEP 4. THANK YOU FOR PARTICIPATING

We are finished. Thank you for helping us. The information you’ve provided will be very helpful for making this survey better.

Interviewer Instructions: Collect materials. Please give the respondent an incentive [$40].