Middle Grades Longitudinal Study (MGLS:2017) Cognitive Interviews and Focus Groups

NCES Cognitive, Pilot, and Field Test Studies System

MGLS 2017 Cog Labs 2014 Vol 3 Protocols

Middle Grades Longitudinal Study (MGLS:2017) Cognitive Interviews and Focus Groups

OMB: 1850-0803

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 1850-0803 can be found here:

Document [docx]

Download: docx | pdf

National Center for Education Statistics

Middle Grades Longitudinal Study of 2016-2017 (MGLS:2017) Cognitive Interviews

Volume III

REVISED Cognitive Laboratory Protocols

OMB# 1850-0803 v. 97

February 27, 2014

MGLS:2017 COGNITIVE INTERVIEW PROTOCOL:

STUDENT MATHEMATICS ASSESSMENT

MGLS: 2017 Cognitive Interview Protocol:
Student Mathematics Assessment

I. Interviewer Welcome Script

The following welcome script should not be read verbatim. You, as the interviewer, need to be familiar enough with the scripts to ask the item probes in a conversational manner. Text written in the box is suggested content for you to be thoroughly familiar with in advance. You should project a warm and reassuring manner toward the participant to develop a friendly rapport and should use conversational language throughout. While the text in this introduction does not need to be read verbatim, all italicized text (notifying student of confidentiality and item-specific probes) is read verbatim to ensure we cover these important topics and ask students these questions in the same way.

Hello, my name is [NAME] and I am working with the U.S. Department of Education on a research study called the Middle Grades Longitudinal Study of 2016-17 (MGLS:2017).

Thank you for coming here today to help us. You may have already talked with your parents about what we are doing here, but just in case you still aren’t completely sure we want to take a second and talk you through what’s going on.

You are here because we asked your [mom/dad] if you might be able to work with us as we create math questions for the MGLS:2017. These questions are going to be given to students all around the country, and what you are helping us with today is essential to creating math questions that help us find out what children are learning in school. After we asked your [mom/dad] they agreed that you could participate and so you are here today.

We need your help selecting good math questions to ask students in grade [student’s grade level]. Today, we’re going to work through some math questions, and while we want you to do your best to answer the questions correctly, what we really want to learn is whether or not the questions make sense. That is, do you understand what the math questions are asking you to do, or is there something about the questions that make them confusing, tricky to understand, or especially hard to answer? Please go through the questions and try to answer them the best that you can. In addition to talking about things you may have found confusing, we would like to ask you about how you answered some of the questions and a little bit about what you were thinking when solving the questions and ways that we might improve some of the questions. It will take about an hour to go through all the questions and talk about them.

Your information and the answers you give today will be kept private. Only people working on the research study will know about your answers and ideas, but they won’t know that it was you who gave those answers and ideas. We will not talk about any of your answers or comments today with your parents, teachers, or school, but please feel free to tell your parents about anything we talk about today. Your help today will only be used for research purposes and may not be used for any other purpose except as required by law.

I will be taking notes, and I would also like to record what we are doing today so I can listen to it later when I write up my notes. No one outside our research team will hear the recording. Is it okay with you if I record our interview?

Just to remind you, your participation is completely voluntary, and if at any point you decide not to continue, that is okay.

Do you have any questions before we get started?

If, for any reason, the student is no longer interested in participating, thank the student for his/her time and end the interview. After answering questions and giving further explanation, begin the interview with the first item.

In addition to the notes you will take when talking with the student after they finish the assessment, make sure to observe the student while taking the assessment, noting signs of restlessness and frustration (such as fidgeting and time spent with eyes wandering around the room instead of on the task at hand). Note also the items for which the student uses a calculator or works on paper (and the location on the paper that has the work for each item). If a student returns to a previous item please note which item they were on, what item they returned to it and if they corrected or completed the item that they returned to.

II. Example Items and Probes

Students will be instructed to complete a set of approximately 20 math questions as they normally would under naturalistic testing conditions (silently, without interruption). They will be asked to circle or highlight words that are difficult to understand and questions that they find confusing. They can skip any item that they find confusing. Then, after the task is completed, using a think-aloud protocol, they will be asked to talk about the questions that they skipped, found confusing, difficult to read or to understand. We then will ask them to try the problem again and talk about what they are thinking as they do the problem. In addition, we will ask questions about select items that have characteristics that we are interested in learning more about.

Some of these questions are for students in later grades and so may be very difficult. I would like you to circle or highlight anything that you find confusing or difficult to understand. You may skip any question that is too difficult to understand. You can use the calculator or extra paper to help you in solving the problems. You can ask me questions at any point.

Ask generic and item-specific probes, as appropriate, before going on to the next identified item.

When probing, think about the following kinds of details that we would like to know about how the student reads and understands each question:

Understanding of the Item: We are interested in learning if students have never seen the word before or if they forgot its meaning, and if they were still able to answer the question. Reassure them that it is okay if they did not know all the words and that their feedback is important – we want to make sure the questions are straight-forward for all students.
1. What questions and/or words were confusing or difficult to read?
  1. Have the student circle or highlight problem words and talk more about them – are they completely unfamiliar?
  2. Ask student to identify a word that may have been less confusing or easier. Ask how their teacher or textbook usually asks these questions, and what words they use.
2. Probe about word problems with more than two sentences in particular. Can they restate in their own words what the question is asking? If not, why?:
  1. Can you tell me, in your own words, what the question is asking?
  2. Have you answered questions like this in school?
  3. How would you [did you] find an answer to this question?
  4. Are there any words, phrases, or (if applicable) pictures/tables/figures/graphs that are difficult for you? If so, which ones?
  5. Is there anything about this question that confuses you? If yes, do you have a suggestion for how to make it easier to understand?
  6. (If applicable) If you tried these items on a computer, and it said to click on the answer, what would you do?
  7. (If applicable) How many answers are there for this question? How did you go about selecting those answers?

Presentation of the Item:
1. Certain items will have graphs or displays associated with the question. How does this affect the student’s understanding? Do they see questions like these in their math books or math class?
2. Probe the student about their familiarity with the way concepts are presented, especially with fractions or equations that can be represented in multiple ways.
  1. Are some conventions easier for the student to understand than others? Why?
3. Use Probes appropriate for item content:
  1. Ask the student to read the question to you (including the numbers). Note words that are difficult. Ask: Can you tell me, in your own words, what the question is asking?
  2. How did you answer this question?
  3. Are there any words, phrases, or (if applicable) pictures/tables/figures/graphs that are difficult for you? If so, which ones?
  4. Did you recognize the symbols in this question? Have you seen your math teacher or textbook use these symbols?[ point to symbols in the item]
  5. [If the item was skipped] Did the symbols confuse you at all? Is there anything that would make it easier to understand the question?

Language:
1. If an item has a proper noun, such as a person or place’s name, does the student recognize that it is a name or are they spending time trying to decode it?
2. For sentences with item clauses (i.e., IF this, THEN that), does the student understand what the question is asking? Ask the student to put the question in their own words. Students can also show in writing what the question is asking, using the paper you provide.
3. Probes:
  1. Can you tell me, in your own words, what the question is asking?
  2. How did you answer this question?
  3. Can you please read the word in these parentheses out loud [point to difficult word(s)]?
  4. Are there any words, phrases, or (if applicable) pictures/tables/figures/graphs that are difficult for you? If so, which ones?
    1. Did it affect how you answered the question? How?
    2. What can you tell me about these words [Point to words mentioned above)]? Did you need to read these words [proper names] in order to answer the question?

Understanding the Questions

With these questions, we are interested in probing whether the student understood what the question is asking.

Please take a moment to look at the following question.

CCSS 7.NS.1 – Applying

MGLS NS053

Starting at the point A marked on the number line below, subtract -3. Plot your result on the number line.

Item-Specific Probes:

Can you tell me, in your own words, what is the question asking?
How did you answer this question?
Are there any words, phrases, or (if applicable) pictures/tables/figures/graphs that are difficult for you? If so, which ones?
If this question was shown to you on a computer, and it asked you to click on the answer, what would you do?
Is there anything about this question that confuses you? If yes, do you have a suggestion for how to make it easier to understand?

Please take a moment to look at the following question.

CCSS 6.EE.1 – Knowing

MGLS EE010

Which of the following are equal to 2³? Choose all that apply.

6
8
9
12
3(2)
2 * 2 * 2
4(2)
2(2 * 2)
3 * 3
(2 * 2) * 3

Item-Specific Probes:

Can you tell me, in your own words, what is the question asking?
How did you answer this question?
How many answers are there for this question?
Are there any words or (if applicable) pictures/tables/figures/graphs that are difficult for you? If so, which ones?
Is there anything about this question that confuses you? If yes, do you have a suggestion for how to make it easier to understand?

Presentation

In this question, we want to observe whether the student recognizes the use of the “*” to represent multiplication, and if the layout of this question is confusing.

Please take a moment to look at the following question.

CCSS 7.EE.3 – Applying

MGLS EE033

Evaluate this expression when P = 7.

Item-Specific Probes:

Ask the student to read the question to you (including the numbers). Note words that are difficult. Ask: Can you tell me, in your own words, what the question is asking?
[If unclear about the meaning of the *] point to * and ask: Have you seen your math teacher or textbook use this symbol or expressions like these?[repeat for ( ) ]
Are there any words or phrases that are difficult for you? If so, which ones?
Is there anything that would make it easier to understand the question?

Language Use (Example Item for Recognizing Proper Nouns / Names and Mathematical Terms)

In this question, we want to observe whether the student spends a lot of time trying to decode the name Pierre. We also will examine how students attempt the problem and understand the phrases “constant rate” and “represent the relationship,” particularly for students in grades 6 and 7 who may not been introduced to grade 8 Functions standards (that is, have not had an Algebra class). We will probe about whether they read all the words or skipped over some (and whether they thought that they needed to read names), and whether the names Pierre or Louis made it difficult to understand.

In this item we will also specifically probe how they interpreted the sentence, “His friend Louis already painted some of the wall for Pierre.” Did they assume that Pierre would repaint that wall or did they understand that the area of the wall painted by Pierre was in addition to the wall that Louis painted?

CCSS 8.F.4a – Reasoning

MGLSF058

Pierre is painting a wall in his house. He paints at a constant rate. His friend Louis already painted some of the wall for Pierre. The table below shows the total area painted after Pierre paints for a certain number of minutes.

number of minutes that Pierre spent painting (m)	total area painted in square feet (a)
1	27.75
6	61.50

Pierre constructed the following function to represent the relationship between a, the total area painted in square feet, and m, the number of minutes that Pierre spent painting.

a = 6.75m

Is Pierre’s equation correct? Answer “yes” or “no”.
Explain your reasoning.

Item-Specific Probes:

Can you tell me, in your own words, what the question is asking?
How did you answer this question?

What can you tell me about these words [Point to (Pierre) and (at a constant rate) and (represent the relationship)]. Did you know what this word/phrase means? Did you need to understand the word/phrase in order to answer the question?

Are there any other words or phrases that are difficult for you? If so, which ones?
1. Did it affect how you answered the question? How?
2. Did you need to read these words in order to answer the question?
Tell me about this part of the question [Point to (His friend Louis already painted some of the wall for Pierre.)]. Can you tell me, in your own words, what this sentence tells you? [Probe to also find out if they considered this part of the total painted area that Pierre had in his table or additional wall that was painted – did the student think Pierre repainted the wall that Louis had painted?]
What kinds of things did you think about in answering this question?

Collect the student’s assessment and review with them any items where the student circled a word they did not understand.

At the end of the interview, ask the student again if there were any other words or phrases that they did not understand or things that they found confusing as they went through the items.

General Probes:

Have you seen this word before in your math book or does your teacher use this word?
Did you need to understand the word in order to answer the question?

How did not knowing this word affect how you answered the question?

How would you change this question to make it easier to understand?

III. Debriefing and Thank You for Participation

Thank student for his/her time. Collect all hard copy forms and scratch paper the student used.

Before we finish, I’d like to hear any other thoughts you have about what you’ve been doing this morning/afternoon.

Is there anything else you would like to tell me about working on the task?

Is there anything you would like to ask me about what we did today? [Answer student questions]

Thank you for helping us to improve our math questions.

MGLS:2017 COGNITIVE INTERVIEW PROTOCOL:

STUDENT EXECUTIVE FUNCTION ASSESSMENT

MGLS:2017 Cognitive Interview Protocol:

Student Executive Function Assessment

I. Interviewer Welcome Script

The following scripts should not be read verbatim. You, as the interviewer, need to be familiar enough with the scripts to introduce the think-aloud process in a conversational manner. Text written in italics is suggested content for you to be thoroughly familiar with in advance. You should project a warm and reassuring manner toward the participant to develop a friendly rapport and should use conversational language throughout. The italicized text should be read verbatim, both in the introduction and the probes. This will ensure that we cover these important topics and ask students these questions in the same way.

Hello, my name is [NAME] and I am working with the U.S. Department of Education on a research study called the Middle Grades Longitudinal Study of 2016-17 (MGLS:2017).

You are here because we asked your [mom/dad] if you might be able to work with us as we develop activities to help us understand how students your age organize, remember, and use information.. These activities are going to be given to students all around the country, and what you are helping us with today is essential to selecting our activities. After we asked your [mom/dad] they agreed that you could participate and so you are here today.

Today, you will complete a few activities or games on the laptop. We need your help to make sure the activities work on the laptop and that they make sense. That is, while we want you to do your best to complete the activities as quickly and correctly as you can, what we really want to learn is whether the instructions are easy to understand, and if you have any problems using the laptop to complete the activity. After you finish, I will ask you to tell me about any activities that were confusing or that did not make sense, so that you can help us make sure these activities are clear for students.

What you tell me today will make these activities better. Your information and the answers you give today will be kept private. Only people working on the research study will know about your answers and ideas, but they won’t know that it was you who gave those answers and ideas. We will not talk about any of your answers or comments today with your parents, teachers, or school, but please feel free to tell your parents about anything we talk about today. Your help today will only be used for research purposes and may not be used for any other purpose except as required by law. I will be taking notes, and I would also like to video record what we are doing today so I can watch it later when I write up my notes about what we learn from you about these activities. No one outside our research team will see the video recording. Is it okay with you if I video record our interview?

Today you will help us with four activities. Each activity should take about five to ten minutes. Please do each carefully. Going through all the activities and my questions will take about 60 minutes.

Just to remind you, your participation is completely voluntary, and if at any point you decide not to continue, that is okay.

Do you have any questions before we get started?

If, for any reason, the student is no longer interested in participating, thank the student for his/her time and end the interview. If the student agrees to be video recorded, make sure to start the recording once you have obtained his or her permission. After answering any student questions, begin the interview with the first activity.

II. Observation Prompts and Probes for Executive Function Tasks

This section will include four measures: the Stop Signal, 3-Back with verbal stimulus, 2-Back with nonverbal stimulus, and the Hearts and Flowers task. Together, these measures will assess inhibitory control, working memory, and shifting in middle grade students. Students will be provided with a laptop and should be instructed to complete the tasks following the directions on the screen. They should complete the tasks as they normally would under naturalistic testing conditions (silently, without interruption). In general, we are interested to know if the instructions were clear and the student understood each task. Interviewers will observe students as they take the executive function assessment and interview the students about their experiences with the tasks. The interviewer will ask students about the instructions after each practice session and before the student attempts the actual task.

As they work through the tasks, the interviewer will observe students and note any evidence of engagement, understanding, boredom, or confusion in students’ facial expressions or body language. Interviewers should observe how students complete each executive function task, using the questions below as a guide.

Interviewer Observation Questions:

1. Does the student express signs of confusion, boredom, or excitement (e.g., they fidget, seem to move attention away from the task and let eyes wander the room)?

2. Does the student rapidly move through the Stop Signal items or take his/her time?

3. How often does the student change/switch their response to the items?

4. What is the start and stop time of each task? Write the information down on the timing form.

5. Does the student put on headphones when directed and check the sound? Does the student have any problems with continuing to wear headphones? Explain.

6. In the Hearts and Flowers and Stop Signal tasks, does the student use correct hand and finger placement on the arrow keys as directed? If not, explain.

7. In the N-back tasks, does the student utilize the space bar as directed? If not, explain.

8. How often do you need to redirect the student to the task? Explain.

9. Does the student appear to need additional directions or practice sessions? Explain.

After the student reads the instructions and completes the practice test for each section, the interviewer will ask them the following questions:

Can you tell me in your own word, how to play this game?
Were the instructions difficult to understand or to read?
How difficult to you think it will be for other students your age to understand the directions?

After students complete the tasks, the interviewer will debrief with the student about his/her experiences. Interviewers should probe students during the debriefing on any notable behaviors observed when students are completing the items in the executive function tasks. For example, if the interviewers see that a student is spending a lot of time on a particular item within a task, they should note that and probe on it later (e.g., “I noticed you spent some time on this item. Can you tell me about…”).

We are particularly interested in learning if students understood what the task was asking them to do and whether they had difficulties with any of the tasks, including any problems with the laptop or program not working correctly.

The italicized text below should be read aloud to the student.

Were any of the instructions difficult to understand or hard to read?
Do you think other students your age would be able to understand the directions?
Was the laptop difficult to use for any parts of the activities?
Was there any other information that would have made these activities easier to understand or complete?
Was it clear how to do these activities?
1. Which activity did you like the most?
2. Which activity did you like the least? Why did you like it less than the others?
Do you have any other questions?

III. Debriefing and Thank You for Participation

Thank student for his/her time.

Before we finish, I’d like to hear any other thoughts you have about what you’ve been doing this morning/afternoon.

Is there anything else you would like to tell me about participating in these activities?

Is there anything you would like to ask me about what we did today? [Answer student questions]

Thank you for helping us.

Make sure to end the recording when the interview is complete.

MGLS:2017 COGNITIVE INTERVIEW PROTOCOL:
STUDENT INTERVIEW

MGLS:2017 Cognitive Interview Protocol:

Student Interview

I. Interviewer Welcome Script

The following Interview Welcome Script should not be read verbatim. You, as the interviewer, need to be familiar enough with the scripts below to introduce the MGLS:2017 study and the purpose of the cognitive interview in a conversational manner. Text in the box is suggested content for you to be thoroughly familiar with in advance. You should project a warm and reassuring manner toward the participant to develop a friendly rapport and should use conversational language throughout. Please note any italicized text for item-specific probes should be read verbatim to ensure we cover these important topics and ask students these questions in the same way.

Hello, my name is [NAME] and I’m from [DIR/Mathematica] and we are working with the U.S. Department of Education on a research study called the Middle Grades Longitudinal Study of 2016-17 (MGLS:2017).

You are here because we asked your [mom/dad] if you might be able to work with us as we develop a student questionnaire for the MGLS:2017. This questionnaire is going to be given to children all around the country, and what you are helping us with today is essential to creating that questionnaire. After we asked your [mom/dad] they agreed that you could participate and so you are here today.

We need your help selecting good questions to ask students in grade [student’s grade level]. Today, I have some questions that middle-grade students like you in many different schools will be asked to answer about themselves, their education, and their families. While we want you to do your best to answer the items, what we really want is your feedback on how easy questions are to understand. That is, do all of the questions make sense? Is there a difficult word that you don’t understand? Is there a better way for us to ask a question so that it makes more sense? There are no right or wrong answers to these questions, but please try your best, because the things you tell us today will be used to make improvements to these questions. This should take about one hour.

What you tell me today will make these questions better. Your information and the answers you give will be kept private. Only people working on the research study will know about your answers and ideas, but they won’t know that it was you who gave those answers and ideas. We will not talk about your answers or comments with anyone outside the research team, including your parents, teachers, or school, but please feel free to tell your parents about anything we talk about today. Your help today will only be used for research purposes and may not be used for any other purpose except as required by law. I will be taking notes, and I would also like to record what we are doing today so I can listen to it later when I write up my notes about what we learn from you about these questions. No one outside our research team will hear the recording. Is it okay with you if I record our interview?

Just to remind you, your participation is completely voluntary, and if at any point you decide not to continue, that is okay.

Do you have any questions before we get started?

If, for any reason, the student is no longer interested in participating, thank the student for his/her time and end the interview. After answering any student questions, begin the interview with the first item.

II. Survey Items and Probes for Non-Assessment Student Questionnaire

The student interview consists of students answering 8 items or item series that were newly created, modified, or need further testing with this age group. Because we are interested in understanding how students interpret and answer these items, this portion of the interview will be interactive: after the student is presented with and answers each item, the interviewer will ask item-specific probes. By asking probes right after each item or item series, students will be able to provide item-specific information while experiencing minimal recall difficulties.

Students will be instructed to complete each item or item series as they will do during the field test and national study (independently without interruption from the interviewer). Then, after the item is completed, they will be asked to describe what they were thinking. Ask item-specific probes before going on to the next item.

1. Students’ Perceptions of Parents’ Behaviors

We are interested in understanding how students answer item series about various things that their parents may say or do in response to the good or bad grades they receive at school. We want to examine students’ comprehension of the items and whether their parents respond to their grades in ways that are not currently listed. We are also interested in how students may respond to items generally about “their parent(s)/guardian(s)” (that is, “do your parent(s)/ guardian(s) encourage you…?”) versus asking questions pertaining to each individual parent (that is, “does your mother encourage you…?”; “does your father encourage you…?).

All italicized text should be read aloud to the student.

Please take a moment to read this question and the answers to the question. As you read through them, please circle any words that are difficult to understand or answers that you were not sure how to respond to.

A1. What do your parents/guardians do when you receive good grades in school?

When I get good grades my parents/guardians ….

	MARK ONE EACH ROW
	Yes	No
a. Give me a reward, like a present or gift	1 □	2 □
b. Give me money or increase my allowance ………	1 □	2 □
c. Take me to some place special, like out to dinner or to a movie	1 □	2 □
d. Give me more opportunities to make decisions for myself	1 □	2 □
e. Encourage me to try harder	1 □	2 □
f. Are less strict with me	1 □	2 □
g. Tell me I am a good student	1 □	2 □
h. Tell me they are proud of me	1 □	2 □
i. Say I should have done even better	1 □	2 □
j. Do not know about my good grades	1 □	2 □
k. Do not care about my good grades	1 □	2 □
l. Do not really do anything	1 □	2 □
m. Other (Please specify)	1 □	2 □

SURVEY: Ginsburg & Bronstein, 1993

Student Item-Specific Probes:

Can you tell me, in your own words, what the question is asking?
Did you have trouble answering any of these statements?
1. Were there any words or statements that were confusing or unclear?
2. NOTE TO INTERVIEWER: Discuss any of the statements or words the student has circled/noted when completing the item series, asking why they found it difficult and if they had suggestions for how to make it easier to understand.
What grades of yours did you think about when answering each question? Did you think about your recent test or homework grades, your most recent report card, this school year, or something else?
What does the phrase “Give me more opportunities to make decisions for myself” mean to you? Can you give me an example of this?
What does the phrase “Are less strict with me” mean to you? Can you give me an example of this?
Are there other things that your parents or guardians say or do when you get good grades that were not listed among the questions you answered?
1. If so: What other things do your parents or guardians say or do when you get good grades?
Who came to mind when you answered the question about how your parents or guardians would respond to the good grades you get?
1. Why did this person come to mind?

SURVEY: Ginsburg & Bronstein, 1993

A2. What do your parents/guardians do when you receive bad grades in school?

When I get bad grades my parents/guardians ….

	MARK ONE EACH ROW
	Yes	No
a. Reduce my allowance	1 □	2 □
b. Give me fewer opportunities to make decisions for myself	1 □	2 □
c. Are more strict with me	1 □	2 □
d. Punish or ground me	1 □	2 □
e. Encourage me to try harder	1 □	2 □
f. Offer me a reward, such as money or a present, if I do better next time	1 □	2 □
g. Make me feel bad	1 □	2 □
h. Offer to help me with my school work	1 □	2 □
i. Offer to find me a tutor	1 □	2 □
j. Do not know about my bad grades	1 □	2 □
k. Do not care about my bad grades	1 □	2 □
l. Do not really do anything	1 □	2 □
m. Other (Please specify)	1 □	2 □

Student Item-Specific Probes:

Can you tell me, in your own words, what the question is asking?
1. PROBE: How do you define “bad” grades?
Did you have trouble answering any of these statements?
1. Were there any words or statements that were confusing or unclear?
2. NOTE TO INTERVIEWER: Discuss any of the statements or words the student have circled/noted when completing the item series, asking why they found it difficult and if they had suggestions for how to make it easier to understand.
What grades of yours did you think about when answering each question? Did you think about your recent test or homework grades, your most recent report card, this school year, or something else?
What does the phrase “give me fewer opportunities to make decisions for myself” mean to you? Can you give me an example of this?
What does the phrase “are more strict with me” mean to you? Can you give me an example of this?
As you read through the statements, did you think there were statements that were really asking the same thing?
1. If so: Which statements felt that way?
Are there other things that your parents or guardians say or do when you get bad grades that were not listed among the questions you answered?
1. If so: what other things do your parents or guardians say or do when you get bad grades?
Who came to mind when you answered these two questions about how your parents or guardians would respond to the grades you get? Why did this person come to mind?
1. Is there another adult who takes care of you who came to mind when answering this question?
  1. If so: Who else takes care of you?

Note to interviewer: If student names only one individual, proceed to question A3 in the interviewer protocol script. If student names two people or more (e.g., mother, father, stepparent, relative, guardian), continue with the following set of probes below. Parent 1, Parent 2, Parent 3, etc. refer to the individuals the student mentions in probes 8 and 8a.

Thinking about the different individuals you named, how did you come up with your answer?
If the question had been specifically about “What does your [PARENT 1] say or do when you get good or bad grades?”, would you have responded (or answered) differently to any of these items?
If the question had been specifically about “What does your [PARENT 2] say or do when you get good or bad grades?”, would you have responded (or answered) differently to any of these items?
If applicable: Continue to ask this question for all individuals mentioned in response to probes 8 and 8a.

Students’ Time Use

We are interested in understanding specific dimensions of students’ time use. With the first item in this section, we are interested in understanding how much time students spend with their friends and how the students differentiate spending time with their friends in person versus online. We would also like to know how students recall and estimate how much time they spend with their friends. With the second item in this section, we are interested in understanding what work or job experiences students may have. We are also interested in learning whether our lists of jobs adequately represent students’ experiences or if additional items need to be added.

All italicized text should be read aloud to the student.

Please take a moment to read this question and the answers to the question.

A3. During a typical weekday during the school year, how many hours do you spend hanging out or socializing with your friends?

SURVEY: High School Longitudinal Study of 2009

MARK ONE ONLY

1 □ Less than 1 hour

2 □ 1 to 2 hours

3 □ 2 to 3 hours

4 □ 3 to 4 hours

5 □ 4 to 5 hours

6 □ 5 or more hours

Student Item-Specific Probes:

Can you tell me, in your own words, what the question is asking?
How did you answer this question? Did you have trouble answering the question?
What does the phrase “hanging out or socializing with your friends” mean to you? Can you give me an example of this?
1. PROBE: Is it something you do in person (e.g., invite your friends over to your home after school, hang out at the mall together, go to the movies together, etc.)?
  1. If so: Can you give me an example of what you are doing when you are hanging out with or socializing with your friends in person?
2. PROBE: Is it something you do online?
  1. If so: Can you give me an example of what you are doing when you are hanging out with or socializing with your friends online?
  2. What do you use to do this (i.e. do you use your phone, computer, etc.)?
  3. What specific websites or applications do you use to hang out or socialize with your friends online?
Who did you think about in answering this question?
1. PROBE: Did you think of friends from school? From your neighborhood? Friends you know from somewhere else?
2. PROBE: Did you think of friends who you know or met online?
  1. If so: How did you meet (i.e. in a chat room, while playing games online, etc.) these online friends?
What does the phrase “typical weekday during the school year” mean to you?
How did you figure out the number of hours you spend hanging out or socializing with your friends during a typical day?
1. PROBE: Did you think about what you typically did last week, last month, or even longer ago than that?
2. PROBE: Was it difficult to separate out how much time you spend hanging out or socializing with your friends on the week days versus how much time you spend with them on the weekends?
  1. If so: How so?

All italicized text should be read aloud to the student.

This next question asks about getting paid for work you may do. This does not include things like doing chores at home or getting allowance. As you read through the question, please circle any words that are difficult to understand or answers that you were not sure how to respond to.

SURVEY: National Education Longitudinal Study, 1988

A4. Which of the categories below describe the kind of work you do/did for pay? (Do not include chores or other work around the house, or an allowance you might receive.)

MARK ALL THAT APPLY

1 □ Lawn work

2 □ Waiter/waitress, dishwasher, or busser

3 □ Newspaper route

4 □ Babysitting or child care

5 □ Farm or agricultural work

6 □ Other manual labor

7 □ Store clerk, salesperson

8 □ Office or clerical

9 □ Odd jobs

10 □ Other (please specify)

11 □ I have not worked for pay

Student Item-Specific Probes:

Can you tell me, in your own words, what the question is asking?
What kinds of things did you think about in answering this question?
What does the phrase “work you do/did for pay” mean to you?
Of the types of work that were listed, were there any that were difficult to understand?
1. If so, which ones were difficult to understand?
Is there something you thought about, but were not sure which category it belonged to?
1. If so, what did you think about?
ASK ONLY IF STUDENT MARKED RESPONSE OPTION 10: Are there other types of work that you have done that were not listed above and that you did not write in yourself?
1. If so, what other type of work have you done?

Students’ Interests/Sparks

We are interested in understanding whether students have a “spark” in their lives. By “spark” we mean any interests, hobbies, and talents the students are excited about. The list of talents, interests, and hobbies were primarily developed from open-ended responses reported in the MADICS survey and additional responses options were created to further capture students’ experiences. We are interested in learning about what talents, interests, or hobbies students engage in, and having students talk about what activities do/do not make this list and why.

All italicized text should be read aloud to the student.

Please take a moment to read the next few questions and the answers to the questions. As you read through them, please circle any words that are difficult to understand or answers that you were not sure how to respond to.

MODIFIED SURVEY: Maryland Adolescent Development In Context Study: Wave 1

A5a. Below is a list of talents, interests, or hobbies that students your age are sometimes excited about. Please select the talents, interests, or hobbies that are very important to you. If you do not see yours listed, use the “Other” area to mark an “X” and write in that talent, interest, or hobby on the line provided.

What talents, interests, or hobbies are you excited about?

MARK ALL THAT APPLY

1 □ Math/Science

2 □ Writing/Reading

3 □ Computers/electronics

4 □ Team sports/athletics (Baseball, Basketball, Football, Gymnastics, Volleyball, etc.)

5 □ Physical activities (Biking, Running, Martial Arts, Skateboarding, Ice Skating etc.)

6 □ Dance

7 □ Music

8 □ Singing/Choir

9 □ Art

10 □ Drama

11 □ Volunteering/service/activism

12 □ Being in nature, caring for animals, or participating in outdoor recreation

13 □ Doing construction, architecture, or other types of mechanics or engineering

14 □ Collecting (trading cards, stamps, models, etc.)

15 □ Cooking/baking

16 □ Other (Specify):

17 □ I don’t have any talents, interests, or hobbies that I am excited about

(PUT DOWN YOUR PENCIL AND WAIT FOR FURTHER INSTRUCTIONS)

A5b. Among the talents, interests, or hobbies you selected above (question A4a), which are you the most excited about?

MARK ONE ONLY

MODIFIED SURVEY: Maryland Adolescent Development In Context Study: Wave 1

1 □ Math/Science

2 □ Writing/Reading

3 □ Computers/electronics

4 □ Team sports/athletics (Baseball, Basketball, Football, Gymnastics, Volleyball, etc.)

5 □ Physical activities (Biking, Running, Martial Arts, Skateboarding, Ice Skating etc.)

6 □ Dance

7 □ Music

8 □ Singing/Choir

9 □ Art

10 □ Drama

11 □ Volunteering/service/activism

12 □ Being in nature, caring for animals, or participating in outdoor recreation

13 □ Doing construction, architecture, or other types of mechanics or engineering

14 □ Collecting (trading cards, stamps, models, etc.)

15 □ Cooking/baking

16 □ Other (Specify):

MODIFIED SURVEY: Maryland Adolescent Development In Context Study: Wave 1

A6. Do you have a parent, mentor, friend, or another caring adult who has helped you pursue the talents, interests, or hobbies you are excited about?

1 □ Yes

0 □ No

Student Item-Specific Probes for A5a-b, and A6:

Can you tell me, in your own words, what the questions are asking?
1. PROBE (IF NOT ADDRESSED IN RESPONSE): What does “excited about” mean to you?
What does the phrase “talents, interests, or hobbies” mean to you?
Were there any talents, interests, or hobbies that you thought about but decided not to include? If so, what else came to mind? Was there a reason why you chose not to include it?
1. PROBE: Were there any talents, interests, or hobbies that you have that you were not sure which category they went under? If so, which one?
On a scale of 1 to 5, with 1 being easy and 5 being difficult, how hard would it be for kids your age to answer this question?

Interviewer Note: If student answered A6 ask:

When you said, [STUDENT’S RESPONSE TO A6], what kinds of things did you think about in answering this question?
Who did you think about in answering the question?
1. PROBE: What does that person/people do to help you with your talents, interests, or hobbies? Can you give me an example?
What does the phrase “helped you pursue the talents, interests, or hobbies you are excited about” mean to you?

Students’ Educational Aspirations and Expectations

With this study question, we are interested in understanding students’ educational goals. Because these items were originally used with older students, we want to know how students in grades 6, 7, or 8 think about their educational goals and if they have any difficulties answering the questions.

All italicized text should be read aloud to the student.

A7. As things stand now, how far in school do you think you will get?

MARK ONE ONLY

MODIFIED SURVEY: High School Longitudinal Study of 2009

1 □ Won’t finish high school

2 □ Will graduate from high school, but won’t go any further

3 □ Will go to vocational, trade, or business school after high school

4 □ Will attend college

5 □ Will graduate from college

6 □ Will attend a higher level of school after graduating from college

7 □ Don’t know

Student Item-Specific Probes:

Can you tell me, in your own words, what the question is asking?
How did you answer this question?
What kinds of things did you think about in answering this question?
What does the phrase “vocational, trade, or business school” mean to you? Can you give me an example of what this means?
What does the phrase “higher level of school after graduating from college” mean to you? Can you give me an example of what this means?
Did you have a hard time picking your answer?
1. If so: Were you deciding between two answers?
2. If so: What helped you figure out which one to choose if you were deciding between these two answer choices?

If student responded 1, 2, or 3 to question A7, skip question A8 and go to the Debriefing and Thank You.

Let’s do another.

A8. How much do you agree with the following statement: I am definitely going to college.

MARK ONE ONLY

MODIFIED SURVEY: High School Longitudinal Study of 2009

1 □ Very Strongly Disagree

2 □ Strongly Disagree

3 □ Disagree

4 □ Agree

5 □ Strongly Agree

6 □ Very Strongly Agree

Student Item-Specific Probes:

Can you tell me, in your own words, what the question is asking?
What kinds of things did you think about in answering this question?
Did you have a hard time deciding between two of these answer choices?
1. If so: What helped you figure out which one to choose if you were deciding between these two answer choices?
On a scale of 1 to 5, with 1 being easy and 5 being difficult, how hard would it be for kids your age to answer this question?

III. Debriefing and Thank You for Participation

Thank student for his/her time.

Before we finish, I’d like to hear any other thoughts you have about the questions you answered for us today.

Is there anything else you would like to tell me about working on the questions?

Is there anything you would like to ask me about what we did today? [Answer student questions]

Thank you for helping us to improve our questionnaire.

MGLS:2017 COGNITIVE INTERVIEW PROTOCOL:
PARENT QUESTIONNAIRE

MGLS:2017 Cognitive Interview Protocol:

Parent Questionnaire

Interviewer Welcome Script

The following scripts should be read verbatim. Text in the box is suggested content for you to thoroughly review in advance and be familiar enough with to present in a conversational manner. You should project a warm and reassuring manner toward the participant to develop a friendly rapport and should use conversational language throughout. Please note that any italicized text for item-specific probes should be read verbatim to ensure we cover these important topics and ask parents these questions in the same way.

Hello, my name is <name>. I’m from Decision Information Resources, Inc. and we are working with the U.S. Department of Education.

Thank you for agreeing to help us today. Decision Information Resources and Mathematica Policy Research are working with the National Center for Education Statistics, part of the U.S. Department of Education, to design a new national longitudinal study of how children learn and progress during sixth, seventh, and eighth grade. This study, the Middle Grades Longitudinal Study of 2016-17 (or MGLS:2017), will collect information from students, parents, teachers, and school administrators.

You should have received a brief letter in the mail or e-mail that included a few questions for you to review. These are questions that parents of middle grade students across the country will be asked to answer about their children’s education, behavior, and family life. Today, I will ask you some follow-up questions about the topics in those questions. This should only take about 30 minutes. Your feedback will help us make these questions clearer for parents who will participate in the study in the future.

We would like to record this interview so the members of our research team can review the recordings later. No one else will hear the recording outside of the research team. What you say is very important to us, and we don’t want to miss anything. Your information will be kept strictly private and will not be shared with anyone outside the research team. Your feedback may be used only for research purposes and may not be used for any other purpose except as required by law. Is it okay if I record our interview?

Do you have any questions before we get started?

If, for any reason, the parent is no longer interested in participating, thank the parent for his/her time and end the interview. After answering questions and giving further explanation, begin the interview with the first item.

II. Survey Topics, Items, and Probes

Parents will be sent the parent survey questions in advance of the interview so that they are aware of the topics we will cover. During the cognitive interview, the participant will read the questions and then the interviewer will follow-up with the probes.

All italicized text should be read verbatim to the parent.

Conversations About Future Planning

We are interested in the conversations that parents have with their children about school, future life events and decisions (careers, schooling after high school, life goals more generally). Here, we want to learn about the frequency and depth of conversations parents have with their children about the courses they take during and across the middle grades in order to develop and modify existing survey items as needed. The participant will read the question and answer the items, then the interviewer will ask the probes immediately after.

A1. Since the beginning of this school year, have you talked with your child about which math courses to take next school year?

MODIFIED SURVEY: High School Longitudinal Study of 2009

1 □ Yes

2 □ No

Item-Specific Probes:

Can you tell me, in your own words, what the question is asking?
1. How did you answer this question?
Who initiated this conversation?
What kinds of things did you discuss?
Is your response based on discussions you had with your child only or did you consider discussions others in the household had with him/her as well?
1. If “others”, who else did you include?
2. What led you to base your response on the discussions of multiple people?
Would you have chosen a different answer if the question had asked “Have you talked with your child about which math courses to take next school year?”
Were there any words in the question that you found confusing? (PROBE AS NEEDED: Was the way the question was worded confusing? What would have made the question clearer?)

Social skills and positive behaviors

Here we are interested in how parents respond to questions about their child’s social skills and positive behaviors. The participant will read and answer statements about the child’s behaviors and the interviewer will follow with probes inquiring whether the questions were clear and easy to understand and to evaluate a new response scale (i.e., 5-point frequency scale).

Next we would like to ask you some questions about your child’s social skills and behaviors. Please read through the list of statements below and then I will ask a series of follow-up questions.

B1. Please read the following list of items that sometimes describe children. For each of the following characteristics please mark how often this occurred for your child over the last six months.

MARK ONE PER ROW	NEVER	RARELY	OCCASIONALLY	OFTEN	ALWAYS
a. He/She is considerate of other people’s feelings.	1 □	2 □	3 □	4 □	5 □
MODIFIED SURVEY: ECLS-K:11 b. He/She is helpful if someone is hurt, upset, or feeling ill	1 □	2 □	3 □	4 □	5 □
c. He/She is nervous in new situations, easily loses confidence	1 □	2 □	3 □	4 □	5 □
d. He/She often offers to help others (parents, teachers, children)	1 □	2 □	3 □	4 □	5 □
e. He/She gets along better with adults than with other youth	1 □	2 □	3 □	4 □	5 □

Item-Specific Probes:

Can you tell me, in your own words, what the question is asking?
1. How did you answer this question?
What behaviors or events came to mind when you heard the questions?
1. Why?
Were there any words in the question that you found confusing? (PROBE AS NEEDED: Was the way the question was worded confusing? What would have made the question clearer?)
Was there a particular statement about your child where you had a hard time deciding between two of these answer choices? If so: What helped you figure out which one to choose if you were deciding between two answer choices?
Did the response categories make the question easy or difficult to understand?
1. Would making your selection have been easier with more answer choices? Less answer choices?
Can you tell me, in your own words, what the phrase “Rarely” means to you? Can you tell me, in your own words, what the phrase “Occasionally” means to you?

PROBE: How did you decide between “Rarely” and “Occasionally”?

Parent involvement in school

With the following questions about parent involvement in school, we are interested in the relevance, understanding, and appropriateness of the response options and the questions for parents of middle grade students. Using the Parental Role Construction for Involvement in the Child’s Day-to-Day Education scale (Hoover-Dempsey), we have drawn questions from Scales 1 and 2 of the Belief Section (Table C1.) and the Behavior Section (Table C.2). Testing will focus on the most appropriate mix of items from those two sections. Additionally, across parenting topics we are interested in whether parent answers are based on his/her own personal behaviors alone or if multiple people’s behaviors (that is, both of the parents and/or other adults) in the home are taken into account.

Next we would like to ask questions about your thoughts on your involvement in your child’s school. Read through and answer the list of statements. I will ask a series of follow-up questions once you are done.

C1. Indicate how much you AGREE or DISAGREE with each of the following statements. Please think about the current school year as you consider each statement.

	MARK ONE PER ROW
	DISAGREE VERY STRONGLY	DISAGREE	DISAGREE JUST A LITTLE	AGREE JUST A LITTLE	AGREE	AGREE VERY STRONGLY
Hoover-Dempsey, K.V., & Sandler, H.M. (2005). Final Performance Report for OERI Grant # R305T010673: The Social Context of Parental Involvement: A Path to Enhanced Achievement. Presented to Project Monitor, Institute of Education Sciences, U.S. Department of Education, March 22, 2005. a. It's my job to explain tough assignments to my child.	1 □	2 □	3 □	4□	5□	6 □
b. It's my job to make sure my child understands his or her assignments.	1 □	2 □	3 □	4□	5□	6 □
c. I make it my business to stay on top of things at school.	1 □	2 □	3 □	4□	5□	6 □
d. I assume my child is doing all right when I don't hear anything from the school.	1 □	2 □	3 □	4□	5□	6 □
e. The teacher has to let me know about a problem before I can do something about it.	1 □	2 □	3 □	4□	5□	6 □
f. I get most of my information about my child's progress from report cards.	1 □	2 □	3 □	4□	5□	6 □
g. My child's learning is mainly up to the teacher and my child.	1 □	2 □	3 □	4□	5□	6 □

Item-Specific Probes:

How did you answer this question?
Can you tell me, in your own words, what the phrase “I assume my child is doing all right when I don't hear anything from the school” is asking?
1. How did you answer this question?
What examples of your involvement in your child’s school came to mind when you heard the questions?
1. Why?
Do these type of things apply for your [6^th/7^th/8^th] grader or his/her school?
Did you have trouble picking a response from the six provided? (Follow-up probe if needed: What if you did not have a hard copy?)
For each item you selected ‘Disagree Very Strongly’ or ‘Disagree’, please tell me if the item simply did not apply to you and why.
Were your responses based on the way you think or act or did you consider the role of others in the household as well?
1. If “others”, who else did you include?
2. What led you to base your response on the discussions of multiple people?
Were there any words in the question that you found confusing? (PROBE AS NEEDED: Was the way the question was worded confusing? What would have made the question clearer?)

Now we would like to ask questions about your involvement in your child’s learning and education. Read through and select a response to statements in the list below. I will ask a series of follow-up questions once you are done.

C2. Indicate HOW OFTEN you have done the following during this school year.

	MARK ONE EACH ROW
	NEVER	ONCE SO FAR THIS YEAR	ABOUT ONCE A MONTH	ONCE EVERY TWO WEEKS	ONCE A WEEK	DAILY
a. I kept an eye on my child's progress.	1 □	2 □	3 □	4 □	5 □	6 □
b. I made sure that my child's homework got done.	1 □	2 □	3 □	4 □	5 □	6 □
c. I helped my child study for tests or quizzes.	1 □	2 □	3 □	4 □	5 □	6 □
d. I talked to my child about what he or she is learning.	1 □	2 □	3 □	4 □	5 □	6 □
e. I took my child to the library, community events, or similar places.	1 □	2 □	3 □	4 □	5 □	6 □
f. I expected the school to notify me if my child had a problem.	1 □	2 □	3 □	4 □	5 □	6 □
g. I expected my child to do his or her homework at school.	1 □	2 □	3 □	4 □	5 □	6 □
h. I relied on the teacher to make sure my child understands his or her assignments.	1 □	2 □	3 □	4 □	5 □	6 □

Item-Specific Probes:

When considering your answer to the questions, how far back did you consider? One month? Six months?
1. Would the question have been easier to answer if it asked about a different timeframe?
Can you tell me, in your own words, what the phrase “I kept an eye on my child's progress” is asking?
1. How did you answer this question?
Can you tell me, in your own words, what the phrase “I talked to my child about what he or she is learning” is asking?
1. How did you answer this question?
Can you tell me, in your own words, what the phrase “I expected the school to notify me if my child had a problem” is asking?
1. How did you answer this question?
Can you tell me, in your own words, what the phrase “I expected my child to do his or her homework at school” is asking?
1. How did you answer this question?
Can you tell me, in your own words, what the phrase “I relied on the teacher to make sure my child understands his or her assignments” is asking?
1. How did you answer this question?
What examples of your involvement in your child’s learning and education came to mind when you heard the questions?
1. Why?
Do these type of things apply for your [6^th/7^th/8^th] grader or his/her school?
Did you have trouble picking a response from the six provided? (Follow-up probe if needed: What if you did not have a hard copy?)
For each item you selected ‘Never’, please tell me if the item simply did not apply to you and why.
Were your responses based on your behavior only or did you consider the involvement of others in the household had with as well?
1. If “others”, who else did you include?
2. What led you to base your response on the discussions of multiple people?
Were there any words in the question that you found confusing? (PROBE AS NEEDED: Was the way the question was worded confusing? What would have made the question clearer?)

III. Debriefing and Thank You for Participation

Thank parent for his/her time.

Before we finish, I’d like to hear any other thoughts you have about the topics we just discussed.

Is there anything else you would like to tell me about on these topics or any others?

Is there anything you would like to ask me about what we did today? [Answer parent questions]

Thank you for helping us to improve our parent questionnaire.

MGLS:2017 Cognitive Interview Protocol:

Mathematics Teacher Focus Group Interview Guide

MGLS:2017 Cognitive Interview Protocol: Mathematics Teacher Focus Group Interview Guide

I. Interviewer Welcome Script (5 minutes)

The following scripts should not be read verbatim. You, as the interviewer, need to be familiar enough with the script to introduce the study and the focus group process in a conversational manner. Text in the box is suggested content for you to be thoroughly familiar with in advance. You should project a warm and reassuring manner toward the participants to develop a friendly rapport and should use conversational language throughout. Please note that any italicized text for item-specific probes should be read verbatim to ensure we cover these important topics and ask teachers these questions in the same way.

Hello, I am [NAME] from Mathematica Policy Research and we are working with the U.S. Department of Education.

Thank you all for agreeing to help us today. Decision Information Resources and Mathematica Policy Research are working with the National Center for Education Statistics, part of the U.S. Department of Education, to design a new national longitudinal study of how students learn and progress during sixth, seventh, and eighth grade. This study, the Middle Grades Longitudinal Study of 2016-17 (or MGLS:2017), will collect information from students, parents, teachers, and school administrators.

A focus of MGLS:2017 is the mathematics instruction that students receive as well as their mathematics achievement. You should have already received a packet of information from us in the mail that includes proposed survey and assessment items. Thank you for reviewing these items, and for taking the time to talk about them with us today.

Today we will be focusing on the content of the student mathematics assessment and questions we will be asking teachers about their mathematics classrooms and students. As we review the survey questions, please remember that there are no right or wrong answers. Your feedback will help us make these questions clearer for teachers who participate in the national study.

The next paragraph should be read verbatim. Keep the participant list nearby and mark each person as he or she gives consent. If you are missing consent from someone, please ask them directly (for example, “Jane, I don’t think I heard from you yet – is it okay with you if I record this interview?”).

We would like to record this session so we can listen to it later as we write up our notes. What you say is very important to us, and we don’t want to miss anything. Your information will be kept strictly private and will not be shared with anyone outside the study team. Your feedback may be used only for research purposes and may not be used for any other purpose except as required by law. Is it okay if I record our interview? Please give your name when you speak so I can confirm that this is okay with everyone on the call.

There are multiple participants on the call today. You could do us a great favor by being sure to speak clearly, and one at a time—this will help ensure that everyone gets a chance to present their perspective, and should minimize the number of times we need to ask people to repeat what they said. If you have a different perspective from the one being presented by another teacher, that is completely fine, and we are absolutely interested in hearing different sides of the issues we will be discussing today. Just remember, please allow your peers to finish making their point, and then you’ll have a chance to voice your perspective. I will lead the discussion. We have a number of topics to discuss during our time together, and we want to be sure to hear from everyone today. As a result, I may call on individuals at times and I also may need to interrupt you. If that happens, I apologize in advance. I want to be sure that we cover everything and hear from everyone in the time we have together today.

Do you have any questions before we start?

After answering any questions:

Good, let’s begin.

Let’s go around the call now and introduce ourselves. Please say your name, the grades and mathematics courses you teach, and the state you are calling from.

II. Mathematics Assessment (25 minutes)

Teachers will be sent the student mathematics assessment questions to review before the focus group. With these follow-up questions, our goal is to make sure the assessment items cover students’ learning in the four assessment domains (number systems, ratios and proportions, expressions and equations, and functions). We want to know what teachers think about the range in difficulty of the items so that we have items appropriate for sixth graders whose mathematics skills are below grade level, as well as eighth graders whose mathematics skills are above grade level.

All italicized text should be read aloud.

We will start with the student mathematics assessment items. To help with the discussion, please have the items we sent in front of you to reference as we talk about them

Do you have any major, overarching concerns to share at this time?
1. Are there any skills or concepts that are missing from the assessment but are critical to understanding student learning in the four selected domains: Number Systems; Ratio and Proportional Relationships; Expressions and Equations (and Algebra); and Functions?
2. Do the items adequately represent these domains?
3. We want to be sure to measure the progress of children from the spring of grade 6 to the spring of grade 8.
  1. For the 6^th grade teacher focus group: Does the difficulty level of the items allow us to capture the mathematics skills of 6^th graders who perform below grade level? What about students who perform well below grade level? Can you give me an example of an item that your students who are performing well below grade level would be able to answer correctly?
  2. For the 8^th grade teacher focus group: Does the difficulty level of the items allow us to capture the mathematics skills of 8^th graders who perform above grade level? What about students who perform well above grade level? Can you give me an example of a question that your students performing well above grade level might find challenging?
Is there anything you wish to discuss that you noted on your hard copy but we have not discussed?

III. Mathematics Teacher Questionnaire (20 minutes)

Teachers will be sent the questions and topics before the focus group. In most cases the moderator will read the question to them and then they will be asked several follow-up questions from the list of item-specific probes.

Thank you for that thoughtful discussion of the student assessment. We would like to move on and talk about the teacher questionnaire items we sent you. As with the assessment items, please have the teacher questionnaire in front of you as we talk about it now.

As noted on page X of the questionnaire, for most of the questions we will discuss, we want you to focus on a single math class. Please think of the same class you had in mind when reading through the questions we sent earlier. Why don’t we take a minute for everyone to tell us the name of that class and the grades represented in it (for example, “I am answering for my grade 7 pre-Algebra class.”). Please also tell the group whether your classes in your school are self-contained (that is, you teach multiple subjects to the same students most or all of the day) or departmentalized (you teach subject matter courses to several classes of different students throughout the day). If mathematics instruction in your school is departmentalized, please indicate whether the section of the class you are answering questions for is taught to students at a particular ability level—for example, a remedial course for students performing below grade level or an honors course for students performing above grade level in mathematics. Finally, if your school is departmentalized, please tell the group what kind of schedule your school uses – for example, all classes meet every day, a block schedule where classes might not meet every day, or a block schedule where students do not take every subject during every quarter or grading period.

Make sure each teacher states:

The name and grade level of the class
Whether instruction is in a self-contained or departmentalized classroom
- If departmentalized, whether the students in the class are in a particular “track” such as regular, remedial, or honors
- If departmentalized, what type of schedule

PROBE as necessary if teachers do not touch on the points above.

1. Content Covered in Each Course

With this item we want to determine the extent to which teachers cover mathematics topics central to the MGLS:2017 student mathematics assessment. In particular, we are interested in learning whether teachers feel they can easily estimate topic coverage over the entire school year, as well as whether the number of class periods is the most useful way of measuring the time spent on a topic. We also want to understand if teachers in schools with different kinds of schedules find the question and response options fitting.

I’d like to talk a little bit about question A1, which dealt with the content you cover in class. Because this question has so many topics, I won’t read them all aloud, but I did want to introduce the item as a way of beginning the discussion.

Start reading the question at bolded sentence starting “To what extent…” and list the response categories and domains (but not each item within a domain). Then proceed to item-specific probes.

SOURCE:
Bill Schmidt’s Course Teacher Content Goals (TCG)

A1. The purpose of this section is to obtain a description of the specific mathematic content areas you covered or plan to cover in your course this academic year.

Following is a list of content areas covering materials that may be taught in grades 6, 7 & 8. Please respond to the entire list so that we may obtain an indication of the topics covered in your class that is as complete and accurate as possible.

Before marking, please read quickly through the entire list to obtain an idea of where various content areas you've taught may be found. (Note: not all areas are necessarily appropriate for your class.)

For each listed content area, indicate the approximate number of class periods during which you taught the content area during this school year to your class.

To what extent have you or will you teach each of the following topics in your course during the 2013-2014 school year?

	NUMBER OF CLASS PERIODS TAUGHT
MARK ONE PER ROW	None	1 or Less	2-5	6-10	11-15	>15
Ratios and Proportional Relationships
a. Understand ratio concepts and use ratio reasoning to solve problems	0 □	1 □	2 □	3 □	4 □	5 □
b. Analyze proportional relationships and use them to solve real-world and mathematical problems	0 □	1 □	2 □	3 □	4 □	5 □
The Number System
c. Apply and extend previous understandings of multiplication and division to divide fractions by fractions	0 □	1 □	2 □	3 □	4 □	5 □
d. Compute fluently with multi-digit numbers and find common factors and multiples	0 □	1 □	2 □	3 □	4 □	5 □
e. Apply and extend previous understandings of numbers to the system of rational numbers	0 □	1 □	2 □	3 □	4 □	5 □
f. Apply and extend previous understandings of operations with fractions to add, subtract, multiply and divide rational numbers	0 □	1 □	2 □	3 □	4 □	5 □
g. Know that there are numbers that are not rational, and approximate them by rational numbers	0 □	1 □	2 □	3 □	4 □	5 □
Functions
h. Define, evaluate and compare functions	0 □	1 □	2 □	3 □	4 □	5 □
i. Use functions to model relationships between quantities	0 □	1 □	2 □	3 □	4 □	5 □
Expressions and Equations
j. Apply and extend previous understandings of arithmetic to algebraic expressions	0 □	1 □	2 □	3 □	4 □	5 □
k. Reason about and solve one-variable equations and inequalities	0 □	1 □	2 □	3 □	4 □	5 □
l. Represent and analyze quantitative relationships between dependent and independent variables	0 □	1 □	2 □	3 □	4 □	5 □
m. Use properties of operations to generate equivalent expressions	0 □	1 □	2 □	3 □	4 □	5 □
n. Solve real-life and mathematical problems using numerical and algebraic expressions and equations	0 □	1 □	2 □	3 □	4 □	5 □
o. Work with radicals and integer exponents	0 □	1 □	2 □	3 □	4 □	5 □
p. Understand the connections between proportional relationships, lines, and linear equations	0 □	1 □	2 □	3 □	4 □	5 □
q. Analyze and solve linear equations and pairs of simultaneous linear equations	0 □	1 □	2 □	3 □	4 □	5 □

All italicized text should be read aloud.

Item-Specific Probes:

Did you have any difficulty estimating the number of class periods devoted to any of these topics?
Did you need to consult past or future lesson plans to answer this question?
When estimating the number of class periods, how did you define a class period that covered a given content area?
1. PROBE IF NEEDED: For example, did you only consider class periods where a certain proportion of the time was devoted to the content area? How did you count class periods where you covered more than one content area?
Do the response categories make it easy or difficult to answer this question (If difficult, why?)
Would it be easier to answer this question by indicating the percent of the school year spent on each topic? The number of weeks? Some other way? Keep in mind that teachers in schools with varying schedules will all be asked the same question.
For how many content areas did you pick more than 15 periods? (Follow-up probe as needed: Which content areas?)
For those content areas where you specified more than 15 class periods, what is the highest number of class periods you taught an area?
Was any of the language used in these items confusing? What would have made the language clearer?
Was anything else about this task confusing? What would have made the task clearer?

2. Curriculum

The set of commonly used mathematics textbooks is fairly limited. The probes for this item helps us understand how teachers use textbooks, and helps to ensure we are not leaving any commonly used textbooks out of our response options.

Let’s talk about question A2, which asked about the textbook you use most in this class.

Do not read item (it is included here for interviewer reference).

SOURCE: TCG Modified response categories A2. Which one of the following textbooks do you use the most in your class as your primary source of instruction? If the textbook you mark has an option to indicate grade level, please circle the grade as indicated below. What additional textbooks do you use to supplement your instruction?
	PRIMARY TEXTBOOK	SUPPLEMENTARY TEXTBOOKS MARK ALL THAT APPLY		GRADE LEVEL (ITEMS M, N, P, AND Q ONLY
	MARK ONE ONLY	SUPPLEMENTARY TEXTBOOKS MARK ALL THAT APPLY	PUBLICATION YEAR	MARK ONE ONLY
a. Holt Mathematics Course 1; Holt, Rinehart & Winston	1 □	1 □	\| \| \| \| \|
b. Holt Mathematics Course 2; Holt, Rinehart & Winston	2 □	2 □	\| \| \| \| \|
c. Holt Mathematics Course 3; Holt, Rinehart & Winston	3 □	3 □	\| \| \| \| \|
d. Holt Pre-Algebra; Holt, McDougal	4 □	4 □	\| \| \| \| \|
e. Holt Algebra I; Holt, McDougal	5 □	5 □	\| \| \| \| \|
f. Saxon Math	6 □	6 □	\| \| \| \| \|
g. Algebra 1; McDougal Littell/Houghton Mifflin	7 □	7 □	\| \| \| \| \|
h. Algebra; Prentice Hall	8 □	8 □	\| \| \| \| \|
i. Algebra 1; Glencoe/McGraw‐Hill	9 □	9 □	\| \| \| \| \|
j. Contemporary Mathematics in Context; Glencoe/McGraw‐Hill	10 □	10 □	\| \| \| \| \|
k. Integrated Mathematics; McDougal Littell/Houghton Mifflin	11 □	11 □	\| \| \| \| \|
l. Everyday Mathematics (UCSMP); Everyday Learning	12 □	12 □	\| \| \| \| \|
m. Math; Harcourt, Brace, Jovanovich	13 □	13 □	\| \| \| \| \|	5 □ 6 □ 7 □ 8 □ 9 □
n. Math; Scott Foresman	14 □	14 □	\| \| \| \| \|	5 □ 6 □ 7 □ 8 □ 9 □
o. Trailblazers; Kendall Hunt	15 □	15 □	\| \| \| \| \|
p. Math Investigations; Pearson/TERC	16 □	16 □	\| \| \| \| \|	5 □ 6 □ 7 □ 8 □ 9 □
q. Other, specify:	17 □	17 □	\| \| \| \| \|	5 □ 6 □ 7 □ 8 □ 9 □
r. I do not use a textbook in teaching this class	18 □

All italicized text should be read aloud.

Item-Specific Probes:

What was your answer to this question?
How easy or difficult was it for you to indicate the textbook you use most?
If you use only one textbook, do you follow it through in order? Do you cover all content by the end of the year?
If you use multiple textbooks, how do you decide which source to use for which topics or lessons?
Are there any other textbooks that you considered using or that you know other teachers use in your school that are not listed here that should be included?

3. Factors for Determining Math Course

This question is designed to elicit information about how students are placed or assigned into different mathematics courses. The probes help us gain insight into how familiar mathematics teachers are with this process, and whether the factors considered vary by grade.

Now we would like to talk about how students are assigned to mathematics classes. Question A3 asked:

SURVEY: HSLS 09 School Counselor Modified response options A3. How important is each of the following factors in placing a student into a mathematics course for the grade you teach? Please indicate whether EACH is not at all important, a little important, somewhat important, or very important.
	MARK ONE PER ROW
	NOT AT ALL IMPORTANT	A LITTLE IMPORTANT	SOMEWHAT IMPORTANT	VERY IMPORTANT
a. Counselor recommendation	1 □	2 □	3 □	4 □
b. Prior teacher recommendation	1 □	2 □	3 □	4 □
c. Courses taken previously	1 □	2 □	3 □	4 □
d. Achievement in previous courses	1 □	2 □	3 □	4 □
e. Results of a district or state end-of-year or end-of-course exams	1 □	2 □	3 □	4 □
f. Results of placement tests	1 □	2 □	3 □	4 □
g. Results of standardized tests	1 □	2 □	3 □	4 □
h. Student career or education plan	1 □	2 □	3 □	4 □
i. Student and/or parent or guardian selection	1 □	2 □	3 □	4 □

All italicized text should be read aloud.

Item-Specific Probes:

Did you already know the information necessary to answer this question? If no, how did you determine your answer? Who would be the best person to ask this question?
Are the response options listed appropriate? Are there any response options that were never used?
Are there other factors that are not listed? (If yes, please tell me what other factors)
Do all of these factors apply to grade [6/7/8]?
Would your response be different for the factors determining math placement for other grades in your school? If so, how?
Were any of the factors listed here not applicable to you? If so, which ones?

4. Use of Technology

Schools use a variety of technologies to support student learning. The probes for this question are meant to assess whether the question covers the full range of resources (such as tablets and Smartboards) and how they are used (for practice or to do research, for example). They serve as a starting point for a broader discussion of how teachers use technology.

Now we would like to talk about the technological resources you use with your students. Recall question A4 asked:

SURVEY: NAEP Teacher Modified to add “tablet”

A4. In your [FILL GRADE] mathematics class this year, how often do your students use a computer or other technological resources, such as a tablet or Smartboard, to do each of the following? Fill in one box on each line.

	MARK ONE PER ROW
	NEVER OR HARDLY EVER	ONCE OR TWICE A MONTH	ONCE OR TWICE A WEEK	EVERY DAY OR ALMOST EVERY DAY
a. Practice or review mathematics topics on the computer	1 □	2 □	3 □	4 □
b. Extend mathematics learning with enrichment activities	1 □	2 □	3 □	4 □
c. Research a mathematics topic on the Internet or CD-ROM	1 □	2 □	3 □	4 □
d. Use a drawing program for geometric shapes	1 □	2 □	3 □	4 □
e. Use a graphing program	1 □	2 □	3 □	4 □
f. Play mathematics computer games	1 □	2 □	3 □	4 □

All italicized text should be read aloud.

Item-Specific Probes:

The question lists computers, tablets, and Smartboards as resources you might use with your students. Are there other technological resources that you use that we did not list?
Are there other ways you use technology that we did not list? In terms of what you do with technology or how you incorporate it into your class?
If you selected “never or hardly ever,” why?

IV. Mathematics Teacher Student Report (5 minutes)

In this section, we present a set of items to be completed for a specific student in a teacher’s mathematics class. We want to understand students’ mathematics skills and behavior from their teachers’ perspectives. A main goal is to confirm mathematics teachers (who may only see a student one period) can adequately answer student-specific questions.

For the following question, please answer for the same student for which you answered questions on the hard copy we sent.

Math classroom performance

This question is meant to capture information about students’ mathematics skills that are difficult to assess in a one-time direct assessment. The probes aim to ascertain whether teachers think we are asking about the skills they believe are necessary to measure when assessing a student’s mathematics abilities.

Question B1 asked:

B1. Please rate this student’s skills in the following areas, as exhibited in your class. Fill in one box on each line.

	OUTSTANDING	VERY GOOD	GOOD	FAIR	POOR	NOT APPLICABLE/ NOT OBSERVED
a. Ability to apply mathematical concepts to “real world” problems	1 □	2 □	3 □	4 □	5 □	6 □
b. Ability to complete or conduct proofs or demonstrations of his/her mathematical reasoning	1 □	2 □	3 □	4 □	5 □	6 □
c. Ability to talk about his/her reasoning or thinking in solving a problem	1 □	2 □	3 □	4 □	5 □	6 □
d. Ability to explain his/her reasoning in solving a problem in writing	1 □	2 □	3 □	4 □	5 □	6 □
e. Ability to use representations to model mathematical ideas	1 □	2 □	3 □	4 □	5 □	6 □
f. Ability to use a calculator to solve problems	1 □	2 □	3 □	4 □	5 □	6 □
g. Ability to use a graphing calculator to complete mathematics assignments	1 □	2 □	3 □	4 □	5 □	6 □

All italicized text should be read aloud.

Item-Specific Probes:

Do the skills listed address the things you monitor to assess students’ math performance?
What other skills do you monitor or assess?
If you chose not applicable/not observable, what skill was that? Why? What would make it applicable or observable?
Are there any skills listed here that are less important for understanding a student’s mathematics skills? If so, which ones?

V. Debriefing and Thank You for Participation (5 minutes)

Thank teachers for their time.

Before we finish, I’d like to hear any other thoughts you have about the assessment and teacher questionnaire materials you have been reviewing.

Is there anything else you would like to tell me about your experience answering the questions?

Is there anything you would like to ask me about regarding what we did today? [Answer questions].

We look forward to receiving your marked-up copies in the mail.

Thank you so much for your time.

MGLS:2017 COGNITIVE INTERVIEW PROTOCOL:

SCHOOL ADMINISTRATOR QUESTIONNAIRE

MGLS:2017 Cognitive Interview Protocol:

School Administrator Questionnaire

Interviewer Welcome Script

The following script should be read verbatim. Text in the box is suggested content for you to be thoroughly familiar with in advance. You should project a warm and reassuring manner toward the participant to develop a friendly rapport and as such should use conversational language throughout. Please note that any italicized text for item-specific probes should be read verbatim to ensure we cover these important topics and ask administrators these questions in the same way.

Hello, my name is [NAME]. I’m from Decision Information Resources, Inc. and we are working with the U.S. Department of Education.

Thank you for agreeing to help us today. Decision Information Resources and Mathematica Policy Research are working with the National Center for Education Statistics, part of the U.S. Department of Education, to design a new national longitudinal study of how students learn and progress throughout the middle grades. This study, the Middle Grades Longitudinal Study of 2016-17 (or MGLS:2017), will collect information from students, parents, teachers, and school administrators.

You should have received a brief letter in the mail that included a few questions for you to review and complete. Today, I will ask you some follow-up questions about those topics. This should only take about 30 minutes. These are questions that administrators across the country will be asked to answer about school factors that may influence student development, motivation, and mathematics learning. Some of the questions we talk about today may not be easy to answer, but there are no right or wrong answers, so please answer the questions the best you can. Your feedback will help us make these questions clearer for administrators who participate in the national study.

We would like to record this interview so members of our research team can review the recordings later. No one else will hear the recording. What you say is very important to us, and we don’t want to miss anything. Your information will be kept strictly private and will not be shared with anyone outside the research team, including your school staff, district personnel, or parents. Your feedback may be used only for research purposes and may not be used for any other purpose except as required by law. Is it okay if I record our interview?

Do you have any questions before we get started?

If, for any reason, the administrator is no longer interested in participating, thank the administrator for his/her time and end the interview. After answering questions and giving any necessary further explanation, begin the interview with the first item.

Survey Topics, Items, and Probes

Administrators will be sent the questions and topics before the interview. Whether they complete them beforehand or not, the interviewer will read the question to them and then they will be asked several follow-up questions.

All italicized text should be read verbatim to the administrator.

We are interested in asking school administrators a series of questions about various aspects of their school, such as school services and supports, course offerings, and placement strategies. We need your help in developing questions that maximize our time with administrators, determine how best to contact and follow-up with them, and ensure that the topics we plan to cover are best suited to be answered by school administrators.

School Climate and Structure

First, we have several questions about the school climate and structure that cover a variety of topics. Many of the topics I am going to read to you could be addressed by an administrator, teachers, or other staff. What I’d like to know is if you are the best person at your school to talk to about these topics. If you are not the best person, please tell me who would be the best person to ask about the topic.

A1. School safety and security measures, such as presence of officer, metal detectors, buzzer system

A2. Organization, such as rules and expectations

A3. Supportive relationships for students with peers and adults, for example, teachers support students and students are supported by their peers

A4. Policies addressing absenteeism

A5. Student mobility

A6. Policies addressing student misconduct and bullying

A7. Number of teachers (full and part time)

A8. Years of teacher experience (average across school’s teachers)

School Services and Supports

We are also interested in the services and supports schools provide to students. Just like the previous question, I will read a list of topics to you and I’d like to know if you are the best person to talk to about these topics. If you are not the best person, please tell me who would be the best person to ask about the topic.

B1. Availability of afterschool/extended learning opportunities

B2. Within-school extended class time

B3. Level of rigor for mathematics courses

B4. Specific course offerings in mathematics and their enrollment

PROBE: Do you have a mathematics department chair? Would that person be better suited to answer questions about the mathematics course offerings?

Reaching School Administrators

C1. The school administrator survey will be administered as a web survey. We are planning to send an invitation to participate in the web survey directly to school administrators through their email. Is an invitation sent directly to your email address the best way to get in contact with you?

PROBE: If we do not get a response to the email invitation, what would be the best way to follow up? For example, calling and leaving a direct voicemail? Going through the school secretary? Sending a hard copy of the survey via regular mail? Can you think of other alternative ways to contact school administrators?

Assisting students with transitions

Above, we were interested in the best person to ask about specific topics on the School Administrator Questionnaire. For this part of the cognitive lab, we are interested in improving our questions. Specifically, in this part of the cognitive lab, we are interested in the types and levels of assistance schools make available to students transitioning from elementary to middle school and middle to high school in general (Table D1. and D3.) and specifically those made available by school counselors (Table D2. And D4.). Interviewers will read a list of types of assistance followed by probes designed to gather more detailed information about those types of assistance. We are also interested to know if there is a difference in the type of assistance provided to students shifting grades within a school or building (e.g., moving from grade 6 to grades 7 and 8 in K-8 school) versus students transitioning to a different physical location (e.g., from a K-5 school to a 6-8 school).

Thank you for helping us by providing information on who we might contact for certain pieces of information and how we might best contact school administrators.

Now we would like your help in making sure we are asking certain questions clearly and capturing the appropriate information. Specifically, we would like to ask you about the types of assistance made available to students transitioning to and from your school and get your feedback on the clarity and construction of the questions You received these questions in your advance-packet and I will read them aloud, now. We plan to ask about what the school does in general and what counselors do specifically.

D1. Does your school organize the transition from the elementary grades to the middle grades in the following ways?

MODIFIED SURVEY: NELS88 Enhancement Survey of Middle Grades Practices	MARK ALL THAT APPLY
No transition—middle grades seamlessly continue directly from elementary grades	1 □
No special activities until students enter middle school	1 □
Middle grades students share information with the elementary grades students	1 □
Elementary grades students visit the middle grades students’ assembly	1 □
Elementary grades students attend regular middle grades courses	1 □
Buddy or big brother/big sister programs that pairs new students with an older student in the fall	1 □
Parents visit the school or middle grades section while students are still in elementary grades	1 □
Parents can attend an orientation in the fall after students start middle grades	1 □
Meetings for elementary grade students during the summer prior to beginning the middle grades	1 □
Middle grades counselors meet with students while they are still in elementary grades	1 □
Middle grades and elementary grades teachers meet together on courses and requirements	1 □
Middle grades and elementary grades administrators meet together on articulation and programs	1 □
Elementary grades counselors meet with middle grades counselors or staff	1 □
Other (describe)	1 □

Item-Specific Probes:

Upon reading these questions, would you say they reflect the types of activities your school does?
1. Are any of the questions irrelevant for your school? Would phrasing the question differently make it more relevant?
If meetings were held during the summer for elementary grade students prior to beginning the middle grades, what did they entail? Who did the students meet with?
[IF SCHOOL CONFIGURATION IS K-8 OR SIMILAR WITH ELEMENTARY THROUGH MIDDLE GRADES IN SAME BUILDING] Does the wording accurately capture the transition activities that occur within the school (as opposed to when students may physically change buildings at these grades)?
What activities might we add to the list for how your school organizes the transition to middle grades?

MODIFIED SURVEY: High School Longitudinal Study of 2009

D2. Do counselors at your school assist students in the transition from elementary grades to middle grades in the following ways?

	MARK ALL THAT APPLY
Elementary grades counselors meet with middle grades counselors or staff	1 □
Elementary grades counselors meet with individual elementary grades students and assist them with selecting middle grades courses while they are still in elementary grades	1 □
Elementary grades counselors present information to elementary grades students' parents or guardians about middle grades courses and registration	1 □
Elementary grades counselors place elementary grades students into middle grades courses based on school or district placement policies	1 □
Elementary grades counselors present information to elementary grades students about middle school courses and registration	1 □
Counselors assist in some other way	1 □
Counselors do not assist students in the transition from elementary grades to middle grades.	1 □

Item-Specific Probes:

Upon reading these questions, would you say the questions are focused on the kinds of assistance counselors at your school offer?
1. Are any of the questions irrelevant for your school? Would phrasing the question differently make it more relevant?
Is “counselors” the most appropriate term for this person at your school? Would another term be better?
What activities might we add to the list for how your school organizes the transition to the middle grades?

D3. Does your school organize the transition from the middle grades to the high school grades in the following ways?

MODIFIED SURVEY: NELS88 Enhancement Survey of Middle Grades Practices	MARK ALL THAT APPLY
No transition—high school grades seamlessly continue directly from middle grades	1 □
No special activities until students enter high school	1 □
High school grades students share information with the middle grades students	1 □
Middle grades students visit the high school grades students’ assembly	1 □
Middle grades students attend regular high school grades courses	1 □
Buddy or big brother/big sister programs that pairs new students with an older student in the fall	1 □
Parents visit the high school grades section while students are still in middle grades	1 □
Parents can attend an orientation in the fall after students start high school grades	1 □
Meetings for middle grades students during the summer prior to beginning the high school grades	1 □
High school grades counselors meet with students while they are still in middle grades	1 □
High school grades and middle grades teachers meet together on courses and requirements	1 □
High school grades and middle grades administrators meet together on articulation and programs	1 □
Middle grades counselors meet with high school grades counselors or staff	1 □
Other (describe)	1 □

Item-Specific Probes:

Upon reading these questions, would you say they reflect the types of activities your school does?
1. Are any of the questions irrelevant for your school? Would phrasing the question differently make it more relevant?
If meetings were held during the summer for middle grades students prior to beginning the high school grades, what did they entail? Who did the students meet with?
[IF SCHOOL CONFIGURATION IS K-12 OR SIMILAR WITH MIDDLE THROUGH HIGH SCHOOL GRADES IN SAME BUILDING] Does the wording accurately capture the transition activities that occur within the school (as opposed to when students may physically change buildings at these grades)?
What activities might we add to the list for how your school organizes the transition to high school grades?

MODIFIED SURVEY: High School Longitudinal Study of 2009

D4. Do counselors at your school assist students in the transition from the middle grades to the high school grades in the following ways?

	MARK ALL THAT APPLY
Middle grades counselors meet with high school grades counselors or staff	1 □
Middle grades counselors meet with individual middle grades students and assist them with selecting high school grades courses while they are still in middle grades	1 □
Middle grades counselors present information to middle grades students' parents or guardians about high school grades courses and registration	1 □
Middle grades counselors place middle grades students into high school grades courses based on school or district placement policies	1 □
Middle grades counselors present information to middle grades students about high school grades courses and registration	1 □
Counselors assist in some other way	1 □
Counselors do not assist students in the transition from middle grades to high school grades.	1 □

Item-Specific Probes:

Upon reading these questions, would you say the questions are focused on the kinds of assistance counselors at your school offer?
1. Are any of the questions irrelevant for your school? Would phrasing the question differently make it more relevant?
Is “counselors” the most appropriate term for this person at your school? Would another term be better?
What activities might we add to the list for how your school organizes the transition to the high school grades?

III. Debriefing and Thank You for Participation

Thank administrator for his/her time.

Before we finish, I’d like to hear any other thoughts you have about the topics we just discussed.

Is there anything else you would like to tell me about on these topics or any others? [Answer questions]

Thank you for helping us to improve our questionnaire.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	2021-01-28