National Assessment of Educational Progress (NAEP) Pretesting of Survey and Cognitive Items for Pilot in 2017 and 2018 Updated Revised

Good <MORNING/AFTERNOON/EVENING>. Is this the <NAME FROM CALL LIST> household? My name is <NAME>, and I am calling from EurekaFacts. We are a local research company administering a study on behalf of the U.S. Department of Education’s National Center for Education Statistics, also known as NCES. We are recruiting students currently in eighth grade to sign up to participate in a research study to help in the development of new survey questions for the National Assessment of Educational Progress, also known as the NAEP assessment. [If parent/legal guardian asks what this is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

Are there any eighth grade students in your household?

Yes (Great! Let me give you some additional information about the study.)

No (Thank you for your time. Have a good <DAY/EVENING>.)

The survey will be administered to students throughout the United States. We are conducting research interviews to explore how students work through sample survey questions to ensure that they are clear and understandable to a broad range of students. Your child will receive a $25 gift card (from a major credit card company) for participating in the research interview. You will also receive a $25 gift card to thank you for your time in bringing your child to and from the interview site.

The research interview will take place at <LOCATION>. It can be scheduled at your convenience and will last no more than 90 minutes. During the interview, your child will be asked to respond verbally to sample survey questions. NCES simply wants to know how he or she would answer the questions if they were included on an actual survey given to students after taking a NAEP test.

Students will be assigned a unique student identifier (ID), and at no time will their names be linked to any of their answers. Your child’s responses will be used only for research purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573]. (INTERVIEWER: This reference is for informational purposes only. If asked what law is being referenced, please provide the information contained within this bracketed text.)

These research interviews are an important stage of the NAEP survey development process. By sharing their feedback with us, students can directly impact the future of this nationwide assessment by helping NCES evaluate the survey questions and instructions to create the best survey possible.

Do you believe your child would be interested in participating?

Yes

No (Thank you for your time. Have a good <DAY/EVENING>.)

Great! To ensure that we interview a broad mix of students, I have a few brief screening questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to identify if (HE/SHE) meets the participation requirements.

Skip to “Screener”.

Script when calling parent/legal guardian who responded to e-mail, advertisement, or invitation.

Good <MORNING/AFTERNOON/EVENING>. My name is <NAME>. I’m calling from EurekaFacts on behalf of the U.S. Department of Education’s National Center for Education Statistics, also known as NCES. Thank you for contacting us about the possibility of your child participating in a research interview to help develop survey questions for the National Assessment of Educational Progress, also known as the NAEP assessment. To ensure that we interview a broad mix of students, I have a few brief screening questions to ask about you and your child. It should only take a few minutes of your time. I want to collect some additional information about your child to identify if (HE/SHE) meets the participation requirements.

Before proceeding, please confirm that you are still interested in the possibility of your child participating in this research interview.

Yes (INTERVIEWER: Proceed to “Screener”.)

No (Thank you for your time. Have a good <DAY/EVENING>.)

Screener:

(INTERVIEWER: Determine recruitment needs in advance. Record all responses. End screener at whatever point the parent or legal guardian gives a response that confirms that his/her group has already been sampled adequately. End screener script: Based on the requirements of this study, we are not able to include your child in the interviews at this time. Thank you for your time. Have a good <DAY/EVENING>.)

What is your child’s name?

What is your child’s gender?

Male
Female

What grade is your child currently in? (INTERVIEWER: Do not read this list.)

 Grades K-7  (Unfortunately, we are only interviewing students currently in grade 8 for this study. Thank you for your time. Have a good <DAY/EVENING>.)

 Grade 8

 Grades 9-12 (Unfortunately, we are only interviewing students currently in grade 8 for this study. Thank you for your time. Have a good <DAY/EVENING>.)

What school does your child attend? (INTERVIEWER: Note if homeschool.)

____________________________

What school district does HE/SHE attend? (INTERVIEWER: Note if parochial, charter, or homeschool.)

____________________________

What is your child’s race or ethnicity? (INTERVIEWER: Only read list if parent/legal guardian doesn’t immediately reply.) Is your child…?

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

 American Indian or Alaska Native

 Asian

 Black or African American

 Native Hawaiian or other Pacific Islander

 White or Caucasian

Just to be sure we're interviewing a broad range of students for this study, does your child participate in the National School Lunch program? [If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine.” And move on to the next question.]

 Yes

 No

(INTERVIEWER: If the participant indicates a preference to not disclose this information, say: “That’s fine”, and move on to the next question.)

Would you consider the area you live in to be …

Urban
Suburban
Rural

What is the highest level of education that you have completed? (INTERVIEWER: Do not read this list.)

Less than high school
High school degree or GED
Some college
Trade school degree
Associate’s or Bachelor’s degree
Post graduate degree

Thank you for answering these questions. Your child is eligible to participate in the NAEP research interviews and thank you for allowing <HIM/HER> to do so.

Please note that you will be required to sign a consent form to confirm that you have agreed to allow your child to participate in the interview. What e-mail or mailing address should we send the consent form to? (INTERVIEWER: Record address.)

Please bring the signed consent form to the interview; if you forgot to bring the consent form one can be provided to you at the interview site. Thank you so much for talking with us today. We look forward to having your child participate in the interview.

IF PARENT/LEGAL GUARDIAN AVAILABLE AND CURRENTLY SCHEDULING INTERVIEWS

We will be holding our current interviews from <DATE> to <DATE>. What would be the best date and time to schedule the interview?

Date ____________________

Time ____________________

Thank you again for talking with us today and for allowing your child to participate in NAEP research interviews. Have a good <DAY/EVENING>.

IF PARENT/LEGAL GUARDIAN AVAILABLE AND NOT CURRENTLY SCHEDULING INTERVIEWS

We are in between interview sessions, but will be scheduling sessions until <DATE>. We will need to call you back when we are actively scheduling the interviews to determine the date and time that will work best, but what are your scheduling preferences? (INTERVIEWER: Select all that apply.)

 Morning

 Afternoon

 After school

 Weekends

Thank you again for talking with us today and for allowing your child to participate in NAEP research interviews. We look forward to speaking with you soon to schedule your child’s interview. Have a good <DAY/EVENING>.

Appendix BX: Teacher and Principal Recruitment Screener

Good <MORNING/AFTERNOON/EVENING>. May I please speak to <NAME>?

Good <MORNING/AFTERNOON/EVENING>. My name is <CONTACT NAME> and I am contacting you from EurekaFacts, a local research company. We are administering an important study for the U.S. Department of Education’s National Center for Education Statistics, also known as NCES.

The National Assessment of Educational Progress, also known as NAEP, has developed survey questions for teachers and principals. After this call, you can find more information about the NAEP assessment online at: http://nces.ed.gov/nationsreportcard/.

NCES needs to know if the survey questions are clear. Your input will help NCES improve the survey so educators like you understand all the questions. During the interview, you will receive a printed copy of multiple-choice questions that you will be asked to answer verbally. The interviewer will ask open-ended questions to follow up on why you selected each answer.

The voluntary research interviews will take place at <LOCATION> and will last no more than 90 minutes. A limited number of interviews may be conducted via phone or WebEx. Your responses will be used only for research purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573]. (INTERVIEWER: This reference is for informational purposes only. If asked what law is being referenced, please provide the information contained within this bracketed text.)

These interviews are an important stage of the NAEP survey development process. Your comments, along with the comments of other teachers and principals, will help NCES improve the survey. You will receive a $40 gift card (from a major credit card company) at the conclusion of the session for your time and effort.

Are you interested in participating in this research study?

If no: Thank you for your time. Have a good <DAY/EVENING>.

If yes: Great, thank you very much. To ensure that we interview a broad mix of educators, I have a few brief screening questions to ask you. It should only take a few minutes of your time.

Screener:

(INTERVIEWER: Determine recruitment needs in advance. Record all responses. End screener at whatever point the teacher or principal gives a response that confirms that their group has already been sampled adequately. End screener script: Based on the requirements of this study, we are not able to include you in the interviews at this time. We greatly appreciate your help. Thank you for your time. Have a good <DAY/EVENING>.)

What is your job title?

Principal (GO TO Part A)
Teacher (GO TO Part B)

Part A.

Are you a U.S. citizen or U.S. permanent resident? (If the principal indicates that he/she is not a U.S. citizen or U.S. permanent resident, thank and terminate: Unfortunately you are not eligible to participate in this study. Thank you for your time. Have a good <DAY/EVENING>.)

At what school are you a principal?

Are you at a school with a population that includes:

Eighth-grade students
Other END SCREENER (Based on the requirements of this study, we are not able to include you in the interviews at this time. We greatly appreciate your help. Thank you for your time. Have a good <DAY/EVENING>.)

How many students are enrolled in your school?
1. 449 students or less
2. 450-699 students
3. 700 students or more

Part B.

Are you a U.S. citizen or U.S. permanent resident? (If the teacher indicates that he/she is not a U.S. citizen or U.S. permanent resident, thank and terminate: Unfortunately you are not eligible to participate in this study. Thank you for your time. Have a good <DAY/EVENING>.)

At what school are you a teacher?

What grade level do you teach?
1. Eighth grade
2. Other END SCREENER (Based on the requirements of this study, we are not able to include you in the interviews at this time. We greatly appreciate your help. Thank you for your time. Have a good <DAY/EVENING>.)

Do you currently (or within this school year) teach civics, geography, or U.S history to eighth graders?
1. Yes  Which classes do you teach? (INTERVIEWER: Record class names.)
2. No END SCREENER (Based on the requirements of this study, we are not able to include you in the interviews at this time. We greatly appreciate your help. Thank you for your time. Have a good <DAY/EVENING>.)

What is the size of the student population at your school?
1. 449 students or less
2. 450-699 students
3. 700 students or more

Eligible Participant Script

Thank you for taking the time to answer the screening questions. You are eligible to participate in the voluntary research interviews. Again, the interview will last no more than 90 minutes and will be held at <LOCATION>. Are you available for the interview at <TIME> on <DATE>?

(INTERVIEWER: If necessary, provide alternate times/dates to find an available time/date for interview and then record time/date of appointment.)

Please note that before your interview starts, you will be asked to sign a consent form to confirm that you have agreed to participate in the interview. You will not be able to participate until the form has been signed.

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time?

(INTERVIEWER: Answer any questions.)

If you do have any questions before your interview scheduled for <DATE/TIME>, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this very important research study.

Appendix BY: Parent or Legal Guardian of Student (under age 18) Participant Consent Form

PARENTAL CONSENT DOCUMENT

FOR PARTICIPATION IN RESEARCH

EurekaFacts

Title of Research Study: Cognitive Laboratory Interviews for NAEP Survey Question Testing

Thank you for agreeing to your child’s participation in the research study described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them if they occur to you later.

What is this study about?

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, EurekaFacts is administering interviews with students currently in eighth grade about new survey questions for the National Assessment of Educational Progress (NAEP). The survey questions will be part of a new digitally-based test given to a sample of students across the country. Your child’s feedback will help NCES improve the survey so students like him or her understand all the questions. (More information about the NAEP assessment is available online at: http://nces.ed.gov/nationsreportcard/)

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION> at an after-school or weekend session.

What will happen during the research interview?

During the interview, your child will receive a printed copy of survey questions that he or she will be asked to answer verbally. The interviewer will ask open-ended questions to follow up on why your child selected each answer. Questions will focus on your child’s understanding of the questions. Your child will also be asked whether he or she has any suggestions on how we might improve the questions. The purpose of these interviews is not to test or grade students. NCES wants to hear their thoughts to improve the survey questions and make the questions easier to understand.

Will you keep information private?

Your child will be assigned a unique student identifier (ID), and at no time will his or her name be linked to any of his or her answers. Your child’s responses will be used only for research purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573]. The sessions will be audio recorded and the recordings will be used only by the researchers conducting the study. No one else will hear the recording.

How long will the research interview last and when will it take place?

The interview will last no more than 90 minutes and requires only one visit to our office. Sessions will take place at a time and date convenient to the participants’ parent(s) or legal guardian(s). <TBD: note days of the week when sessions are scheduled and the window of time when the sessions will be held>.

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his/her school, and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this voluntary study.

Who can I contact with questions or for further information?

<CONTACT NAME> is administering this study at <EUREKAFACTS> on behalf of NCES. If you have any questions, please call <HIM/HER> at <CONTACT PHONE> or by e-mail <CONTACT E-MAIL>.

Can participation end early?

Participation in this study is completely voluntary. Your child has the right to refuse to answer particular questions. Your child may elect to withdraw from this study at any time and can end the session early if he or she wishes.

By signing below, you agree that your child, _______________________________, may participate in this research study and that you have read and understood the information provided above. Your child will receive a $25 gift card (from a major credit card company) at the conclusion of the session for his or her time and effort. If you provide transportation (whether via car, train, bus, or some other transportation) to the location of the interview for your child to participate in the cognitive interview session, you, the parent/legal guardian, will receive a $25 gift card as well. In addition, you will receive a thank you letter/e-mail for allowing your child to participate in the study. We hope that you will give your consent for your son or daughter to participate in the voluntary NAEP interview by signing this form. Without your consent, your child will not be able to participate in the interview.

Print Name: ________________________________________

Signature:______________________________________________Date:________________________

E-mail address: _____________________________________________

Internal use only:

Payment Card Transaction Number (Student): __________________________

Payment Card Transaction Number (Parent): __________________________

Appendix BZ: Teacher and Principal Consent Form

PARTICIPATION CONSENT DOCUMENT

FOR PARTICIPATION IN RESEARCH

EurekaFacts

Title of Research Study: Cognitive Laboratory Interviews for NAEP Survey Question Testing

Thank you for your interest in participating in the research study described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them if they occur to you later.

What is this study about?

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, EurekaFacts is administering interviews with teachers and principals in schools with students currently in eighth grade about new survey questions for the National Assessment of Educational Progress (NAEP). The survey questions will be part of a new Technology-based assessment given to a sample of students across the country. Your feedback will help NCES improve the survey so educators like you understand all the questions. (More information about the NAEP assessment is available online at: http://nces.ed.gov/nationsreportcard/)

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION> at an after-school or weekend session. A limited number of interviews may be conducted via phone or WebEx.

What will happen during the research interview?

During the interview, you will receive a printed copy of survey questions that you will be asked to answer verbally. The interviewer will ask open-ended questions to follow up on why you selected each answer. Questions will focus on your understanding of the questions. You will also be asked whether you have any suggestions on how we might improve the questions. NCES wants to hear your thoughts to improve the survey questions and make the questions easier to understand.

Will you keep information private?

You will be assigned a unique identifier (ID), and at no time will your name be linked to any of your responses. Your responses will be used only for research purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573]. The sessions will be audio recorded and the recordings will be used only by the researchers administering the study. No one else will hear the recording.

How long will the research interview last and when will it take place?

The interview will last no more than 90 minutes and, if conducted in-person, requires only one visit to our office.

Will the results go to my school?

Your answers will not be disclosed to your school, and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this voluntary study.

Who can I contact with questions or for further information?

<CONTACT NAME> is administering this study at EurekaFacts on behalf of NCES. If you have any questions, please call <HIM/HER> at <CONTACT PHONE> or by e-mail <CONTACT E-MAIL>.

Can my participation end early?

Your participation in this study is completely voluntary. You have the right to refuse to answer particular questions. You may elect to withdraw from this study at any time.

By signing below, you agree that you will participate in this research study and that you have read and understood the information provided above. You will receive a $40 gift card (from a major credit card company) at the conclusion of the session for your time and effort. We thank you for your voluntary participation and ask you to sign the form below.

Print Name: _____________________________________________

Signature:___________________________________________________Date:___________________

E-mail address: _____________________________________________

For IRS reporting purposes, check one of these boxes:

I am a U.S. citizen/U.S. permanent resident. I have not received payment from ETS for any other service performed during the current calendar year [2015/2016] and I am not an employee of ETS.

I am a U.S. citizen/U.S. permanent resident. I have received payment from ETS during the current calendar year [2015/2016] that will NOT meet or exceed $600 in total and I am not an employee of ETS.

I am a U.S. citizen/U.S. permanent resident. I have received payment from ETS during the current calendar year [2015/2016] that WILL meet or exceed $600 in total and I am not an employee of ETS.

I am not a U.S. citizen or U.S. permanent resident. I am not eligible to participate in this study.

Internal use only:

Payment Card Transaction Number: __________________________

Appendices – Cognitive Items Playtesting

Appendix CA: ETS Recruitment Phone Script for Schools of Potential Student Participants: Reading and Mathematics Playtesting

Hello. May I please speak to <NAME OF CONTACT AT SCHOOL>?

Good morning/afternoon/evening. My name is <CONTACT NAME>, and I am contacting you from Educational Testing Service (ETS). We are carrying out an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are recruiting students who are currently in grades 4, 8 or 12 to participate in research interviews supporting the development of new items in reading and mathematics for the National Assessment of Educational Progress (NAEP). The research interviews will explore how students work through items on a <tablet computer or computer> to ensure that the items are clear and understandable to a broad range of students. NCES simply wants to know how students would respond to the items if they were on an actual test and would like to hear students’ thoughts to improve the items and make them easier to understand.

The voluntary research interviews will last no more than <60 or 90> minutes. Students will be asked to respond to reading and mathematics items. Research staff will take notes of students’ reactions to these items and will subsequently use students’ feedback to revise and refine the items. We will observe and ask about the students’ level of interest in, and understanding of, the items. Students will also be asked whether they have any suggestions about how to enhance the items.

Student responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law. [Recruiter note: If asked what law is being referenced, please provide its name: Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573.]

We would like your help in distributing and posting informational flyers. We will follow up directly with those who contact us. We will hold some sessions at <LOCATION>, <ADDRESS>; however, if the parents/guardians of students cannot provide transportation to <LOCATION>, <ADDRESS> we would like to set up sessions at your school.

Each participating student will receive a $25 gift card (from a major credit card company) in appreciation of the participants’ time. In addition, if transportation to the site is involved, a parent/guardian of each student will also receive a $25 gift card as appreciation for bringing the child to and from the interview site.

These interviews are an important stage of the NAEP test development process. The comments of students attending your school will help NCES improve test questions.

Would your school be interested in participating in or helping us to recruit students for this research study?

If NO: Thank you for your time. Have a wonderful day/evening. [END CALL]

If YES: Good, thank you very much. To ensure that we interview a broad mix of students and determine the best interview location, I have a few brief screening questions to ask you. It should take only a few minutes.

Screener

[Recruiter: Determine recruitment needs in advance. Look in public documents and school website to find what grades a school includes, the size of each grade, the SES, and racial/ethnic makeup of the student body. Record all responses to Screener questions. End Screener at whatever point the school staff member gives a response that confirms that their group has already been sampled adequately. End Screener Script: “Based on the requirements of this study, we are not able to include your school in the interviews at this time. Thank you for your time. Have a good day/evening.” [Interviewer: End call]]

Do you have <##> [specify recruitment needs] 4th graders who might be available to participate in the study?
Do you have <##> [specify recruitment needs] 8th graders who might be available to participate in the study?
Do you have <##> [specify recruitment needs] 12th graders who might be available to participate in the study?

Thank you for taking the time to answer the screening questions. Your school is generally eligible to participate in the research interviews.

Would you be able to distribute and post flyers regarding this study? After receiving a contact of interest, an ETS staff member would follow up with the parent/legal guardian by phone and ask them some screening questions to ensure that a diverse sample of students is included in this study.

If NO: Thank you for your time. Have a good day/evening. [Interviewer: End call]

If YES: Would you be able to make a room at your school available where ETS could conduct some <60- or 90-> minute research sessions?

If YES: That’s great, we will contact you to schedule some sessions at your school after we have some students signed up. (Interviewer: go to Closing]

If NO: (Interviewer: go to Closing]

(Closing)

Thank you for agreeing to disseminate information about this important research study. We will send you an email with the informational flyers attached.

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time?

[Recruiter: Answer any questions.]

If you have any (additional) questions, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this important research project.

Appendix CB: ETS Recruitment Phone Script for Parents/Legal Guardians of Student Participants (Under Age 18): Reading and Mathematics Playtesting

Script when returning calls from potential participants, or calling parents from a previously compiled list:

Hello. May I please speak to <NAME OF Parent/Legal Guardian>?

Good morning/afternoon/evening. Is this the <NAME FROM CALL LIST> household? My name is <CALLER NAME>, and I am calling from Educational Testing Service (ETS). We are carrying out a study on behalf of the U.S. Department of Education’s National Center for Education Statistics. We are recruiting students currently in grades 4, 8 or 12 to sign up to participate in a research study supporting the development of new items in reading and mathematics for the National Assessment of Educational Progress or NAEP. [If parent asks what this is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

Are there any fourth-, eighth-, or twelfth- grade students in your household?

If Yes: (Great! Let me give you some additional information about the study.)

If No: (Thank you, have a good day/evening.)

We are carrying out research interviews to explore how students work through sample reading and mathematics items on a <tablet computer or computer> to ensure that they are clear and understandable to a broad range of students. NCES simply wants to know if these items were included in an actual NAEP assessment, how students would respond to them.

If selected to participate, your child will receive a $25 gift card (from a major credit card company) as a thank you for his/her time. In addition, if transportation to the site is involved, you will also receive a $25 gift card as appreciation for bringing your child to and from the interview site.

The research interview will take place at <NAME OF INTERVIEW SITE AND LOCATION>. It can be scheduled at your convenience and will last no more than <60 or 90 minutes>. These research interviews are an important stage of the NAEP test development process. By sharing their feedback with us, students can directly guide the future of this nationwide assessment by helping NCES evaluate the items to create the best test possible.

Do you believe that your child would be interested in participating?

If No: (Thank you anyway, have a good day/evening.) (END CALL)

If Yes: Great! To ensure that we interview a broad mix of students, I have a few brief questions to ask about you and your child. It should take only a few minutes. First, I want to collect some additional information about your child to find out whether he/she meets the participation requirements.

(Skip to Screener, on following page.)

Script when calling a parent/legal guardian who responded to email or letter:

Hello. May I please speak to <NAME OF PARENT/LEGAL GUARDIAN>?

Good morning/afternoon/evening. My name is <NAME>. I’m calling from Educational Testing Service (ETS) on behalf of the National Center for Education Statistics, part of the U.S. Department of Education. Thank you for contacting us about the possibility of your child participating in a research interview to help develop items in reading and mathematics for the National Assessment of Educational Progress, or NAEP. To ensure that we interview a broad mix of students, I have a few brief screening questions to ask about you and your child. It should take only a few minutes. First, I want to collect some additional information about your child to find out whether he/she meets the participation requirements.

Screener

(INTERVIEWER: Determine recruitment needs in advance. Record all responses. End Screener at whatever point the parent/legal guardian gives a response that confirms that their group has already been sampled adequately. End Screener script: “Thank you for answering my questions. Based on the requirements of this study, we are not able to include your child in the interviews at this time. Thank you for your time. Have a good day/evening. [Interviewer: End call]”)

What is your child’s name?

What grade is your child currently in? (RECRUITER: DO NOT READ LIST)

 K – 3^rd  (Unfortunately, we are interviewing only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 4^th – Good.

 5^th – 7^th  (Unfortunately, we are interviewing only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.) 8^th – Good.

 9^th – 11^th (Unfortunately, we are interviewing only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12^th – Good.

What school does your child attend? (INTERVIEWER: Note if homeschooled)

____________________________________________________________

What school district does he/she attend? (INTERVIEWER: Note if parochial, charter, homeschooled) __________________________________________________

Is your child currently enrolled in any of the following types of courses? (select all that apply)

 Advanced placement (i.e., AP courses)

 Honor courses

 Remedial courses

 Other courses

What is your child’s race or ethnicity? (INTERVIEWER: Read list only if parent doesn’t immediately reply. “Are you…?”)

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

 American Indian or Alaska Native

 Asian

 Black or African American

 Hispanic or Latino

 Native Hawaiian or other Pacific Islander

 White or Caucasian

Just to be sure we're interviewing a broad range of students for this study, does your child participate in the National School Lunch program? [If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine.” And move on to the next question.]

 Yes

 No

Would you consider the area you live in to be …

 Urban

 Suburban

 Rural

Ok. Thank you and what is the zip code of the area that you live in?

________________________

Thank you. We would like to speak with your child to confirm if he/she is interested in participating in the research interview. It will take only a couple of minutes and then I will ask to speak with you again for another moment. Is your child available to speak with us now or should we call him/her back at another time?

 Yes, the child is available for screening questions  (SKIP TO SCRIPT FOR FOURTH- or EIGTH- or TWELFTH- GRADE STUDENTS)

 No, please call back.

Is the phone number you gave me earlier the best number to call?

Other phone numbers: _________________________________

Is there a good time for me to call? When would be best for you?

___________________________________________________

INTERVIEWER: IF STUDENT NEEDS TO RECEIVE A CALL BACK, RECORD NAME, PHONE NUMBER, DATE/TIME

Parent/Legal Guardian Name: ______________________________________________________________

Child Name: ____________________________________________________________________________

Parent/Legal Guardian Phone ______________________________________________________________

Date: __________________________________________________________________________________

Time: __________________________________________________________________________________

SCRIPT FOR GRADE 4, 8, or 12 STUDENTS

Hello. Is this <NAME OF STUDENT>?

Good morning/afternoon/evening. My name is <NAME>, and I’m calling from Educational Testing Service (ETS). We are doing an important study for the National Center for Education Statistics, part of the U.S. Department of Education.

We have spoken to your parent/legal guardian and we are pleased that you may be interested in helping us develop items in reading and mathematics for a nationwide student assessment, the National Assessment of Educational Progress, or NAEP. Before I tell you more about what you would be asked to do, I need to ask you one question:

What grade are you currently in? (RECRUITER: DO NOT READ LIST)

 K – 3^rd  (Unfortunately, we are interviewing only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 4^th – Good.

 5^th – 7^th (Unfortunately, we are interviewing only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 8^th – Good.

 9^th – 11^th (Unfortunately, we are interviewing only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12^th – Good.

Thank you. For the research interview, you will be asked to visit <LOCATION>, <ADDRESS> and respond to sample items on a <tablet computer or computer>. Your responses will not be graded. The National Center for Education Statistics simply wants to know how you would do each item if it were to be included in the National Assessment of Educational Progress. The items focus on reading and mathematics and are interesting and fun to do. Your input will help evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). The interview will last no more than <60 or 90 minutes>. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your interview responses and all other interview materials, and we will not use your name to identify you.

Would you like to participate?

 Yes

 No  (THANK AND END CALL)

If NO: Thank you for your time. I will need to let your parent/legal guardian know that you would not like to participate. Is he/she available?

TO PARENT/LEGAL GUARDIAN: Your child has decided not to participate, but I would like to thank you very much for taking the time to speak with us about this opportunity.

If YES: Thank you for agreeing to participate in this important research study. I will need to speak to your parent/legal guardian to talk about the date/time of the interview. Is your parent/legal guardian available?

 Yes

 No

INTERVIEWER: IF CALLING PARENT/LEGAL GUARDIAN BACK, RECORD NAME, PHONE NUMBER, DATE/TIME

Parent/Legal Guardian Name: _______________________________________________________

Child Name: _____________________________________________________________________

Parent/Legal Guardian Phone: ______________________________________________________

Date: __________________________________________________________________________

Time: __________________________________________________________________________

IF PARENT/LEGAL GUARDIAN AVAILABLE AND CURRENTLY SCHEDULING INTERVIEWS

Thank you again for allowing your child to participate in the NAEP research interviews. We will be holding our current interviews from <DATE> to <DATE>. What would be the best date/time to schedule the interview?

Date ____________________

Time ____________________

Thank you. We will follow up with a confirmation email/letter and a consent form. What is your email or postal address for us to send these to?

____________________________________________________________________________________

IF PARENT/LEGAL GUARDIAN AVAILABLE AND NOT CURRENTLY SCHEDULING INTERVIEWS

Thank you again for allowing your child to participate in the NAEP research interviews. We are in between interview sessions and will need to call you back when we are actively scheduling the interviews to determine the date and time that will work best. What are your scheduling preferences? (SELECT ALL THAT APPLY)

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix CC: ETS Recruitment Phone Script for Students Age 18 or Over Who Responded to Email, Advertisement, or Invitation: Reading and Mathematics Playtesting

Hello. May I please speak to <NAME OF STUDENT>?

Good morning/afternoon/evening. My name is <NAME>. I’m calling from Educational Testing Service (ETS) on behalf of the National Center for Education Statistics, part of the U.S. Department of Education. Thank you for contacting us about the possibility of participating in a research interview to help develop reading and mathematics items for the National Assessment of Educational Progress, or NAEP. [If the student asks what this is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

To ensure that we interview a broad mix of students, I have a few brief screening questions to ask you. It should take only a few minutes. First, I want to collect some additional information about you to find out whether you meet the participation requirements.

Screener

(INTERVIEWER: Determine recruitment needs in advance. Record all responses. End Screener at whatever point the response confirms that this group has already been sampled adequately. End Screener script: “Thank you for answering my questions. Based on the requirements of this study, we are not able to include you in the interviews at this time. Thank you for your time. Have a good day/evening. [Interviewer: End call]”)

What grade are you currently in at school? (INTERVIEWER: DO NOT READ LIST)

 K – 11^th  (Unfortunately, we are interviewing only students currently in grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12^th – Good.

What school do you attend? (INTERVIEWER: Note if homeschooled)

____________________________________________________________

What school district do you attend? (INTERVIEWER: Note if parochial, charter, homeschooled) __________________________________________________

What is your race or ethnicity? (INTERVIEWER: Read list only if student doesn’t immediately reply. “Are you…?”)

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

 American Indian or Alaska Native

 Asian

 Black or African American

 Hispanic or Latino

 Native Hawaiian or other Pacific Islander

 White or Caucasian

Are you currently enrolled in any of the following types of courses? (select all that apply)

Advanced placement (i.e., AP courses)

Honor courses

Remedial courses

Other courses

Just to be sure we're interviewing a broad range of students for this study, do you participate in the National School Lunch program?

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Would you consider the area you live in to be …

Urban

Suburban

Rural

Ok. Thank you and what is the zip code of the area that you live in?

________________________

Thank you. For the research interview, you will be asked to visit <LOCATION>, <ADDRESS> and respond to sample items on a <tablet computer or computer>. Your responses will not be graded. The National Center for Education Statistics simply wants to know how you would do each item if it were to be included in the National Assessment of Educational Progress. The items focus on reading and mathematics and are interesting and fun to do. Your input will help evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). The interview will last no more than 90 minutes. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your interview responses and all other interview materials, and we will not use your name to identify you.

Would you like to participate?

 Yes

 No  (THANK AND END CALL)

IF CURRENTLY SCHEDULING INTERVIEWS

Thank you again for agreeing to participate in the NAEP research interviews. We will be holding our current interviews from <DATE> to <DATE>. What would be the best date/time to schedule the interview?

Date ____________________

Time ____________________

IF NOT CURRENTLY SCHEDULING INTERVIEWS

Thank you again for agreeing to participate in the NAEP research interviews. We are in between interview sessions and will need to call you back when we are actively scheduling the interviews to determine the date and time that will work best. What are your scheduling preferences? (SELECT ALL THAT APPLY)

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix CD: ETS Consent Form for Parents/Legal Guardians of Student Participants (Under Age 18): Reading and Mathematics Playtesting

PARENTAL CONSENT FORM

FOR PARTICIPATION IN RESEARCH

Educational Testing Service (ETS)

Title of Research Study: Playtesting for NAEP Reading and Mathematics

Thank you for your interest in having your child participate in the research study described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them later as they occur to you.

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting informal interviews called Playtesting with students currently in grades 4, 8, and 12 about new test questions for the National Assessment of Educational Progress (NAEP). The questions we are testing focus on reading and mathematics and will be part of a digitally-based test. In these sessions, students will see early versions of digitally-based questions. We may ask students whether they have any suggestions about how we might improve the questions.

What will happen during the playtesting sessions?

Each participant will take an early version of a new NAEP reading and mathematics item on a computer. We will ask questions that focus on your child’s level of interest and ability to understand the questions presented to them. We may ask whether they have any suggestions about how we might improve the questions. Their responses will not be graded.

ETS staff will audio record and take notes about your child’s reactions to these questions to revise and refine test content. There will be more than one adult present.

Where will the playtesting sessions take place?

The sessions will take place at <NAME OF SESSION location>, located at <ADDRESS>.

[(Insert if session takes place at a school) For research activities conducted in schools, NAEP will not conduct pretesting sessions during school hours. If any pretesting sessions are conducted at schools, after regular school hours, we do not anticipate any disruption in school services to students.]

How long will the playtesting session last?

The session will last no more than <60 or 90> minutes. Each session requires only one visit; however your child may participate in multiple sessions, if interested.

Will you keep the information private and confidential?

Your child will be assigned a unique student identifier (ID), and at no time will his/her name be linked to any of his/her answers. Your child’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573]. These sessions will be audio recorded, as long as you agree, and only used by the researchers.

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his/her school, and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this research.

Can participation end early?

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this session at ETS on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>.

By signing below, you agree that your child, _________________________________, may participate in this research study. Your child will receive a $25 gift card (from a major credit card company) at the conclusion of the session. In addition, if transportation to the interview site is involved, you will also receive a $25 gift card as appreciation for the time bringing your child to and from the site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: ___________________________________________________________________________________

Signature: ______________________________________________________Date:__________________________

Email address: _________________________________________________________________________________

School student attends: _________________________________________________________________________

To be completed by ETS staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendix CE: ETS Consent Form for Student Participants (Age 18 or Over): Reading and Mathematics Playtesting

CONSENT FORM

FOR PARTICIPATION IN RESEARCH

Educational Testing Service (ETS)

Title of Research Study: Playtesting for NAEP Reading and mathematics

What is this study about?

What will happen during the playtesting sessions?

Each participant will take an early version of a new NAEP reading and mathematics item on a computer. We will ask questions that focus on your level of interest and your ability to understand the questions presented to you. We may ask whether you have any suggestions about how we might improve the questions. Your responses will not be graded.

ETS staff will audio record and take notes about your reactions to these questions to revise and refine test content.

Where will the playtesting sessions take place?

The sessions will take place at <NAME OF SESSION location>, located at <ADDRESS>.

How long will the playtesting sessions last?

The sessions will last no more than 90 minutes. Each session requires only one visit; however you may participate in multiple sessions, if you are interested.

Will you keep information private and confidential?

You will be assigned a unique student identifier (ID), and at no time will your name be linked to any of your answers. All of your responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. The interviews will be audio recorded, as long as you agree, and only used by the researchers.

Will the results go to my school?

Your answers will not be disclosed to your school, and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this research.

Can my participation end early?

Participation in this study is completely voluntary. You have the right to refuse to answer particular questions. You may also elect to withdraw from this study at any time and you can end early if you wish.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this session at ETS on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>.

By signing below, you agree that you will participate in this research study. You will receive a $25 gift card (from a major credit card company) at the conclusion of each session. In addition, if transportation to the site is involved, your parent/guardian will also receive a $25 gift card in appreciation for the time bringing you to and from the playtesting site. Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: __________________________________________________________________________________

Signature: ______________________________________________________Date: _________________________

Email address: _________________________________________________________________________________

School student attends: _________________________________________________________________________

To be completed by ETS staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendices – Cognitive Items Cognitive Interviews

Appendix CF: ETS Consent Form for Parents/Legal Guardians of Student Participants (Under Age 18): Reading and Mathematics Cognitive Interviews

PARENTAL CONSENT FORM

FOR PARTICIPATION IN RESEARCH

Educational Testing Service (ETS)

Title of Research Study: Cognitive Laboratory Interviews for NAEP

Reading and mathematics

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting research interviews with students currently in grades 4, 8, and 12 about new test questions for the National Assessment of Educational Progress (NAEP). The questions we are testing focus on reading and mathematics and will be part of a digitally-based test. In these interviews, students will see early versions of digitally-based items. We will ask questions that focus on students’ ability to understand the questions presented to them. Students will also be asked whether they have any suggestions about how we might improve the items.

What will happen during the research interview?

Each participant will work through an early version of a new NAEP reading and mathematics item on a tablet computer. An interviewer will ask what your child was thinking as he/she completed the item, and whether he/she believes the item is clear and understandable. Your child’s responses will not be graded. In some cases, we may use a camera (also known as an eye-tracker) attached to the computer to record where on the screen students are looking while working. Interviewers will also take notes about students’ reactions to the items so that items can be revised and refined. There will be more than one adult present.

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <ADDRESS>.

[(Insert if INTERVIEW takes place at a school) For research activities conducted in schools, NAEP will not conduct pretesting interviews during school hours. If any pretesting interviews are conducted at schools, after regular school hours, we do not anticipate any disruption in school services to students.]

How long will the research interview last?

The interview will last no more than <60 or 90> minutes. The interview requires only one visit to the interview site.

Will you keep information private and confidential?

Your child will be assigned a unique student identifier (ID), and at no time will his/her name be linked to any of his/her answers. Your child’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573]. Audio and screen activity will be recorded for analysis, but no personal identifying information will be recorded or retained.

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his/her school and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at ETS on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>.

By signing below, you agree that your child, _______________________________, may participate in this study. Your child will receive a $25 gift card (from a major credit card company) at the conclusion of the interview.

In addition, if transportation to the interview site is involved, you will also receive a $25 gift card as appreciation for the time bringing your child to and from the site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: ________________________________________________________________________________

Signature: ______________________________________________________Date: _______________________

Email address: ______________________________________________________________________________

School student attends: _______________________________________________________________________

To be completed by ETS staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendix CG: ETS Consent Form for Student Participants (Age 18 or Over): Reading and Mathematics Cognitive Interviews

CONSENT FORM

FOR PARTICIPATION IN RESEARCH

Educational Testing Service (ETS)

Title of Research Study: Cognitive Laboratory Interviews for NAEP

Reading and Mathematics

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting research interviews with students currently in grades 4, 8, and 12 about new test questions for the National Assessment of Educational Progress (NAEP). The questions we are testing focus on reading and mathematics and will be part of a digitally-based test. In these interviews, students will see early versions of digitally-based questions. We may ask students whether they have any suggestions about how we might improve the questions.

What will happen during the research interview?

Each participant will work through an early version of a new NAEP reading and mathematics item on a tablet computer. An interviewer will ask what you were thinking as you completed the item, and whether you believe that the item is clear and understandable. Your responses will not be graded. In some cases, we may use a camera (also known as an eye-tracker) attached to the computer to record where on the screen students are looking while working. Interviewers will also take notes about your reactions to the items so that items can be revised and refined. We will observe and ask you questions about your understanding of the items.

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <ADDRESS>.

How long will the research interview last?

The interview will last no more than 90 minutes. The interview requires only one visit to the interview site.

Will you keep information private and confidential?

You will be assigned a unique student identifier (ID), and at no time will your name be linked to any of your answers. Your responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573]. Audio and screen activity will be recorded for analysis, but no personal identifying information will be recorded or retained.

Will the results go to my school?

Your answers will not be disclosed to your school and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Your participation in this study is completely voluntary. You have the right to refuse to answer particular questions. You may elect to withdraw from this study at any time.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at ETS on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>.

By signing below, you agree that you will participate in this research study. You will receive a $25 gift card (from a major credit card company) at the conclusion of the interview. In addition, if transportation to the site is involved, your parent/guardian will also receive a $25 gift card in appreciation for the time bringing you to and from the interview site. Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: ________________________________________________________________________________

Signature: ______________________________________________________Date: _______________________

Email address: ______________________________________________________________________________

School student attends: _______________________________________________________________________

To be completed by ETS staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendix CH: EurekaFacts Recruitment Phone Script for Schools of Potential Student Participants: Reading and Mathematics Cognitive Interviews

Hello. May I please speak to <NAME of contact at school>?

Good morning/afternoon/evening. My name is <CONTACT NAME> and I am contacting you from EurekaFacts, a local research company. We are carrying out an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are recruiting students to participate in research sessions supporting the development of new reading and mathematics items in reading and mathematics for the National Assessment of Educational Progress (NAEP). The research sessions will explore how students work through the items on a tablet computer to ensure that the items are clear and understandable to a broad range of students. NCES simply wants to know how students would do the items if they were on an actual test, and would like to hear students’ thoughts to improve the items and make them easier to understand.

The voluntary research sessions will last no more than <60 or 90> minutes. Students will be asked to respond to reading and mathematics items and will be asked about them. EurekaFacts staff will take notes of students’ reactions to the items, and students’ feedback will be used to revise and refine the items. We will observe and ask about the students’ understanding of the items. Students may also be asked whether they have any suggestions about how to enhance the items.

Student responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law.

[INTERVIEWER note: If asked what law is being referenced, please provide its name: Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573.]

We would like your help in distributing flyers to students, and we will follow up directly with any respondents who contact us. We will hold some sessions at <LOCATION>; however, if the parents/guardians of students cannot provide transportation to <LOCATION>, we would like to set up sessions at your school or we can arrange to conduct the session at a nearby location.

For each participating student, a $25 gift card (from a major credit card company) will be issued in appreciation for participants’ time. In addition, if transportation to the site is involved, a parent or guardian of each student will receive a $25 gift card as appreciation for bringing the child to and from the research site.

These sessions are an important stage of the NAEP test development process. The comments of students attending your school will help NCES improve test questions.

Would your school be interested in participating in or helping us to recruit students for this research study?

If NO: Thank you for your time. Have a good day/evening.

If YES: Great, thank you very much. To ensure that we work with a broad mix of students and determine the best session location, I have a few brief screening questions to ask you. It should take only a few minutes.

Screener

[INTERVIEWER: Determine recruitment needs in advance. Look in public documents and school website to find what grades a school includes, the size of each grade, the socioeconomic status (SES), and racial/ethnic makeup of the student body. Record all responses to screener questions. End screener at whatever point the school staff member gives a response that confirms that their group has already been sampled adequately. End screener script: “Based on the requirements of this study, we are not able to include you in the study at this time. Thank you for your time. Have a good day/evening. [Interviewer: End call]”]

Are you at a school with a population that includes:
1. Grade 4 students
2. Grade 8 students
3. Grade 12 students

How many students are enrolled in your school?
1. 299 students or fewer
2. 300-599 students
3. 600 students or more

Eligible Participant Script

Thank you for taking the time to answer the screening questions. Students at your school are generally eligible to participate in the research sessions. After receiving a contact of interest, a EurekaFacts staff member will follow up with the parent/guardian by phone and ask them some screening questions to ensure that a diverse sample of students is included in this study.

Would you be able to send flyers to parents of potential student participants?

If NO: Thank you for your time. Have a good day/evening. [Interviewer: End call.]

If YES: Would it be possible to have a room at your school available where EurekaFacts could conduct <60 or 90>-minute research sessions?

If YES: That’s great, we will contact you to schedule some sessions at your school after we have some students signed up. [Interviewer: go to Closing]

If NO: That’s alright, we can arrange to have a session scheduled at a location near the school (e.g. library). [Interviewer: go to Closing]

(Closing)

Thank you for agreeing to disseminate information about this important research study. We will send you an email with informational flyers.

Is [INSERT EMAIL ADDRESS] the best email address to reach you? OR What is the best email address to reach you?

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time? [INTERVIEWER: Answer any questions.]

If you do have any questions, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this important research project. Have a wonderful day/evening.

Appendix CI: EurekaFacts Recruitment Phone Script for Parents/Legal Guardians of Student Participants (Under Age 18): Reading and Mathematics Cognitive Interviews

Script when calling parents from previously compiled list:

Good morning/afternoon/evening. Is this the <NAME FROM CALL LIST> household? My name is <CALLER NAME>, and I am calling from EurekaFacts, a local research company. We are carrying out a study on behalf of the U.S. Department of Education’s National Center for Education Statistics.

We are recruiting students currently in grades 4, 8 or 12 to participate in a research study supporting the development of new questions for the National Assessment of Educational Progress or NAEP. [If parent asks what this is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

Are there any students in your household?

No: (Thank you, have a good day/evening.)

Yes: (Great! Let me give you some additional information about the study.)

We are looking for students to help us test new computer-delivered reading and mathematics items. These items will be part of the National Assessment of Educational Progress (NAEP) delivered to students nationwide.

The research session will take place at <NAME OF SESSION SITE AND LOCATION>. It can be scheduled at your convenience and will last about <60 or 90> minutes. During the session, your child will be asked to respond to sample reading and mathematics items on a tablet computer. NCES simply wants to know how students would do the items if they were on an actual test, and would like to hear students’ thoughts to improve the items and make them easier to understand.

If selected to participate, your child will receive a $25 gift card (from a major credit card company) as thanks for his or her time. In addition, if transportation to the site is involved, you will also receive a $25 gift card as appreciation for bringing your child to and from the research site.

These research sessions are an important stage of the NAEP test development process. By sharing their feedback with us, students can directly guide the future of this nationwide assessment by helping NCES evaluate the test questions and instructions to create the best test possible.

Do you believe that your child would be interested in participating?

No: (Thank you anyway, have a good day/evening.)

Yes: (Great! Thank you so much. To ensure that we work with a broad mix of students, I have a few brief questions to ask about you and your child. It should take only a few minutes. First, I want to collect some additional information about your child to find out whether he or she meets the participation requirements.)

(Skip to Screener, on following page.)

Script when calling a parent/guardian who responded to email or letter

Good morning/afternoon/evening. Is this the <NAME FROM CALL LIST> household? My name is <NAME>. I’m calling from EurekaFacts on behalf of the National Center for Education Statistics, part of the U.S. Department of Education. Thank you for contacting us about the possibility of your child participating in a research session to help assess test questions for the National Assessment of Educational Progress, or NAEP. To ensure that we work with a broad mix of students, I have a few brief screening questions to ask about you and your child. It should only take a few minutes. First, I want to collect some additional information about your child to find out whether he or she meets the participation requirements.

Screener

[INTERVIEWER: Determine recruitment needs in advance. Record all responses. End screener at whatever point the parent or legal guardian gives a response that confirms that their group has already been sampled adequately. End Screener script: “Thank you for answering my questions. Based on the requirements of this study, we are not able to include your child in the sessions at this time. Thank you for your time. Have a good day/evening. [Interviewer: End call]”)

What is your child’s name?

What grade is your child currently in? (INTERVIEWER: DO NOT READ LIST)

 K-3^rd  (Unfortunately, we need only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 4^th- Good

 5^th – 7^th  (Unfortunately, we need only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 8^th - Good

 9^th – 11^th (Unfortunately, we need only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12^th - Good

What school does your child attend? (INTERVIEWER: Note if homeschooled)

____________________________________________________________

What school district does he or she attend? (INTERVIEWER: Note if parochial, charter, homeschooled) __________________________________________________

Is your child currently enrolled in any of the following types of courses (select all that apply)

Advanced placement (i.e., AP courses)

Honor courses

Remedial courses

Other courses

What is your child’s race or ethnicity? (INTERVIEWER: Read list only if parent doesn’t immediately reply. “Are you…?”)

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

 American Indian or Alaska Native

 Asian

 Black or African American

 Hispanic or Latino

 Native Hawaiian or other Pacific Islander

 White or Caucasian

Just to be sure we're interviewing a broad range of students for this study, does your child participate in the National School Lunch program?

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine.” and move on to the next question.]

Would you consider the area you live in to be …

Urban

Suburban

Rural

Ok. Thank you and what is the zip code of the area that you live in?

______________

 Yes, the child is available for screening questions  (SKIP TO SCRIPT FOR 4^th, 8th or 12th GRADE STUDENTS)

 No, please call back.

Is the phone number you gave me earlier the best number to call?

Other phone numbers: _________________________________

Is there a good time for me to call? When would be best for you?

___________________________________________________

INTERVIEWER: IF CHILD NEEDS TO RECEIVE A CALL BACK, RECORD NAME, PHONE NUMBER, DATE/TIME

Parent/Legal Guardian Name ___________________________________________________________

Child Name ____________________________________________________________________

Date ____________________

Time ____________________

SCRIPT FOR 4^th, 8th or 12th GRADE STUDENTS

Good morning/afternoon/evening. Is this <NAME OF STUDENT>? My name is <NAME>, and I’m calling from EurekaFacts. We are doing an important study for the National Center for Education Statistics, part of the U.S. Department of Education.

What grade are you currently in? (RECRUITER: DO NOT READ LIST)

 K-3^rd  (Unfortunately, we need only students currently in grade 4, 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 4^th – Good.

 5^th – 7^th  (Unfortunately, we are interviewing only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 8^th – Good.

 9^th – 11^th (Unfortunately, we are interviewing only students currently in grade 8 or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12^th – Good.

Thank you. For the research interview, you will be asked to visit <LOCATION>, <ADDRESS> and respond to sample reading or mathematics items on a tablet computer. Your responses will not be graded. The National Center for Education Statistics simply wants to know how you would do each task if it were to be included in the National Assessment of Educational Progress. The items focus on reading and mathematics and are interesting and fun to do. Your input will help evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). The interview will last no more than <60 or 90> minutes. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your interview responses and all other interview materials, and we will not use your name to identify you.

Would you like to participate?

 Yes

 No : Thank you for your time. I will need to let your parent/legal guardian know that you would not like to participate. Is he/she available?

If NO:

TO PARENT/LEGAL GUARDIAN: Your child has decided not to participate, but I would like to thank you very much for taking the time to speak with us about this opportunity.

If YES: Thank you for agreeing to participate in this important research study. I will need to speak to your parent/legal guardian to talk about the date/time of the interview. Could you please pass the phone back to your parent/legal guardian?

IF PARENT/LEGAL GUARDIAN AVAILABLE AND CURRENTLY SCHEDULING INTERVIEWS

Date ____________________

Time ____________________

Thank you. We will follow up with a confirmation email/letter and a consent form. What is your email or postal address for us to send these to?

____________________________________________________________________________________

IF PARENT/LEGAL GUARDIAN AVAILABLE AND NOT CURRENTLY SCHEDULING INTERVIEWS

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix CJ: EurekaFacts Recruitment Phone Script for Students (Age 18 or Over): Reading and Mathematics Cognitive Interviews

Script when calling students from previously compiled list:

Good morning/afternoon/evening. Is this <NAME FROM CALL LIST>? My name is <CALLER NAME>, and I am calling from EurekaFacts, a local research company. We are carrying out a study on behalf of the U.S. Department of Education’s National Center for Education Statistics.

We are recruiting students to participate in a research study supporting the development of new questions for the National Assessment of Educational Progress or NAEP.

Are you a student 18 or over?

No: (Thank you, have a good day/evening.)

Yes: (Great! Let me give you some additional information about the study.)

We are looking for students to help us test new computer-delivered reading and mathematics items. These items will be part of the National Assessment of Educational Progress (NAEP) delivered to students nationwide.

The research session will take place at <NAME OF SESSION SITE AND LOCATION>. It can be scheduled at your convenience and will last about 90 minutes. During the session, you will be asked to respond to sample reading and mathematics items on a tablet computer. NCES simply wants to know how students would do the items if they were on an actual test, and would like to hear students’ thoughts to improve the items and make them easier to understand.

If selected to participate, you will receive a $25 gift card (from a major credit card company) to thank you for your time. In addition, if transportation to the site is involved, your parent/guardian will also receive a $25 gift card as appreciation for bringing you to and from the research site.

These research sessions are an important stage of the NAEP test development process. By sharing your feedback with us, students can directly guide the future of this nationwide assessment by helping NCES evaluate the test questions and instructions to create the best test possible.

Would you be interested in participating?

No: (Thank you anyway, have a good day/evening.)

Yes: (Great! Thank you so much. To ensure that we work with a broad mix of students, I have a few brief questions to ask you. It should take only a few minutes. First, I want to collect some additional information to find out whether you meet the participation requirements.)

(Skip to Screener, on following page.)

Script when calling a student age 18 or over who responded to email, advertisement, or invitation

Good <MORNING/AFTERNOON/EVENING>. Is this <NAME OF STUDENT>? My name is <NAME>. I’m calling from EurekaFacts on behalf of the U.S. Department of Education’s National Center for Education Statistics, also known as NCES. Thank you for contacting us about the possibility of participating in a research session to help test new test questions for the National Assessment of Educational Progress, also known as the NAEP assessment. To ensure that we interview a broad mix of students, I have a few brief screening questions to ask you about yourself. It should take only a few minutes of your time. First, I want to collect some additional information about you to identify if you meet the participation requirements.

Before proceeding, please confirm that you are still interested in the possibility of participating in this research interview?

Yes: (INTERVIEWER: Proceed to “Screener.”)

No: (Thank you for your time. Have a good <DAY/EVENING>.)

Screener:

[INTERVIEWER: Determine recruitment needs in advance. Record all responses. End screener at whatever point the student gives a response that confirms that their group has already been sampled adequately. End Screener script: “Thank you for answering my questions. Based on the requirements of this study, we are not able to include you in the sessions at this time. Thank you for your time. Have a good day/evening. [Interviewer: End call]”)

What is your name?

What grade are you currently in? (If the student indicates that he/she is in any grade other than 8^th or 12^th, thank and end call: “Unfortunately we are interviewing only students currently in grades 8 or 12 for this study. Thank you for your time. Have a good <DAY/EVENING>.”)

What school do you attend? (INTERVIEWER: Note if homeschooled)

____________________________________________________________

What school district do you attend? (INTERVIEWER: Note if parochial, charter, homeschooled) __________________________________________________

Are you currently enrolled in any of the following types of courses? (select all that apply)

Advanced placement (i.e., AP courses)

Honor courses

Remedial courses

Other courses

What is your race or ethnicity? (INTERVIEWER: Read list only if respondent doesn’t immediately reply. “Are you…?”)

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

American Indian or Alaska Native

Asian

Black or African American

Hispanic or Latino

Native Hawaiian or other Pacific Islander

White or Caucasian

Just to be sure we're interviewing a broad range of students for this study, do you participate in the National School Lunch program?

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Would you consider the area you live in to be …

Urban

Suburban

Rural

Ok. Thank you and what is the zip code of the area that you live in?

________________________

Would you like to participate?

Yes

No  (THANK AND END CALL)

If NO: Thank you for your time, and have a wonderful <DAY/EVENING.>

If YES:

Thank you for agreeing to participate in this important research study. For the research session, you will be asked to go to <LOCATION> and respond to a sample of test questions. Your responses will not be graded. The questions focus on reading and mathematics and are interesting and fun to answer. You will be asked to answer the questions on a tablet computer.

Your input will help evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). If your parent/ guardian brings you to and from our office, he or she will also receive a $25 gift card. The session will last about 90 minutes. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your responses and all other session materials, and we will not use your name to identify you.

IF CURRENTLY SCHEDULING INTERVIEWS

Thank you again for agreeing to participate in the NAEP research sessions. We will be holding our current sessions from <DATE> to <DATE>. What would be the best date/time to schedule the session?

Date ____________________

Time ____________________

IF NOT CURRENTLY SCHEDULING SESSIONS

Thank you again for agreeing to participate in the NAEP research sessions. We are in between research sessions, and will need to call you back when we are actively scheduling the sessions to determine the date and time that will work best. What are your scheduling preferences? (SELECT ALL THAT APPLY)

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix CK: EurekaFacts Consent Form for Parents/Legal Guardians of Student Participants (Under Age 18): Reading and Mathematics Cognitive Interviews

PARENTAL CONSENT FORM

FOR PARTICIPATION IN RESEARCH

EurekaFacts, LLC

Title of Research Study: Cognitive Laboratory Interviews for NAEP Reading and Mathematics

Thank you for your interest in having your child participate in the research project described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them later as they occur to you.

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting research interviews with students currently in grades 4, 8 and 12 about new test questions for the National Assessment of Educational Progress (NAEP). The questions we are testing focus on reading and mathematics and will be part of a digitally-based test. In these interviews, students will see early versions of digitally-based questions. EurekaFacts staff will ask questions that focus on students’ level of interest and their ability to understand the questions presented to them. Students will also be asked if they have any suggestions about how we might improve the questions.

What will happen during the research interview?

Each participant will work through an early version of a new NAEP reading and mathematics item on a tablet computer. An interviewer will ask what your child was thinking as he or she completed the task, and whether he or she believes the item is clear and understandable. Your child’s responses will not be graded. EurekaFacts staff will use computer software to record what students do on screen, as well as microphones to record students’ reactions to items, descriptions of their work, and responses to interviewer questions. In some cases, we may use a camera (also known as an eye-tracker) attached to the computer to record where on the screen students are looking during the task. Interviewers will also take notes about students’ reactions to the items so that items can be revised and refined. We will observe and ask about the students’ understanding of the items. There will be more than one adult present.

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

How long will the research interview last and when it will take place?

The interview will last up to <60 or 90> minutes and will require only one visit to our office. Interviews can be scheduled at your convenience.

Will you keep information private and confidential?

Your child will be assigned a unique student identifier (ID), and at no time will his/her name be linked to any of his/her answers. Your child’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573]. Audio and screen activity will be recorded for analysis, but no personal identifying information will be recorded or retained.

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his/her school, and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Your child’s participation in this study is completely voluntary. Your child has the right to refuse to answer particular questions. You or your child may elect to withdraw from this study at any time.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at EurekaFacts on behalf of NCES. If you have any questions, please call him/her at <CONTACT PHONE> or by email <CONTACT EMAIL>.

In addition, if transportation to the site is involved, you will receive a $25 gift card as appreciation for the time bringing your child to and from the research site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: _________________________________________________________________________

Signature: ___________________________________________________________________________

Date:_______________________________________________________________________________

School that student attends: ____________________________________________________________

To be completed by EurekaFacts staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendix CL: EurekaFacts Consent Form for Student Participants (Age 18 and Over): Reading and Mathematics Cognitive Interviews

CONSENT FORM

FOR PARTICIPATION IN RESEARCH

EurekaFacts, LLC

Title of Research Study: Cognitive Laboratory Interviews for NAEP Reading and mathematics

Thank you for your interest in participating in the research project described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them later as they occur to you.

What is this study about?

What will happen during the research interview?

Each participant will work through an early version of a new NAEP reading and mathematics item on a tablet computer. An interviewer will ask what you were thinking as you completed the item, and whether you believe that the item is clear and understandable. Your responses will not be graded. EurekaFacts staff will use computer software to record what students do on screen, as well as microphones to record students’ reactions to items, descriptions of their work, and responses to interviewer questions. In some cases, we may use a camera (also known as an eye-tracker) attached to the computer to record where on the screen students are looking while working. Interviewers will also take notes about your reactions to the reading and mathematics items so that items can be revised and refined. We will observe and ask you questions about your understanding of the items.

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

How long will the research interview last and when it will take place?

The interview will last up to 90 minutes and will require only one visit to our office. Interviews can be scheduled at your convenience.

Will you keep information private and confidential?

Will the results go to my school?

Your answers will not be disclosed to your school, and will not be linked to any personally identifiable information such as the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Your participation in this study is completely voluntary. You have the right to refuse to answer particular questions. You may elect to withdraw from this study at any time.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at EurekaFacts on behalf of NCES. If you have any questions, please call him/her at <CONTACT PHONE> or by email <CONTACT EMAIL>.

By signing below, you agree that you will participate in this study. You will receive a $25 gift card (from a major credit card company) at the conclusion of the session.

In addition, if transportation to the site is involved, your parent/guardian will also receive a $25 gift card as appreciation for bringing you to and from the research site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: _________________________________________________________________________

Signature: ___________________________________________________________________________

Date:_______________________________________________________________________________

School that you attend: ________________________________________________________________

To be completed by EurekaFacts staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendices – Cognitive Items Tryouts

Appendix CM: EurekaFacts Recruitment Phone Script for Schools of Potential Student Participants: Reading and Mathematics Tryouts

Hello. May I please speak to <NAME of contact at school>?

Good morning/afternoon/evening. My name is <CONTACT NAME> and I am contacting you from EurekaFacts, a local research company. We are carrying out an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are recruiting students to participate in research sessions supporting the development of new items in reading and mathematics for the National Assessment of Educational Progress (NAEP). The research sessions will explore how students work through the items on a tablet computer to ensure that the items are clear and understandable to a broad range of students.

The voluntary research sessions will last no more than <60 or 90> minutes. During the session, the student will be asked to respond to test questions on a tablet computer. NCES simply wants to know how students would answer the questions if they were to be included on an actual NAEP assessment.

These question tryout sessions are an important stage of the NAEP test development process. By sharing their feedback with us, students can directly impact the future of this nationwide assessment by helping NCES evaluate the test questions to create the best test possible.

Student responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law.

[INTERVIEWER note: If asked what law is being referenced, please provide its name: Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573.]

These sessions are an important stage of the NAEP test development process. The comments of students attending your school will help NCES improve test questions.

Would your school be interested in participating in or helping us to recruit students for this research study?

If NO: Thank you for your time. Have a good day/evening.

Screener

Are you at a school with a population that includes:

Grade 4 students
Grade 8 students
Grade 12 students

How many students are enrolled in your school?
1. 299 students or fewer
2. 300-599 students
3. 600 students or more

Eligible Participant Script

Would you be able to send flyers to parents of potential student participants?

If NO: Thank you for your time. Have a good day/evening. [Interviewer: End call.]

If YES: Would it be possible to have a room at your school available where EurekaFacts could conduct <60- or 90>-minute research sessions?

If YES: That’s great, we will contact you to schedule some sessions at your school after we have some students signed up. [Interviewer: go to Closing]

If NO: That’s alright, we can arrange to have a session scheduled at a location near the school (e.g. library). [Interviewer: go to Closing]

(Closing)

Thank you for agreeing to disseminate information about this important research study. We will send you an email with informational flyers.

Is [INSERT EMAIL ADDRESS] the best email address to reach you? OR What is the best email address to reach you?

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time? [INTERVIEWER: Answer any questions.]

If you do have any questions, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this important research project. Have a wonderful day/evening.

Appendix CN: EurekaFacts Recruitment Screener – Parents/Legal Guardians of Student Participants (Under Age 18): Reading and Mathematics Tryouts

Script when calling parents from previously compiled list:

Good morning/afternoon/evening. Is this the <NAME FROM CALL LIST> household? My name is <NAME>, and I am calling from EurekaFacts. We are a local research company carrying out a study on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are recruiting students to sign up to participate in a research study supporting the development of new items in reading and mathematics for the National Assessment of Educational Progress, or NAEP. [If parent asks what this is: This national assessment is administered to students in elementary and middle schools throughout the United States.]

Are there any students in your household?

Yes: (Great! Let me give you some additional information about the study.)

No: (Thank you, have a good day/evening.)

We are carrying out question tryout sessions to explore how students work through sample reading and mathematics test questions to ensure that they are clear and understandable to a broad range of students. If selected to participate, your child will receive a $25 gift card (from a major credit card company). You will also receive a $25 gift card to thank you for your time in bringing your child to and from the session site.

The question tryout session will take place at <LOCATION>. It will last no more than <60 or 90> minutes and can be scheduled at your convenience. During the session, your child will be asked to respond to test questions on a tablet computer. NCES simply wants to know how students would answer the questions if they were to be included on an actual NAEP assessment.

Yes: (Great! To ensure that we conduct this research with a broad mix of students, I have a few brief questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to identify if <HE/SHE> meets the participation requirements.

(Skip to Screener, on following page.)

No: (Thank you, have a good day/evening. END CALL.)

Script when calling a parent who responded to email, advertisement, or invitation

Good morning/afternoon/evening. Is this <NAME OF STUDENT>? My name is <NAME>. I’m calling from EurekaFacts on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education.

Thank you for contacting us about the possibility of your child participating in a tryout session to help develop test questions for the National Assessment of Educational Progress, or NAEP. To ensure that we conduct research with a broad mix of students, I have a few brief questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to identify if <HE/SHE> meets the participation requirements.

Screener

What is your child’s name?

What grade is your child currently in? (INTERVIEWER: DO NOT READ LIST)
- K – 3^rd (Unfortunately, we are only interviewing students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)
- 4^th - Good
- 5^th – 7th  (Unfortunately, we are only interviewing students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)
- 8th - Good
- 9th – 11th  (Unfortunately, we are only interviewing students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)
- 12^th - Good

What school does your child attend? (INTERVIEWER: Note if homeschooled)

____________________________

What school district does he/she attend? (INTERVIEWER: Note if parochial, charter, homeschooled)

____________________________

Is your child currently enrolled in any of the following types of courses (select all that apply)

Advanced placement (i.e., AP courses)

Honor courses

Remedial courses

Other courses

What is your race or ethnicity? (INTERVIEWER: Read list only if respondent doesn’t immediately reply. “Are you…?”)

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

American Indian or Alaska Native

Asian

Black or African American

Hispanic or Latino

Native Hawaiian or other Pacific Islander

White or Caucasian

Just to be sure we're interviewing a broad range of students for this study, do you participate in the National School Lunch program?

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Would you consider the area you live in to be …
- Urban
- Suburban
- Rural

Ok. Thank you and what is the zip code of the area that you live in?

_________________

Thank you for your time. Is the phone number you gave me earlier the best number to call?

Other phone numbers: ____________________________

Is there a good time for me to call? When would be best for you?

Parent Name _____________________________________________

Child Name _______________________________________________

Date ____________________________________________________

Time ____________________________________________________

SCRIPT FOR 4^th, 8th or 12th GRADE STUDENTS

What grade are you currently in? (RECRUITER: DO NOT READ LIST)

 K – 3^rd – (Unfortunately, we are interviewing only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

4^th^ - Good

 5^th – 7^th  (Unfortunately, we are interviewing only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 8^th – Good.

 9^th – 11^th (Unfortunately, we are interviewing only students currently in grade 4, grade 8, or grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12^th – Good.

Thank you. For the research interview, you will be asked to visit <LOCATION>, <ADDRESS> and respond to sample reading and mathematics items on a tablet computer. Your responses will not be graded. The National Center for Education Statistics simply wants to know how you would do each task if it were to be included in the National Assessment of Educational Progress. The items focus on reading and mathematics and are interesting and fun to do. Your input will help evaluate the test that will be given to students throughout the nation. To thank you for participating, you will receive a $25 gift card (from a major credit card company). The interview will last no more than <60 or 90> minutes. Your participation is totally voluntary. If you participate, no one but the study’s researchers will see your interview responses and all other interview materials, and we will not use your name to identify you.

Would you like to participate?

 Yes

 No  Thank you for your time. I will need to let your parent/legal guardian know that you would not like to participate. Is he/she available?

If NO:

TO PARENT/LEGAL GUARDIAN: Your child has decided not to participate, but I would like to thank you very much for taking the time to speak with us about this opportunity.

IF PARENT/LEGAL GUARDIAN AVAILABLE AND CURRENTLY SCHEDULING INTERVIEWS

Date ____________________

Time ____________________

Thank you. We will follow up with a confirmation email/letter and a consent form. What is your email or postal address for us to send these to?

____________________________________________________________________________________

IF PARENT/LEGAL GUARDIAN AVAILABLE AND NOT CURRENTLY SCHEDULING INTERVIEWS

 Morning

 Afternoon

 After school

 Weekends

Thank you so much for talking with us today. We look forward to communicating with you soon.

Appendix CO: EurekaFacts Recruitment Screener – Students Age 18 or Over: Reading and Mathematics Tryouts

Script when calling students from previously compiled list:

Good morning/afternoon/evening. Is this <NAME FROM CALL LIST>? My name is <NAME>, and I am calling from EurekaFacts. We are a local research company carrying out a study on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. We are recruiting students to sign up to participate in a research study supporting the development of new reading and mathematics test questions for the National Assessment of Educational Progress, or NAEP. [If student asks what this is: This national assessment is administered to students in elementary and middle schools throughout the United States.]

Are you a student over the age of 18?

Yes: (Great! Let me give you some additional information about the study.)

No: (Thank you, have a good day/evening.)

We are carrying out question tryout sessions to explore how students work through sample reading and mathematics test questions to ensure that they are clear and understandable to a broad range of students. If selected to participate, you will receive a $25 gift card (from a major credit card company). If your parent or guardian accompanies you they will also receive a $25 gift card to thank them for your time in bringing you to and from the session site.

The question tryout session will take place at <LOCATION>. It will last no more than 90 minutes and can be scheduled at your convenience. During the session, you will be asked to respond to test questions on a tablet computer. NCES simply wants to know how students would answer the questions if they were to be included on an actual NAEP assessment.

These question tryout sessions are an important stage of the NAEP test development process. By sharing your feedback with us, students can directly impact the future of this nationwide assessment by helping NCES evaluate the test questions to create the best test possible. Would you be interested in participating?

Yes: (Great! To ensure that we conduct this research with a broad mix of students, I have a few brief questions to ask about you. It should only take a few minutes of your time. First, I want to collect some additional information to identify if you meet the participation requirements.

(Skip to Screener, on following page.)

No: (Thank you, have a good day/evening. END CALL.)

Script when calling students who responded to email, advertisement, or invitation

Thank you for contacting us about the possibility of you participating in a tryout session to help develop test questions for the National Assessment of Educational Progress, or NAEP. To ensure that we conduct research with a broad mix of students, I have a few brief questions to ask you. It should only take a few minutes of your time. First, I want to collect some additional information to identify if you meet the participation requirements.

Screener

What is your name?

What grade are you currently in? (INTERVIEWER: DO NOT READ LIST)

K-11^th  (Unfortunately, we are only interviewing students currently in grade 12 for this study. Thank you for your time. Have a good day/evening.)

 12^th - Good

What school do you attend? (INTERVIEWER: Note if homeschooled)

____________________________

What school district do you attend? (INTERVIEWER: Note if parochial, charter, homeschooled)

____________________________

Are you taking any of the following types of courses (select all that apply)

Advanced placement (i.e., AP courses)

Honor courses

Remedial courses

Other courses

What is your race or ethnicity? (INTERVIEWER: Only read list if parent doesn’t immediately reply. Are you…?)

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

American Indian or Alaska Native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian or other Pacific Islander
White or Caucasian

Just to be sure we're interviewing a broad range of students for this study, do you participate in the National School Lunch program?

[If the respondent indicates they prefer not to disclose this information, the interviewer should say “That’s fine” and move on to the next question.]

Would you consider the area you live in to be …

 Urban

 Suburban

 Rural

Ok. Thank you and what is the zip code of the area that you live in?

_________________

Thank you for your time. Is the phone number you gave me earlier the best number to call?

Other phone numbers: ____________________________

Is there a good time for me to call? When would be best for you?

Participant Name _______________________________

Date _________________________________________

Time _________________________________________

Appendix CP: EurekaFacts Consent Form for Parents/Legal Guardians of Student Participants (Under Age 18): Reading and Mathematics Tryouts

PARENTAL CONSENT FORM

FOR PARTICIPATION IN RESEARCH

EurekaFacts, LLC

Title of Research Study: Tryouts for NAEP Reading and Mathematics

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting a study with students currently in grade 4, 8 or grade 12 to try out new reading and mathematics test questions for the National Assessment of Educational Progress (NAEP). NCES needs to know if the questions on the test are clear, specifically questions on reading and mathematics. Your child’s input will help NCES improve the test so that students like your child will understand it.

What will happen during the tryout?

Each participant will take an early version of a new NAEP reading and mathematics test on a tablet computer. Participants will be asked to answer the questions as if it were an actual test. Participants will not be graded. In some cases, we may use a camera (also known as an eye-tracker) attached to the computer to record where on the screen students are looking while working.

Where will the tryouts take place?

The tryouts will take place at <NAME OF TRYOUT LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

How long will the tryouts last and when will it take place?

The tryout session will last no more than <60 or 90> minutes and can be scheduled at your convenience. Each tryout requires only one visit to our office.

Whose consent do you require for participation?

For students under the age of 18, we require written informed consent from a parent/legal guardian in order for the eligible student to participate. We also ask the student for his/her verbal consent to participate during the prescreening process. Participation in the study is completely voluntary. The parent/legal guardian or student may withdraw consent for the student’s participation in the study at any time. The student also has the right to refuse to answer particular questions.

Students age 18 or over can sign the consent form directly; but again, participation in the study is completely voluntary and students have the right to withdraw intent to participate and/or refuse to answer particular questions.

Will you keep information private and confidential?

All of your child’s responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. Your child will be assigned a unique student identifier (ID), and at no time will his/her name be linked to any of his/her answers. The tryouts will be audio and video recorded, as long as you agree, and are also used only for research purposes.

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his/her school and will not be linked to any personally identifiable information such as the school’s name.

Are there rewards or incentives for participating?

Yes, the student will receive a $25 gift card (from a major credit card company) from us at the end of his/her tryout. The student’s parent/guardian will also receive a $25 gift card to thank him/her for bringing their child to and from the tryout site.

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at EurekaFacts on behalf of NCES. If you have any questions, please call him/her at <CONTACT PHONE> or by email <CONTACT EMAIL>.

In addition, if transportation to the site is involved, you will also receive a $25 gift card as appreciation for the time bringing your child to and from the research site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: _________________________________________________________________________

Signature: ___________________________________________________________________________

Date:_______________________________________________________________________________

School that student attends: ____________________________________________________________

To be completed by EurekaFacts staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

Appendix CQ: EurekaFacts Consent Form for Student Participants (Age 18 or Over): Reading and Mathematics Tryouts

CONSENT FORM

FOR PARTICIPATION IN RESEARCH

EurekaFacts, LLC

Title of Research Study: Tryouts for NAEP Reading and mathematics

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting a study with students currently in grade 4, 8 or grade 12 to try out new reading and mathematics test questions for the National Assessment of Educational Progress (NAEP). NCES needs to know if the questions on the test are clear, specifically questions on reading and mathematics. Your input will help NCES improve the test so that students like you will understand it.

What will happen during the research interview?

Where will the tryouts take place?

The tryouts will take place at <NAME OF TRYOUT LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION>.

How long will the tryouts last and when will it take place?

The tryout session will last no more than 90 minutes and can be scheduled at your convenience. Each tryout requires only one visit to our office.

Whose consent do you require for participation?

Will you keep information private and confidential?

All of your responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. You will be assigned a unique student identifier (ID), and at no time will your name be linked to any of your answers. The tryouts will be audio and video recorded, as long as you agree, and are also used only for research purposes.

Will the results go to my school?

Your answers will not be disclosed to your school and will not be linked to any personally identifiable information such as the school’s name.

Are there rewards or incentives for participating?

What are the possible risks of being in the study?

There are no known risks to participating in this study.

Can participation end early?

Your participation in this study is completely voluntary. You have the right to refuse to answer particular questions. You may elect to withdraw from this study at any time.

Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this study at EurekaFacts on behalf of NCES. If you have any questions, please call him/her at <CONTACT PHONE> or by email <CONTACT EMAIL>.

By signing below, you agree that you will participate in this study. You will receive a $25 gift card (from a major credit card company) at the conclusion of the session.

In addition, if transportation to the site is involved, your parent/guardian will also receive a $25 gift card as appreciation for the time bringing you to and from the research site.

Your signature below will indicate that your questions have been answered satisfactorily, and that you have read and understood the information provided above.

Print Name: _________________________________________________________________________

Signature: ___________________________________________________________________________

Date:_______________________________________________________________________________

School that you attend: ____________________________________________________________

To be completed by EurekaFacts staff:

Student Payment Card Transaction Number: ________________________________________________________

Parent/Guardian Payment Card Transaction Number: _________________________________________________

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	NATIONAL CENTER FOR EDUCATION STATISTICS
Author	joc
File Modified	0000-00-00
File Created	2021-01-23