National Assessment of Educational Progress (NAEP) Background Questions Cognitive Interviews for Students, Teachers, and School Administrators for NAEP 2013

An interviewer, on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting interviews with <fourth-/eighth-/twelfth-> grade students about survey questions for the National Assessment of Educational Progress (NAEP). The interviews will take no longer than <30/60/90> minutes. The purpose of these interviews is not to test or grade students. We want to hear their thoughts about the survey instructions and questions to improve how the survey is administered and make the questions easier to understand.

The interviewer will meet one-on-one with each student to ask for his or her input. These sessions will be recorded for the researchers on this study to review. No one else will see or hear the recording. Your child’s answers may be used only for statistical purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573].

These interviews are an important stage of the NAEP survey development process. The comments that students provide help NCES to improve the survey questions and directions to make the best survey possible.

We hope that you will give your consent for your son or daughter to participate in the NAEP interview by signing this form. Without your consent, your child will not be able to participate in the interview.

Respectfully,

Donnell Butler

Assessment Division

Educational Testing Service

I hereby give my permission for _____________________________________________ to

(Student’s Name)

participate in the NAEP survey interview.

____________________________________ ________________________

Signature Date

Appendix B: Legal Guardian Consent Form (Parent Recruited)

Dear Parent or Legal Guardian: <DATE>

These interviews are an important stage of the NAEP survey development process. The comments students provide help NCES improve the survey questions and directions to make the best survey possible. Your child will be given a $25 major gift card as thanks for their time.

Respectfully,

Donnell Butler

Assessment Division

Educational Testing Service

I hereby give my permission for _____________________________________________ to

(Student’s Name)

participate in the NAEP survey interview.

____________________________________ ________________________

Signature Date

Appendix C: Adult (18 or older) Student Consent Form (School Recruited)

Dear Student: <DATE>

An interviewer, on behalf of The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting interviews with students about survey questions for the National Assessment of Educational Progress (NAEP). The interviews will take no longer than <30/60/90> minutes.

NCES needs to know if the survey instructions and questions are clear. Your ideas will help NCES improve the survey so students like you understand all the questions. You will be given questions that are very similar to the actual questions that will be asked during the national testing. The interviewer will pay very close attention to your opinions and feelings about all aspects of the survey. He or she will also ask what you think about the questions after you complete them.

You will not be judged on how well you answer the questions, and you will not receive a grade. Other students like you will also participate in one-on-one sessions just like these. These sessions will be recorded for the researchers on this study to review. No one else will see or hear the recording. Your answers may be used only for statistical purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573].

These interviews are an important stage of the NAEP survey development process. Your comments, along with the comments of other students, will help NCES improve the survey. We thank you for your voluntary participation and ask you to please sign the form below.

Thank you,

Donnell Butler

Assessment Division

Educational Testing Service

I hereby accept the terms described above and agree to participate voluntarily in the interview.

__________________________________

Print Name

____________________________________ ________________________

Signature Date

Appendix D: Adult (18 or older) Student Consent Form (Parent or Student Recruited)

Dear Student: <DATE>

These interviews are an important stage of the NAEP survey development process. Your comments, along with the comments of other students, will help NCES improve the survey. In addition, you will be given a $25 gift card in compensation for your time and effort. We thank you for your voluntary participation and ask you to please sign the form below.

Thank you,

Donnell Butler

Assessment Division

Educational Testing Service

I hereby accept the terms described above and agree to participate voluntarily in the interview.

__________________________________

Print Name

____________________________________ ________________________

Signature Date

Appendix E: Teacher and School Administrator Consent Form

Dear Educator: <DATE>

An interviewer, on behalf of The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting interviews with teachers and school administrators about survey questions for the National Assessment of Educational Progress (NAEP). The interviews will take no longer than <60/90> minutes.

NCES needs to know if the survey instructions and questions are clear. Your ideas will help NCES improve the survey so educators like you understand all the questions. You will be given questions that are very similar to the actual questions that will be asked of teachers and school administrators whose students participate in future NAEP assessments. The interviewer will pay very close attention to your opinions and feelings about all aspects of the survey. He or she will also ask what you think about the questions after you complete them.

Other educators like you will also participate in one-on-one sessions just like these. These sessions will be recorded for the researchers on this study to review. No one else will hear the recording, or see your responses or comments. Your answers may be used only for statistical purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573].

These interviews are an important stage of the NAEP survey development process. Your comments, along with the comments of other educators, will help NCES improve the survey. In addition, you will be given a $40 gift card in compensation for your time and effort. We thank you for your voluntary participation and ask you to please sign the form below.

Thank you,

Donnell Butler

Assessment Division

Educational Testing Service

I hereby accept the terms described above and agree to participate voluntarily in the interview.

__________________________________

Print Name

____________________________________ ________________________

Signature Date

Appendix F: Interviewer / Researcher Confidentiality Agreement

Agreement: Item and Test Security Requirements for National Assessment of Educational Progress (NAEP) Questionnaires

Under this agreement, you will be reviewing, creating, or handling secure test material belonging to the National Assessment of Educational Progress (NAEP) and the National Center for Educational Statistics (NCES) in the United States Department of Education. These survey materials are confidential and may not be discussed with any person who has not signed this agreement. Any materials associated with the NAEP questionnaires may not be copied, published, announced or in any other way be made public, and you may not grant permission for anyone else to share any information. If you are creating secure questionnaire material under this agreement, for purposes of copyright, it is understood and agreed that any material produced will be considered a work made for hire. As such, the Department of Education or its designee will be the sole and exclusive owner of the material you produce and will copyright thereon in the United States and throughout the world, and you will retain no rights in such materials.

By signing this agreement, you acknowledge that the questionnaire materials are subject to this agreement and are proprietary and confidential materials of the U.S. Department of Education. You further understand that any disclosure, unauthorized use or reproduction of the materials would damage the confidentiality of NAEP. By entering into this agreement, you agree to comply with the following:

Confidential survey materials must be only used for the sole purpose of performing professional services for NCES. Any other use of these materials would require prior written consent of NCES.
Confidential materials must be kept in a locked container when they are not being used.
Confidential materials must not be shared with any parties who have not signed this agreement.
Confidential survey materials must not be copied in part or whole, and all copies of materials must be returned promptly to the U.S. Department of Education or its designee upon task completion.
In the event that confidential survey materials are lost, stolen, or an unauthorized party has gained access to these materials, the U.S. Department of Education must be notified immediately.

ACCEPTED AND AGREED TO THE REQUIREMENTS

NAME:___________________________________________________________

SIGNATURE:______________________________________________________

DATE:____________________________________________________________

Appendix G: Script for Recruiting Schools: Existing Subject-Specific

[If E-mail] Subject: U.S. Department of Education needs your input

Dear <NAME>;

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting interviews with students, teachers and school administrators (i.e., principals or assistant principals) about new survey questions for the National Assessment of Educational Progress (NAEP). Before administering the new survey questions, NCES needs to know if the survey instructions and questions are clear. Your assistance will help NCES improve the survey.

Respondents will be asked to answer several survey questions and describe their thinking as they answer the question. NCES simply wants to know how people would answer the questions if they were included in an actual survey. NCES wants to hear the thoughts of students, teachers, and school administers about the survey instructions and questions to improve how the survey is administered and make the questions easier to understand. All answers may be used only for statistical purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573].

To thank you for your time, your school will receive a $25 gift card per student interview conducted. Your school will also receive a $25 gift card for each adult non-participant observer (such as a teacher assistant) provided during the student interviews. Participating teachers and school administrators will receive a $40 gift card.

Participant requirements

NCES needs the following interviewees from each participating school: two students (one student must be a boy and one a girl, and for sample diversity purposes one of the students must be non-white), one adult non-participant observer for student interviews, one teacher, and one school administrator (principal or assistant principal). The interviews would take place at your school.

Study room requirements

The interviews will be individually administered and should be conducted in a small, quiet, private room. The room for the interviews must accommodate at least 3 individuals (i.e., interviewee, interviewer, and observer), be private enough that the participant can feel comfortable speaking aloud, and quiet enough that the digital recorder does not pick up extraneous noise.

Time requirements

Student and teacher interviews will last approximately 30 minutes to 1 hour. School administrator interviews will last approximately 1 hour. We are scheduling data collection to occur from October 4 through December 9. If you choose to participate, an interviewer will come to your school to conduct each interview at a time convenient for you, the students, observer, teacher, administrator, and your school’s schedule. The interviewer will be available to conduct the interviews all school day and/or after school.

These interviews are an important stage of the NAEP survey development process. Your comments, along with the comments of other students, teachers and school administrators, will help NCES improve the survey.

If you are interested in participating in this research project, please call [phone number] or e-mail [e-mail address].

Thank you,

Donnell Butler

Assessment Division

Educational Testing Service

Appendix H: Script Directly Recruiting Parents of Student Participants: Existing Subject-Specific

[If E-mail] Subject: U.S. Department of Education needs your child’s input

Dear <NAME>;

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting interviews with <fourth-/eighth-/twelfth- > grade students about new survey questions for the National Assessment of Educational Progress (NAEP). Before giving the new survey to all students, NCES needs to know if the survey instructions and questions are clear. Your input will help NCES improve the survey so students like your child will understand all the questions.

The interviews will take place at <LOCATION> and will last approximately 30 minutes to 1 hour. Your child will be asked to respond to several survey questions. NCES simply wants to know how he or she would answer the questions if it were an actual survey that was given to students after taking a NAEP test. All of your child’s answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573].

If your child is interested in participating in this research project, please call [phone number] or e-mail [e-mail address].

Thank you,

Donnell Butler

Assessment Division

Educational Testing Service

Appendix I: Script for Directly Recruiting Parents of Student Participants: TEL

[If E-mail] Subject: Seeking Student Participants for Education Research

Dear <NAME>;

We are recruiting students who are currently in the 8^th grade to participate in research interviews supporting the development of new survey questions for the National Assessment of Educational Progress (NAEP), which will be administered to future 8^th graders throughout the U.S. The research interviews will explore how students work through the survey questions to ensure that the questions are clear and understandable to a broad range of middle school students. Your child will be given a $25 gift card as thanks for their time. In addition, you will be given a $25 gift card for your time to bring the participating student to and from the cognitive laboratory site.

The interviews will take place at <LOCATION> and will last no more than 90 minutes. During the interview, your child will be asked to respond to several survey questions. NCES simply wants to know how he or she would answer the questions if they were included as an actual survey given to students after taking a NAEP test. All of your child’s answers will be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573].

These interviews are an important stage of the NAEP survey development process. The comments students provide help NCES improve the survey questions and directions to create the best survey possible.

If your child is interested in participating in this research project, please call [phone number] or e-mail [e-mail address]. We look forward to speaking with you soon!

Thank you,

EurekaFacts, LLC

Appendix J: Phone Script for Recruiting Parents of Student Participants: Existing Subject-Specific

Good morning/afternoon. May I please speak to <NAME>?

Good morning/afternoon. My name is <INTERVIEWER>. I’m calling from Educational Testing Service on behalf of the National Center for Educational Statistics or NCES, part of the U.S. Department of Education. [Interviewer: If this is a follow-up call based on a response to the e-mail or other outreach, then skip to the Screener after reading the following: Thank you for contacting us about the possibility of your child participating in a project to help develop survey questions for NAEP.]

We are recruiting students to participate in research interviews supporting the development of new survey questions for the National Assessment of Educational Progress (NAEP). The survey will be administered to future students throughout the U.S. The research interviews will explore how students work through the survey questions to ensure that they are clear and understandable to a broad range of students. The students who qualify for and participate in the study will be given a $25 gift card as thanks for their time. In addition, you will be given a $25 gift card for your time to bring the participating student to and from the interview site.

The interviews will take place at <LOCATION> and will last about 30 minutes to 1 hour. During the interview, your child will be asked to respond to several survey questions. NCES simply wants to know how he or she would answer the questions if they were included on an actual survey given to students after taking a NAEP test.

These interviews are an important stage of the NAEP survey development process. The comments students provide help NCES improve the survey questions and directions to create the best survey possible.

Would your child be interested in participating in this research project?

[If No] Thank you for taking the time to speak with us today about this important project. Have a wonderful day/evening.

[If Yes] Great, thank you very much. To ensure that we interview a broad mix of students, I have a few brief screening questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to be certain he or she is a good fit for participating.

Screener

[Interviewer: Determine recruitment needs in advance. Record all responses. End screener at whatever point parent gives a response that confirms that their group has already been sampled adequately. End screener script: Based on the requirements of this study, we are not able to include you in the interviews at this time.]

What is your child’s name?
Is <STUDENT NAME> a boy or a girl?
What grade is <STUDENT NAME> currently in?
Is <STUDENT NAME> either Hispanic or Latino origin?
Is <STUDENT NAME>:

American Indian or Alaska Native
Asian
Black or African American
Native Hawaiian or Other Pacific Islander, or
White?

You may select one or more

What is the highest level of education that you have completed?

Less than high school degree
High school degree or GED
Some college
Trade school degree
Associate’s or Bachelor’s degree, or
Post graduate degree?

What is your current zip code?
And has <STUDENT NAME> said <SHE, HE> would be interested in participating in the interview session which will take 30 minutes to 1 hour?

Eligible Participant Script

Thank you for taking the time to answer the screening questions. Your child is eligible to participate in the research interviews.

In the session, we will be asking <STUDENT NAME> to respond to several survey questions. NCES needs to make sure that students will understand what the survey is asking them to do.

Participation is entirely voluntary, and your child’s answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. Interviewer note: this reference is for informational purposes only. If asked what law is being referenced, please provide the information contained within this bracketed text.]

Your child’s input will help NCES improve the survey that will be given nationwide. To thank <STUDENT NAME>, <SHE, HE> will receive a $25 gift card. In addition, you will receive a $25 gift card for your time to bring the <STUDENT NAME> to and from the interview site.

Again, the interviews will last about 30 minutes to 1 hour and will be held at <LOCATION>. Do you think <STUDENT NAME> would like to participate?

[If No] Thank you for taking the time to speak with us today about this important project.

[If Yes] And would you be willing to sign a form indicating your approval for <STUDENT NAME> to participate?

[If No] Thank you for taking the time to speak with us today about this important project.

[If Yes] Are you and your child available for the interview at <TIME> on <DATE>?

[Interviewer: if necessary, provide alternate times/dates to find an available time/date for interview and then record time/date of appointment]

Thank you. We will need to send you a confirmation e-mail and letter that includes the directions, address, and interview consent form. What e-mail and mailing address would you prefer us to use to send you this information?

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time?

[Interviewer: Answer any questions.]

If you or <STUDENT NAME> have any questions before your interview scheduled for <DATE/TIME>, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this very important research project.

Appendix K: Phone Script for Recruiting Parents of Student Participants: TEL

Good morning/afternoon. May I please speak to <NAME>?

Good morning/afternoon. My name is <INTERVIEWER>. I’m calling from EurekaFacts on behalf of the National Center for Educational Statistics or NCES, part of the U.S. Department of Education. [Interviewer: If this is a follow-up call based on a response to the e-mail or other outreach, then skip to the Screener after reading the following: Thank you for contacting us about the possibility of your child participating in a project to help develop survey questions for NAEP. To ensure that we interview a broad mix of students, I have a few brief screening questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to identify if <she/he> meets the participation requirements.]

We are recruiting students who are currently in the 8th grade to participate in research interviews supporting the development of new survey questions for the National Assessment of Educational Progress (NAEP). The survey will be administered to future 8th graders throughout the U.S. The research interviews will explore how students work through the survey questions to ensure that they are clear and understandable to a broad range of middle school students. The student who qualifies for and participates in the study will be given a $25 gift card as thanks for their time. In addition, you will be given a $25 gift card for your time to bring the participating student to and from the interview site.

The interviews will take place at <LOCATION> and will last no more than 90 minutes. During the interview, your child will be asked to respond to several survey questions. NCES simply wants to know how he or she would answer the questions if they were included on an actual survey given to students after taking a NAEP test.

Would your child be interested in participating in this research project?

[If No] Thank you for taking the time to speak with us today about this important project. Have a wonderful day/evening.

[If Yes] Great, thank you very much. To ensure that we interview a broad mix of students, I have a few brief screening questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to identify if <she/he> meets the participation requirements.

Screener

1. What is your child’s name?

2. Is <STUDENT NAME> a boy or a girl?

3. What grade is <STUDENT NAME> currently in?

4. Is <STUDENT NAME> either Hispanic or Latino origin?

5. Is <STUDENT NAME>:

a. American Indian or Alaska Native

b. Asian

c. Black or African American

d. Native Hawaiian or Other Pacific Islander, or

e. White?

You may select one or more

6. Has <STUDENT NAME> taken any classes in school or out of school related to technology or engineering?

7. What is the highest level of education that you have completed?

a. Less than high school degree

b. High school degree or GED

c. Some college

d. Trade school degree

e. Associate’s or Bachelor’s degree, or

f. Post graduate degree?

8. What is your current zip code?

And has <STUDENT NAME> said <SHE, HE> would be interested in participating in the interview session, which will take 60 to 90 minutes?

Eligible Participant Script

Thank you for taking the time to answer the screening questions. You are eligible to participate in the research interviews.

In the session, we will be asking <STUDENT NAME> to respond to several survey questions. NCES needs to make sure that students will understand what the survey is asking them to do.

Participation is entirely voluntary and your child’s answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. Interviewer note: this reference is for informational purposes only. If asked what law is being referenced, please provide the information contained within this bracketed text.]

Your child’s input will help NCES improve the survey that will be asked nationwide. To thank <STUDENT NAME>, <SHE, HE> will receive a $25 gift card. In addition, you will receive a $25 gift card for your time to bring the participating student to and from the interview site.

Again, the interviews will last about 60 to 90 minutes and will be held at <LOCATION>. Do you think <STUDENT NAME> would like to participate?

[If No] Thank you for taking the time to speak with us today about this important project.

[If Yes] And would you be willing to sign a form indicating your approval for <STUDENT NAME> to participate?

[If No] Thank you for taking the time to speak with us today about this important project.

[If Yes] Are you and your child available for the interview at <TIME> on <DATE>?

[Interviewer: if necessary, provide alternate times/dates to find an available time/date for interview and then record time/date of appointment]

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time?

[Interviewer: Answer any questions.]

If you or <STUDENT NAME> have any questions before your interview scheduled for <DATE/TIME>, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this very important research project.

Appendix L: Script for Directly Recruiting Teachers and School Administrators: Existing Subject-Specific

[If E-mail] Subject: U.S. Department of Education needs your input

Dear <NAME>;

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting interviews with teachers and school administrators about new survey questions for the National Assessment of Educational Progress (NAEP). Before administering the new survey questions, NCES needs to know if the survey instructions and questions are clear. Your input will help NCES improve the survey so educators like you will understand all the questions.

The interviews will take place <at LOCATION / by phone> and will last approximately 30–60 minutes. You will be asked to respond to survey questions. NCES simply wants to know how you would answer them if it was an actual survey. NCES wants to hear your thoughts about the survey instructions and questions to improve how the survey is administered and make the questions easier to understand. All of your answers may be used only for statistical purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. To thank you for your time, you will receive a $40 gift card.

These interviews are an important stage of the NAEP survey development process. Your comments, along with the comments of other educators, will help NCES improve the survey.

If you are interested in participating in this research project, please call [phone number] or e-mail [e-mail address].

Thank you,

Donnell Butler

Assessment Division

Educational Testing Service

Appendix M: Script for Directly Recruiting School Administrators: TEL

[If E-mail] Subject: U.S. Department of Education needs your input

Dear <NAME>;

We are currently recruiting school administrators to participate in research interviews supporting the development of new survey questions for the National Assessment of Educational Progress (NAEP) and would like to invite you or another member of your school’s administration to participate in this research effort.

The interviews will explore how school administrators work through the survey questions to ensure that they are clear and understandable to a broad range of educators. To thank participants for their time, each participant will receive a $40 gift card.

The interviews will take place at <LOCATION / or by phone> and will last approximately 60–90 minutes. During the interview, you will be asked to respond to several survey questions. NCES simply wants to know how you would answer them if it was an actual survey. Additionally, NCES wants to hear your thoughts about the survey instructions and questions to improve how the survey is administered and make the questions easier to understand. All of your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573].

These interviews are an important stage of the NAEP survey development process. Your comments, along with the comments of other educators, will help NCES improve the survey.

If you are interested in participating in this research project, please call [phone number] or e-mail [e-mail address].

Thank you,

EurekaFacts, LLC

Appendix N: Phone Script for Recruiting Teachers and School Administrators: Existing Subject-Specific

Good morning/afternoon. May I please speak to <NAME>?

Good morning/afternoon. My name is <INTERVIEWER NAME> from Educational Testing Service calling on behalf of the National Center for Educational Statistics or NCES, part of the U.S. Department of Education. The National Assessment of Educational Progress (NAEP) has developed new survey questions for teachers and school administrators. Before giving the new survey to all teachers and school administrators whose students participate in a NAEP assessment, NCES needs to know if the survey instructions and questions are clear. Your input will help NCES improve the survey so educators like you understand all the questions and the things you are being asked to do. To thank participants for their time, each participant will receive a $40 gift card.

The interviews will take place <at LOCATION / by phone> and will last approximately 30–60 minutes. You will be asked about the survey instructions and to respond to several questions. NCES simply wants to know how you would answer them if it was an actual survey. NCES wants to hear your thoughts about the survey instructions and questions to improve how the survey is administered and make the questions easier to understand. All your answers may be used only for statistical purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. Interviewer note: this reference is for informational purposes only. If asked what law is being referenced, please provide the information contained within this bracketed text.]

These interviews are an important stage of the NAEP survey development process. Your comments, along with the comments of other teachers and school administrators, will help NCES improve the survey.

Are you interested in participating in this research project?

The interview will last approximately 30 minutes – 1 hour. It will take place <at LOCATION / by phone>.

Do you have any questions at this time?

[Answer any questions.]

Are you available for the interview at <TIME> on <DATE>? If you do have any questions before then, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this very important research project.

Appendix O: Phone Script for Recruiting School Administrators: TEL

Good morning/afternoon. May I please speak to <NAME>?

Good morning/afternoon. My name is <INTERVIEWER>. I’m calling from EurekaFacts on behalf of the National Center for Educational Statistics or NCES, part of the U.S. Department of Education.

We are currently recruiting school administrators to participate in research interviews supporting the development of new survey questions for the National Assessment of Educational Progress (NAEP). The research interviews will explore how school administrators whose students participate in a NAEP assessment work through the survey. We want to ensure that the questions and instructions are clear and understandable to a broad range of educators. To thank participants for their time, each participant will receive a $40 gift card.

The interviews will take place <at LOCATION / by phone> and will last approximately 60–90 minutes . During the interview, you will be asked to respond to several survey questions. NCES simply wants to know how you would answer them if it was an actual survey. Additionally, NCES wants to hear your thoughts about the survey instructions and questions in order to improve how the survey is administered and make the questions easier to understand. All of your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573]. Interviewer note: this reference is for informational purposes only. If asked what law is being referenced, please provide the information contained within this bracketed text.]

These interviews are an important stage of the NAEP survey development process. Your comments, along with the comments of other educators, will help NCES improve the survey.

Are you interested in participating in this research project?

If no: Thank you for your time. Have a wonderful day/evening.

If yes: Great, thank you very much. To ensure that we interview a broad mix of educators, I have a few brief screening questions to ask you. It should only take a few minutes of your time.

Screener

[Interviewer: Determine recruitment needs in advance. Record all responses. End screener at whatever point administrator gives a response that confirms that their group has already been sampled adequately. End screener script: Based on the requirements of this study, we are not able to include you in the interviews at this time.]

At what school are you an administrator?

Are you at a school with a population that includes eighth-grade students?
1. If yes  CONTINUE
2. If no  END SCREENER (Based on the requirements of this study, we are not able to include you in the interviews at this time. We greatly appreciate your help. Thank you for your time.)

What is the size of the student population at your school?
1. 299 students or less (recruit ~5)
2. 300-599 students (recruit ~5)
3. 600 students or more (recruit ~5)

Does your school offer technology and/or design/engineering courses for eighth-grade students?
1. Yes (recruit ~5)
2. No

Eligible Participant Script

Thank you for taking the time to answer the screening questions. You are eligible to participate in the research interviews. Again, the interview will last about 1 -1 ½ hours and will be held at <LOCATION>. Are you available for the interview at <TIME> on <DATE>?

[Interviewer: record time/date of appointment]

Thank you. We will need to send you a confirmation e-mail that includes the directions, address, and interview consent form. What e-mail address would you prefer us to use to send you this information?

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time?

[Interviewer: Answer any questions.]

If you do have any questions before your interview scheduled for <DATE/TIME>, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this very important research project.

Appendix P: Script and Parent Flyer for Recruiting Participants via Organizational Outreach: TEL

[If E-mail] Subject: Seeking Student Participants for Education Research

Hello,

My name is <FIELD MANAGER NAME> and I am contacting you from EurekaFacts, a local research company conducting an important study for the U.S. Department of Education’s National Center for Education Statistics (NCES). As a <ORGANIZATION PROFILE>, we are contacting you to ask for your help in spreading the word about this exciting research opportunity.

We are currently recruiting 8^th grade students to participate in paid research interviews supporting the development of a new National Assessment of Educational Progress (NAEP) technology and engineering literacy skills assessment that will be administered to future 8^th graders throughout the U.S.

We would like to ask for your assistance in letting parents and students know about how they can help shape this national test. The research interviews will explore how students work through the test to ensure that the questions are clear and understandable to a broad range of middle school students. The student who qualifies for and participates in the study will be given a $25 gift card as thanks for their time. In addition, parents will be given a $25 gift card for their time to bring their child to and from the interview site.

I want to assure you that this research incorporates very strict confidentiality and privacy standards. Students’ answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573].

I have included a short description of this research study below in the hope that you might consider passing this along to parents and guardians of 8^th grade students at your school.

Thanks again for your consideration. Please do not hesitate to contact me with any questions regarding our firm or this project.

EurekaFacts, LLC

EurekaFacts LLC

451 Hungerford Drive, Suite 515

Rockville, Maryland, 20850

Tel: (240) 403-4800

Toll Free (866) My-Facts

Fax: (301) 610-0640

Web: wwwEurekaFacts.com

Information for Parents

U.S. Department of Education 8^th Grader Educational Research Study

We are a local educational research company conducting an important study for the U.S. Department of Education. If your child is in the 8th grade we need his or her help to evaluate questions associated with a new national educational test aimed at middle school students.

What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting interviews with eighth-grade students about new test items for the National Assessment of Educational Progress (NAEP).

Where and when will these interviews be held?

The interviews will take place at <LOCATION> and will last no more than 90 minutes. During the interview, your child will be asked to respond to several survey questions aimed at better understanding students’ use of technology and computers. NCES simply wants to know how students would answer the questions if they were included as an actual survey given to students after taking a NAEP test. Your child’s input will help improve this assessment by ensuring that its content will be understood by all students.

Will you keep information private?

These sessions will be recorded. All of your child’s answers will be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573].

Will participants receive compensation?

These interviews are an important stage of the NAEP test development process. The comments students provide help NCES improve the test items to make the best test possible. If selected, your child will be given a $25 gift card as thanks for their time. In addition, parents will be given a $25 gift card for their time to bring the participating student to and from the interview site.

How do we sign up?

If you and your child are interested in participating in this research project, please call or text us at 240-86 FACTS (240-863-2287) or e-mail us at panel@eurekafacts.com.

If you e-mail or text us, please provide your name, preferred e-mail address or telephone number, and the best time to reach you, and we will respond promptly.

Appendix Q: Thank You Letter to Student Participants

<DATE>

Dear <Name>;

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, I would like to thank you for your participation in our survey research study. Thanks to you, we are better able to develop the best possible survey questionnaires for the National Assessment of Educational Progress (NAEP).

Your assistance helped us better understand how students approach both the survey questions and directions. Your open, candid, and insightful comments were very helpful.

Again, thank you for your assistance.

Sincerely,

Donnell Butler

Assessment Division

Educational Testing Service

Appendix R: Thank You Letter to Parents of Non-Adult (Age 17 or younger) Student Participants

<DATE>

Dear <Name>;

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, thank you for allowing your child to participate in our survey research study. Your child’s input will help us develop the best possible survey questionnaires for the National Assessment of Educational Progress (NAEP).

The assistance provided by your child helped us to better understand how students approach both the survey questions and directions. Your child’s input, combined with input from other students, was incredibly useful and informative.

Again, thank you for your assistance.

Sincerely,

Donnell Butler

Assessment Division

Educational Testing Service

Appendix S: Thank You Letter to Teachers and School Administrators

<DATE>

Dear <Name>;

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, I would like to thank you for your participation in our research study. Thanks to you, we are better able to develop the best possible survey questionnaires for the National Assessment of Educational Progress (NAEP).

Your assistance helped us better understand how <teachers/school administrators> approach both the survey questions and directions. Your open, candid, and insightful comments were very valuable. Your input, combined with input from other <teachers/school administrators> like you, was incredibly useful and informative.

Again, thank you for your assistance.

Sincerely,

Donnell Butler

Assessment Division

Educational Testing Service

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Appendices (A-R)
Author	mgisbert
File Modified	0000-00-00
File Created	2021-01-31

National Assessment of Educational Progress (NAEP) Background Questions Cognitive Interviews for Students, Teachers, and School Administrators for NAEP 2013

NCES Cognitive, Pilot, and Field Test Studies System

Appendix A-S NAEP Background Cog Labs for 2013