National Assessment of Educational Progress (NAEP) Core Background Questions Cognitive Interviews for Students, Teachers, and School Administrators

An interviewer from Abt Associates, on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting interviews with <fourth-/eighth-/twelfth-> grade students about survey questions for the National Assessment of Educational Progress (NAEP). The interviews will take about <60/90/90> minutes. The purpose of these interviews is not to test or grade students. We want to hear their thoughts about the survey instructions and questions to improve how the survey is administered and make the questions easier to understand.

The interviewer will meet one-on-one with each student to ask for his or her input. These sessions will be recorded for the researchers on this study to review. No one else will see or hear the recording. Your child’s answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

These interviews are an important stage of the NAEP test development process. The comments students provide help NCES improve the survey questions and directions to make the best test possible. Your child will be given a $25 gift card as thanks for their time. In addition, you will be given a $25 gift card for your time to bring the participating student to and from the cognitive laboratory site.

We hope that you will give your consent for your son or daughter to participate in the NAEP interview by signing this form. Without your consent, your child will not be able to participate in the interview.

Respectfully,

Johnny Blair

Principal Scientist

Abt Associates, Inc.

I hereby give my permission for _____________________________________________ to

(Student’s Name)

participate in the NAEP survey interview.

____________________________________ ________________________

Signature Date

Appendix B: Adult Consent Form—Students

Dear Student: <DATE>

An interviewer from Abt Associates, on behalf of The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting interviews with students about survey questions for the National Assessment of Educational Progress (NAEP). The interviews will take about 90 minutes.

NCES needs to know if the survey instructions and questions are clear. Your ideas will help NCES improve the survey so students like you understand all the questions. You will be given questions that are very similar to the actual questions that will be asked during the national testing. The interviewer will pay very close attention to your opinions and feelings about all aspects of the survey. He or she will also ask what you think about the questions after you complete them.

You will not be judged on how well you answer the questions, and you will not receive a grade. Other students like you will also participate in one-on-one sessions just like these. These sessions will be recorded for the researchers on this study to review. No one else will see or hear the recording. Your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

These interviews are an important stage of the NAEP test development process. Your comments, along with the comments of other students, will help NCES improve the survey. In addition, you will be given a $25 gift card in compensation for your time and effort. If your parent will bring you to and from the cognitive laboratory site,, your parent will also receive a $25 gift card for their time and effort We thank you for your voluntary participation and ask you to please sign the form below.

Thank you,

Johnny Blair

Principal Scientist

Abt Associates, Inc.

I hereby accept the terms described above and agree to participate voluntarily in the interview.

__________________________________

Print Name

____________________________________ ________________________

Signature Date

Appendix C: Adult Consent Form—Teachers and School Administrators

Dear Educator: <DATE>

An interviewer from Abt Associates, on behalf of The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting interviews with <teachers/school administrators> about survey questions for the National Assessment of Educational Progress (NAEP). The interviews will take about <90/120> minutes.

NCES needs to know if the survey instructions and questions are clear. Your ideas will help NCES improve the survey so <teachers/school administrators> like you understand all the questions. You will be given questions that are very similar to the actual questions that will be asked of <teachers/school administrators> whose students participate in future NAEP assessments. The interviewer will pay very close attention to your opinions and feelings about all aspects of the survey. He or she will also ask what you think about the questions after you complete them.

Other <teachers/school administrators> like you will also participate in one-on-one sessions just like these. These sessions will be recorded for the researchers on this study to review. No one else will hear the recording, or see your responses or comments. Your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

These interviews are an important stage of the NAEP test development process. Your comments, along with the comments of other <teachers/school administrators>, will help NAEP improve the survey. In addition, you will be given a $40 gift card in compensation for your time and effort. We thank you for your voluntary participation and ask you to please sign the form below.

Thank you,

Donnell Butler

Assessment Division

Educational Testing Service

I hereby accept the terms described above and agree to participate voluntarily in the interview.

__________________________________

Print Name

____________________________________ ________________________

Signature Date

Appendix D: Interviewer Confidentiality Agreement

Agreement: Item and Test Security Requirements for National Assessment of Educational Progress (NAEP) Questionnaires

Under this agreement, you will be reviewing, creating, or handling secure test material belonging to the National Assessment of Educational Progress (NAEP) and the National Center for Educational Statistics (NCES) in the United States Department of Education. These survey materials are confidential and may not be discussed with any person who has not signed this agreement. Any materials associated with the NAEP questionnaires may not be copied, published, announced or in any other way be made public, and you may not grant permission for anyone else to share any information. If you are creating secure questionnaire material under this agreement, for purposes of copyright, it is understood and agreed that any material produced will be considered a work made for hire. As such, the Department of Education or its designee will be the sole and exclusive owner of the material you produce and will copyright thereon in the United States and throughout the world and you will retain no rights in such materials.

By signing this agreement, you acknowledge that the questionnaire materials are subject to this agreement and are proprietary and confidential materials of the U.S. Department of Education. You further understand that any disclosure, unauthorized use or reproduction of the materials would damage the confidentiality of NAEP. By entering into this agreement, you agree to comply with the following:

Confidential survey materials must be only used for the sole purpose of performing professional services for NCES. Any other use of these materials would require prior written consent of NCES.
Confidential materials must be kept in a locked container when they are not being used.
Confidential materials must not be shared with any parties who have not signed this agreement.
Confidential survey materials must not be copied in part or whole, and all copies of materials must be returned promptly to the U.S. Department of Education or its designee upon task completion.
In the event that confidential survey materials are lost, stolen or an unauthorized party has gained access to these materials, the U.S. Department of Education must be notified immediately.

ACCEPTED AND AGREED TO THE REQUIREMENTS

NAME:___________________________________________________________

SIGNATURE:______________________________________________________

DATE:____________________________________________________________

Appendix E: Researcher Confidentiality Agreement

Agreement: Item and Test Security Requirements for National Assessment of Educational Progress (NAEP) Questionnaires

Under this agreement, you will be reviewing, creating, or handling secure test material belonging to the National Assessment of Educational Progress (NAEP) and the National Center for Educational Statistics (NCES) in the United States Department of Education. These survey materials are confidential and may not be discussed with any person who has not signed this agreement. Any materials associated with the NAEP questionnaires may not be copied, published, announced or in any other way be made public, and you may not grant permission for anyone else to share any information. If you are creating secure test material under this agreement, for purposes of copyright, it is understood and agreed that any material produced will be considered a work made for hire. As such, the Department of Education or its designee will be the sole and exclusive owner of the material you produce and will copyright thereon in the United States and throughout the world, and you will retain no rights in such materials.

By signing this agreement, you acknowledge that the questionnaire materials are subject to this agreement are proprietary and confidential materials of the U.S. Department of Education. You further understand that any disclosure, unauthorized use or reproduction of the materials would damage the confidentiality of NAEP. By entering into this agreement, you agree to comply with the following:

Confidential survey materials must be only used for the sole purpose of performing professional services for NCES. Any other use of these materials would require prior written consent of NCES.
Confidential materials must be kept in a locked container when they are not being used.
Confidential materials must not be shared with any parties who have not signed this agreement.
Confidential survey materials must not be copied in part or whole, and all copies of materials must be returned promptly to the U.S. Department of Education or its designee upon task completion.
In the event that confidential survey materials are lost, stolen or an unauthorized party has gained access to these materials, the U.S. Department of Education must be notified immediately.

ACCEPTED AND AGREED TO THE REQUIREMENTS

NAME:___________________________________________________________

SIGNATURE:______________________________________________________

DATE:____________________________________________________________

Appendix F: E-mail for Recruiting Parents of Student Participants

E-mail subject line: U.S. Department of Education needs your input

Dear <NAME>;

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting interviews with fourth-, eighth-, and twelfth-grade students about new survey questions for the National Assessment of Educational Progress (NAEP). Before giving the new survey to all students, NCES needs to know if the survey instructions and questions are clear. Your input will help NCES improve the survey so students like your child will understand all the questions.

The interviews will take place in <LOCATION> and will last approximately <60/90/90> minutes. Your child will be asked to respond to several survey questions. NCES simply wants to know how they would answer the questions if it were an actual survey that was given to students after taking a NAEP test. All of your child’s answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

If your child is interested in participating in this research project, please call [phone number] or e-mail [e-mail address].

Thank you,

Johnny Blair

Principal Scientist

Abt Associates, Inc.

Appendix G: E-mail for Recruiting Adult Participants—Teachers and School Administrators

E-mail subject line: U.S. Department of Education needs your input

Dear <NAME>;

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting interviews with <teachers/school administrators> about new survey questions for the National Assessment of Educational Progress (NAEP). Before administering the new survey questions, NCES needs to know if the survey instructions and questions are clear. Your input will help NCES improve the survey so <teachers/school administrators> like you will understand all the questions.

The interviews will take place in <LOCATION> and will last approximately <one-and-a-half / two> hours. You will be asked to respond to survey questions. NCES simply wants to know how you would answer them if it was an actual survey. NCES wants to hear your thoughts about the survey instructions and questions to improve how the survey is administered and make the questions easier to understand. All of your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573]. To thank you for your time, you will receive a $40 gift card.

These interviews are an important stage of the NAEP test development process. Your comments, along with the comments of other <teachers/school administrators>, will help NAEP improve the survey.

If you are interested in participating in this research project, please call [phone number] or e-mail [e-mail address].

Thank you,

Donnell Butler

Assessment Division

Educational Testing Service

Appendix H: Phone Script for Recruiting and Screening—Parents

Good morning/afternoon. May I please speak to <name of parent> about the National Assessment of Educational Progress or NAEP Questionnaires?

Good morning/afternoon. My name is <name> from UserWorks calling on behalf of the National Center for Educational Statistics or NCES, part of the U.S. Department of Education. Thank you for contacting us about the possibility of your child participating in a project to help develop survey questions for NAEP.

First, I want to collect some additional information about your child to be certain they are a good fit for participating. Then I’ll tell you more about what <she, he> will be asked to do.

What is your child’s name?
What grade is <student name> currently in?
Is <student name> either Hispanic or Latino origin?
Is < student name >:

- American Indian or Alaska Native

- Asian

- Black or African American

- Native Hawaiian or Other Pacific Islander, or

- White?

(You may select one or more)

And has <student name> said <she’d, he’d> be interested in participating in the interview session which will take <60/90> minutes?

In the session, we will be asking <student name> to respond to several survey questions. NCES needs to make sure that students will understand what the survey is asking them to do.

Participation is entirely voluntary and your child’s answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

Your child’s input will help NCES improve the survey that will be asked nationwide. To thank <student name>, <she, he> will receive a $25 gift card. In addition, you will receive a $25 gift card for your time to bring the participating student to and from the cognitive laboratory site.

The interview will last approximately <60/90> minutes and will be recorded. It will take place in downtown Bethesda at Abt Associates Inc., located at 4550 Montgomery Avenue, Suite 800 North.

Do you have any questions?

[Answer any questions.]

Do you think <student name> would like to participate?

[If No] Thank you for taking the time to speak with us today about this important project.

[If Yes] And would you be willing to sign a form indicating your approval for <student name> to participate?

[If No] Thank you for taking the time to speak with us today about this important project.

[If Yes]

In the next day or so, we will call <her, him> to be sure <student name> understands what we’re asking <her, him> to do, confirm that <student name> is still interested, and schedule a time for the session. What would be the best phone number to call and time of day to call?

If you or < student name> have any questions before then, please call <phone number>.

Thank you again for agreeing to be part of this very important research project.

Appendix I: Phone Script for Recruiting and Screening—Grade 4 and Grade 8 Students

Good morning/afternoon. May I please speak to <name of student> about the National Assessment of Educational Progress or NAEP Questionnaires interview?

Good morning/afternoon. My name is <name> from UserWorks calling on behalf of the National Center for Educational Statistics or NCES, part of the U.S. Department of Education. We have spoken with your <mother, father> and we are pleased that you and <she, he> have agreed that you can help in developing survey questions for NAEP.

Before we schedule your session, I need to confirm some information we have and tell you more about what you’ll be asked to do.

What grade are you currently in?
Are you of either Hispanic or Latino origin?
Are you:

- American Indian or Alaska Native

- Asian

- Black or African American

- Native Hawaiian or Other Pacific Islander, or

- White?

(You may select one or more)

And are you male or female?

In the session, we will be asking you to respond to several survey questions. NCES needs to make sure that students like you will understand what the survey is asking you to do. Your input will help NCES improve the survey that will be administered nationwide. To thank you for your time, you will receive a $25 gift card. If your parent will bring you to and from the cognitive laboratory site, your parent will also receive a $25 gift card for their time and effort.

The interview will last approximately <60/90> minutes and will be recorded. It will take place in downtown Bethesda at Abt Associates Inc., located at 4550 Montgomery Avenue, Suite 800 North. Your participation is entirely voluntary, and your responses may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

Do you have any questions? [Answer any questions.]

Would you like to participate?

[If No] Thank you for taking the time to speak with us today about this important project.

[If Yes] We would like to schedule your session for <date, time>. Is that day and time ok? Can you put your parent on the phone to confirm that this time would be ok?

[For Parents] We just wanted to confirm that we plan to see your child at <time> on <date>. If you or your child has any questions before then, please call <phone number>.

Thank you again for agreeing to be part of this very important research project.

Appendix J: Phone Script for Recruiting and Screening—Grade 12 Students

Good morning/afternoon. May I please speak to <name of student> about the National Assessment of Educational Progress or NAEP Questionnaires interview?

Before we schedule your session, I need to confirm some information we have and tell you more about what you’ll be asked to do.

What grade are you currently in?
How old are you?
Are you of either Hispanic or Latino origin?
Are you:

- American Indian or Alaska Native

- Asian

- Black or African American

- Native Hawaiian or Other Pacific Islander, or

- White?

(You may select one or more)

And are you male or female?

The interview will last approximately 90 minutes and will be recorded. It will take place in downtown Bethesda at Abt Associates Inc., located at 4550 Montgomery Avenue, Suite 800 North. Your participation is entirely voluntary and your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law[Education Sciences Reform Act of 2002, 20 U.S.C §9573].

Do you have any questions? [Answer any questions.]

Would you like to participate?

[If No] Thank you for taking the time to speak with us today about this important project.

[If Yes] We would like to schedule your session for <date, time>. Is that day and time ok?

We plan to see you at <time> on <date>. If you do have any questions before then, please call <phone number>.

Thank you again for agreeing to be part of this very important research project.

Appendix K: Phone Script for Recruiting and Screening—Teachers and School Administrators

Good morning/afternoon. May I please speak to <name>.

Good morning/afternoon. My name is <name> from Educational Testing Service calling on behalf of the National Center for Educational Statistics or NCES, part of the U.S. Department of Education. The National Assessment of Educational Progress (NAEP) has developed new survey questions for <teachers/school administrators>. Before giving the new survey to all <teachers/school administrators> whose students participate in a NAEP assessment, NCES needs to know if the survey instructions and questions are clear. Your input will help NCES improve the survey so <teachers/school administrators> like you understand all the questions and the things you are being asked to do.

The interviews will take place in <LOCATION> and will last approximately <one-and-a-half / two> hours. You will be asked about the survey instructions and to respond to several questions. NCES simply wants to know how you would answer them if it was an actual survey. NCES wants to hear your thoughts about the survey instructions and questions to improve how the survey is administered and make the questions easier to understand. All your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573]. To thank you for your time, you will receive a $40 gift card.

These interviews are an important stage of the NAEP test development process. Your comments, along with the comments of other <teachers/school administrators>, will help NAEP improve the survey.

Are you interested in participating in this research project?

The interview will last approximately <one-and-a-half / two> hours. It will take place in <LOCATION>.

Do you have any questions at this time?

[Answer any questions.]

Are you available for the interview at <time> on <date>? If you do have any questions before then, please call <phone number>.

Thank you again for agreeing to be part of this very important research project.

Appendix L: Confirmation Letter to Student Participants

<DATE>

Dear <NAME>;

Thank you for agreeing to help try out the new survey questions for the National Assessment of Educational Progress (NAEP). Before giving the new survey to all students, the National Center for Education Statistics (NCES), part of the U.S. Department of Education, needs to know if the survey instructions and questions are clear. Your input will help NCES improve the survey so students like you understand all the questions. All the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

You have been scheduled for an interview at <TIME> on <DATE>. The interview will last approximately <60/90/90> minutes. You will be asked about the instructions and to complete a few tasks. To thank you for your time, you will receive a $25 gift card. If your parent will bring you to and from the cognitive laboratory site,, your parent will also receive a $25 gift card for their time and effort.

The interviews will take place at:

These interviews are an important stage of the NAEP test development process. Your comments, along with the comments of other students, will help NCES improve its survey questions.

Thank you for volunteering in this very important research for NAEP. Our interviewer is looking forward to meeting with you. If you have any questions, please contact [contact information].

Thank you,

Johnny Blair

Principal Scientist

Abt Associates, Inc.

Appendix M: Confirmation Letter to Parents of Student Participants

<DATE>

Dear <Name>;

Thank you for agreeing for allowing your child to help try out the new survey questions for the National Assessment of Educational Progress (NAEP). Before giving the new survey to all students, the National Center for Education Statistics (NCES), part of the U.S. Department of Education, needs to know if the survey instructions and questions are clear. Your child’s input will help NCES improve the survey so all students understand the directions and questions. All the information your child provides may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

Your child has been scheduled for an interview at <TIME> on <DATE>. The interview will last approximately <60/90/90> minutes. Your child will receive a $25 gift card as a token of appreciation for his or her time. In addition, you will receive a $25 gift card for your time to bring the participating student to and from the cognitive laboratory site.

The interviews will take place at:

These interviews are an important stage of the NAEP test development process. Your child’s comments, along with the comments of other students, will help NCES improve the survey.

Thank you for volunteering in this very important research for NAEP. Our interviewer is looking forward to meeting with your child. If you have any questions, please contact [contact information].

Thank you,

Johnny Blair

Principal Scientist

Abt Associates, Inc.

Appendix N: Confirmation Letter to Adult Participants—Teachers and School Administrators

<DATE>

Dear <NAME>;

Thank you for agreeing to help try out the new survey questions for the National Assessment of Educational Progress (NAEP). Before giving the new survey to <teachers/school administrators>, the National Center for Education Statistics (NCES), part of the U.S. Department of Education, needs to know if the survey instructions and questions are clear. Your input will help NCES improve the survey so <teachers/school administrators> like you understand all the questions. All the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002, 20 U.S.C §9573].

You have been scheduled for an interview at <TIME> on <DATE>. The interview will last approximately <one-and-a-half / two> hours. You will be asked about the instructions and to complete a few tasks. To thank you for your time, you will receive a $40 gift card.

The interviews will take place at:

These interviews are an important stage of the NAEP test development process. Your comments, along with the comments of other students, will help NAEP improve its survey questions.

Thank you for volunteering in this very important research for NAEP. Our interviewer is looking forward to meeting with you. If you have any questions, please contact [contact information].

Thank you,

Donnell Butler

Assessment Division

Educational Testing Service

Appendix O: Confirmation Phone Script for Students, Teachers, and School Administrators

Good morning/afternoon. May I please speak to <name> about the National Assessment of Educational Progress (NAEP) survey interview?

Good morning/afternoon. My name is <name> from <Abt Associates/ ETS / ETS> calling about the National Assessment of Educational Progress (NAEP) survey interview. We are so pleased that you agreed to help us develop the best possible survey questionnaire possible. We have scheduled you for <time> on <date>.

The interview will last approximately <TIME> minutes. It will take place at <LOCATION>.

We will be asking you about the survey directions and some of the survey questions. We need to make sure that <students/teachers/school administrators> like you will understand what the survey is asking you to do. Your input will help us improve the surveys that will be asked nationwide. To thank you for your time, you will receive a <INCENTIVE>.

Do you have any questions at this time?

[Answer any questions.]

We plan to see you at <time> on <date>. If you have any questions before then, please call <phone number>.

Thank you again for agreeing to be part of this very important research project.

Appendix P: Thank You Letter to Student Participants

<DATE>

Dear <Name>;

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, I would like to thank you for your participation in our survey research study. Thanks to you, we are better able to develop the best possible survey questionnaires for the National Assessment of Educational Progress (NAEP).

Your assistance helped us better understand how students approach both the survey questions and directions. Your open, candid, and insightful comments were very helpful.

Again, thank you for your assistance.

Sincerely,

Johnny Blair

Principal Scientist

Abt Associates, Inc.

Appendix Q: Thank You Letter to Parents of Student Participants

<DATE>

Dear <Name>;

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, thank you for allowing your child to participate in our survey research study. Your child’s input will help us develop the best possible survey questionnaires for the National Assessment of Educational Progress (NAEP).

The assistance provided by your child helped us to better understand how students approach both the survey questions and directions. Your child’s input, combined with input from other students, was incredibly useful and informative.

Again, thank you for your assistance.

Sincerely,

Johnny Blair

Principal Scientist

Abt Associates, Inc.

Appendix R: Thank You Letter to Adult Participants—Teachers and School Administrators

<DATE>

Dear <Name>;

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, I would like to thank you for your participation in our research study. Thanks to you, we are better able to develop the best possible survey questionnaires for the National Assessment of Educational Progress (NAEP).

Your assistance helped us better understand how <teachers/school administrators> approach both the survey questions and directions. Your open, candid, and insightful comments were very valuable. Your input, combined with input from other <teachers/school administrators> like you, was incredibly useful and informative.

Again, thank you for your assistance.

Sincerely,

Donnell Butler

Assessment Division

Educational Testing Service

Volume I: Appendix page 20

File Type	application/msword
File Title	Appendices (A-R)
Author	mgisbert
Last Modified By	Authorised User
File Modified	2011-01-06
File Created	2011-01-04

National Assessment of Educational Progress (NAEP) Core Background Questions Cognitive Interviews for Students, Teachers, and School Administrators

NCES Cognitive, Pilot, and Field Test Studies System

Appendix-NAEP_Background_Cog_Lab