National Assessment of Educational Progress (NAEP) 2022 Social Science and Economics Questionnaire Cognitive Interviews

National Center for Education Statistics

National Assessment of Educational Progress

Appendices

Communication Materials for Survey Questionnaire Pretesting

National Assessment of Educational Progress (NAEP) 2022

Social Science and Economics Questionnaire Cognitive Interviews

OMB# 1850-0803 v.241

October 2018

Table of Contents

Appendix A: Recruitment Email/Letter to Youth Organizations 3

Appendix B: Recruitment Phone Script for Community Centers and Organizations 4

Appendix C: Recruitment Email/Letter to Parents or Legal Guardians of Student Participants (Under Age 18) 6

Appendix D: Recruitment Phone Script for Parents/Legal Guardians of Student Participants (Under Age 18) 7

Appendix E: Recruitment Phone Script for Student Participants (Under Age 18) 12

Appendix F: Recruitment Email/Letter to Student Participants (Age 18 or Over) 14

Appendix G: Recruitment Phone Script for Student Participants (Age 18 or Over) 15

Appendix H: Recruitment Web-Surveyor Intake Form—Link Provided in Emails/Letters and Flyers to Parents/Legal Guardians/Students (Age 18 or over) Interested in Participating 20

Appendix I: Flyer/Information Brochure 23

Appendix J: Postcard/Newspaper Advertisement/Internet Recruitment 24

Appendix K: Frequently Asked Questions (FAQs) 25

Appendix L: Consent Form for Parent/Legal Guardian of Student Participants (Under Age 18) 27

Appendix M: Consent Form for Student Participants (Age 18 or Over) 29

Appendix N: Confirmation of Interview Email/Day-of Interview Reminder Email 31

Appendix O: Thank You Email/Letter to Parent/Legal Guardian of Student Participants (Under Age 18)/Student Participants (Age 18 or Over) 32

Appendix P: Recruitment Email/Teachers, Department Heads, and School Administrators Professional Organizations, Education-Related Graduate Programs, Teachers’ Unions, Educational Non-Profit/Community Based Organizations 33

Appendix Q: Recruitment Email/Letter to Teachers, Department Heads, and School Administrators 34

Appendix R: Recruitment Phone Script for Teachers, Department Head, and School Administrators 35

Appendix S: Flyer/Information Brochure for Teachers, Department Heads, and School Administrators Professional Organizations, Education-Related Graduate Program, Teachers’ Unions, Educational Non-Profit/Community Based Organizations 40

Appendix T: Frequently Asked Questions (FAQs) for Teachers, Department Heads, and School Administrators 41

Appendix U: Consent Form for Teacher, Department Head, and School Administrator 42

Appendix V: Confirmation of Interview Email/Day of Interview Reminder Email for Teachers, Department Heads, and School Administrators 44

Appendix W: Thank You Email/Letter to Teachers, Department Heads, and School Administrators 45

Appendix X: Example Cognitive Lab Interview Tracking Schedule: Cognitive Interviews for Survey Questionnaires 46

Appendix A: Recruitment Email/Letter to Youth Organizations

[IF SENT AS EMAIL—Subject: Education Research Opportunity for Students in <8^th or 12^th> grade]

<DATE>

Dear <NAME>,

We are administering an important study on behalf of the U.S. Department of Education’s National Center for Education Statistics (NCES). Because your organization plays an active role in the community, we need your help.

We are looking for students in <8^th or 12^th> grade to help us test new computer-delivered survey questions in <civics, geography, U.S. History, or economics> that will be part of the National Assessment of Educational Progress (NAEP) test. Student participation is vital in helping us develop the best questions possible for the test.

The voluntary research interviews will last no more than 60 minutes. During the interview with a researcher, each student will be asked to answer <civics, geography, U.S. History, or economics> survey questions using a tablet as if it were an actual survey. There will be more than one researcher present during each research interview. Their feedback is important for us to develop the best ways of assessing students’ knowledge of <civics, geography, U.S. History, or economics>. Because we care about student privacy, we will assign each student a unique student identifier. Student names will not be linked to their answers.¹

Each student will receive a $25 gift card at the end of the interview as a thank you. In addition, the student’s parent or legal guardian will receive a $25 gift card as a thank you for bringing the student to and from the testing site.

Students play a key role in helping us improve our surveys. In fact, their participation enhances the experience of thousands of students all over the United States who will take the surveys.

We would like to ask for your assistance in letting parents/legal guardians and students know how they can be involved in this important study. I have included a flyer with a brief description of our study in the hopes that you can post it for everyone to see. I have also included a Frequently Asked Questions (FAQs) page with additional information about the study.

[If off-site interviews are possible: To make it more convenient for students, we would like to hear from you if you have space available and a group of interested students who are unable to commute to <location>.]

We appreciate your consideration of this request and invite you to please contact us with any questions regarding this project. We look forward to speaking with you soon!

Sincerely,

<CONTACT NAME>

<JOB TITLE>

EurekaFacts, LLC

<CONTACT PHONE>

ATTACHED: Flyer and Frequently Asked Questions (FAQs) (Appendices I and K)

Appendix B: Recruitment Phone Script for Community Centers and Organizations

Hello. May I please speak to <NAME OF CONTACT AT COMMUNITY CENTER>?

Good <morning/afternoon/evening>. My name is <CONTACT NAME>. I am contacting you from EurekaFacts, a local research company, and we need your help. We are carrying out an important study on behalf of the U.S. Department of Education’s National Center for Education Statistics (NCES). Because your organization plays an active role in the community, we are asking for your help in spreading the word about this exciting research opportunity for students.

We are recruiting students who are in <8^th or 12^th> grade to help us test new <civics, geography, U.S. History, or economics> survey questions. These questions are part of a test called the National Assessment of Educational Progress, or NAEP, which is delivered to students nationwide. [Recruiter: If community center or organization asks what NAEP is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

Students play a key role in helping us improve our surveys. In fact, their participation enhances the experience of thousands of students all over the United States who will take the surveys.

Participation would only require one visit to the testing location, and interviews will last up to 60 minutes. Each student will receive a $25 gift card at the end of the interview as a thank you. In addition, the student’s parent or legal guardian will receive a $25 gift card as a thank you for bringing the student to and from the testing site.

Would your <community center OR organization> be interested in advertising and/or helping us to recruit students for this study?

If NO: Thank you for your time. Have a wonderful <day/evening>. [Recruiter: END CALL]

If YES: Good, thank you very much. To ensure that we interview a broad mix of students and determine the best interview location, I have a few brief screening questions to ask you. It should take only a few minutes.

Student participation is vital in helping us improve our survey questions and develop the best ways of surveying stduents’ knowledge of <civics, geography, U.S. History, or economics>. EurekaFacts research staff will record audio for analysis and take notes of students’ reactions to these survey questions. The study will not retain any personally identifiable information. Prior to the start of the study, students will be notified that their participation is voluntary and that the information they provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

We would like to ask for your assistance in letting parents/legal guardians and students know how they can be involved in this important study. We will hold some interviews at <LOCATION>, <ADDRESS>; however, if the parents/legal guardians of students cannot provide transportation to <LOCATION>, <ADDRESS>, we would like to set up interviews at your <community center OR organization>, if possible.

Screener

[Recruiter: Determine recruitment needs in advance. Look in public documents and the website to find what grades the community center or organization includes, as well as the size, socioeconomic status (SES), and racial/ethnic makeup of the student population. Record all responses to Screener questions. End Screener at whatever point the staff member gives a response that confirms that his/her group has already been sampled adequately. End Screener Script: Based on the requirements of this study, we are not able to include your students in the interviews at this time. Thank you for your time. Have a good <day/evening>. [Recruiter: End call]]

1. Do you work with students who are in <8^th or 12^th> grade who might be available to participate in the study?

If NO: Thank you for your time. Have a wonderful <day/evening>. [Recruiter: END CALL]

If YES: Great.

2. Would you be able to distribute and post flyers regarding this study? [Recruiter: Interested parties would contact EurekaFacts directly. We would follow up with them by phone and ask them some screening questions to ensure that a diverse sample of students is included in this study.]

If NO: Thank you for your time. Have a good <day/evening>. [Recruiter: End call]

If YES: Great! Thank you. Would you also be able to organize a room at your <community center OR organization> where EurekaFacts could conduct some research interviews?

If YES: That’s great! We will contact you to schedule some interviews at your <community center OR organization> after we have some students signed up. [Recruiter: go to Closing]

If NO: That’s fine. If you are able to distribute and post flyers regarding the study, we would greatly appreciate that. EurekaFacts staff will follow up with interested parents/legal guardians by phone to make interview arrangements. [Recruiter: go to Closing]

Thank you for taking the time to answer the screening questions. Your <community center OR organization> is generally eligible to participate in the research interview.

Closing

Thank you for agreeing to share the information about this important study. We will send you an email with the informational flyers attached.

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time? [Recruiter: Answer any questions.]

If you have any <additional> questions, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this important research project.

Appendix C: Recruitment Email/Letter to Parents or Legal Guardians of Student Participants (Under Age 18)

[IF SENT AS EMAIL—Subject: Paid Education Research Opportunity]

<DATE>

Dear <NAME>:

We are reaching out to parents of <8^th or 12^th> students just like you because we need your help. Our company, EurekaFacts, is administering an important education study on behalf of the National Center for Education Statistics (NCES), which is part of the U.S. Department of Education.

NCES is looking for students who are in <8^th or 12^th> grade to help us test new computer-delivered survey questions. These questions will be given to students nationwide as a part of the National Assessment of Educational Progress (NAEP).⁰ Students’ participation and opinions will help us improve survey questions before other students take the surveys nationwide.

The research interview will take place at <LOCATION> and will last for no more than 60 minutes. During the interview with a researcher, your child will be asked to answer <civics, geography, U.S. History, or economics> survey questions using a tablet as if it were an actual survey. There will be more than one adult present during each research interview.

Your child will receive a $25 gift card at the end of the interview as a thank you. The parent or legal guardian will also receive a $25 gift card as a thank you for bringing the student to and from the testing site.

Because we care about your child’s privacy, we will assign him/her a unique student identifier. At no time will your child’s name be linked to his/her answers.⁰

This study is voluntary and has many interviews available. Your child can be scheduled at a time that works best for you.

We have included a Frequently Asked Questions (FAQs) sheet with additional information about this important study.

If your child is interested in this study, please <fill out an intake form at <LINK> OR call <CONTACT PHONE> OR email <CONTACT EMAIL>>. We look forward to speaking with you soon!

Thank you,

<CONTACT NAME>

<JOB TITLE>

EurekaFacts, LLC

<CONTACT PHONE>
<Website Link>

ATTACHED: Frequently Asked Questions (FAQs) (Appendix K)

Appendix D: Recruitment Phone Script for Parents/Legal Guardians of Student Participants (Under Age 18)

Script (when calling parents/legal guardians from a previously compiled list)

Good <morning/afternoon/evening>. Is this the <NAME FROM CALL LIST> household? My name is <CALLER NAME>, and I am calling from EurekaFacts, a local research company. Our company is administering an important education study on behalf of the National Center for Education Statistics (NCES), which is part of the U.S. Department of Education, and we need your help.

We are recruiting students who are in <8^th or 12^th> grade to help us test new computer-delivered <civics, geography, U.S. History, or economics> survey questions. If your child qualifies, your child will receive a $25 gift card at the end of the interview as a thank you. The parent or legal guardian will also receive a $25 gift card as a thank you for bringing the student to and from the testing site.

I’d like to tell you a little bit more about the study first and ask you some questions; is now a good time? (YES or NO and call back later.)

1. Are there any students in your household who are in <8^th or 12^th> grade?

	SELECT ONE
If YES	1	(CONTINUE)
If NO	2	(THANK AND TERMINATE)

We are administerng a study to test new computer-delivered <civics, geography, U.S. History, or economics> survey questions. During an interview with a researcher, your child will answer <civics, geography, U.S. History, or economics> survey questions using a tablet as if it were an actual survey. We will observe and ask about your child’s level of interest in and understanding of the questions, and whether they have any suggestions on how to improve the questions. There will be more than one researcher present during each research interview. These questions will be given to students who take the National Assessment of Educational Progress, or NAEP, which is delivered to students nationwide. [Recruiter: If parent/legal guardian asks what NAEP is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.] Students’ participation and opinions will help us improve survey questions before other students take the surveys nationwide.

The study interview will take place at <NAME OF INTERVIEW SITE AND LOCATION>. It can be scheduled at your convenience and will last no more than 60 minutes. The study will not retain any personally identifiable information. Prior to the start of the study, your child will be notified that his/her participation is voluntary and that the information he/she provides may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). Audio will be recorded for analysis and EurekaFacts staff may take notes about your child’s reactions to the survey questions, however, no personally identifying information will be recorded or retained.

2. If your child qualifies, do you give your permission for us to invite your child to participate?

   	SELECT ONE
   If YES	1	(CONTINUE)
   If NO	2	(THANK AND TERMINATE)

3. What is your child’s name?

RECORD NAME:

______________________________________________________________________________

4. What grade is your child in? [If not in grade <8 OR 12> (THANK AND TERMINATE)]

RECORD GRADE:

______________________________________________________________________________

5. What is your child’s sex? (READ THE QUESTION AND RESPONSE OPTIONS)

	SELECT ONE
Male	1
Female	2
Prefer not to answer	3

6. What type of school does your child currently attend? (READ LIST)

	SELECT ONE
Public school	1	(CONTINUE)
Private school	2
Charter school	3
Homeschool	4
Don’t know	5	(THANK AND TERMINATE)

The next few questions are for classification purposes only.

7. Is your child Hispanic or Latino? (DO NOT READ LIST)

   	SELECT ONE
   Yes	1
   No	2
   Prefer not to answer	3

8. Which of the following categories best describes your child’s racial background? (READ LIST)

	SELECT ALL THAT APPLY
American Indian or Alaska Native	1
Asian	2
Black or African American	3
Native Hawaiian or Other Pacific Islander	4
White	5
Prefer not to answer	6

9. What is your total annual household income range based on the following ranges?

   	SELECT ONE
   Less than $30,000	1
   $30,000 to $99,999	2
   $100,000 or more	3
   Prefer not to answer	4

10. Does your child participate in the National School Lunch program?

    	SELECT ONE
    Yes	1
    No	2
    Prefer not to answer	3

11. Do you live in a rural, suburban, or urban area?

	SELECT ONE
Rural	1
Suburban	2
Urban	3
Don’t know	4
Prefer not to answer	5

12. Is your child enrolled in any English Language Learner (ELL) classes at this time?

	SELECT ONE
Yes	1	(CONTINUE TO 13)
No	2
Prefer not to answer	3

13. Do you speak a language other than English at home?

	SELECT ONE
Yes	1	(CONTINUE TO 13A)
No	2
Prefer not to answer	3

13A. What is this language? If the respondent is unsure how to answer read the following (For example: Korean, Italian, Spanish, Vietnamese)

14. [For Grade 12 students only] Is your child enrolled in an economics course or another course with economics topics in school?

	SELECT ONE
Yes	1	(CONTINUE TO 14A)
No	2
Prefer not to answer	3

14A. What is this economics course or course with economics topics? If the respondent is unsure how to answer read the following. (For example: Introduction to Business, Accounting, AP Macroenomics, AP Microenomics)

15. We are conducting study interviews between <DATE> and <DATE>. If your child chooses to participate, what day and time would work best for you to take <HIM/HER> to the study interview?

[CHECK SCHEDULE FOR RESPONDENT’S LOCATION FOR AVAILABLE TIMES. IF NECESSARY, SUGGEST A DIFFERENT PLACE OR TIME.]

RECORD DATE AND TIME:

______________________________________________________________________________

16. To attend the interview and receive the $25 gift card, we must receive the parental consent form. You can scan the consent form and email it to us, or you can bring it with you and give it to the researcher. Do you understand that you must have a completed and signed consent form for your child to participate?

	SELECT ONE
Yes	1
No	2

17. Would you like me to send the consent form by email?

If Yes:
email

17A. What is your email address?

RECORD PARENT’S/LEGAL GUARDIAN’S EMAIL ADDRESS:

_____________________________________________________________________________

Thank you.

18. If we need to follow up with you about this study, will you please provide us with your contact information?

RECORD PARENT/LEGAL GUARDIAN FIRST NAME: ____________________________________

RECORD PARENT/LEGAL GUARDIAN LAST NAME: _____________________________________

RECORD EMAIL ADDRESS: ____________________________________

MOBILE PHONE NUMBER: ____________________________________

HOME PHONE NUMBER: _____________________________________

19. Now that you have agreed for your child to participate in the study if he/she is selected, I would like to be able to ask your child a few questions, during which you are welcome to stay on the line. Is he/she available to speak with me at this time?

(If child is not available, get a call back time and call back.)

DATE: _____________________________________

TIME: _____________________________________

(If child is available) Again, you are welcome to stay on the line, but please allow your child to speak for him/herself. Thank you.

Appendix E: Recruitment Phone Script for Student Participants (Under Age 18)

RECRUITER: Please determine how talkative and articulate the student is. It is critical that we speak with students who are comfortable expressing themselves with someone they are meeting for the first time. Please use your best judgment, and recruit only students who are thoughtful and freely express themselves in a manner that is relevant to the topic being discussed.

All recruits MUST be articulate students who can answer open ended questions and can provide support or explanation for their responses (e.g., “I like it because...”; “I don’t like it because...”).

DO NOT recruit any students who have difficulty answering these questions, whose parent/legal guardian answers FOR them, who seem hesitant or shy speaking with you (as an adult they do not know), who answer “I don’t know,” or who CANNOT provide an explanation to support their responses.

Hi, <STUDENT’S NAME>. My name is <CALLER NAME>, and I have been speaking with your parent/legal guardian for a few minutes.

As I explained to your parent/legal guardian, I work for a research company. We are administering an important study to test new computer-delivered <civics, geography, U.S. History, or economics> survey questions for a nationwide student assessment, the National Assessment of Education Progress, or NAEP. For the study interview, if you agree to participate, you would be asked to visit <LOCATION> and complete computer-based <civics, geography, U.S. History, or economics> survey questions. Your responses would not be graded. The National Center for Education Statistics, or NCES, simply wants to know how you would respond to each question or the questions.

During an interview with a researcher, you will answer <civics, geography, U.S. History, or economics> survey questions using a tablet as if it were an actual survey. We will observe and ask about your level of interest in and understanding of the questions, and whether you have any suggestions on how to improve the questions. There will be more than one adult present during each research interview. Your participation will help us improve survey questions before other students take these surveys nationwide. The interview will take no more than 60 minutes and you will receive a $25 gift card to thank you for participating in the research interview. If a parent or legal guardian brings you to and from the interview site, he/she will also receive a $25 gift card to thank him/her for his/her time and effort.

As I said before, your responses would be used to improve the national test so that it works well for all students. We would not use your name to identify you in any of our records.

1. Do you think you would be interested in participating if selected?

	SELECT ONE
Yes	1	(CONTINUE)
No	2	(READ THE FOLLOWING)

Thank you for your time. I will need to let your parent/legal guardian know that you would not like to participate. Is he or she available?

TO PARENT/LEGAL GUARDIAN: Your child has decided not to participate, but I would like to thank you very much for taking the time to speak with me about this opportunity.

2. What grade are you in?

RECORD GRADE:

[MUST BE in grade <8 or 12>. OTHERWISE THANK AND TERMINATE.]

3. What type of school do you currently attend? (READ LIST)

	SELECT ONE
Public school	1	(CONTINUE)
Private school	2
Charter school	3
Homeschool	4
Don’t know	5	(THANK AND TERMINATE)

4. You qualify to participate in a study interview with a researcher in <LOCATION>. Your parent/legal guardian is available to bring you to the interview on <DAY/DATE AND TIME>. The interview will last about 60 minutes. You will receive a $25 gift card for participating. Your parent/legal guardian gave permission for you to attend if you are available. Will you be able to attend the interview?

	SELECT ONE
Yes	1	(CONTINUE)
No	2	(THANK AND TERMINATE)

5. To attend the interview and receive the $25 gift card, we must receive the parental consent form that we will email to your parent/legal guardian. Your parent/legal guardian can scan and email the consent form to us, or you can bring it with you and give it to the researcher. Do you understand that you must have a completed and signed parental consent form to participate?

	SELECT ONE
Yes	1
No	2

Thank you. We look forward to seeing you at the interview.

[END OF SCREENER]

Appendix F: Recruitment Email/Letter to Student Participants (Age 18 or Over)

[IF SENT AS EMAIL—Subject: Paid Education Research Opportunity]

<DATE>

Dear <NAME>,

We are reaching out to 12^th grade students just like you because we need your help. Our company, EurekaFacts, is administering an important education study for the National Center for Education Statistics (NCES), which is part of the U.S. Department of Education.

NCES is looking for students who are in 12^th grade to test new computer-delivered <civics, geography, U.S. History, or economics> survey questions. These questions will be given to students for the National Assessment of Educational Progress (NAEP).¹ Students’ participation and opinions will help us improve survey questions before other students take the surveys nationwide.

The research interview will take place at <LOCATION> and will last for no more than 60 minutes. During the interview with a researcher, you will be asked to answer <civics, geography, U.S. History, or economics> survey questions using a tablet as if it were an actual survey. There will be more than one researcher present during each research interview.

You will receive a $25 gift card at the end of the interview as a thank you for your participation. A parent or legal guardian will also receive a $25 gift card as a thank you if they bring you to and from the testing site.

Because we care about your privacy, we will assign you a unique student identifier. At no time will your name be linked to your answers.¹

We have included a Frequently Asked Questions (FAQs) sheet with additional information about this important study.

If you are interested in participating in this voluntary study, please <fill out an intake form at <LINK> OR call <CONTACT PHONE> OR email <CONTACT EMAIL>>. This study is voluntary and has many interviews available. We look forward to speaking with you soon!

Thank you,

<NAME>

<JOB TITLE>

EurekaFacts, LLC

<CONTACT PHONE>

ATTACHED: Frequently Asked Questions (FAQs) (Appendix K)

¹ The National Center for Education Statistics (NCES) is authorized to conduct NAEP by the National Assessment of Educational Progress Authorization Act (20 U.S.C. §9622). All of the information provided by school staff and students may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20

U.S.C. §9573 and 6 U.S.C. §151). Audio will be recorded for analysis and EurekaFacts staff may take notes about students’ reactions to the survey questions, however, no personally identifying information will be recorded or retained.

Appendix G: Recruitment Phone Script for Student Participants (Age 18 or Over)

RECRUITER: Please determine how talkative and articulate the student is. It is critical that we speak with students who are comfortable expressing themselves with someone they are meeting for the first time. Please use your best judgment and recruit only students who are thoughtful and freely express themselves in a manner that is relevant to the topic being discussed.

All recruits MUST be articulate students who can answer open-ended questions and can provide support or explanation for their responses (e.g., “I like it because...”; “I don’t like it because...”).

DO NOT recruit any students who have difficulty answering these questions, whose parent/legal guardian answers FOR them, who seem hesitant or shy speaking with you (as an adult they do not know), who answer “I don’t know,” or who CANNOT provide an explanation to support their responses.

Script (when calling from a previously compiled list)

Good <morning/afternoon/evening>. Is this the <NAME FROM CALL LIST> household? My name is <CALLER NAME>, and I am calling from EurekaFacts, a local research company. Our company is administering an important education study on behalf of the National Center for Education Statistics (NCES), which is part of the U.S. Department of Education, and we need your help.

We are recruiting students who are in 12^th grade to help us test new computer-based <civics, geography, U.S. History, or economics> survey questions. If you qualify, you will receive a $25 gift card at the end of the interview as a thank you. Your parent or legal guardian will also receive a $25 gift card as a thank you if they bring you to and from the testing site.

I’d like to tell you a little bit more about the study first and ask you some questions; is now a good time? (YES or NO and call back later.)

1. Are there any students in your household who are in 12^th grade?

	SELECT ONE
Yes	1	(CONTINUE)
No	2	(THANK AND TERMINATE)

During an interview with a researcher, you will answer <civics, geography, U.S. History, or economics> survey questions using a tablet as if it were an actual survey. We will observe and ask about your level of interest in and understanding of the questions, and whether you have any suggestions on how to improve the questions. There will be more than one researcher present during each research interview. These questions will be given to students in the National Assessment of Educational Progress, which is delivered to students nationwide. [If students ask what NAEP is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.] Your participation and opinions will help us improve survey questions before other students take the surveys nationwide.

The study interview will take place at <NAME OF INTERVIEW SITE AND LOCATION>. It can be scheduled at your convenience and will last no more than 60 minutes. The study will not retain any personally identifiable information. Prior to the start of the study, you will be notified that their participation is voluntary and that the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). Audio will be recorded for analysis and EurekaFacts staff may take notes about your reactions to the survey questions, however, no personally identifying information will be recorded or retained.

2. If you qualify, are you willing to participate?

	SELECT ONE
Yes	1	(CONTINUE)
No	2	(THANK AND TERMINATE)

3. What is your name?

RECORD NAME:

______________________________________________________________________________

4. What grade are you in? [If not in grade 12  (THANK AND TERMINATE)]

RECORD GRADE:

______________________________________________________________________________

5. What is your sex?  (READ THE QUESTION AND RESPONSE OPTIONS)

	SELECT ONE
Male	1
Female	2
Prefer not to answer	3

6. What type of school do you currently attend? (READ LIST)

	SELECT ONE
Public school	1	(CONTINUE)
Private school	2
Charter school	3
Homeschool	4
Don’t know	5

The next few questions are for classification purposes only.

7. Are you Hispanic or Latino?

   	SELECT ONE
   Yes	1
   No	2
   Prefer not to answer	3

8. Which of the following categories best describes your racial background? (READ LIST)

	SELECT ALL THAT APPLY
American Indian or Alaska Native		1
Asian		2
Black or African American		3
Native Hawaiian or Other Pacific Islander		4
White		5
Prefer not to answer		6

9. What is your total annual household income range based on the following ranges?

   	SELECT ONE
   Less than $30,000	1
   $30,000 to $99,999	2
   $100,000 or more	3
   Prefer not to answer	4

10. Do you participate in the National School Lunch program?

	SELECT ONE
Yes	1
No	2
Prefer not to answer	3

11. Do you live in a rural, suburban, or urban area?

	SELECT ONE
Rural	1
Suburban	2
Urban	3
Don’t know	4
Prefer not to answer	5

12. Is your child enrolled in any English Language Learner (ELL) classes at this time?

	SELECT ONE
Yes	1	(CONTINUE TO 13)
No	2
Prefer not to answer	3

13. Do you speak a language other than English at home?

	SELECT ONE
Yes	1	(CONTINUE TO 13A)
No	2
Prefer not to answer	3

13A. What is this language? Note: if the respondent has trouble answering read the following (For example: Korean, Italian, Spanish, Vietnamese)

14. You qualify to participate in a study interview in <LOCATION>. Your interview will be on <DAY/DATE AND TIME> and will last no more than 60 minutes. You will receive a $25 gift card for participating, and your parent/legal guardian will also receive a $25 gift card to thank him/her if they bring you to and from the testing site. Will you be able to attend the interview?

	SELECT ONE
Yes	1	(CONTINUE TO 15)
No	2	(CONTINUE TO 16)

15. We are conducting study interviews between <DATE> and <DATE>. If you choose to participate, what day and time would work best for you?

[CHECK SCHEDULE FOR RESPONDENT’S LOCATION FOR AVAILABLE TIMES. IF NECESSARY, SUGGEST A DIFFERENT PLACE OR TIME.]

RECORD DATE AND TIME:

______________________________________________________________________________

Thank you. We look forward to seeing you at the interview.

16. To attend the interview and receive the $25 gift card we must receive a signed consent form. You can scan the consent form and email it to us, or you can bring it with you and give it to the researcher. Do you understand that you must have a completed and signed consent form for you to participate?

	SELECT ONE
Yes	1
No	2

17. Would you like me to send the consent form by email?

	SELECT ONE
Yes	1
No	2

email

16A. What is your email address?

RECORD EMAIL ADDRESS:

_____________________________________________________________________________

Thank you.

18. If we need to follow up with you about this study, will you please provide us with your contact information?

RECORD FIRST NAME: ____________________________________

RECORD LAST NAME: _____________________________________

RECORD EMAIL ADDRESS: _________________________________

MOBILE PHONE NUMBER: _________________________________

HOME PHONE NUMBER: __________________________________

Appendix H: Recruitment Web-Surveyor Intake Form—Link Provided in Emails/Letters and Flyers to Parents/Legal Guardians/Students (Age 18 or over) Interested in Participating

Student Name (Last, First): ____________________________________________

[If 18 or over skip Parent/Legal Guardian Information  Student’s Grade]

Parent/Legal Guardian Name (Last, First): ________________________________

Parent/Legal Guardian Phone Number: __________________________________

Student’s Grade: 1^st__ 2^nd__ 3^rd__ 4^th__ 5^th__ 6^th__ 7^th__ 8^th__ 9^th__ 10^th__ 11^th__ 12^th__

Student’s School/District: _____________________________________________

Student’s Gender: ___________________________________________________

Student’s Ethnicity/Race:

Ethnicity:

 Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

 American Indian or Alaska Native

 Asian

 Black or African American

 Native Hawaiian or other Pacific Islander

 White

 Prefer not to answer

Which language did the student learn when he or she first began to talk? ____________________________

Which language does the student most frequently speak at home? _________________________________

Is the student enrolled in any English language learner (ELL) classes at this time? ______________________

Is student currently enrolled in any of the following types of courses? (select all that apply)

 Advanced placement (e.g., AP, IB, or other college-level courses)

 Honor courses

 Remedial courses

 Other courses

Does student participate in the National School Lunch program?

Yes

No

Prefer not to answer

What is the total annual household income range based on the following ranges?

Less than $30,000

$30,000 to $99,999

$100,000 or more

Prefer not to answer

Does student live in a rural, suburban, or urban area?

Rural

Suburban

Urban

Don’t Know

Prefer not to answer

Possible Time For 60-Minute Interview:

Please mark all the interviews that would work for student if they qualify to participate. Not all of these times will be used. We are trying to schedule a limited number of interviews that will be convenient for the largest number of participants.

<DATE>	<TIME>	<TIME>	<TIME>
<DATE>	<TIME>	<TIME>	<TIME>
<DATE>	<TIME>	<TIME>	<TIME>
<DATE>	<TIME>	<TIME>	<TIME>

If some other time during these weeks would be better, what 60-minute time frame would that be?

________________________________________________________________________________

If your student qualifies to participate, we will need to follow up with you for scheduling purposes. Please provide us with your contact information.

FIRST NAME: ___________________________________________________

LAST NAME: ___________________________________________________

EMAIL ADDRESS: _______________________________________________

MOBILE PHONE NUMBER: ________________________________________

ALTERNATE PHONE NUMBER: _____________________________________

To complete the submission of this form, please click the “Submit” button. You will be contacted within <TIMEFRAME> to let you know whether or not the student has been selected to participate in this study.

(SUBMIT BUTTON)

Appendix I: Flyer/Information Brochure

Appendix J: Postcard/Newspaper Advertisement/Internet Recruitment

EurekaFacts invites 8^th and 12^th grade students to participate in an important study for the U.S. Department of Education’s National Center for Education Statistics (NCES). Interviews will take place on <DATE>. Students will get a $25 gift card and another $25 gift card for the parent or legal guardian as a thank you for bringing the student to and from the testing site.

Appendix K: Frequently Asked Questions (FAQs)

Frequently Asked Questions

1. What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting a study with students who are in <8^th or 12^th> grade to test new computer-delivered <civics, geography, U.S. History, or economics> survey questions for the National Assessment of Educational Progress (NAEP). Students’ participation will help NAEP improve survey questions before other students take the surveys nationwide.

2. Where will the research interviews take place?

The study interview will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>.

3. What will happen during the research interview?

Each participant will work through an early version of new NAEP survey questions in <civics, geography, U.S. History, or economics> on a tablet. A researcher will ask what the student was thinking as he or she completed the questions, whether he or she thinks the questions are clear and understandable, and whether he or she has any suggestions on how to improve the questions.

<Civics, Geography, U.S. Hsitory, or Economics>-specific knowledge is not required, and participants’ responses will not be graded. There will be more than one adult present during each research interview.

4. How long will the research interview last?

The interview will last no more than 60 minutes. The interview requires only one visit to the site.

5. Will information be kept confidential?

The participating student will be assigned a unique student identifier, and at no time will his/her name be linked to any of his/her answers. The study will not retain any personally identifiable information. Prior to the start of the study, students will be notified that their participation is voluntary and that the information they provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). Audio will be recorded for analysis, but no personally identifying information will be recorded or retained.

6. Will the results go to the participating students’ schools?

The participating students’ answers will not be disclosed to their schools and will not be linked to any personally identifiable information, such as a school’s name.

7. What are the possible risks of being in the study?

There are no known risks to participating in this study.

8. Is participation required?

Participation in this study is completely voluntary. The participating student has the right to refuse to answer particular questions. The student may elect to withdraw from this study at any time and can end early if he/she wishes.

9. What is needed or required for participation?

For students under the age of 18, we require written, informed consent from a parent/legal guardian in order for the eligible student to participate. We also ask the student for his or her verbal consent to participate during the prescreening process.

Participation in the study is completely voluntary. The parent or legal guardian or student may withdraw consent for the student’s participation in the study at any time. The student also has the right to refuse to answer particular questions.

Students age 18 or over can sign the consent form directly; again, participation in the study is completely voluntary and students have the right to withdraw intent to participate and/or refuse to answer particular questions.

Additionally, a valid picture ID of a parent or legal guardian, or the student if 18 or older, is required for participation. If the student uses eyeglasses for reading or watching TV, he/she should bring them to the study interview.

10. Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this interview at EurekaFacts on behalf of NCES. If you have any questions, please <fill out an intake form at <LINK> OR call <CONTACT PHONE> OR email <CONTACT EMAIL>>.

Appendix L: Consent Form for Parent/Legal Guardian of Student Participants (Under Age 18)

PARENTAL CONSENT FORM

FOR PARTICIPATION IN RESEARCH

Thank you for your interest in participating in the study described below. This consent form explains the study. Please read it carefully and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them later as they occur to you.

1. What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting a study with students who are in <8^th or 12^th> grade to test new computer-delivered <civics, geography, U.S. History, or economics> survey questions for the National Assessment of Educational Progress (NAEP). Students’ participation will help NAEP improve survey questions before other students take the surveys nationwide.

2. Where will the study interviews take place?

The study interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>.

3. What will happen during the research interview?

Each participant will work through an early version of new NAEP survey questions in <civics, geography, U.S. History, or economics> on a tablet. A researcher will ask what the student was thinking as he or she completed the questions, whether he or she thinks the questions are clear and understandable, and whether he or she has any suggestions on how to improve the questions.

<Civics, Geography, U.S. Hsitory, or Economics>-specific knowledge is not required, and participants’ responses will not be graded. There will be more than one adult present during each research interview.

4. How long will the research interview last?

The interview will last no more than 60 minutes. The interview requires only one visit to the site.

5. Will information be kept confidential?

Your child will be assigned a unique student identifier and at no time will his/her name be linked to any of his/her answers. The study will not retain any personally identifiable information. Prior to the start of the study, students will be notified that their participation is voluntary and that the information they provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

Audio will be recorded for analysis, but no personally identifying information will be recorded or retained.

6. Will the results go to my child’s school?

Your child’s answers will not be disclosed to his/her school and will not be linked to any personally identifiable information, such as the school’s name.

7. What are the possible risks of being in the study?

There are no known risks to participating in this study.

8. Is participation required?

Participation in this study is completely voluntary. Your child has the right to refuse to answer particular questions. Your child may elect to withdraw from this study at any time and can end early if he/she wishes.

Your child will still receive a gift card if he or she refuses to answer particular questions or elects to end the research interview early.

9. What is needed or required for participation?

For students under the age of 18, we require written, informed consent from a parent/legal guardian in order for the eligible student to participate. We also ask the student for his or her verbal consent to participate during the prescreening process.

Participation in the study is completely voluntary. The parent or legal guardian or student may withdraw consent for the student’s participation in the study at any time. The student also has the right to refuse to answer particular questions.

Additionally, a valid picture ID of a parent or legal guardian is required for participation. If your child uses eyeglasses for reading or watching TV, please bring them to the study interview.

10. Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this interview at EurekaFacts on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>. Additionally, you can fill out an intake form for your child at <LINK>.

____ Yes, I agree to allow my child to participate in this study.

____ No, I do not want my child to participate in this study.

By signing below, you agree that your child, _______________________________, may participate in this study and that we may record your child’s conversation with the researcher. Your child will receive a $25 gift card at the conclusion of the interview.

In addition, if transportation to the site is involved, you will receive a $25 gift card to thank you for bringing your child to and from the interviewsite.

Your signature below will indicate that your questions have been answered satisfactorily and that you have read and understood the information provided above.

Print Name: ___________________________________________________________________________

Signature: ___________________________________________________ Date: __________________

Email address: _________________________________________________________________________

To be completed by EurekaFacts staff:

Student Payment Card Transaction Number: _______________________________________________________

Parent/Legal Guardian Payment Card Transaction Number: ___________________________________________________________________________________________

Appendix M: Consent Form for Student Participants (Age 18 or Over)

STUDENT (AGE 18 OR OVER) CONSENT FORM

FOR PARTICIPATION IN RESEARCH

Thank you for your interest in participating in the study described below. This consent form explains the study. Please read it carefully and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them later as they occur to you.

1. What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is conducting a study with students who are in 12^th grade to test new computer-delivered <civis, geography, U.S History, or economics> survey questions for the National Assessment of Educational Progress (NAEP). Students’ participation will help us improve survey questions before other students take the surveys nationwide.

2. Where will the study interviews take place?

The study interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>.

3. What will happen during the research interview?

Each participant will work through an early version of new NAEP survey questions in <civis, geography, U.S History, or economics> on a tablet. A researcher will ask what the student was thinking as he or she completed the questions, whether he or she thinks the questions are clear and understandable, and whether he or she has any suggestions on how to improve the questions.

<Civics, Geography, U.S. History, or Economics>-specific knowledge is not required, and your responses will not be graded. There will be more than one adult present during each research interview.

4. How long will the research interview last?

The interview will last no more than 60 minutes. The interview requires only one visit to the site.

5. Will information be kept confidential?

You will be assigned a unique student identifier and at no time will your name be linked to any of your answers. The study will not retain any personally identifiable information. Prior to the start of the study, students will be notified that their participation is voluntary and that the information they provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

Audio will be recorded for analysis, but no personally identifying information will be recorded or retained.

6. Will the results go to my school?

Your answers will not be disclosed to your school and will not be linked to any personally identifiable information, such as your school’s name.

7. What are the possible risks of being in the study?

There are no known risks to participating in this study.

8. Is participation required?

Participation in this study is completely voluntary. You have the right to refuse to answer particular questions. You may elect to withdraw from this study at any time and can end early if you wish.

You will still receive a gift card if you refuse to answer particular questions or elect to end the research interview early.

9. What is needed or required for participation?

Students age 18 or over can sign the consent form directly.

Participation in the study is completely voluntary. You may withdraw consent for your participation in the study at any time. You also have the right to refuse to answer particular questions.

Additionally, a valid picture ID is required for participation. If you use eyeglasses for reading or watching TV, please bring them to the study interview.

10. Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this interview at EurekaFacts on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>. Additionally, you can fill out an intake form at <LINK>.

  __ Yes, I agree to participate in this study.

  __ No, I do not agree to participate in this study.

By signing below, you agree that you, _______________________________, may participate in this study and that we may record your conversation with the researcher. You will receive a $25 gift card at the conclusion of the interview.

In addition, if transportation to the site is involved, your parent or legal guardian will receive a $25 gift card to thank him/her for bringing you to and from the interview site.

Your signature below will indicate that your questions have been answered satisfactorily and that you have read and understood the information provided above.

Print Name: ___________________________________________________________________________

Signature: __________________________________________________ Date: __________________

Email address: _________________________________________________________________________

To be completed by EurekaFacts staff:

Student Payment Card Transaction Number: _______________________________________________________

Parent/Legal Guardian Payment Card Transaction Number: ___________________________________________

Appendix N: Confirmation of Interview Email/Day-of Interview Reminder Email

Dear <NAME>:

Thank you for agreeing to help us test the new NAEP computer-delivered <civics, geography, U.S. History, or economics> survey questions for the National Assessment of Educational Progress (NAEP).

The interview will last about 60 minutes. We will begin promptly at <TIME>, and it is important that you arrive 15 minutes early to check in. If you are late, we cannot guarantee that you will be interviewed and receive the $25 gift card.

This is a <friendly reminder OR confirmation> that <your child has OR you have> been scheduled to participate in a study at the following location and date/time:

LOCATION: <Location Address>

DATE: <Day of Week, Month, Day, Year>

TIME: <XX:XX AM/PM>

GIFT CARD: <Your child OR You> will receive a $25 gift card to thank <them for their OR you for your> participation. If <you bring your child OR your parent or legal guardian brings you> to and from the interview, <you OR he OR she> will also receive a $25 gift card as a thank you.

PLEASE BRING THE FOLLOWING ITEMS TO THE RESEARCH INTERVIEW:

• Parent/guardian valid picture ID—this is required for participation.

• Signed parent/guardian consent form (attached). Your child can either bring the signed form to the interview or you can scan it and email it back. If not, we have copies available in person when you arrive. Without a signed form, your child cannot participate in the interview and receive the $25 gift card.

• If your child uses eyeglasses for reading or watching TV, they should wear them during the study interview.

OR

• Your valid picture ID—this is required for participation.

• Signed consent form (attached), if not we have copies available in person when you arrive.

• If you use eyeglasses for reading or watching TV, you should wear them during the study interview.

Thank you for volunteering in this very important research for NAEP. We look forward to seeing you. If you have any questions about the study, please contact <EurekaFacts CONTACT NAME> at <CONTACT INFORMATION>.

Sincerely,

<NAME>

<EurekaFacts, LLC>

Appendix O: Thank You Email/Letter to Parent/Legal Guardian of Student Participants (Under Age 18)/Student Participants (Age 18 or Over)

[IF SENT AS EMAIL—Subject: Thank you—NAEP Research Study]

<DATE>

Dear <NAME>,

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, EurekaFacts would like to thank you <If Parent/Legal guardian: for allowing <CHILD’S NAME> to participate OR If Student Age 18 or Over: for participating> in the study for the National Assessment of Educational Progress (NAEP).

<If Parent/Legal guardian: <CHILD’S NAME>’s OR If Student Age 18 or Over: Your> participation helped us test the new NAEP computer-delivered <civics, geography, U.S. History, or economics> survey questions. The assistance <he/she OR you> provided, combined with input from other students, was incredibly useful and informative for improving the survey experience for students nationwide. We greatly appreciate you taking the time <If Parent/Legal guardian: to bring <CHILD’S NAME> to and from the interview location> to participate.

For more information on NAEP, please visit the NCES NAEP website at: http://nces.ed.gov/nationsreportcard.

Again, thank you for your assistance.

Sincerely,

<CONTACT NAME>

EurekaFacts, LLC

<CONTACT INFORMATION >

Appendix P: Recruitment Email/Teachers, Department Heads, and School Administrators Professional Organizations, Education-Related Graduate Programs, Teachers’ Unions, Educational Non-Profit/Community Based Organizations

[IF SENT AS EMAIL—Subject: Education Research Opportunity for Educators]

<DATE>

Dear <NAME>,

We are administering an important study on behalf of the U.S. Department of Education’s National Center for Education Statistics (NCES). Because your organization plays an active role in the community, we need your help.

We are looking for <teachers, department heads, or school administrators> in schools with students in <8^th and/or 12^th> grade to help us test new computer-delivered survey questions in <civics, geography, U.S. History, or economics> that will be part of the National Assessment of Educational Progress (NAEP) questionanire. Educator participation is vital in helping us develop the best survey questions possible for the NAEP questionnaire.

Participants will receive a $60 gift card at the end of the study to thank them for their time and effort.

The voluntary interviews will last no more than 60 minutes. During the interview, the <teacher, department head, or school administrator> will receive a copy of multiple choice questions that he/she will be asked to answer verbally. A researcher will ask open-ended questions to follow up on why he/she selected each answer. There will be more than one researcher present during each research interview. NCES wants to hear his/her thoughts on improving the survey questions and making the questions easier to understand. A unique identifier (ID) will be assigned, and names will not be linked to answers.⁰

Educators play a key role in helping us improve our NAEP questionnaires. In fact, their participation enhances the experience of thousands of educators all over the United States who will later participate in these questionnaires.

We would like to ask for your assistance in letting <teachers, department heads, or school administrators> know how they can be involved in this important study. I have included a flyer with a brief description of our study in the hopes that you can post it for everyone to see. I have also included a Frequently Asked Questions (FAQs) page with additional information about the study.

To make it more convenient for <teachers, department heads, or school administrators>, a limited number of interviews may be completed via phone or a digital video conference such as, WebEx. The interviews can be scheduled at a time that works best for them.

We appreciate your consideration of this request and invite you to please contact us with any questions regarding this study. We look forward to speaking with you soon!

Sincerely,

<CONTACT NAME>

<JOB TITLE>

EurekaFacts, LLC

<CONTACT PHONE>

ATTACHED: Flyer and Frequently Asked Questions (FAQs) (Appendices S and T)

Appendix Q: Recruitment Email/Letter to Teachers, Department Heads, and School Administrators

[IF SENT AS EMAIL—Subject: Paid Education Research Opportunity]

Dear <NAME>: <DATE>

My name is <CONTACT NAME>, and I am contacting you from EurekaFacts, a local research company. Our company is administering an important education study for the National Center for Education Statistics (NCES), which is part of the U.S. Department of Education.

NCES is looking for <teachers, department heads, or school administrators> to participate in voluntary interviews for supporting the development of survey questions for the National Assessment of Educational Progress (NAEP).⁰ <Teachers’, Department Heads’, or School administrators’> participation and opinions will help us improve survey questions so educators like you understand all the questions.

You will receive a $60 gift card at the end of the study as a thank you for your time and effort.

The interviews will take place at <LOCATION> and will last no more than 60 minutes. A limited number of interviews may be conducted via phone or a digital video conference, such as WebEx. The interview can be scheduled at a time that works best for you. During the interview, you will receive a copy of multiple choice questions that you will be asked to answer verbally. A researcher will ask open-ended questions to follow up on why you selected each answer. There will be more than one researcher present during each research interview. NCES wants to hear your thoughts on improving the survey questions and making the questions easier to understand.

You will be assigned a unique identifier (ID), and your name will not be linked to your answers.⁰ The interviews will be audio recorded and the recordings will be used only by the researchers administering the study to write up the findings.

These voluntary research interviews are an important stage of the NAEP questionnaire development process. Your comments, along with the comments of other educators, will help NCES improve the survey.

If you are interested in participating in this study, please call <CONTACT PHONE> or email <CONTACT EMAIL>. We look forward to speaking with you soon!

Thank you,

<CONTACT NAME>

EurekaFacts, LLC

<CONTACT INFORMATION >

Appendix R: Recruitment Phone Script for Teachers, Department Head, and School Administrators

Good <morning/afternoon/evening>. May I please speak to <NAME>?

My name is <CALLER NAME>, and I am contacting you from EurekaFacts, a local research company. The U.S. Department of Education is recruiting <teachers, department heads, or school administrators> to participate in a small study for which you will receive a token of our appreciation in the form of a gift card. Is this a good time? It will only a take a few minutes. (YES or NO and call back later.)

Our company is administering an important education study on behalf of the National Center for Education Statistics (NCES), which is part of the U.S. Department of Education, and we need your help. We are recruiting <teachers, department heads, or school administrators> to help us test new survey questions. If you qualify, you will receive a $60 gift card at the end of the study as a thank you.

We are administering a study to test new <civics, geography, U.S. History, or economics> survey questions. During the interview, you will receive a copy of multiple-choice questions that you will be asked to answer verbally. A researcher will ask open-ended questions to follow up on why you selected each answer and if you have any suggestions on how to improve the questions. These questions will be given to educators as a part of the National Assessment of Educational Progress, or NAEP, questionnaires. There will be more than one researcher present during each research interview. [If teachers, department heads, or school administrators ask what NAEP is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.] Your participation and opinions will help us improve survey questions before other educators take the questionnaires nationwide.

The voluntary research interviews will take place at <NAME OF INTERVIEW LOCATION> and will last no more than 60 minutes. A limited number of interviews may be conducted via phone or WebEx video conference. Your participation is voluntary and the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). Audio activity will be recorded for analysis and a researcher may take notes about your reactions to the survey questions, however, no personally identifying information will be recorded or retained.

1. Are you interested in participating in this study?

	SELECT ONE
If YES	1	(CONTINUE TO QUESTION 3)
If NO	2	(CONTINUE TO QUESTION 2)

2. Can you tell me why you chose not to participate in this study? Please choose your top reason.

	SELECT ONE
The timeframe between December 2018 and February 2019 is not ideal.	1	 (THANK AND TERMINIATE)
The 60-minute interview time commitment is too long.	2	 (THANK AND TERMINIATE)
The commuting time commitment is too long.	3	 (THANK AND TERMINIATE)
The compensation is too low. I chose not to participate for another reason. Reason: ____________________________	4 5	 (THANK AND TERMINIATE)  (ASK PARTICIPANT FOR REASON)(THANK AND TERMINATE)(THANK AND TERMINATE)(THANK AND TERMINATE)(THANK AND TERMINATE)
I prefer not to answer.	6	 (THANK AND TERMINIATE)

3. What is your job title?

	SELECT ONE
Principal	1	 (CONTINUE TO A)
Department Head	2	 (CONTINUE TO D)
Teacher	3	 (CONTINUE TO I)
Other	4	 (THANK AND TERMINIATE)(THANK AND TERMINATE)(THANK AND TERMINATE)(THANK AND TERMINATE)(THANK AND TERMINATE)

PRINCIPAL

A. At what school are you a principal? _________________________________________

B. Are you at a school with a population that includes: (READ LIST)

	SELECT ONE OR MORE
Eighth-grade students	1
Twelfth-grade students	2
Other	3	 (THANK AND TERMINATE)

C. How many students are enrolled in your school?

	SELECT ONE
499 students or fewer	1
450-699 students	2
700 students or more	3

DEPARTMENT HEAD

D. At what school are you a department head? _________________________________________

E. Are you at a school with a population that includes: (READ LIST)

	SELECT ONE OR MORE
Eighth-grade students	1
Twelfth-grade students	2
Other	3	 (THANK AND TERMINATE)

F. What <department or subject> are you the <head or supervisor> of? ______________________________

	SELECT ONE
Social Studies	1
Economics	2	 (CONTINUE TO G)
Other	3	 (THANK AND TERMINATE)

G. Which economics classes are offered at your school? _____________________________

H. How many students are enrolled in your school?

	SELECT ONE
499 students or fewer	1
450-699 students	2
700 students or more	3

TEACHER

I. At what school are you a teacher? _________________________________________

J. What grade level do you teach?

	SELECT ONE OR MORE
Eighth-grade students	1
Twelfth-grade students	2
Other	3	 (THANK AND TERMINATE)

K. Do you currently teach <civics, geography, U.S History, or economics> to <8^th or 12^th> graders?

	SELECT ONE
Yes	1	(CONTINUE TO L)
No	2	 (THANK AND TERMINATE)

L. Which classes do you teach? ______________________________________________________

M. What is the size of the student population at your school?

	SELECT ONE
499 students or fewer	1
450-699 students	2
700 students or more	3

4. We are doing interviews between <DATE> and <DATE>. If you choose to participate, what day and time would work best for you?

[CHECK SCHEDULE FOR RESPONDENT’S LOCATION FOR AVAILABLE TIMES. IF NECESSARY, SUGGEST A DIFFERENT PLACE OR TIME.]

RECORD DATE AND TIME:

______________________________________________________________________________

5. You qualify to participate in a study interview in <LOCATION>. The interview will last no more than 60 minutes. Are you available for the interview at <TIME> on <DATE>? You will receive a $60 gift card for participating.

	SELECT ONE
Yes	1
No	2	(CONTINUE TO 8)

6. To attend the interview and receive the $60 gift card, we must receive the consent form that we will email to you. You can scan the consent form and email it to us, or you can bring it with you and give it to the researcher. Do you understand that you must have a completed and signed consent form to participate?

	SELECT ONE
Yes	1
No	2

7. Would you like me to send the consent form by email?

	SELECT ONE
Yes	1
No	2

email

7A. What is your email address?

RECORD EMAIL ADDRESS:

_____________________________________________________________________________

8. If we need to follow up with you about this study, will you please provide us with your contact information?

RECORD FIRST NAME: _____________________________________

RECORD LAST NAME: _____________________________________

RECORD EMAIL ADDRESS: _____________________________________

MOBILE PHONE NUMBER: _____________________________________

HOME PHONE NUMBER: _____________________________________

Thank you for helping us today. We look forward to seeing you at the study interview.

[END OF SCREENER]

Appendix S: Flyer/Information Brochure for Teachers, Department Heads, and School Administrators Professional Organizations, Education-Related Graduate Program, Teachers’ Unions, Educational Non-Profit/Community Based Organizations

Teachers, Department Heads, and School administrators will test new versions of survey questions for an important study for the U.S. Department of Education’s National Center of Education Statistics (NCES).

The research interview will last no more than 60 minutes. Teachers, Department Heads, and School administrators will receive a $60 gift card as a thank you for participating in this voluntary education study.

Survey responses will not be shared with schools and participants’ names will not be included in the write up of the results.*

For more information, please call us at

(866-693-2287), or go to the website:

EurekaFacts.com/Educational-Research

*All of the information provided by school staff and students may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any purpose except as required by law. (20 U.S.C. §9573 and 6 U.S.C. §151).

8^TH AND 12^TH

GRADE EDUCATORS

8^TH AND 12^TH

GRADE EDUCATORS

Appendix T: Frequently Asked Questions (FAQs) for Teachers, Department Heads, and School Administrators

Frequently Asked Questions

1. What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is administering a study with <teachers, department heads, or school administrators> in schools with students in <8^th or 12^th> grade to test new <civics, geography, U.S. History, or economics> survey questions for the National Assessment of Educational Progress (NAEP). <Teachers’, Department Heads’, or School administrators’> participation will help them improve survey questions before other educators take the questionnaire nationwide.

2. Where will the research interviews take place?

The study interview will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>.

3. What will happen during the research interview?

During the interview, the <teacher, department head, or school administrator> will receive a copy of multiple choice questions that he/she will be asked to answer verbally. A researcher will ask open-ended questions to follow up on why he/she selected each answer. There will be more than one researcher present during each research interview. NCES wants to hear his/her thoughts on improving the survey questions and making the questions easier to understand.

4. How long will the research interview last?

The interview will last no more than 60 minutes. The interview requires only one visit to the site.

5. Will information be kept confidential?

The participating <teacher, department head, or school administrator> will be assigned a unique identifier (ID), and at no time will his/her name be linked to any of his/her answers. The study will not retain any personally identifiable information. Prior to the start of the study, <teachers, department heads, or school administrators> will be notified that their participation is voluntary and that the information they provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). Audio activity will be recorded for analysis, but no personally identifying information will be recorded or retained.

6. Will the results go to the participating <teachers’, department heads’, or school administrators’> schools?

The participating <teachers’, department heads’, or school administrators’> answers will not be disclosed to their school and will not be linked to any personally identifiable information, such as a school’s name.

7. What are the possible risks of being in the study?

There are no known risks to participating in this study.

8. Is participation required?

Participation in this study is completely voluntary. The participating <teacher, department head, or school administrator> has the right to refuse to answer particular questions. The <teacher, department head, or school administrator> may elect to withdraw from this study at any time and can end early if he/she wishes.

9. Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this interview at EurekaFacts on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>.

Appendix U: Consent Form for Teacher, Department Head, and School Administrator

CONSENT FORM

FOR PARTICIPATION IN RESEARCH

Thank you for your interest in participating in the study described below. This consent form explains the study. Please read it carefully and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them later as they occur to you.

1. What is this study about?

The National Center for Education Statistics (NCES), part of the U.S. Department of Education, is administering a study with <teachers, department heads, or school administrators> in schools with students in <8^th or 12^th> grade to test new <civics, geography, U.S. History, or economics> survey questions for the National Assessment of Educational Progress (NAEP). <Teachers’, Department Heads’, or School administrators’> participation will help them improve survey questions before other educators take the questionnaire nationwide.

2. Where will the research interviews take place?

The study interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>.

3. What will happen during the research interview?

During the interview, the <teacher, department head, or school administrator> will receive a copy of multiple choice questions that he/she will be asked to answer verbally. A researcher will ask open-ended questions to follow up on why he/she selected each answer. There will be more than one researcher present during each research interview. NCES wants to hear his/her thoughts on improving the survey questions and making the questions easier to understand.

4. How long will the research interview last?

The interview will last no more than 60 minutes. The interview requires only one visit to the site.

5. Will information be kept confidential?

You will be assigned a unique identifier (ID) and at no time will your name be linked to any of your answers. The study will not retain any personally identifiable information. Prior to the start of the study, you will be notified that your participation is voluntary and that the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).Audio will be recorded for analysis, but no personally identifying information will be recorded or retained.

6. Will the results go to my school?

Your answers will not be disclosed to your school and will not be linked to any personally identifiable information, such as the school’s name.

7. What are the possible risks of being in the study?

There are no known risks to participating in this study.

8. Is participation required?

Participation in this study is completely voluntary. You have the right to refuse to answer particular questions. You may elect to withdraw from this study at any time and can end early if you wish.

You will still receive a gift card if you refuse to answer particular questions or elect to end the study early.

9. Whom can I contact with questions or for further information?

<CONTACT NAME> is coordinating this interview at EurekaFacts on behalf of NCES. If you have any questions, please call <CONTACT PHONE> or email <CONTACT EMAIL>.

  __ Yes, I agree to participate in this study

  __ No, I do not agree to participate in this study

By signing below, you agree that you, _______________________________, may participate in this study and that we may record your conversation with the researcher. You will receive a $60 gift card at the conclusion of the study.

Your signature below will indicate that your questions have been answered satisfactorily and that you have read and understood the information provided above.

Print Name: ___________________________________________________________________________

Signature: __________________________________________________ Date: __________________

Email address: _________________________________________________________________________

To be completed by EurekaFacts staff:

Teacher, Department Head, or School Adminstrator Payment Card Transaction Number: _______________________

Appendix V: Confirmation of Interview Email/Day of Interview Reminder Email for Teachers, Department Heads, and School Administrators

Dear <NAME>:

Thank you for agreeing to participate in this voluntary interview to provide feedback on survey questions for the National Assessment of Educational Progress (NAEP).

You have been scheduled for an interview at the following location and date/time:

Location: [For in-person interview: <ADDRESS>]

[For phone/WebEx interview: The interview will take place via <phone or WebEx> at <PHONE NUMBER or WEBEX URL>.]

Date: <DATE>

Time: <TIME>

Gift Card: You will receive a $60 gift card to thank you for your participation.

We have attached a consent form to this reminder.

[For in-person interview: Please sign, date, and return the form when you come to the interview, agreeing to participate in NAEP research.]

[For phone/WebEx interview: Please sign, date, and return the form prior to the interview via email at <EMAIL> or fax at <FAX NUMBER>, agreeing to participate in NAEP research.]

Thank you for volunteering in this very important research for NAEP. We look forward to seeing you. If you have any questions about the study, please contact <EurekaFacts CONTACT NAME> at <CONTACT INFORMATION>.

Sincerely,

<NAME>

<EurekaFacts, LLC>

Appendix W: Thank You Email/Letter to Teachers, Department Heads, and School Administrators

<DATE>

[IF SENT AS EMAIL—Subject: Thank you—NAEP Research Study]

Dear <NAME>,

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, EurekaFacts would like to thank you for participating in the study for the National Assessment of Educational Progress (NAEP).

Your participation helped us better understand how <teachers, department heads, or school administrators> approach the survey questions. Your open, candid, and insightful comments were valuable. Your input, combined with input from other <teachers, department heads, or school administrators> like you, provides incredibly useful information, and is helping us develop the best possible survey questions for the National Assessment of Educational Progress (NAEP).

For more information on NAEP, please visit the NCES NAEP website at: http://nces.ed.gov/nationsreportcard.

Again, thank you for your assistance.

Sincerely,

<CONTACT NAME>

EurekaFacts, LLC

<CONTACT INFORMATION >

Appendix X: Example Cognitive Lab Interview Tracking Schedule: Cognitive Interviews for Survey Questionnaires

An example tracking sheet to show how responses to screener questions (Appendices D, E, and R) will be tracked in order to be used to determine the targeted sample, including diversification on key characteristics.

1 The National Center for Education Statistics (NCES) is authorized to conduct NAEP by the National Assessment of Educational Progress Authorization Act (20 U.S.C. §9622). All of the information provided by school staff and students may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20

U.S.C. §9573 and 6 U.S.C. §151). Audio will be recorded for analysis and EurekaFacts staff may take notes about students’ reactions to the survey questions; however, no personally identifying information will be recorded or retained.

0 NAEP is the largest nationally representative assessment of what America's students know and can do in various subject areas. More information about the NAEP assessment is available online at: http://nces.ed.gov/nationsreportcard/.

0 The National Center for Education Statistics (NCES) is authorized to conduct NAEP by the National Assessment of Educational Progress Authorization Act (20 U.S.C. §9622). All of the information provided by school staff and students may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20

U.S.C. §9573 and 6 U.S.C. §151). Audio will be recorded for analysis and EurekaFacts staff may take notes about students’ reactions to the survey questions, however, no personally identifying information will be recorded or retained.

0 The National Center for Education Statistics (NCES) is authorized to conduct NAEP by the National Assessment of Educational Progress Authorization Act (20 U.S.C. §9622). All of the information provided by school staff and students may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20

U.S.C. §9573 and 6 U.S.C. §151). Audio will be recorded for analysis and EurekaFacts staff may take notes about school staff’s reactions to the survey questions; however, no personally identifying information will be recorded or retained.

0 NAEP is the largest nationally representative assessment of what America's students know and can do in various subject areas. More information about the NAEP assessment is available online at: http://nces.ed.gov/nationsreportcard/.

0 The National Center for Education Statistics (NCES) is authorized to conduct NAEP by the National Assessment of Educational Progress Authorization Act (20 U.S.C. §9622). All of the information provided by school staff and students may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). Audio will be recorded for analysis and EurekaFacts staff may take notes about school staff’s reactions to the survey questions; however, no personally identifying information will be recorded or retained.