Appendices for NAEP Questionnaires Cog Interviews

My name is <CONTACT NAME> and I am contacting you from <CRP, INC./EUREKAFACTS> (<http://www.crpcorp.com / http://www.eurekafacts.com>), a local research company. We are conducting an important study for the National Center for Education Statistics (NCES), part of the U.S. Department of Education.

We are recruiting <FOURTH-/EIGHTH-/TWELFTH-> grade students to participate in voluntary research interviews supporting the development of survey questions for the National Assessment of Educational Progress (NAEP) assessment. (More information about the NAEP assessment is available online at: http://nces.ed.gov/nationsreportcard/).

NCES needs to know if the survey questions are clear. Student feedback will help NCES improve the survey so students like your child understand all the questions. During the interview, your child will receive a printed copy of multiple-choice questions that he or she will be asked to answer verbally. The interviewer will ask open-ended questions to follow up on why your child selected each answer. The purpose of these interviews is not to test or grade students. NCES wants to hear their thoughts to improve the survey questions and make the questions easier to understand.

The interview for your child will take place at <LOCATION> and will last no more than 90 minutes. The interview can be scheduled at the convenience of you and your child. Students will be assigned a unique student identifier (ID), and their names will not be linked to their answers. Your child’s answers will be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573]. The sessions will be audio recorded and the recordings will be used only by the researchers conducting the study. No one else will hear the recordings.

Your child will receive a $25 Visa gift card at the conclusion of the session for his or her time and effort. If you provide transportation to and from the location of the interview for your child to participate in the cognitive interview session, you, the parent/legal guardian, will receive a $25 Visa gift card as well. In addition, you will receive a thank you letter/email for taking your time to bring your child to the testing site.

If your child is interested in participating in this voluntary research study, please call <CONTACT PHONE> or email <CONTACT EMAIL>. We look forward to speaking with you soon!

Thank you,

<NAME>

<CRP, INC./EUREKAFACTS>

Appendix B: Email Recruiting Teachers and School Administrators

Subject: Education Research Study Opportunity for Teachers and School Administrators

Dear <NAME>:

My name is <CONTACT NAME> and I am contacting you from <CRP, INC./EUREKAFACTS> (<http://www.crpcorp.com / http://www.eurekafacts.com>), a local research company. We are conducting an important study for the U.S. Department of Education’s National Center for Education Statistics (NCES).

We are recruiting teachers and school administrators to participate in voluntary research interviews supporting the development of survey questions for the National Assessment of Educational Progress (NAEP) assessment. (More information about the NAEP assessment is available online at: http://nces.ed.gov/nationsreportcard/).

NCES needs to know if the survey questions are clear. Your feedback will help NCES improve the survey so educators like you understand all the questions. During the interview, you will receive a printed copy of multiple-choice questions that you will be asked to answer verbally. The interviewer will ask open-ended questions to follow up on why you selected each answer.

The interviews will take place at <LOCATION> and will last no more than 90 minutes. A limited number of interviews may be conducted via phone or WebEx. The interview can be scheduled at your convenience. Your responses will be used only for research purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573]. The sessions will be audio recorded and the recordings will be used only by the researchers conducting the study. No one else will hear the recordings.

These voluntary research interviews are an important stage of the NAEP survey development process. Your comments, along with the comments of other educators, will help NCES improve the survey. You will receive a Visa gift card ($40 for in-person interview or $25 for phone or WebEx interview) at the conclusion of the session as a thank you for your time and effort.

If you are interested in participating in this voluntary research study, please call <CONTACT PHONE> or email <CONTACT EMAIL>. We look forward to speaking with you soon!

Thank you,

Appendix C: Recruitment Email/Letter to Adults who Requested Information on Research Study Opportunities

Subject: Education Research Study Opportunity for Students in Grades 4, 8, and 12

<DATE>

Hello,

My name is <CONTACT NAME> and I am contacting you from <CRP, INC./EUREKAFACTS>, a local research company. You are receiving this <EMAIL/LETTER> because you have previously given us your contact information in order to receive information regarding upcoming research opportunities.

If you are the parent or legal guardian of a child who is currently in fourth-, eighth-, or twelfth grade, or if you know anyone who is, we would like you to know that we are currently conducting an important research study on behalf of the U.S. Department of Education’s National Center for Educational Statistics (NCES). This study supports the development of new survey questions for the National Assessment of Educational Progress, also known as NAEP (an assessment given to students in elementary, middle, and high schools throughout the nation).

We are conducting research interviews to explore how students work through sample survey questions to ensure that the questions and instructions are clear and understandable to a broad range of students. All interviews will be scheduled for no more than 90 minutes. Students will receive a $25 Visa gift card for participating in the research interview. Parents or legal guardians (one per participating student) will also receive a $25 Visa gift card to thank them for their time in bringing their child to and from the interview site.

Students’ interview responses or materials will not be graded or shared with their school or anyone other than the researchers. These research interviews are an important stage of the NAEP survey development process. By sharing their feedback with us, students can directly impact the future of this national assessment by helping NCES create the best survey possible.

Students will be assigned a unique student identifier (ID), and their names will not be linked to their answers. Students’ responses will be used only for research purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573]. The sessions will be audio recorded and the recordings will be used only by the researchers conducting the study. No one else will hear the recordings.

I have included a brochure with a short description of this research study in the hope that you might pass it along to anyone who may be interested.

Thanks again for your consideration and assistance. We look forward to speaking with you, and please do not hesitate to contact me with any questions regarding this project or our company.

ATTACH: Brochure^¹

Sincerely,

<CRP, INC./EUREKAFACTS>

Appendix D: Recruitment Email/Letter to Parent-Teacher-Student Associations

Subject: Education Research Study Opportunity for Students in Grades 4, 8, and 12

<DATE>

Hello,

My name is <CONTACT NAME> and I am contacting you from <CRP, INC./EUREKAFACTS>, a local research company conducting an important study for the U.S. Department of Education’s National Center for Education Statistics (NCES). As a local <NAME OF ORGANIZATION> member, we are contacting you to ask for your help in spreading the word about this exciting research opportunity.

We are recruiting students who are currently in fourth-, eighth-, and twelfth grades to participate in research interviews supporting the development of new survey questions for the National Assessment of Educational Progress, also known as NAEP (an assessment given to students in elementary, middle, and high schools throughout the nation). The research interviews will explore how students work through the sample survey questions to ensure that the questions are clear and understandable to a broad range of students. All interviews will be scheduled for no more than 90 minutes. Students will receive a $25 Visa gift card for participating in the research interview. The student’s parent or legal guardian will also receive a $25 Visa gift card to thank him or her for bringing their child to and from the interview site.

We would like to ask for your assistance in letting parents/legal guardians and students know how they can help shape this national assessment. The feedback we receive from students on these survey questions will help to evaluate the questions and instructions and create the best survey possible.

I have included a brochure with a short description of this research study in the hope that you might pass it along to anyone who may be interested.

Thanks again for your consideration and assistance. We look forward to speaking with you, and please do not hesitate to contact me with any questions regarding this project or our company.

ATTACH: Brochure^²

Sincerely,

<CRP, INC./EUREKAFACTS>

Appendix E: Recruitment Email/Letter to Youth Organizations/Principals

Subject: Education Research Opportunity for Students in Grades 4, 8, and 12

<DATE>

Dear <NAME>,

My name is <CONTACT NAME> and I am contacting you from <CRP, INC./EUREKAFACTS>, a local research company conducting an important study for the U.S. Department of Education’s National Center for Education Statistics (NCES). Because you are a <leader in youth outreach/principal at an elementary, middle, or high school>, we are contacting you to ask for your help in spreading the word about this exciting research opportunity.

I have included a brochure with a short description of this research study in the hope that you might pass it along to anyone who may be interested.

Thanks again for your consideration and assistance. We look forward to speaking with you, and please do not hesitate to contact me with any questions regarding this project or our company.

ATTACH: Brochure^³

Sincerely,

<CRP, INC./EUREKAFACTS>

Appendix F: CRP Recruitment Flyer, Grade 4

Appendix G: CRP Recruitment Flyer, Grade 8

Appendix H: CRP Recruitment Flyer, Grade 12

Appendix I: CRP Recruitment Flyer, Teachers and School Administrators

Appendix J: EurekaFacts Recruitment Internet/Newspaper Advertisement/Flyer

EurekaFacts, a local research company, is conducting an important study for the U.S. Department of Education’s National Center for Education Statistics (NCES). If your child is currently in 4^th, 8^th, or 12^th grade, we need his or her help to evaluate new survey questions for the National Assessment of Educational Progress (NAEP) survey for elementary, middle, and high school students. Students’ input will help researchers evaluate the survey by ensuring that the questions and instructions are clear and understandable to a broad range of students.

EurekaFacts will provide a $25 Visa gift card to each student who participates in the study, plus a $25 Visa gift card to the parent/legal guardian who brings their child to and from the interview location (one interview per student). Interviews will begin this <SPECIFY SESSION OR MONTH> and may be scheduled at your convenience after school and on weekends. The interviews will last no more than 90 minutes. Please call, text, or email us now for more information about this opportunity!

Call or text: <CONTACT PHONE> or

Email: <EMAIL ADDRESS> or

Visit our website for more information: <WEBSITE>

Appendix K: Text for Information Brochure

<CRP, INC. /EUREKAFACTS> is a local research company conducting an important study for the U.S. Department of Education’s National Center for Education Statistics (NCES). If your child is currently in 4^th, 8^th, or 12^th grade, we need his or her help to evaluate new survey questions for the National Assessment of Educational Progress (NAEP) survey for elementary, middle, and high school students. Students’ input will help researchers evaluate the survey by ensuring that the questions and instructions are clear and understandable to a broad range of students.

<CRP, INC./EUREKAFACTS> will provide a $25 Visa gift card to each student who participates in the study, plus a $25 Visa gift card to the parent/legal guardian who brings their child to and from the interview location (one interview per student). Interviews will begin this <SPECIFY SESSION OR MONTH> and may be scheduled at your convenience after school and on weekends. The interviews will last no more than 90 minutes. Please call, text, or email us now for more information about this opportunity!

Call or text: <CONTACT PHONE> or

Email: <EMAIL ADDRESS> or

Visit our website for more information: <WEBSITE>

Appendix L: Text for Informational Bookmarks

Participate! <CRP, INC./EUREKAFACTS>, a local research company, is conducting an important study for the U.S. Department of Education’s National Center for Education Statistics (NCES). Help evaluate new survey questions for the National Assessment of Educational Progress (NAEP) to improve the survey. Call or text <CONTACT NUMBER>. Students in fourth-, eighth-, and twelfth- grades wanted - research study. Receive a gift card. Visit our website for more information: <WEBSITE>

Appendix M: CRP Youth Organizations/Principals Follow-Up Script

Good <MORNING/AFTERNOON>. May I please speak to <NAME OR NAME’S ASSISTANT>?

^M^yⁿ^a^m^e^is^<NAME>
fr^o^m^C^R^P^.^I^am^cal^lⁱⁿ^g^t^o^f^o^l^l^o^w^u^p^oⁿ<AN EMAIL/A LETTER>^th^a^t^w^as^s^ent^oⁿ^<DATE>regarding interviews CRP has been asked to conduct on behalf of the United States Department of Education’s National Center for Education Statistics, also known as NCES, in collaboration with the Educational Testing Service, also known as ETS. The survey interviews are being conducted through the National Assessment of Educational Progress, to aid in assessment survey questionnaire development. Students from your <SCHOOL/ORGANIZATION> have been invited to participate in the cognitive interview.

Are you interested in having your <4th/8th/12^th> grade students participate in this research study?

^Ifⁿ^o^:^Thaⁿ^k^yo^u^f^o^r^y^o^u^r
tⁱ^m^e.Have a good <DAY/EVENING>

^If^y^e^s^:^G^r^ea^t.^Thaⁿ^k^yo^u^v^e^r^y^m^u^ch.^We^w^o^u^ld^ap^p^recⁱ^ate^it^if^yo^u^c^o^u^ld^dis^s^e^mⁱⁿ^a^t^e^the^stu^d^ent
f^l^y^ers^pr^o^vⁱ^d^edto students, parents, and legal guardians. Interested students, parents, and legal guardians may contact <NAME>, of CRP, at 1-866-277-4772 to confirm their participation, ask questions, and obtain consent forms and logistics information. Would you like me to repeat the contact information? (If yes, do so.)

^{If
we ne}^e^d^to^r^e^ach^y^o^{u
again, is}^t^h^eⁿ^u^m^b^er^I
cal^l^ed
the^b^estⁿ^u^m^b^er^t^o^contact
you?

Do you have any questions at this time?

^Thaⁿ^k^y^o^u^agaⁱⁿ^f^o^r
a^g^r^e^eⁱⁿ^g^t^o^be^part^o^f^t^h^is^v^eryⁱ^m^p^o^r^t^aⁿ^t^r^es^e^arch^s^t^ud^y^.Have a good <DAY/EVENING>

Appendix N: Parent or Legal Guardian of Student (Under Age 18) Participant Recruitment Screener

Cold-calling script for parents/legal guardians:

Good <MORNING/AFTERNOON/EVENING>. Is this the <NAME FROM CALL LIST> household? My name is <NAME>, and I am calling from <CRP, INC./EUREKAFACTS>. We are a local research company conducting a study on behalf of the U.S. Department of Education’s National Center for Educational Statistics, also known as NCES. We are recruiting students currently in fourth-, eighth-, and twelfth grades to sign up to participate in a research study to help in the development of new survey questions for the National Assessment of Educational Progress, also known as the NAEP assessment. [If parent/legal guardian asks what this is: This national assessment is administered to students in elementary, middle, and high schools throughout the United States.]

Are there any fourth-, eighth-, or twelfth grade students in your household?

Yes (Great! Let me give you some additional information about the study.)

No (Thank you for your time. Have a good <DAY/EVENING>.)

The survey will be administered to students throughout the United States. We are conducting research interviews to explore how students work through sample survey questions to ensure that they are clear and understandable to a broad range of students. Your child will receive a $25 Visa gift card for participating in the research interview. You will also receive a $25 Visa gift card to thank you for your time in bringing your child to and from the interview site.

The research interview will take place at <LOCATION>. It can be scheduled at your convenience and will last no more than 90 minutes. During the interview, your child will be asked to respond verbally to sample survey questions. NCES simply wants to know how he or she would answer the questions if they were included on an actual survey given to students after taking a NAEP test.

Students will be assigned a unique student identifier (ID), and their names will not be linked to their answers. Your child’s responses will be used only for research purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573]. The sessions will be audio recorded and the recordings will be used only by the researchers conducting the study. No one else will hear the recordings. (INTERVIEWER: This reference is for informational purposes only. If asked what law is being referenced, please provide the information contained within this bracketed text.)

These research interviews are an important stage of the NAEP survey development process. By sharing their feedback with us, students can directly impact the future of this nationwide assessment by helping NCES evaluate the survey questions and instructions to create the best survey possible.

Do you believe your child would be interested in participating?

Yes

No (Thank you for your time. Have a good <DAY/EVENING>.)

Great! To ensure that we interview a broad mix of students, I have a few brief screening questions to ask about you and your child. It should only take a few minutes of your time. First, I want to collect some additional information about your child to identify if (HE/SHE) meets the participation requirements.

Skip to “Screener”

Script when calling parent/legal guardian who responded to email, advertisement, or invitation.

Good <MORNING/AFTERNOON/EVENING>. My name is <NAME>. I’m calling from <CRP,INC. /EUREKAFACTS> on behalf of the U.S. Department of Education’s National Center for Educational Statistics, also known as NCES. Thank you for contacting us about the possibility of your child participating in a research interview to help develop survey questions for the National Assessment of Educational Progress, also known as the NAEP assessment. To ensure that we interview a broad mix of students, I have a few brief screening questions to ask about you and your child. It should only take a few minutes of your time. I want to collect some additional information about your child to identify if (HE/SHE) meets the participation requirements.

Before proceeding, please confirm that you are still interested in the possibility of your child participating in this research interview?

Yes (INTERVIEWER: Proceed to “Screener.”)

No (Thank you for your time. Have a good <DAY/EVENING>.)

Screener:

(INTERVIEWER: Determine recruitment needs in advance. Record all responses. End screener at whatever point the parent or legal guardian gives a response that confirms that his/her group has already been sampled adequately. End screener script: Based on the requirements of this study, we are not able to include your child in the interviews at this time. Thank you for your time. Have a good <DAY/EVENING>.)

What is your child’s name?

What is your child’s gender?

Male
Female

What grade is your child currently in? (INTERVIEWER: Do not read this list.)

 Grades K-3  (Unfortunately, we are only interviewing students currently in grades 4, 8, or 12 for this study. Thank you for your time. Have a good <DAY/EVENING>.)

 Grade 4

 Grades 5-7  (Unfortunately, we are only interviewing students currently in grades 4, 8, or 12 for this study. Thank you for your time. Have a good <DAY/EVENING>.)

 Grade 8

 Grades 9-11 (Unfortunately, we are only interviewing students currently in grades 4, 8, or 12 for this study. Thank you for your time. Have a good <DAY/EVENING>.)

 Grade 12 (INTERVIEWER: Ask parent/legal guardian if his/her child is age 18 or over. If yes, give the parent/legal guardian the option to have the child come to the phone to respond to the “Students Age 18 or Over Recruitment Screener” (see Appendix O, starting with question 3). If the student is under age 18, or if the student is age 18 or over and the parent/legal guardian wants to proceed with the screener himself/herself, proceed with question 4 below.)

What school does your child attend? (INTERVIEWER: Note if homeschool.)

____________________________

What school district does HE/SHE attend? (INTERVIEWER: Note if parochial, charter, or homeschool.)

____________________________

What is your child’s race or ethnicity? (INTERVIEWER: Only read list if parent/legal guardian doesn’t immediately reply.) Is your child…?

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

 American Indian or Alaska Native

 Asian

 Black or African American

 Native Hawaiian or other Pacific Islander

 White or Caucasian

Just to be sure we're interviewing a broad range of students for this study, what is your total annual household income?

(INTERVIEWER: If the participant indicates a preference to not disclose this information, say: “That’s fine” and move on to the next question.)

Would you consider the area you live in to be …

Urban
Suburban
Rural

What is the highest level of education that you have completed? (INTERVIEWER: Do not read this list.)

Less than high school
High school degree or GED
Some college
Trade school degree
Associate’s or Bachelor’s degree
Post graduate degree

(INTERVIEWER: Questions 10 and 11 below are only for parents/legal guardians of grade 12 students.)

10. Does your child currently take a math class?

 Yes  What is the name of your class? (INTERVIEWER: Record class name.)

 No

11. Does your child currently take an English course?

 Yes  What is the name of your class? (INTERVIEWER: Record class name.)

 No

Thank you for answering these questions. Your child is eligible to participate in the NAEP research interviews and thank you for allowing <HIM/HER> to do so.

Please note that you will be required to sign a consent form to confirm that you have agreed to allow your child to participate in the interview. What email or mailing address should we send the consent form to? (INTERVIEWER: Record address.)

Please bring the signed consent form to the interview; otherwise your child will not be able to participate. Thank you so much for talking with us today. We look forward to having your child participate in the interview.

IF PARENT/LEGAL GUARDIAN AVAILABLE AND CURRENTLY SCHEDULING INTERVIEWS

We will be holding our current interviews from <DATE> to <DATE>. What would be the best date and time to schedule the interview?

Date ____________________

Time ____________________

Thank you again for talking with us today and for allowing your child to participate in NAEP research interviews. Have a good <DAY/EVENING>.

IF PARENT/LEGAL GUARDIAN AVAILABLE AND NOT CURRENTLY SCHEDULING INTERVIEWS

We are in-between interview sessions, but will be scheduling sessions until <DATE>. We will need to call you back when we are actively scheduling the interviews to determine the date and time that will work best, but what are your scheduling preferences? (INTERVIEWER: Select all that apply.)

 Morning

 Afternoon

 After school

 Weekends

Thank you again for talking with us today and for allowing your child to participate in NAEP research interviews. We look forward to speaking with you soon to schedule your child’s interview. Please provide the best phone number to reach you at. Have a good <DAY/EVENING>.

Appendix O: Students Age 18 or Over Recruitment Screener

Script when calling student age 18 or over who responded to email, advertisement, or invitation.

Good <MORNING/AFTERNOON/EVENING>. My name is <NAME>. I’m calling from <CRP, INC./EUREKAFACTS> on behalf of the U.S. Department of Education’s National Center for Educational Statistics, also known as NCES. Thank you for contacting us about the possibility of participating in a research interview to help develop survey questions for the National Assessment of Educational Progress, also known as the NAEP assessment. To ensure that we interview a broad mix of students, I have a few brief screening questions to ask you. It should only take a few minutes of your time. First, I would like to collect some information about you to determine if you meet the participation requirements.

Before proceeding, please confirm that you are still interested in the possibility of participating in this research interview?

Yes (INTERVIEWER: Proceed to “Screener.”)

No (Thank you for your time. Have a good <DAY/EVENING>.)

Screener:

(INTERVIEWER: Determine recruitment needs in advance. Record all responses. End screener at whatever point the student gives a response that confirms that his/her group has already been sampled adequately. End screener script: Based on the requirements of this study, we are not able to include you in the interviews at this time. We greatly appreciate your help. Thank you for your time. Have a good <DAY/EVENING>.)

What is your name?

What grade are you currently in at school? (If the student indicates that he/she is in any grade other than 12^th, thank and terminate: Unfortunately we are only interviewing students currently in grades 4, 8, or 12 for this study. Thank you for your time. Have a good <DAY/EVENING>.)

What school do you attend? (INTERVIEWER: Note if homeschool.)

____________________________

What school district do you attend? (INTERVIEWER: Note if parochial, charter, or homeschool.)

____________________________

RECORD GENDER (only ask if necessary)

____________________________

What is your race or ethnicity? (INTERVIEWER: Only read list if participant doesn’t immediately reply.) Are you…?

Ethnicity:  Hispanic or Latino  Not Hispanic or Latino

Race (one or more of the categories below):

 American Indian or Alaska Native

 Asian

 Black or African American

 Native Hawaiian or other Pacific Islander

 White or Caucasian

Would you consider the area you live in to be …

Urban
Suburban
Rural

Just to be sure, we're interviewing a broad range of students for this study, what is your total annual household income? Use your best judgment; however it is ok if you do not want to answer the question.

(INTERVIEWER: If the participant indicates a preference to not disclose this information, say: “That’s fine,” and move on to the next question.)

Do you currently take a math class?

 Yes  What is the name of your class? (INTERVIEWER: Record class name.)

 No

Do you currently take an English course?

 Yes What is the name of your class? (INTERVIEWER: Record class name.)

 No

Thank you for answering these questions. You are eligible to participate in the NAEP research interviews.

For the research interview, you will be asked to visit <LOCATION> to respond to a sample of survey questions. It can be scheduled at your convenience and will last no more than 90 minutes. You will be assigned a unique student identifier (ID), and your name will not be linked to your answers. Your responses will be used only for research purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573]. (INTERVIEWER: This reference is for informational purposes only. If asked what law is being referenced, please provide the information contained within this bracketed text.)

You will not be graded; the National Center for Educational Statistics simply wants to know how you would answer the questions if they were included on an actual survey given to students after taking a NAEP test. Your input will help evaluate the survey that will be given to students throughout the nation.

To thank you for your participation, you will receive a $25 Visa gift card. If your parent or legal guardian brings you to and from our office, he or she will also receive a $25 Visa gift card. The interview will last about 90 minutes and will be audio recorded. Your participation is totally voluntary. Your interview responses and all other interview materials will only be used for research purposes and will not use your name to identify you.

Please note that before your interview starts, you will be asked to sign a consent form to confirm that you have agreed to participate in the interview. You will not be able to participate until the form has been signed.

IF CURRENTLY SCHEDULING INTERVIEWS

We will be holding our current interviews from <DATE> to <DATE>. What would be the best date and time to schedule the interview?

Date ____________________

Time ____________________

Thank you again for talking with us today and for agreeing to participate in NAEP research interviews. Have a good <DAY/EVENING>.

IF NOT CURRENTLY SCHEDULING INTERVIEWS

 Morning

 Afternoon

 After-school

 Weekends

Thank you again for talking with us today and for agreeing to participate in NAEP research interviews. We look forward to speaking with you soon to schedule your interview. Please provide the best phone number to reach you at. Have a good <DAY/EVENING>.

Appendix P: Teacher and School Administrator Recruitment Screener

Good <MORNING/AFTERNOON/EVENING>. May I please speak to <NAME>?

Good <MORNING/AFTERNOON/EVENING>. My name is <CONTACT NAME> and I am contacting you from <CRP, INC./EUREKAFACTS>, a local research company. We are conducting an important study for the U.S. Department of Education’s National Center for Education Statistics, also known as NCES.

The National Assessment of Educational Progress, also known as NAEP, has developed survey questions for teachers and school administrators. After this call, you can find more information about the NAEP assessment online at: http://nces.ed.gov/nationsreportcard/.

NCES needs to know if the survey questions are clear. Your input will help NCES improve the survey so educators like you understand all the questions. During the interview, you will receive a printed copy of multiple-choice questions that you will be asked to answer verbally. The interviewer will ask open-ended questions to follow up on why you selected each answer.

The voluntary research interviews will take place at <LOCATION> and will last no more than 90 minutes. A limited number of interviews may be conducted via phone or WebEx. Your responses will be used only for research purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573]. (INTERVIEWER: This reference is for informational purposes only. If asked what law is being referenced, please provide the information contained within this bracketed text.)

These interviews are an important stage of the NAEP survey development process. Your comments, along with the comments of other teachers and school administrators, will help NCES improve the survey. You will receive a Visa gift card ($40 for in-person interview or $25 for phone or WebEx interview) at the conclusion of the session as a thank you for your time and effort.

Are you interested in participating in this research study?

If no: Thank you for your time. Have a good <DAY/EVENING>.

If yes: Great, thank you very much. To ensure that we interview a broad mix of educators, I have a few brief screening questions to ask you. It should only take a few minutes of your time.

Screener:

(INTERVIEWER: Determine recruitment needs in advance. Record all responses. End screener at whatever point the teacher or administrator gives a response that confirms that their group has already been sampled adequately. End screener script: Based on the requirements of this study, we are not able to include you in the interviews at this time. We greatly appreciate your help. Thank you for your time. Have a good <DAY/EVENING>.)

What is your job title?

Principal/Administrator (GO TO Part A)
Teacher (GO TO Part B)

Part A.

At what school are you an administrator?

Are you at a school with a population that includes:

Fourth-grade students
Eighth-grade students
Twelfth-grade students

How many students are enrolled in your school?
1. 299 students or less
2. 300-599 students
3. 600 students or more

Part B.

At what school are you a teacher?

What grade level do you teach?
1. Fourth grade
2. Eighth grade
3. Twelfth grade
4. Other END SCREENER (Based on the requirements of this study, we are not able to include you in the interviews at this time. We greatly appreciate your help. Thank you for your time. Have a good <DAY/EVENING>.)

Do you currently (or within this school year) teach math or reading/English to fourth/eighth graders?
1. Yes  Which classes do you teach? (INTERVIEWER: Record class names.)
2. No END SCREENER (Based on the requirements of this study, we are not able to include you in the interviews at this time. We greatly appreciate your help. Thank you for your time. Have a good <DAY/EVENING>.)

What is the size of the student population at your school?
1. 299 students or less
2. 300-599 students
3. 600 students or more

Eligible Participant Script

Thank you for taking the time to answer the screening questions. You are eligible to participate in the voluntary research interviews. Again, the interview will last no more than 90 minutes and will be held at <LOCATION>. Are you available for the interview at <TIME> on <DATE>?

(INTERVIEWER: If necessary, provide alternate times/dates to find an available time/date for interview and then record time/date of appointment.)

Please note that before your interview starts, you will be asked to sign a consent form to confirm that you have agreed to participate in the interview.

For in-person interviews:

You will not be able to participate until the form has been signed.

For virtual interviews:

What email or mailing address should we send the consent form to? (INTERVIEWER: Record address.) Please return the signed consent form via email or mail prior to the interview; otherwise you will not be able to participate.

We may need to reach you by telephone. Is the number I called the best number to use?

Do you have any questions at this time?

(INTERVIEWER: Answer any questions.)

If you do have any questions before your interview scheduled for <DATE/TIME>, please call <PHONE NUMBER>.

Thank you again for agreeing to be part of this very important research study.

Appendix Q: Parent or Legal Guardian of Student (Under Age 18) Participant Consent Form

PARENTAL CONSENT DOCUMENT

FOR PARTICIPATION IN RESEARCH

<CRP, INC./EUREKAFACTS>

Title of Research Study: Cognitive Laboratory Interviews for NAEP Survey Question Testing

Thank you for agreeing to your child’s participation in the research study described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them if they occur to you later.

What is this study about?

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, <CRP, INC./EUREKAFACTS> is conducting interviews with students currently in fourth-, eighth-, and twelfth grades about new survey questions for the National Assessment of Educational Progress (NAEP). The survey questions will be part of a new computer-based test given to a sample of students across the country. Your child’s feedback will help NCES improve the survey so students like him or her understand all the questions. (More information about the NAEP assessment is available online at: http://nces.ed.gov/nationsreportcard/)

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION> at your convenience at an after-school or weekend session.

What will happen during the research interview?

During the interview, your child will receive a printed copy of survey questions that he or she will be asked to answer verbally. The interviewer will ask open-ended questions to follow up on why your child selected each answer. Questions will focus on your child’s understanding of the questions. Your child will also be asked whether he or she has any suggestions on how we might improve the questions. The purpose of these interviews is not to test or grade students. NCES wants to hear their thoughts to improve the survey questions and make the questions easier to understand.

Will you keep information confidential?

Your child will be assigned a unique student identifier (ID), and his or her name will not be linked to his or her answers. Your child’s responses will be used only for research purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573]. The sessions will be audio recorded and the recordings will be used only by the researchers conducting the study. No one else will hear the recordings.

How long will the research interview last and when will it take place?

The interview will last no more than 90 minutes and requires only one visit to our office. Sessions will take place at a time and date convenient to the participants’ parent(s) or legal guardian(s). <TBD: note days of the week when sessions are scheduled and the window of time when the sessions will be held>.

Will the results go to my child’s school?

Your child’s answers will not be disclosed to his or her school, and will not be linked to any personally identifiable information or to the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this voluntary study.

Who can I contact with questions or for further information?

<CONTACT NAME> is conducting this study at <CRP, INC. /EUREKAFACTS> on behalf of NCES. If you have any questions, please call <HIM/HER> at <CONTACT PHONE> or by email <CONTACT EMAIL>.

Can participation end early?

Participation in this study is completely voluntary. Your child has the right to refuse to answer particular questions. Your child may elect to withdraw from this study at any time and can end the session early if he or she wishes.

By signing below, you agree that your child, _______________________________, may participate in this research study and that you have read and understood the information provided above. Your child will receive a $25 Visa gift card at the conclusion of the session as a thank you for his or her time and effort. If you provide transportation to and from the location of the interview for your child to participate in the cognitive interview session, you, the parent/legal guardian, will receive a $25 Visa gift card as well. In addition, you will receive a thank you letter/email for taking your time to bring your child to the testing site. We hope that you will give your consent for your child to participate in the voluntary NAEP interview by signing this form. Please bring the signed consent form to the interview; otherwise your child will not be able to participate.

Print Name: ________________________________________

Signature:______________________________________________Date:________________________

Email address: _____________________________________________

Appendix R: Student (Age 18 or Older) Consent Form

PARTICIPATION CONSENT DOCUMENT

FOR PARTICIPATION IN RESEARCH

<CRP, INC. /EUREKAFACTS>

Title of Research Study: Cognitive Laboratory Interviews for NAEP Survey Question Testing

Thank you for your interest in participating in the research study described below. This consent form explains the research study. Please read it carefully, and feel free to ask questions about anything you do not understand. If you do not have questions now, you may ask them if they occur to you later.

What is this study about?

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, <CRP, INC./EUREKAFACTS> is conducting interviews with students currently in fourth-, eighth-, and twelfth grades about new survey questions for the National Assessment of Educational Progress (NAEP). The survey questions will be part of a new computer-based test given to a sample of students across the country. Your feedback will help NCES improve the survey so students like you understand all the questions. (More information about the NAEP assessment is available online at: http://nces.ed.gov/nationsreportcard/)

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION> at your convenience at an after-school or weekend session.

What will happen during the research interview?

During the interview, you will receive a printed copy of survey questions that you will be asked to answer verbally. The interviewer will ask open-ended questions to follow up on why you selected each answer. Questions will focus on your understanding of the questions. You will also be asked whether you have any suggestions on how we might improve the questions. The purpose of these interviews is not to test or grade students. NCES wants to hear your thoughts to improve the survey questions and make the questions easier to understand.

Will you keep my information confidential?

You will be assigned a unique student identifier (ID), and your name will not be linked to your answers. Your responses will be used only for research purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573]. The sessions will be audio recorded and the recordings will be used only by the researchers conducting the study. No one else will hear the recordings.

How long will the research interview last and when will it take place?

The interview will last no more than 90 minutes and requires only one visit to our office. <TBD: note days of the week when sessions are scheduled and the window of time when the sessions will be held>.

Will the results go to my school?

Your answers will not be disclosed to your school, and will not be linked to any personally identifiable information or to the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this voluntary study.

Who can I contact with questions or for further information?

<CONTACT NAME> is conducting this study at <CRP, INC./EUREKAFACTS> on behalf of NCES. If you have any questions, please call <HIM/HER> at <CONTACT PHONE> or by email <CONTACT EMAIL>.

Can my participation end early?

Participation in this study is completely voluntary. You have the right to refuse to answer particular questions. You may elect to withdraw from this study at any time.

By signing below, you agree that you will participate in this research study and that you have read and understood the information provided above. You will receive a $25 Visa gift card at the conclusion of the session to thank you for your time and effort. Your parent or legal guardian will also receive a $25 Visa gift card if he or she drives you to and from the interview session. We thank you for your voluntary participation and ask you to sign the form below.

Print Name: _____________________________________________

Signature:______________________________________________Date:________________________

Email address: _____________________________________________

Appendix S: Teacher and School Administrator Consent Form

PARTICIPATION CONSENT DOCUMENT

FOR PARTICIPATION IN RESEARCH

<CRP, INC. /EUREKAFACTS>

Title of Research Study: Cognitive Laboratory Interviews for NAEP Survey Question Testing

What is this study about?

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, <CRP, INC./EUREKAFACTS> is conducting interviews with teachers and administrators in schools with students currently in fourth-, eighth-, and twelfth grades about new survey questions for the National Assessment of Educational Progress (NAEP). The survey questions will be part of a new Technology-based assessment given to a sample of students across the country. Your feedback will help NCES improve the survey so educators like you understand all the questions. (More information about the NAEP assessment is available online at: http://nces.ed.gov/nationsreportcard/)

Where will the research interviews take place?

The interviews will take place at <NAME OF INTERVIEW LOCATION>, located at <LOCATION>, near <NEARBY LANDMARK OR METRO STATION> at an after-school or weekend session. A limited number of interviews may be conducted via phone or WebEx.

What will happen during the research interview?

During the interview, you will receive a printed copy of survey questions that you will be asked to answer verbally. The interviewer will ask open-ended questions to follow up on why you selected each answer. Questions will focus on your understanding of the questions. You will also be asked whether you have any suggestions on how we might improve the questions. NCES wants to hear your thoughts to improve the survey questions and make the questions easier to understand.

Will you keep my information confidential?

You will be assigned a unique identifier (ID), and your name will not be linked to your responses. Your responses will be used only for research purposes and may not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573]. The sessions will be audio recorded and the recordings will be used only by the researchers conducting the study. No one else will hear the recordings.

How long will the research interview last and when will it take place?

The interview will last no more than 90 minutes and, if conducted in-person, requires only one visit to our office. Virtual interviews require only one phone call or WebEx session.

Will the results go to my school?

Your answers will not be disclosed to your school, and will not be linked to any personally identifiable information or to the school’s name.

What are the possible risks of being in the study?

There are no known risks to participating in this voluntary study.

Who can I contact with questions or for further information?

<CONTACT NAME> is conducting this study at <CRP, INC./EUREKAFACTS> on behalf of NCES. If you have any questions, please call <HIM/HER> at <CONTACT PHONE> or by email <CONTACT EMAIL>.

Can my participation end early? Your participation in this study is completely voluntary. You have the right to refuse to answer particular questions. You may elect to withdraw from this study at any time.

By signing below, you agree that you will participate in this research study and that you have read and understood the information provided above. You will receive a Visa gift card ($40 for in-person interview or $25 for phone or WebEx interview) at the conclusion of the session as a thank you for your time and effort. We thank you for your voluntary participation and ask you to sign the form below.

Print Name: _____________________________________________

Signature:___________________________________________________Date:___________________

Email address: _____________________________________________

Kindly return the signed form to <EMAIL ADDRESS> OR <MAILING ADDRESS> prior to your interview, or bring the signed form with you to the interview location, if applicable.

Appendix T: Email/Letter Confirmation to Parent or Legal Guardian of Student (under age 18) Participants

Subject: Confirmation of Participation

<DATE>

Dear <NAME>:

Thank you for agreeing to allow your child, <STUDENT NAME>, to participate in this voluntary research interview to provide feedback on survey questions for the National Assessment of Educational Progress (NAEP). All the information your child provides will be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573].

Your child has been scheduled for a research interview at <TIME> on <DATE>. The interview will last no more than 90 minutes. Your child will receive a $25 Visa gift card at the conclusion of the session as a thank you for <HIS/HER> time and effort. If you provide transportation to and from the location of the interview for your child to participate in the cognitive interview session, you, the parent/legal guardian, will receive a $25 Visa gift card as well. In addition, you will receive a thank you letter/email for taking your time to bring your child to the testing site.

The interview will take place at <LOCATION> at:

We have <ATTACHED/INCLUDED> a consent form <TO/WITH> this confirmation. Please sign, date, and return the form when you bring your child to the interview, granting consent for your child to participate in the voluntary NAEP research. Without your consent, your child will not be able to participate in the interview.

We look forward to having <STUDENT NAME> participate in this very important research study. If you or your child has any questions, please contact <CONTACT INFORMATION>.

Sincerely,

<NAME>

<CRP, INC./EUREKAFACTS>

Appendix U: Email/Letter Confirmation to Student (Age 18 or Older) Participants

Subject: Confirmation of Participation

<DATE>

Dear <NAME>:

Thank you for agreeing to participate in this voluntary research interview to provide feedback on survey questions for the National Assessment of Educational Progress (NAEP). All the information you provide will be used only for research purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573].

You have been scheduled for a research interview at <TIME> on <DATE>. The interview will last no more than 90 minutes. You will receive a $25 Visa gift card at the conclusion of the session as a thank you for your time and effort.

The interview will take place at <LOCATION> at:

We have <ATTACHED/INCLUDED> a consent form <TO/WITH> this confirmation. Please sign, date, and return the form when you come to the interview, agreeing to participate in the voluntary NAEP research.

We look forward to having you participate in this very important research study. If you have any questions, please contact <CONTACT INFORMATION>.

Sincerely,

<NAME>

<CRP, INC./EUREKAFACTS>

Appendix V: Email/Letter Confirmation to Teachers and School Administrators

Subject: Confirmation of Participation

<DATE>

Dear <NAME>:

Thank you for agreeing to participate in this voluntary research interview to provide feedback on survey questions for the National Assessment of Educational Progress (NAEP). All the information you provide will be used only for research purposes and will not be disclosed, or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C. § 9573].

You have been scheduled for a research interview at <TIME> on <DATE>. The interview will last no more than 90 minutes. You will receive a Visa gift card ($40 for in-person interview or $25 for phone or WebEx interview) at the conclusion of the session as a thank you for your time and effort.

[TEXT FOR IN-PERSON INTERVIEW] The interview will take place at <LOCATION> at:

[TEXT FOR PHONE/WEBEX INTERVIEW] The interview will take place via <PHONE/WEBEX> AT:

We have <ATTACHED/INCLUDED> a consent form <TO/WITH> this confirmation.

[TEXT FOR IN-PERSON INTERVIEW] Please sign, date, and return the form when you come to the interview, agreeing to participate in the voluntary NAEP research.

[TEXT FOR PHONE/WEBEX INTERVIEW] Please sign, date, and return the form prior to the interview via email at <EMAIL> or fax at <FAX NUMBER>, agreeing to participate in the voluntary NAEP research.

We look forward to having you participate in this very important research study. If you have any questions, please contact <CONTACT INFORMATION>.

Sincerely,

<NAME>

<CRP, INC./EUREKAFACTS>

Appendix W: Thank You Letter to Parent or Legal Guardian of Student (under age 18) Participants

<DATE>

Dear <NAME>:

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, I would like to thank you for allowing your child to participate in our survey research study.

The assistance provided by your child helped us to better understand how students approach the survey questions. Your child’s input, combined with input from other students, is incredibly useful and informative, and is helping us develop the best possible survey questions for the National Assessment of Educational Progress (NAEP).

Again, thank you for your assistance.

Sincerely,

<NAME>

<CRP, INC. /EUREKAFACTS>

Appendix X: Thank You Letter to Student (age 18 or older) Participants

<DATE>

Dear <NAME>:

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, I would like to thank you for your participation in our survey research study.

Your assistance helped us better understand how students approach the survey questions. Your open, candid, and insightful comments were very valuable. Your input, combined with input from other students like you, is incredibly useful and informative, and is helping us develop the best possible survey questions for the National Assessment of Educational Progress (NAEP).

Again, thank you for your assistance.

Sincerely,

<NAME>

<CRP, INC./EUREKAFACTS>

Appendix Y: Thank You Letter to Teachers and School Administrators

<DATE>

Dear <NAME>:

On behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education, I would like to thank you for your participation in our survey research study. Thanks to you, we are better able to develop the best possible survey questions for the National Assessment of Educational Progress (NAEP).

Your assistance helped us better understand how <TEACHERS/SCHOOL ADMINISTRATORS> approach the survey questions. Your open, candid, and insightful comments were very valuable. Your input, combined with input from other <TEACHERS/SCHOOL ADMINISTRATORS> like you, is incredibly useful and informative.

Again, thank you for your assistance.

Sincerely,

<NAME>

<CRP, INC./EUREKAFACTS>

1 Note: Text for the brochure can be found in Appendix K.

2 Note: Text for the brochure can be found in Appendix K.

3 Note: Text for the brochure can be found in Appendix K.

File Type	application/msword
File Title	NATIONAL CENTER FOR EDUCATION STATISTICS
Author	Laurie Policastro
Last Modified By	U.S. Department of Education
File Modified	2014-11-06
File Created	2014-10-31

Appendices for NAEP Questionnaires Cog Interviews

Appendices NAEP Questionnaires Cog Interviews.doc

NCES Cognitive, Pilot, and Field Test Studies System