National Household Education Surveys Program (NHES) Screener Opt-out Cognitive Interviews

Attachment 2

National Household Education Surveys Program (NHES)

Screener Opt-out Cognitive Interviews

Study Materials

OMB# 1850-0803 v.231

May 2018

National Center for Education Statistics (NCES)

Contents

NHES Screener Opt-Out Cognitive Interview Protocol - English 2

NHES Screener Opt-Out Cognitive Interview Protocol – Spanish 7

NHES Screener Opt-Out Screener Cover Letter – Bilingual 12

NHES Screener Opt-Out Cognitive Interview Protocol - English

Interview ID:		Date of Interview:
First Name:		Interviewer

STEP 1(PRIOR TO INTERVIEW): INTERVIEWER DETERMINES INTERVIEW PATH

Interviewer Instructions: Each recruited participant is given an Interview ID number. This number indicates the path the the interviewer should take. Different paths are needed so that each of the three versions of the opt-out screener are tested first within the series an equal number of times. The most valuable information that we will collect in any one given interview will be about the first instrument that the participant sees; therefore, we will use the path variable below to make sure that each instrument is tested first within the series as often as possible. The path will include the form that will be first tested in the interview and then the two additional forms that will be tested after the first. Below is a table to guide the interviewer as to which form will be tested first and then second.

Interview IDs	First Form	Second Forms
1, 4, 7, 10, 13, 16	A	B, C
2, 5, 8, 11, 14, 17	B	A, C
3, 6, 9, 12, 15	C	A, B

NOTE: This is subject to change due to cancellations. Interviewers please be aware of changes to type of protocol you should use before preparing for your interview.

STEP 2: WELCOME AND INTRODUCTION SCRIPT

Interviewer Instructions: The following scripts should NOT be read verbatim. Text in italics is suggested content with which the interviewer should be thoroughly familiar in advance. Text in brackets [ ] presents instructions for the interviewer.

Hello, my name is <your name > and I work for American Institutes for Research. It’s nice to meet you, thanks for coming to help us out today.

[Create small talk to build rapport with the respondent]

You are here today to help us with a survey that we are working on for the U.S. Department of Education. I will ask you to complete the survey as if you were taking a real survey at home. There are no right or wrong answers, and it’s ok if you do not know some of the answers. As you are answering the questions I will ask you about your answers. This will help us make the survey better and clearer for other people like you.

To make sure we learn everything we can from this session, I am going to be taking notes while you are answering the survey and while we are talking [INSERT IF THERE IS AN OBSERVER: and my colleague will also be observing our session today behind the glass/by video].

Also, this interview will be audio recorded so researchers can review the recording later. Any information that refers to you personally, like your name or names of others in your househould, will not be included in our report, and your responses will be used for research purposes only. Is all of this OK?

Remember, you are not being tested; all we want is to learn how well the survey works.

If at any time you want to stop, just let me know.

[If, for any reason, the participant is no longer interested in participating, thank the participant for his/her time and end the interview.]

Before we continue, do you have any questions? [Answer any questions they may ask.]

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form (or read it to him/her, depending on respondent preference), answer any questions, and have the respondent sign the form. Leave a separate copy of the form with the respondent.

If the respondent has consented to recording, start the recorder.]

STEP 3: COMPLETION OF THE QUESTIONNAIRE

[Hand respondent a copy of the cover letter and first form (see table above for correct form type for this interview) and a pen or pencil.]

Okay. Now I would like you to complete the questionaire, please look at the materials and do what you would do with them if you were at home and received these materials in the mail. As you go through the survey, if any question sticks out to you in any way (maybe you think a question is hard to answer, hard to understand, in a strange place, or you just don’t like it)—anything at all that you would like to say about an item, please tell me so that we can discuss it. Remember that I’ll be stopping you here and there as you complete the survey so that we can discuss it.

[REMEMBER TO TELL THEM THROUGHOUT THE QUESTIONNAIRE THAT THEY ARE DOING A GREAT JOB THINKING ALOUD, AND THEIR ANSWERS TO YOUR QUESTIONS ARE REALLY HELPFUL.]

SCREENER (Form __ ) [Please fill out the form letter here for your notes]
	[HAND RESPONDENT THE SCREENER PACKAGE.] I’m about to hand you the survey package. If your address had been randomly chosen for this study, you would have already received other letters to complete the survey by web. Let’s pretend like you did not respond to those survey requests, and now you received this package. Okay, please do with this survey package what you would do if you received it at home.
Letter	[INTERVIEWER RECORD OBSERVATIONS ABOUT HOW MUCH TIME WAS SPENT ON THE LETTER: __ __ minutes] Let’s first look at the letter. I would like to ask about your reaction to the letter. Please remember that there are no right or wrong answers, we are just trying to learn how you and others like you understand this letter. In general, what is this letter for? Who is the letter from? What are you expected to do next? What information in this letter stood out the most to you? What is the information gathered from this study going to be used for? Is there anything that is unclear or awkward in the letter? If yes, do you have recommendations to make it clear? What are those recommendations?
Screener Cover	[WHEN FINISHED – IF THEY RESPOND “NO” TO THE COVER ITEM THEY ARE DONE WITH THE SURVEY (NO CHILDREN)] Now let’s look at the cover. Tell me what you think of this cover. Was it clear? How did you decide what to mark? Did you notice the instructions? Is there anything that is unclear or awkward on the cover? If yes, do you have recommendations to make it clear? What are those recommendations? [If they answered “No”] Once you completed the survey what would you do next?
Inside (If applicable)	Was there anything confusing, awkward, or unclear about the rest of this form? Once you completed the survey what would you do next?
ALL	Was there anything confusing, awkward, or unclear about this form that you have not already noted above you would like to tell me about?

STEP 4. OTHER FORM TESTING

[This section is testing to see how 2 other covers work. It is to gauge the respondents reaction to the covers compared to the cover that was tested in step 3. The covers are being rotated since secondary reactions to materials can be biased]

We are now interested to hear your thoughts on these two additional covers compared to the cover you received.

SCREENER (Form __ ) [Please fill out the form letter here for your notes]
Cover	[Hand the participant one of the other forms] Let’s look at the cover. Tell me what you think of this cover. Between the first cover and this cover which do you like better? Why? Is there anything that is unclear or awkward on this second cover? If yes, do you have recommendations to make it clear? What are those recommendations?

SCREENER (Form __ ) [Please fill out the form letter here for your notes]
Cover	[Hand the participant one of the last form] Let’s look at the cover. Tell me what you think of this cover. Between the first cover and this cover which do you like better? Why? Of all three covers which would you prefer? Why? Is there anything that is unclear or awkward on this second cover? If yes, do you have recommendations to make it clear? What are those recommendations?

FINAL QUESTION

Is there anything else you would like to tell me about these forms that you haven’t had a chance to mention?

(IF APPLICABLE) REMEMBER TO ASK OBSERVER FOR INPUT/ADDITIONAL QUESTIONS

STEP 5. THANK YOU FOR PARTICIPATING

We are finished. Thank you for helping us. The information you’ve provided will be very helpful for improving this survey.

Interviewer Instructions: Collect materials and escort respondent out of the interviewing room and to the lobby. Please give the respondent incentive envelope [$40] and have them sign the receipt, and thank them for their time and effort.

Useful Probes—Use only as needed when response is unclear or R appears confused or unsure

Can you tell me in your own words what that question was asking?

What does the word [term] in this question mean to you?

How easy or difficult is it to remember [topic]?

How easy or hard was it to choose an answer?

Tell me what you are thinking.

How did you come up with that answer?

You said [answer]. Can you tell me more about that?

[In response to R actions or comments]: Tell me what happened? Tell me what you were doing?

[If you pick up on a visual cue like a furrowed brow or a puzzled look]: Tell me what you are thinking.

[If you pick up on a visual cue that indicates confusion like paper flipping or rereading an old question before answering this question]: Tell me what just happened. Tell me about what you were just doing.

NHES Screener Opt-Out Cognitive Interview Protocol – Spanish

Interview ID:		Date of Interview:
First Name:		Interviewer

STEP 1(PRIOR TO INTERVIEW): INTERVIEWER DETERMINES INTERVIEW PATH

Interviewer Instructions: Each recruited participant is given an Interview ID number. This number indicates the path the the interviewer should take. Different paths are needed so that each of the three versions of the opt-out screener are tested first within the series an equal number of times. The most valuable information that we will collect in any one given interview will be about the first instrument that the participant sees; therefore, we will use the path variable below to make sure that each instrument is tested first within the series as often as possible. The path will include the form that will be first tested in the interview and then the two additional forms that will be tested after the first. Below is a table to guide the interviewer as to which form will be tested first and then second.

Interview IDs	First Form	Second Forms
1, 4,	A	B, C
2, 5, 7	B	A, C
3, 6, 8	C	A, B

NOTE: This is subject to change due to cancellations. Interviewers please be aware of changes to type of protocol you should use before preparing for your interview.

STEP 2: WELCOME AND INTRODUCTION SCRIPT

Interviewer Instructions: The following scripts should NOT be read verbatim. Text in italics is suggested content with which the interviewer should be thoroughly familiar in advance. Text in brackets [ ] presents instructions for the interviewer.

Hola, mi nombre es <su nombre> y trabajo para [COMPAÑIA]. Mucho gusto de conocerle. Gracias por venir a ayudarnos el día de hoy.

[Platique de algún tema casual para establecer una buena relación con el participante.]:

Usted está aquí hoy para ayudarnos con un nuevo estudio de investigación que estamos realizando para el Departamento de Educación de los Estados Unidos. Como parte de este estudio, le voy a pedir que complete una encuesta como que si estuviera completando una encuesta real en su casa. No existen respuestas correctas o incorrectas y está bien si usted no sabe algunas de las respuestas. A medida que usted vaya respondiendo a las preguntas, yo le voy a hacer algunas preguntas acerca de sus respuestas. Esto nos ayudará a que las preguntas en la encuesta sean mejores y más claras para otras personas como usted.

Debido a que la información que usted nos dará es muy importante, voy a estar tomando notas mientras estamos hablando [INSERTE SI HAY UN OBSERVADOR: y mi colega también va a observar nuestra sesión de hoy a través del cristal/por vídeo].

Además, esta entrevista será grabada en audio para que los investigadores del estudio puedan revisar la grabación más tarde. Cualquier información que se refiera a usted personalmente, tal como su nombre o empleador, no será incluida en nuestro informe y sus respuestas serán utilizadas únicamente para fines de investigación. ¿Está usted de acuerdo con lo que he mencionado?

Recuerde que esto no es un examen. Queremos que nos diga lo que usted piensa acerca de las preguntas de la encuesta.

Si en algún momento usted desea parar, solo déjemelo saber.

[Si por algún motivo el participante ya no tiene interés en participar, agradézcale por su tiempo y termine la entrevista.]

Antes de que continuemos, ¿tiene alguna pregunta sobre lo que acabo de decir? [Responda a cualquier pregunta que pueda hacer el participante.] Remember, you are not being tested; all we want is to learn how well the survey works.

DELE AL ENCUESTADO UNA COPIA DEL FORMULARIO DE CONSENTIMIENTO. PEDIRLE QUE LEA LA FORMA (O LEASELA, DEPENDINEDO DE LA PREFERENCIA DEL ENCUESTADO) RESPONDA CUALQUIER PREGUNTA Y PEDIRLE QUE FIRME LA FORMA. DELE UNA COPIA DEL FORMULARIO DE CONSENTIMIENTO AL ENCUENTADO

SI EL ENCUESTADO HA DADO SU CONSENTIMIENTO EMPIECE LA GRABACION

STEP 3: COMPLETION OF THE QUESTIONNAIRE

[Hand respondent a copy of the cover letter and first form (see table above for correct form type for this interview) and a pen or pencil.]

Bueno. Ahora me gustaría que completara el cuestionario, revise los materiales y haga lo que haría con ellos si estuviera en su casa y recibiera estos materiales por correo. A medida que avance en la encuesta, si surge alguna pregunta de algún modo (tal vez piense que una pregunta es difícil de responder, difícil de entender, en un lugar extraño, o simplemente no le gusta): cualquier cosa que quiera decir acerca de una pregunta dime para que podamos hablar sobre eso Recuerde que me detendré aquí y allá mientras complete la encuesta para que podamos hablar al respecto.

[RECUERDA HABLARLOS DURANTE LA ENCUESTA QUE ESTÁN HACIENDO UN GRAN TRABAJO Y QUE SUS RESPUESTAS A TUS PREGUNTAS SON REALMENTE ÚTILES.]

SCREENER (Form __ ) [Please fill out the form letter here for your notes]
	[HAND RESPONDENT THE SCREENER PACKAGE.] Estoy a punto de darte el paquete de la encuesta. Si su dirección se eligió al azar para este estudio, usted habría recibido otras cartas para completar la encuesta en línea. Supongamos que no respondió a las solicitudes de encuesta y ahora recibió este paquete. Por favor, haga con este paquete de encuestas lo que haría si lo recibiera en casa.
Letter	[INTERVIEWER RECORD OBSERVATIONS ABOUT HOW MUCH TIME WAS SPENT ON THE LETTER: __ __ minutes] Primero miremos a la carta. Me gustaría preguntar acerca de su reacción a la carta. Por favor, recuerde que no hay respuestas correctas o incorrectas, sólo estamos tratando de saber esta carta puede ser interpretada. En general, ¿para qué es esta carta? ¿Quién manda esta carta? ¿Qué se espera que haga usted después de leer la carta? ¿Qué información en esta carta destacó más para ti? ¿Para qué se va a usar la información recolectada por este estudio? ¿Hay algo que no está claro en la carta? En caso afirmativo, ¿tiene recomendaciones para dejarlo en claro? ¿Cuáles son las recomendaciones?
Screener Cover	[WHEN FINISHED – IF THE RESPOND “NO” TO THE COVER ITEM THEY ARE DONE WITH THE SURVEY (NO CHILDREN)] Ahora veamos la portada. Cuéntame lo que piensas de esta cubierta de encuesta. ¿Estaba claro? ¿Cómo decidiste qué marque? ¿Notaste las instrucciones? ¿Hay algo que no esté claro en esta cubierta de la encuesta? En caso afirmativo, ¿tiene recomendaciones para dejarlo en claro? ¿Cuáles son las recomendaciones? [If they answered “No”] Una vez que completó la encuesta, ¿qué haría después?
Inside (If applicable)	¿Hay algo que no está claro en esta encuesta? Una vez que completó la encuesta, ¿qué haría después?
ALL	¿Hay algo que no está claro en esta encuesta que usted no haya mencionado anteriormente del que quisiera informarme?

STEP 4. OTHER FORM TESTING

[This section is testing to see how 2 other covers work. It is to gauge the respondents reaction to the covers compared to the cover that was tested in step 3. The covers are being rotated since secondary reactions to materials can be biased]

Ahora estamos interesados ​​en escuchar sus opiniones sobre estas dos encuestas adicionales en comparación con la encuesta anterior.

SCREENER (Form __ ) [Please fill out the form letter here for your notes]
Cover	[Hand the participant one of the other forms] Ahora veamos la portada. Cuéntame lo que piensas de esta cubierta de encuesta. ¿Entre la primera portada de la encuesta y la encuesta que más te gusta? ¿Por qué? ¿Hay algo que no esté claro en esta segunda cubierta de la encuesta? En caso afirmativo, ¿tiene recomendaciones para dejarlo en claro? ¿Cuáles son las recomendaciones?

SCREENER (Form __ ) [Please fill out the form letter here for your notes]

Cover

[Hand the participant one of the last form] Ahora veamos la portada. Cuéntame lo que piensas de esta cubierta de encuesta. ¿Entre la primera portada de la encuesta y la encuesta que más te gusta? ¿Por qué? De las tres portadas de encuesta que preferirías? ¿Por qué? ¿Hay algo que no está claro en esta última encuesta? En caso afirmativo, ¿tiene recomendaciones para dejarlo en claro? ¿Cuáles son las recomendaciones?

FINAL QUESTION

¿Hay algo más sobre lo que le gustaría hablar acerca de estos materiales? ¿Hay algo acerca de lo que no ha tenido oportunidad de hablar?

(SI APLICA) RECUERDE PREGUNTAR AL OBSERVADOR SI TIENE ALGO QUE AÑADIR O PREGUNTAS ADICIONALES

STEP 5. THANK YOU FOR PARTICIPATING

Hemos terminado. Gracias por ayudarnos. La información que nos ha dado será muy útil para mejorar las cartas y la encuesta.

Interviewer Instructions: Collect materials and escort respondent out of the interviewing room and to the lobby. Please give the respondent incentive envelope [$40] and have them sign the receipt, and thank them for their time and effort.

Preguntas útiles—Utilicelas solo cuando las respuestas no son claras o el encuestado parece confundido

En sus propias palabras, ¿qué dice esta pregunta?

¿Qué significa para usted la palabra [término] en esta pregunta?

¿Qué tan fácil o difícil es recordar [tema]?

¿Qué tan fácil o difícil fue escoger una respuesta?

Dígame lo que está pensando.

¿Cómo llegó a esa respuesta?

Usted dijo [respuesta]. ¿Puede decirme un poco más acerca de cómo llego a esa respuesta?

[En respuesta a las acciones o comentarios del encuestado]: Por favor, dígame que sucedió. Dígame que estaba usted haciendo.

[Si nota alguna señal visual, como cejas fruncidas o una expresión de confusión]: Por favor, dígame que sucedió. Dígame que estaba usted haciendo. Dígame lo que está pensando

[Si nota alguna señal visual que indica confusión, como pasar páginas adelante y atrás, volver a preguntas anteriores]: Por favor, dígame que sucedió. Dígame que estaba usted haciendo. Dígame lo que está pensando

NHES Screener Opt-Out Screener Cover Letter – Bilingual

		UNITED STATES DEPARTMENT OF COMMERCE Economics and Statistics Administration U.S. Census Bureau Washington, DC 20233-0001 OFFICE OF THE DIRECTOR

Dear <City> Household:

We have not yet received your completed survey for the 2019 National Household Education Survey.

Please fill out the enclosed survey and return it in the postage-paid envelope as soon as possible.

Did you know?

All households that were selected for the survey that do not respond harm the accuracy of the study’s findings. Whether your household is large or small, has children or not, your response is needed to provide the best possible data about education in America.

Call us toll-free at 1–888–840–8353 with any questions or to complete the survey with an interviewer. If there are no children in your household, this call will take 5 minutes or less.

Thank you for taking part in this important study.

Sincerely,

Ron S. Jarmin

Performing the Non-Exclusive Functions

and Duties of the Director, U.S. Census Bureau

Enclosures

Si usted prefiere la versión en español, vea al dorso de la carta.

census.gov

		UNITED STATES DEPARTMENT OF COMMERCE Economics and Statistics Administration U.S. Census Bureau Washington, DC 20233-0001 OFFICE OF THE DIRECTOR

Estimado hogar de <CITY>

Todavía no hemos recibido su encuesta completa para la Encuesta Nacional en Hogares sobre Educación del 2019.

Complete esta encuesta y envíela en el sobre prepagado que le hemos proporcionado lo antes posible.

¿Sabía qué?

Los hogares que fueron seleccionados para la encuesta, que no respondan, dañan la exactitud de los hallazgos del estudio. Si su familia es grande o pequeña, tiene hijos o no, su respuesta es necesaria para proporcionar la mejor información posible sobre la educación en Estados Unidos.

Llámenos gratis al 1-888-840-8353 con cualquier pregunta o para completar la encuesta con un entrevistador. Si no hay niños en su hogar, esta llamada demorará 5 minutos o menos.

Gracias por participar en este estudio importante.

Atentamente,

Ron S. Jarmin

Realizar las funciones no exclusivas

y deberes del director, Oficina del Censo de los Estados Unidos

Vea encuesta adjunta

If you prefer English please see the letter on the other side.

census.gov

16