Protocol

NHIS Cognitive Interview Protocol

Participant ID #: ________________________

Interview Date: |__|__| / |__|__| / |__|__|__|__| (mm/dd/yyyy)

Interviewer initials: _____

Start Time: ____________ AM / PM End Time: ____________ AM / PM

Double-check that you have the following information ready prior to the start of the interview:

• Basics of the household roster (e.g., participant lives with extended family, unrelated roommates, etc.)

All documents can be found in:

L:\Inter-group Projects\NHIS Self-Response\Cognitive Testing

MAke sure to conduct a “Tech check” with participant

IF PARTICIPANT SHOWS ANY TYPE OF DISTRESS DURING THE INTERVIEW, GO TO SECTION VIII.

SECTION I: INTRODUCTION

[have your camera on when you log on]

Hello, my name is ________________, and I work for the Census Bureau. Thank you for agreeing to participate in our study. [IF OBSERVER PRESENT: I also want to let you know that this interview is being observed by [DESCRIBE OBSERVER’S ROLE: e.g., someone who works on the survey]. They are observing the interview because they are also interested in how the questions work for people like you.]

[If their camera is off] If you wouldn’t mind, we would like to use the camera on Teams so we can better see you. If your camera is not working, that is ok too.

The Census Bureau counts the population in the U.S and also conducts different kinds of surveys.

Today, we are working on improving questions for a survey conducted by the National Center for Health Statistics, which is part of the Centers for Disease Control and Prevention, or CDC. You will help us test questions that ask about health. It is important that these questions make sense, are easy to answer, and that everyone understands the questions the same way. Remember, there are no wrong answers, because only you know what you are thinking. Please answer these questions as you would if an interviewer had come to your home to complete the survey. Do you have any questions before we begin?

If the participant does not log into the Teams meeting:

1. call them at the number in the schedule sheet.

2. Talk them through joining the teams meeting.

3. If needed Ask what email they want the invitation sent to.

4. encourage them to respond on a laptop or desktop, they only need to join through a browser. If they only have a mobile device TELL them they will need to install the teams app.

5. When they join the meeting, they should click the “Join as guest” and enter a name

SECTION II: CONSENT FORM

• If they did sign before the interview, check in qualtrics

Before we start, I noticed that you had not yet signed the document in the link we e-mailed to you; I just wanted to check did you receive that e-mail? [If yes] Did you have any issues opening the link or with signing the document?

PARTICIPANT MUST SIGN CONSENT FORM OR WE CANNOT PROCEED WITH THE INTERVIEW

Before we start, I just wanted to thank you for signing the document in the link I sent before the interview. Just as a quick reminder, that document explained a little bit about this interview and provided information about your rights as a participant, such as that all information you provide is confidential and we won’t use your name or specific circumstances in any report. The link had a signature line to acknowledge the $40 you will digitally redeem for participating. It also asked for your permission to have this session audio recorded as well as a video recording of your computer screen and image. This is so that I can concentrate on what you are saying rather than having to take extensive notes. Your name will not be associated with the recording or any of the other data collected during the session. May I turn on the recorder? [If they say yes turn on SNAGIT recorder and backup audio recorder. ALSO mention for participant to turn off camera to help with the connection] For our purposes, we need to capture your consent orally on the recording. Please state, “I give consent to participate in this research and to be recorded.”

• PARTICIPANT AGREES

• IF PARTICIPANT DOESN’T AGREE TO BE RECORDED, THE INTERVIEW CAN CONTINUE

Thank you for that.

Do you have any other questions about the document or the interview?

Your participation in this interview is very important because it will help the Census Bureau with improving these survey questions. Your participation in this interview is voluntary, and you may decline to answer any question, or stop the interview at any time. We are scheduled to be here for about 60 minutes, and I won’t keep you any longer.

SECTION III: INSTRUCTIONS AND SET UP

While you are answering questions for the survey, I would like you to think aloud. While I am interested in your answers, I am even more interested in the process you go through when you answer the questions. So, as much as possible, I’d like you to think aloud for me; meaning, talk me through your thought process as you answer each question. From time to time, I may also ask you what you are thinking to remind you to do this.

I find it helps people to get the idea of thinking aloud while they are answering a question if they do a practice question first. Let’s begin with a practice question. Remember to think aloud as you answer.

How many windows are there in the house or apartment where you live?

_____________ WINDOWS

IF NEEDED: Try to visualize the place where you live and think about how many windows there are in that place. As you count up the windows, tell me what you are seeing and thinking about.

PROBES:

• How did you come up with that answer?

• Tell me more about that. Why did you say [ANSWER]?

• I noticed that you hesitated. Tell me what you were thinking.]

Great, that is what we will want you to do for today’s session.

Do you have any other questions about the interview?

FOR WEB ONLY

• ask PARTICIPANT to share their screen.

• They need to give teams permission to share their screen. if theY are on a work computer, they may not be able to share their screen. this has to be done in their computer OR BROWSER’s settings. USUALLY, teams will prompt them to do this.

Now I am going to send you a link to the survey and have you share your screen with me. Remember to close anything you don’t want me to see, or you can only share your browser window

send QUALTRICS link in the chat

[QUALTRICS LINK]

1. Have them log into the survey landing PAGE. Once they get to this point WITH SCREEN SHARING ON, HAVE THEM BEGIN ANSWERING QUESTIONS.

2. IF THEY CAN SHARE SCREEN, DO NOT HAVE CAMERAS ON.

3. IF THEY CANNOT SHARE SCREEN, HAVE CAMERAS ON.

4. INTERVIEWER SHOULD HAVE THEIR VERSION OF THE QUALTRICS PROTOCOL OPEN TO FOLLOW ALONG

SECTION IV: NHIS

We will now start the next section of the survey.

You can go ahead and begin answering the questions now. Please remember to think aloud as you go along.

(See Attachment II for the questions of the National Health Interview Survey to be tested. The respondents will answer the survey while thinking aloud. There will be no probing during the survey. If a respondent asks for clarification during the survey, interviewers will encourage the respondent to answer the question as they would if the interviewer was not able to provide clarification. After the full survey has been administered, the interviewer will administer the probing questions in Section 5.)

WHEN THEY START THE ROSTER SECTION

For the purpose of the interview, please pretend that this address is where you live.

123 Main Street

Anywhere, MD, 55555

SECTION V: COGNITIVE INTERVIEW

[CONTINUE IN QUALTRICS. DISPLAY SURVEY QUESTIONS FOR THE PROBES RELATED TO EACH QUESTION.]

Thank you for answering those questions. Now I’d like to ask you some questions about the answers you just provided.

GENERAL PROBES

1. The survey that you completed currently asks questions via face-to-face interviews. Do you think it would have been easier to answer these questions via a face-to-face interview or in the web mode you just completed? [IF NEEDED]: Can you please tell me more about that?

ROSTER PROBES

NAME_FNAME

2. In your own words, what does “living or staying” mean in this question?

3. Was there anyone you thought about including, but didn’t? [IF YES]: Can you please tell me more about that?

USUALRES

4. How often is “usually” in this question? [IF NEEDED] About how many nights a month would you say is “usually”?

MISPERS_MCHILD

5. What does “babies or small children” mean to you in this question? What ages were you thinking of for babies or small children when you answered this question?

MISPERS_MELSE

6. Was there anyone you didn’t add but thought about when answering this question?

7. In your own words, what does “people who are not related” mean to you in this question?

8. In your own words, what does “people who are away traveling” mean to you in this question?

HHRESP

9. Can you tell me what this question is asking?

10. [IF NEEDED] How did you decide who can answer questions about people in your household?

ALLFAM

11. Can you tell me what “family” means to you in this question?

ROSTER GENERAL PROBE

12. [IF NEEDED] Do you think these questions allowed you to correctly report who lives in your household?

HEALTH INSURANCE PROBES

GENERAL HEALTH INSURANCE

13. Were you thinking about current coverage, recent coverage, or past coverage when answering these questions? [IF NEEDED] What time frame were you thinking of for these questions?

14. This question mentions “health insurance” and “health care coverage”. Do they mean the same thing to you, or do they mean something different?

15. These questions mention “deductible”. Are you familiar with what a deductible is? Can you tell us more about that?

16. Do you know the difference between a deductible and copay?

17. Would it have been helpful to have instructions in any of these questions that encouraged you to look at your insurance card? Can you tell me more about that?

HICOVKIND

18. Let’s take a look at the response options for this question. Was it easy or difficult to find your answer among these options?

19. For this question, you answered [RESPONSE SELECTED]. Can you tell us more how you got your current coverage? What, if anything, did you have to do to sign up for this? [IF NEEDED] If someone else got this coverage for you, who was that and do you know how they signed up?

20. [If P mentions something implying marketplace]: Did you sign up for this coverage online?

21. [If private health insurance plan selected] What does private health insurance plan mean to you in this question?

22. [IF SELECTED STATE SPONSORED PLAN] Are you familiar with a ‘state-sponsored plan’? Can you tell me more about that? Do you feel like you were able to report the plan in this question?

23. [IF P SAYS “I DON’T KNOW”]: Can you say more about that? How do you know you are covered?

24. When answering this question, did you find the instruction helpful? [IF NEEDED]: Was the instruction easy or difficult to understand?

MCAREPRB_A

1. Are you familiar with Medicare? [IF NEEDED] In your own words, can you tell me what Medicare is? [IF YES TO MCAREPRB_A] Are you “currently” covered by Medicare?

MCAIDPRB_A

2. Are you familiar with Medicaid? [IF NEEDED]: In your own words, can you tell me what Medicaid is?

SINCOVDE/SINCOVVS/SINCOVRX

3. Can you tell me about what you were thinking when answering this question?

4. Can you tell me what ‘separate plan’ means to you in this question?

5. [IF UNCLEAR THAT SERVICES/PRESCRIPTIONS WERE SEPARATE] Were these [services/prescriptions] covered by a separate plan?

MCPART_A

6. Can you tell me about what you were thinking when answering this question?

7. Are you familiar with Medicare Part A and Part B coverage?

MCCHOICE_A

8. Are you familiar with Medicare Advantage plans? [IF YES] Can you tell me more what Medicare Advantage plans are?

MCHMO_A

9. Are you familiar with Health Maintenance Organizations or HMOs? [IF YES] Can you tell me more about Health Maintenance Organizations or HMOs are?

MAXCHNG_A

10. What does Health Insurance Marketplace mean to you in this question?

MAHDHP_A

11. Can you tell me about how you came up with your answer? [IF NEEDED] Did you think of the exact amount it costs or did you make a guess?

HIPNAM1_A

12. When answering this question, did you find the instructions helpful? [IF NEEDED]: Was the instruction easy or difficult to understand?

HIPNAM2_A

13. Can you tell me more about the plan you wrote in here?”

HICOSTN_A

14. Was there anything that was difficult when filling out this table?

HSAHRA_A

15. [IF YES] Can you tell me more about the special accounts or funds you used?

PRDNCOV

16. [IF NO TO MEDICATIONS] If you needed to have any medications prescribed, would your plan pay for any of the costs? [IF NO TO SERVICES] If you needed [dental care/vision care], would your plan pay for any of the costs?

MILSPC_A

17. How familiar are you with (not selected response options: VA health care; TRICARE; CHAMP-VA)?

HILAST_A

18. Let’s take a look at the response options for this question. I see you chose response option [FILL]. Can you tell me how many months/years you were thinking when you answered this question?

HISTOPJOB, ETC.

19. Let’s take a look at the response options for this question. Was it easy or difficult to find your answer among these options? [IF NEEDED] Can you tell me more about?

RSNHICOST_A

20. Let’s now look at this question [HISTOP, ETC.] and this question [RSNHICOST_A]. Did you see the difference between the two questions? Can you tell us more about that?

MARITAL STATUS PROBES

MARITAL_A

21. How do you think someone who is still married but living with someone else they are in a relationship with would answer this question? How about someone who is in an unmarried couple but not living with their partner?

22. [IF NEEDED]: How easy or difficult was it to answer this question?

SPOUSEP_A

23. In your own words, can you tell me what this question is asking? [IF NEEDED]: In your own words, what does “legally separated” mean in this question?

LEGALSTAT_A

24. What does “current legal marital status” mean to you in this question?

WIDIVSEP_A

25. What does “current legal marital status” mean to you in this question? How do you think someone who has both been widowed and divorced would answer this question? How about someone who was previously divorced and who recently separated from a different person?

GENERAL PROBE:

26. Do you feel these questions allowed you to correctly report your current relationship status?

UTILIZATION PROBES

LASTDR_A

27. Can you tell me more what this last appointment was? Can you tell me more what these instructions in italics are asking?

RETAILHC12M_A

28. [If 1 or more] Can you tell me more about the retail health clinic(s) you have went to? What types of services did you receive?

WELLNESS_A

29. What does “wellness visit, physical, or general purpose check-up” mean to you in this question?

USUALPL_A

30. Can you tell me what this question is asking? [IF NEEDED] How did you answer this question?

HOSPONGT_A

31. [IF YES]: What were you thinking about when you answered this question?

32. [IF YES]: When answering this question, did you find the instructions helpful? [IF NEEDED]: Was the instruction easy or difficult to understand?

MEDDL12M_A/MEDNG12M_A

33. In your opinion, what is the difference between these two questions?

CIGARETTE AND E-CIGARETTE PROBES

SMKEV_A

34. Can you tell me more about what you were thinking when answering this question? [IF NEEDED]: Were you including e-cigarettes (e.g., vaping) in your answer?

SMKAGE_A

35. What does “fairly regularly” mean to you in this question? [If needed] If you had to put a number to “fairy regularly”, how many would you say that is in a regular month?

CIGNOW_A

36. What does the word “now” mean to you in this question? Would you prefer the word “now” or currently”?

ECIGNOW_A

37. Can you please tell me more about your answer to this question?

CARDIOVASCULAR CONDITION PROBES

CHDEV_A – STREV_A

38. How easy or difficult was it to answer these questions in this grid format? [IF NEEDED]: Can you please tell me more about that?

39. [FOR ANGEV_A]: In your own words, can you tell me what this question is asking?

40. [IF ANY DIFFICULTY ANSWERING]: Do you have any suggestions for how we might make these questions easier to answer?

OTHER CHRONIC CONDITION PROBES

COPDEV_A – DEPEV_A

41. How easy or difficult was it to answer these questions in this grid format? [IF NEEDED]: Can you please tell me more about that?

42. Looking at part d, “any type of anxiety disorder”, was there any specific anxiety disorders you were thinking of when answering?

43. Looking at part e, “any type of depression”, which types of depression were you thinking of when answering?

44. [IF ANY DIFFICULTY ANSWERING]: Do you have any suggestions for how we might make these questions easier to answer?

CANCER PROBES

CANKIND_A

45. Were you able to find you answer from the list of answers provided? Can you tell me more about that?

SECTION VI: Debriefing Questions

In general, was this survey easy or difficult to complete on web? [IF NEEDED]: Can you please tell me more about that?

Do you think other people might find any of these questions particularly difficult or sensitive?

• [IF YES] Which ones?

• [IF NEEDED] What about the question do you think people would find difficult/sensitive?

Is there anything else you would like to tell us that you haven’t had a chance to mention yet?

Those are all the questions I have for you. Thank you for participating in this study. Your feedback has been very helpful.

TURN OFF RECORDER

SECTION VII: COMPENSATION FOR PARTICIPATION

If signed consent/voucher before interview

You will be receiving an email to redeem your $40 digital incentive in the next few days.

[If send again] Just to confirm, what e-mail address would you like me to send that to?

Thank you very much for your time today and we will be e-mailing out that $40 digital incentive as soon as you sign that document.

Close out steps

Check recording is the right length and save to the APPROPRIATE folder.

MAKE A NOTE OF THE TIME IT TOOK FOR PARTICIPANT TO COMPLETE THE nhis survey QUESTIONS.

Make a note of any technological issues you had during the interview.

SECTION VIII: DISTRESS GUIDANCE

Level 1 Mild Distress:

Respondent actions:

• Change in voice tone or volume

• Change in focus

• Hesitancy to answer questions or probes

• Fidgeting

• Use of inappropriate language

• Non-relevant answers

• Says, does not want to continue with interview

Interviewer actions:

• Pause give respondent time to recover/compose themselves

• Check if the respondent wants to continue if they have not already indicated that they want to stop – terminate interview if necessary

• Change topic/move to next question/topic

• Inform PI and operations team lead

• Offer appropriate helpline numbers at the end of the interview

Level 2: Moderate distress

Respondent actions:

Level one signs plus…

• Long pauses and sighing

• Tearful / crying

• Extreme forms of agitation

• Non-responsiveness

• Nonsensical responses to questions or probes

• Asks for your advice/help

Interviewer actions:

• Stop interview and offer helpline numbers

• Inform PI and Operations team lead

• Complete respondent distress report form

• Operations team lead to submit incident report to NCHS ERB

Level 3: Severe distress

Respondent actions:

• Mentions suicidal thoughts

• Talks about wishing another person was dead or talks about committing harm to another person

• Respondent asks for immediate help from a health care professional

Interviewer actions:

• Stop interview and offer helpline numbers

• Inform PI and Operations team lead

• Complete respondent distress report form

• Operations team lead to submit incident report to NCHS ERB

Helpline telephone numbers

National Suicide Prevention & Crisis Lifeline; 988; can call, text, or chat; the toll-free numbers still work (1-800-273-TALK (8255) English, 1-888-628-9454 Spanish); available 24/7; https://988lifeline.org

The operations team lead will be responsible for reporting any signs of moderate or severe distress to the NCHS ERB as soon as possible, using the signature (1379) and incident report (1254) forms.

16