Cognitive Interview Protocol

NTIA Internet Use Survey Cognitive Interview Protocol

Participant ID #: ________________________

Interview Date: |__|__| / |__|__| / |__|__|__|__| (mm/dd/yyyy)

Interviewer initials: _____

Start Time: ____________ AM / PM End Time: ____________ AM / PM

Double-check that you have the following information ready prior to the start of the interview:

• Household Roster

• List of relations of household members (up to 3) to the participant

All documents can be found in:

L:\Inter-group Projects\CPS Projects & Info\Supplements\Computer & Internet Use Supplements\Computer & Internet Use 2024\Interview Materials

MAke sure to conduct a “Tech check” with participant five minutes prior to interview

SECTION I: INTRODUCTION

Hello, my name is ________________, and I work for the Census Bureau. Thank you for agreeing to participate in our study. [IF OBSERVER PRESENT: I also want to let you know that this interview is being observed by [DESCRIBE OBSERVER’S ROLE: e.g., someone who works on the survey]. They are observing the interview because they are also interested in how the questions work for people like you.]

The Census Bureau counts the population in the U.S and also conducts different kinds of surveys.

Today, we are working on improving questions for a part of the Current Population Survey. You will help us test questions that ask about how members of your household use technology. It is important that these questions make sense, are easy to answer, and that everyone understands the questions the same way. Remember, there are no wrong answers, because only you know what you are thinking. Please answer these questions as you would if an interviewer had come to your home to complete the survey. Do you have any questions before we begin?

SECTION II: CONSENT FORM

• If they did sign before the interview, check in qualtrics

Before we start, I just wanted to thank you for signing the document in the link I sent before the interview. Just as a quick reminder, that document explained a little bit about this interview and provided information about your rights as a participant, such as that all information you provide is confidential and we won’t use your name or specific circumstances in any report. The link had a signature line to acknowledge the $40 you will receive by mail for participating. It also asked for your permission to have this session audio recorded as well as a video recording of your computer screen and image. This is so that I can concentrate on what you are saying rather than having to take extensive notes. Your name will not be associated with the recording or any of the other data collected during the session. May I turn on the recorder? [If they say yes turn on SNAGIT recorder and backup audio recorder] For our purposes, we need to capture your consent orally on the recording. Please state, “I give consent to participate in this research and to be recorded.”

• PARTICIPANT AGREES

Thank you for that.

Do you have any other questions about the document or the interview?

• If they did NOT sign before the interview:

Before we start, I noticed that you had not yet signed the document in the link we e-mailed to you; I just wanted to check did you receive that e-mail? [If yes] Did you have any issues opening the link or with signing the document? [if they say yes talk them through it. IF UNABLE TO GET SIGNED CONSENT, DISPLAY CONSENT FORM OR RELY ON ORAL CONSENT USING SCRIPT BELOW] I will go over what is in that document and obtain consent orally. However, the document also contains an acknowledgement that you will receive $40 by mail for participating in this research. You can still participate right now, but you will need to sign that document before we can send the $40.

The document explains a little bit about the purpose of this study, which is to test questions for the NTIA Internet Use Survey. The document provides information about your rights as a participant, such as that all information you provide is confidential and we won’t use your name or specific circumstances in any report. It also asks for your permission to have this session audio recorded as well as a video of your computer screen and image so that I can concentrate on what you are saying rather than having to take extensive notes. May I turn on the recorder? [If they say yes turn on Snagit recorder and backup audio recorder] For our purposes, we need to capture your consent orally on the recording. Please state, “I give consent to participate in this research and to be recorded.”

• PARTICIPANT AGREES

Thank you for that.

Your participation in this interview is very important because it will help the Census Bureau with improving these survey questions. Your participation in this interview is voluntary, and you may decline to answer any question, or stop the interview at any time. We are scheduled to be here for about 60 minutes, and I won’t keep you any longer.

• IF PARTICIPANT IS NOT A FEDERAL EMPLOYEE: As a thank you for your help, we will mail you $40 cash after the interview is completed.

SECTION III: INSTRUCTIONS AND SET UP

FOR WEB ONLY

• ask PARTICIPANT to share their screen.

• They need to give teams permission to share their screen. if theY are on a work computer, they may not be able to share their screen. this has to be done in their computer OR BROWSER’s settings. Usually teams will prompt them to do this.

Now I am going to send you a link to the survey and have you share your screen with me. Remember to close anything you don’t want me to see, or you can only share your browser window

SECTION IV: HOUSEHOLD ROSTER

FOR CAPI ONLY; web will be self-administered

Okay, let’s get started. Please answer the questions as you would if an interviewer had come to your home.

Ask R1 – R11 for 1^st HH member. Record information on Household Roster. Then ask for next HH member. Repeat for every member of HH.

R1. NAME First I will ask you some questions about the people who live here. What are the names of all persons living or staying here? Let’s start with you.

R2. SEX What is [your/[NAME]’s] sex?

1. Male

2. Female

R3. REL

IF NOT RESPONDENT: How is [NAME] related to you?

1. Opposite sex spouse (husband/wife)

2. Opposite sex unmarried partner

3. Same sex spouse (husband/wife)

4. Same sex unmarried partner

5. Child

6. Grandchild

7. Parent

8. Brother/ sister

9. Other relative (aunt, uncle, cousin, in law)

10. Foster child

11. Housemate/roommate

12. Roomer/boarder

13. Other nonrelative

14. DK

15. REF

99. self

R4. AGE What is [your/NAME’s] date of birth?

R5. EDUCATION

What is the highest level of school [you have /[NAME has]] completed or the highest degree [you have/[NAME has]] received?

1. 1st, 2nd, 3rd, or 4th grade

2. 5th or 6th grade

3. 7th or 8th grade

4. 9th grade

5.10th grade, 11th grade

6. 12th grade, no diploma

7. High school graduate-high school diploma or equivalent

8. Some college – no diploma

9. Associates degree – occupational / vocational program

10. Associates degree – academic program

11. Bachelor’s degree (BA, BS, AB)

12. Master’s degree (MS, MA, MBA, MEd, MEng, MSW)

13. Professional degree (MD, DDS, DMV, LLB, JD)

14. Doctorate degree (PhD, EdD)

R6. HISPA (Are/Is)(Name/You) Spanish, Hispanic or Latino?

1. Yes

   

2. No

3. Don’t Know Go to R9. RACE

4. Refuse

R7. HISPB (Are/Is)(Name/you) Mexican, Mexican American, Chicano, Puerto Rican, Cuban American, or some other Spanish, Hispanic, or Latino Group?

1. Mexican

2. Mexican American

3. Chicano Go to R7. Race

4. Puerto Rican

5. Cuban American

6. Some Other > Go to R6. S_OROTSP

R8. O_HISP What is the name of (your/his/her) other Spanish, Hispanic, or Latino group?

R9. RACEA I am going to read you a list of five race categories. Please choose one or more races that (NAME/you) (considers yourself/consider NAME/considers himself/considers herself) to be: White; Black or African American; American Indian or Alaska Native; Asian; OR Native Hawaiian or Other Pacific Islander.

1. White

2. Black or African American

3. American Indian or Alaska Native Go to R11. Work

4. Asian

5. Native Hawaiian or Other Pacific Islander

6. Other – DO NOT READ > Go to R10. RACEB

R10. RACEB What is your race?

R11. WORK LAST WEEK, did (name/you) do ANY work for (pay/either pay or profit)?

1. Yes

2. No

3. Retired

4. Disabled

5. Unable to work

SECTION V: NTIA Internet Use Survey

Thank you for providing that information about your household.

We will now start the next section of the survey.

[NOTE TO INTERVIEWERS: ‘IF NEEDED’ TEXT SHOULD ONLY BE READ IF PARTICIPANT HESITATES INITIALLY WHEN ANSWERING THE QUESTION]

[SWITCH TO NTIA INTERNET USE SURVEY DOCUMENT AND ADMINISTER THE QUESTIONNAIRE]

FOR USEINT, ADMINISTER THE QUESTIONS ABOUT THE FOLLOWING

HH Member #1: Name: _________________, Relation to Participant: _________________,

HH Member #2*: Name: _________________, Relation to Participant: _________________,

HH Member #3*: Name: _________________, Relation to Participant: _________________,

[FOR QUESTIONS USEINT TO HOMIOT, SELECT HOUSEHOLD MEMBER THAT WAS ASSIGNED BY PROJECT LEAD. THERE WILL BE A FIRST PRIORITY, SECOND PRIORITY, AND THIRD PRIORITY. IF RELATIONSHIPS DIFFER IN INTERVIEW THAN WHAT WAS COLLECTED IN SCREENING, PRIORITIZE THOSE THAT ARE EITHER UNRELATED OR HAS A DISTANT RELATIONSHIP TO THE PARTICIPANT. THIS CAN INCLUDE GRANDPARENT/GRANDCHILD, AUNT/UNCLE/NEPHEW/NIECE, COUSIN, ETC. HOUSEHOLD MEMBERS MUST BE 15+ YEARS OLD TO BE ADMINISTERED USEINT TO HOMIOT.]

SECTION V: COGNITIVE INTERVIEW

[BRING UP POWERPOINT OF SURVEY QUESTIONS. SHOW SURVEY QUESTIONS FOR THE PROBES RELATED TO EACH QUESTION. NOTE ONLY SHOW QUESTIONS AS SCRIPTED IN PROTOCOL]

Thank you for answering the questions. Now I’d like to ask you some questions about the answers you just provided.

GENERAL PROBES FOR SELF-ADMINISTERED WEB

In general, was this survey easy or difficult to complete on Web? Please tell me more about that.

This survey that you completed currently administers questions via face to face or telephone. Do you think it would have been easier to answer questions via face-to-face/telephone or in the web mode you just completed? Tell me more about that.

I noticed you commented on (INSERT QUESTION). Can you tell me more about what you were thinking?

I noticed you hesitated when answering (INSERT QUESTION). Can you tell me more about what were you thinking?

There were a series of questions where you were asked if anyone in your household participated in certain activities using the Internet. Then you had to select who in the household participated in those activities specifically. Was this easy or difficult to do? Were there any questions where it was easier to answer for one person in the household than another? (IF YES) Can you tell me more about that?

PROBES FOR INTERVIEWER-ADMINISTERED AND SELF-ADMINISTERED

First, we asked about the computers and other internet-connected devices that members of your household have used at least occasionally during the past 6 months.

DESKTP

Earlier we asked, “does anyone in this household use a desktop computer at least occasionally.”

In your own words, what does “desktop computer” mean to you in this question?

I am now showing the full question wording. Please focus on the wording that describes what a desktop is. Does this definition match with how you think of a desktop computer? (IF NO) Can you please tell me how you would describe a desktop?

TABLET

Earlier we asked if you had a tablet or e-book reader.

(IF ANY DIFFICULTY OR ‘IF NEEDED’ TEXT WAS READ) Can you tell me more about your answer to this question?

In your own words, what is the difference between a laptop and tablet, if any?

WEARAB

Earlier we asked if you had a wearable device that is connected to the Internet, such as a smart watch or fitness band. Was this question easy or difficult to answer?

[IF DIFFICULT] Can you tell me more about your answer to this question?

Can you tell me some examples of a smart watch or fitness band?

Are there any other types of devices you consider to be “wearables” other than smart watches and fitness bands?

(ONLY ASK IF THERE WAS ANY DIFFICULTY IN RESPONSE TO PROBE, HESITATION ANSWERING IN SURVEY, OR ‘IF NEEDED’ TEXT WAS READ)

I am now showing the full question wording. Please focus on the examples of what a smart watch and fitness band is. [Were these examples helpful/Would these examples have been helpful] when you answered the question originally?

TVBOX

Earlier we asked “Does anyone in this household] use a smart TV, a game system, or another device that connects to the Internet and plays through a TV? Examples include an Apple TV, Xbox, or Roku that can access the Internet.”

Was this question easy or difficult to answer? [IF DIFFICULT] Can you tell me more about difficulty you experienced answering this question?

[IF NEEDED] Can you tell me more about your answer?

Other than Xbox, Apple TV, or Roku, are there any other devices you know of that connect to the Internet and play through a TV?

DEVQUA

Earlier, we asked, “Thinking about all the different devices we just discussed, how well do the computers and other Internet-connected devices used by this household work overall? Please do not include home Wi-Fi routers or similar equipment.”

Was this question easy or difficult to answer?

Tell me more about your answer to this question.

This question asked how well computers and other Internet-connected devices work overall. The options to this question were “they do not work at all”, “they work poorly”, “they somewhat work”, “they work well”, and “they work perfectly”. What does the phrase “work poorly” mean to you in this question?

DEVSTA

Earlier we asked “at any point during the past six months, were [you/members of your household] unable to use a computer or other Internet-connected device because it was in use by someone else, located elsewhere, or otherwise inaccessible.”

Was this question easy or difficult to answer? [IF NEEDED] Can you tell me more about the difficulty you experienced answering this question?

Can you tell me more about your answer to this question?

INHOME to INOTHR General Probes

Earlier, you answered a series of questions about [your/your household’s] Internet use at different locations including at home, work, school, coffee shop, etc. Which devices were [you/your household] thinking of as you answered these questions?

INTRAV

Next, we asked questions about where [you/members of your household] currently use the Internet.

We asked if you/anyone in the household uses the Internet while going from place to place. Examples include using the Internet while riding public transit or using Google Maps or Waze on your cell phone while driving.

Was this question easy or difficult to answer? (IF NEEDED) Can you please tell me more about the difficulty you experienced answering this question?

(IF NEEDED) Can you tell me more about your answer to this question?

(IF NEEDED) Other than using Internet on public transit or using Google Maps/Waze, were there any other ways of using Internet from place to place you thought of when answering this question?

MOBDAT

Earlier we asked if anyone in the household accessed the Internet using a data plan for a cell phone, smartphone, tablet, mobile hotspot, or other device. Was this question easy or difficult to answer?

Can you tell me more about your answer?

I am now showing the full question wording. Please focus on the text that mentions how this type of Internet service is provided. Was this text helpful when you answered the question originally?

HOMTE

Earlier we asked you about the ways people access the internet from their homes. A couple ways I mentioned included “high speed Internet services” and “Satellite Internet service”.

[IF YES TO ‘SOME OTHER SERVICE’ IN HOMTE]. I also asked if you had “some other service” and you said [INSERT ANSWER FROM HOMTE/TEOTHR]. Can you tell me more about this “other service”? Can you think of any other examples that would fit under “other service”?

[IF NO TO SOME OTHER SERVICE IN HOMTE] Can you think of any other types of ways that people can access the Internet?

EMAIL to MEDMON Proxy Reporting (IF NEEDED)

Earlier, you answered a series of questions about [INSERT NAME]’s Internet use. These questions covered a variety of Internet use activities like checking email, watching videos, working, providing services, etc. I will show you these questions on screen.

How easy or difficult was it to answer these various Internet use questions about [INSERT NAME]?

[IF NEEDED] Which questions were easy to answer? [IF NEEDED] Can you tell me more about that?

[IF NEEDED] Which questions were difficult to answer? [IF NEEDED] Can you tell me more about that?

CONFER

Earlier we asked if [INSERT NAME] participated in video or voice calls or conferencing over the Internet. Was this question easy or difficult to answer?

(ONLY ASK IF THERE WAS ANY DIFFICULTY) Can you tell me more about your answer?

I am now showing the full question wording. Are there any other video call applications that you were thinking about that are not mentioned here?

FINANC

Earlier we asked if [INSERT NAME] used the Internet for financial services such as banking, investing, paying bills online, or sending money to other people. Was this question easy or difficult to answer?

(ONLY ASK IF THERE WAS ANY DIFFICULTY IN RESPONSE TO PROBE, HESITATION ANSWERING IN SURVEY IN TERMS OF COMPREHENSION OR ‘IF NEEDED’ TEXT WAS READ)

Can you tell me more about your answer?

I am now showing the full question wording. Please focus on the examples of using the Internet for financial services. [Were these examples helpful/Would these examples have been helpful] when you answered the question originally?

HOMIOT

Earlier we asked if [INSERT NAME] interacted with household equipment or appliances that are connected to the Internet, such as a smart thermostat, light bulb, or security system. Was this question easy or difficult to answer?

(ONLY ASK IF THERE WAS ANY DIFFICULTY IN RESPONSE TO PROBE OR HESITATION ANSWERING IN SURVEY IN TERMS OF COMPREHENSION)

Can you tell me more about your answer?

I am now showing two versions of this question. One version is the one that you answered during the interview [SHOW TESTED VERSION] and the other is an alternate version [SHOW ORIGINAL VERSION] of a question. Is one of these questions easier to answer than the other?

MEDINT

Earlier, we asked a series of questions about whether your household used the Internet for health-related activities during the past six months. These questions asked about access to health records, communicating with doctors, researching health information online, and whether an electronic health monitoring service was used.

Did you answer these questions thinking about your household in general or just about your own experience?

(IF ANSWERING ON BEHALF OF HOUSEHOLD) How easy or difficult was it to answer about the household?

MEDDOC

Earlier we asked if [INSERT NAME] communicated with a doctor or other health professional using the Internet. Was this question easy or difficult to answer?

Can you tell me more about your answer?

(ONLY ASK IF THERE WAS ANY DIFFICULTY IN RESPONSE TO PROBE, HESITATION ANSWERING IN SURVEY IN TERMS OF COMPREHENSION OR ‘IF NEEDED’ TEXT WAS READ)

I am now showing the full question wording. Please focus on the examples of using the Internet to communicate with a doctor or other health professional. [Were these examples helpful/Would these examples have been helpful] when you answered the question originally?

PSPRE

[INTERVIEWER NOTE: PROBE ON THE FOLLOWING QUESTIONS IF THEY HESITATED OR STRUGGLED ANSWERING]

1. Conducting financial transactions such as banking, investing, or paying bills online?

2. Buying goods or services online?

3. Posting photos, updates, or other information on social networks?

4. Expressing an opinion on a controversial or political issue, such as on a blog or social network, or in an email?

5. Finding information using Google Search, Microsoft Bing, or another search engine?

Next, we asked if anyone in the household had concerns about privacy or security that stopped them from doing specific activities online.

I noticed that you hesitated when answering [READ OPTION FROM LIST]. Can you tell me more about what you were thinking as you answered this question?

SECTION VI: Debriefing Questions

Overall, how easy or difficult was it to answer questions for your household?

Do you think other people might find any of these questions sensitive?

• [IF YES] Which ones?

• [IF NEEDED] What about the question do you think people would find sensitive?

Is there anything else you would like to tell us that you haven’t had a chance to mention yet?

Those are all the questions I have for you. Thank you for participating in this study. Your feedback has been very helpful.

TURN OFF RECORDER

SECTION VII: COMPENSATION FOR PARTICIPATION

If signed consent/voucher before interview

We will be mailing out that $40 cash incentive in the next few days.

If did not sign consent/voucher form before interview

Just as a reminder, you need to sign the document in the link before we can send the $40 cash incentive. Would you like me to send the link again and/or talk you through any problems you may be having with it?

[If send again] Just to confirm, what e-mail address would you like me to send that to?

Thank you very much for your time today and we will be mailing out that $40 cash incentive as soon as you sign that document.

Close out steps

Check recording is the right length and save to the participant’s folder.

PLEASE WRITE DOWN THE TIME IT TOOK FOR PARTICIPANT TO COMPLETE THE NTIA Internet use survey.

Make a note of any technological issues you had during the interview below:

19