Protocol Interview

SmaRT Testing of Decennial Mail Materials – Instrument Testing

RESPONDENT ID #: _________________________ DATE: ____ / ____ / _____

INTERVIEWER’S NAME: _________________________________________________

START TIME: ______: ______ AM/PM

Session start

For the remainder of the session you’ll answer the 2025 Census Survey that you read about in the letters. Then we will have you answer a few questions about your experience with the survey. I want you to respond to it naturally but just like you have been doing so far, remember to think aloud as you do it. If you fall silent, I may remind you to think aloud by saying “keep talking” or “what are you thinking?”

[HAVE RESPONDENT COMPLETE THE ONLINE QUALTRICS DECENNIAL QUESTIONNAIRE;

Please take out Mailing 4 and take a look at the paper questionnaire. Please use the information there to respond to the survey.

Make a note of the mode they would use. For not-large HH, after noting mode, have them complete paper form. For large households (more than 5 people), if they start to answer the paper questionnaire after Q1, make a note, but then redirect them to the online form.

For online questionnaire

• Make note if they had any difficulty with the address screen.

• Make note of if the address standardizer found their address [if all capital letters in address that is returned to them] and how they answered the question.

• Make a note of any comments on live or sleep in the roster questions

• Make note of how they responded to the time capsule.

For paper questionnaire

• Make a note of any comments on live or sleep in the roster questions

• Take notes on how they answered the age question and if there were any spontaneous comments on the checkbox. Identify if they had a child less than 3 years and how they answered the question.

  • Did they comment that they should enter age in months for older individuals or did they understand the intent was for very young children 36 months and younger.

I’m going to put a link in the chat for you to click on.

[THIS LINK CONTAINS THE SATISFACTION QUESTIONS, THEN DEBRIEFING, THEN KNOWLEDGE CHECK QUESTIONS]

[INSERT LINK]

Thank you. And I’m now going to have you answer some satisfaction questions about the survey you just completed.

Debriefing

Now I just have a few more questions for you about the survey. Please go to the next page.

[Show Ancestry Language Letter 2] In the second letter (Mailing 3), you saw this message with the red arrow pointing to it. We talked about it earlier. On the next page, you will see several alternative messages. Think about which message is the clearest. We also want to know if any of the messages motivate you to respond to the census.

[Show Ancestry alternatives] and have them rank the choices. [Observe any comments on the use of “your”, any comments on the use of “2102” for the date, and any comments on “72 years”. Probe on those three messages if no comments were elicited] Tell me more about what you think of the message [X] that reads [read aloud].

[Show FAQ What is the 2025 Census Survey] In the first and fourth mailing there was an insert with answers to frequently asked questions. Look at this question and answer. What does it mean in your own words? Do you have any feedback on the answer?

[Show FAQ Are my answers confidential] Please read this question and the third paragraph in the answer. What does it mean in your own words? Do you have any feedback on that paragraph?

[Show Finding the ID on Mailing 4] Pretend you got to the survey login screen (on the right side). Please read the screen. Pretend you wanted to find your Census ID. Look at Mailing 4 and let us know what you would do to find the ID.

[Show Census ID] The ID you entered was 12 digits. For security reasons, sometimes IDs need to include letters and numbers like the example on the left. Sometimes to meet security requirements, IDs need to have more digits, like the 16-digit ID on the right side. Do you have any preference for IDs? That is, would you prefer to enter an ID with letters and numbers or a longer ID with just numbers, or does not matter to you? Please tell me more about your preference.

[Show Question 1 with large household instruction] On the screen is Question 1 on the first page of the paper questionnaire. You can also look at your paper questionnaire if you’d like.

Please tell me in your own words what the instruction beneath the question means.

[If they filled out the paper questionnaire and didn’t spontaneously comment on the instruction, ask:) Did you see the instruction when you completed the paper questionnaire, or did you not see it?

Do you have any other thoughts about the instruction?

[Show Question 8 Age Instruction] On the screen is a picture of the age and date of birth question on the paper questionnaire. You can also look at your paper questionnaire if you’d like.

Please tell me in your own words how (respondents should answer/you answered) that question.

Now I’m going to give you a few scenarios:

• If a household member was 10 years old, how would you answer this question for them?

• If a household member was 1 year old, how would you answer this question for them?

• If a household member was 6 months old, how would you answer this question for them?

[Show Question 8 Alternate Age Instruction] Now take a look at this alternative design. On the screen is a picture of a different age and date of birth question.

Please tell me in your own words how (respondents should answer/you answered) that question.

Now I’m going to give you a few scenarios:

• If a household member was 10 years old, how would you answer this question for them?

• If a household member was 1 year old, how would you answer this question for them?

• If a household member was 6 months old, how would you answer this question for them?

[Show Side by Side] What are your thoughts about the two different designs? Is there one you feel is easier to understand?

[Show Landing Page for the Online Survey] Please go to the following URL that will take you to the landing page.

Imagine you wanted to know more about what questions the survey would ask. What would you do?

Imagine you later realized that you made a mistake in the online form. Use the FAQs to tell me what you would do in this situation.

Imagine you wanted to get a job at the Census Bureau. What would you do?

What other questions do you have about the 2025 Census Survey?

[Have participant navigate Vacancy screens] We have a few scenarios and we want to know how you will answer some questions based on the task.

• Pretend the occupant moved out and the home is being renovated before it goes on the market. How would you answer this question?

• What about if the occupant got sick and had to move to a rehab facility for a while. How would you answer this question?

• Now pretend that the home that received the Census Survey invitation is a vacation home. How would you answer this question?

• What does the word “vacant” mean to you in the first option? Can you give some examples that this might refer to?

[Show the Other_Vacancy screen]

Take a look at the response choices and give me an example or two of situations that might fall into each option.

[Show Satisfaction questions] At the end of the online questionnaire, we allow users to rate their experience. Please take a look at two versions of this question and let me know what you think of each one and if you prefer one over the other.

Knowledge Check

Now we would like you to complete a few more short questions before we wrap up. Please go to the next page.

Okay, that’s it. Is there anything else that you’d like to share with us about the questions you have answered for us today?