Protocol

AIES Communications Cognitive Interview Protocol

Research Questions:

• How do respondents comprehend our communications?

  • Do respondents understand that the AIES is mandatory?

  • Do respondents understand that the AIES is replacing some annual surveys?

  • Do respondents understand how to access their surveys based on instructions embedded in the communications?

• Do respondents have preferences regarding the mode of communications?

• How are respondents currently receiving our communications?

• What component of the messaging, if any, motivates respondents to start the survey?

Materials: Protocol, consent form, mailings/emails

Method: We will conduct the interviews by Microsoft Teams (or phone)

Expected length of interview: 1 hour (60 minutes) maximum

General probes that may be used throughout the interview:

• What else can you tell me about this?

• Can you tell me more about that?

• What looked unclear or is confusing here?

Introduction:

If necessary: I sent you a consent form in an email today – did you get a chance to sign that? If not, please do so now, and then we’ll get started.

Thank you so much for agreeing to talk with me today!

I am part of a research group within the Census Bureau that provides feedback to other parts of the Bureau about ways to improve the overall experience. I’m talking with you today to evaluate some of our communications about one of our surveys.

Remember, my job is to improve the survey experience. Please be candid and frank in your responses. Our interview is being conducted under the authority of Title 13, which means that your responses are confidential, and neither your name nor the name or identifying information about your company will be included in any of our findings.

Do you have any questions before we get started?

I’d like to record our session today so that when I go to analyze the results of these interviews, I can use the recording to pick up on anything I may have missed in my notes. Do I have your permission to record our session today?

<<Turn on recorder>>

Company Background:

• Tell me a little bit about your business. What types of goods or services does this business provide?

• What is your role in the company?

  • How long have you been with the company?

• What is your role in the process for responding to Census Bureau surveys?

  • Examples: gathering data, entering data, consulting with data providers, etc.

Communications Routing:

• Are you typically the person within your company that receives survey requests?

  • Can you tell me about that process?

• Do the Census Bureau letters typically go directly to you, or do they have to get routed to you by others?

  • How long does that process generally take?

• Do the Census Bureau emails typically go directly to you, or do they have to get routed to you by others?

  • How long does that process generally take?

Communication Materials

(Each respondent will only be asked about a subset of either the letters OR the emails. Each respondent will be asked about the support materials, time permitting. Respondents will be provided with copies of each communication they are being asked about.)

LETTERS

Advanced Mailing

Here is the first letter. Please take your time to look it over.

• Does anything about this letter stand out to you?

• What do you think the purpose of this letter is?

• Based on this letter, what do you think the Annual Integrated Economic Survey is?

  • If you wanted more information about the AIES, what would you do?

• Is there anything in this letter that is confusing to you, or might be confusing to others?

• Would you take any action after receiving this letter? If so, why and what would you do?

  • Would you visit the website?

Initial Request Mailing

• What are your first impressions?

• Does anything about this letter stand out to you?

• What do you think the purpose of this letter is?

• Is there anything in this letter that is confusing to you, or might be confusing to others?

• Would you take any action after receiving this letter? If so, why and what would you do?

• Did you notice any reasons for doing the survey?

  • What were they?

  • What did you think about these reasons?

• Do you think you have to do this survey?

• Is there any information missing from this letter that you would have found helpful?

Due Date Reminder Mailing

• Does anything about this letter stand out to you?

• What do you think the purpose of this letter is?

• Is there anything in this letter that is confusing to you, or might be confusing to others?

• Would you take any action after receiving this letter? If so, why and what would you do?

1^st Past Due Follow-up Mailing

• Does anything about this letter stand out to you?

• What do you think the purpose of this letter is?

• Is there anything in this letter that is confusing to you, or might be confusing to others?

• Would you take any action after receiving this letter? If so, why and what would you do?

EMAILS

Advanced Email

Here is the first email. Please take your time to look it over.

• What are your first impressions?

• Does anything about this email stand out to you?

• What do you think the purpose of this email is?

• Based on this email, what do you think the Annual Integrated Economic Survey is?

  • If you wanted more information about the AIES, what would you do?

• Is there anything in this email that is confusing to you, or might be confusing to others?

• Would you take any action after receiving this email? If so, why and what would you do?

  • Would you visit the website?

• Do you recall the subject line? Do you feel the subject line is appropriate?

Initial Request Email

• What are your first impressions?

• Does anything about this email stand out to you?

• What do you think the purpose of this email is?

• Is there anything in this email that is confusing to you, or might be confusing to others?

• Would you take any action after receiving this email? If so, why and what would you do?

• Did you notice any reasons for doing the survey?

  • What were they?

  • What did you think about these reasons?

• Do you think you have to do this survey?

• Is there any information missing from this email that you would have found helpful?

• Do you recall the subject line? Do you feel the subject line is appropriate?

Due Date Reminder Email

• What are your first impressions?

• Does anything about this email stand out to you?

• What do you think the purpose of this email is?

• Is there anything in this email that is confusing to you, or might be confusing to others?

• Would you take any action after receiving this email? If so, why and what would you do?

• Do you recall if the subject line grabbed your attention? Do you feel the subject line was appropriate?

1^st Past Due Follow-up Email

• What are your first impressions?

• Does anything about this email stand out to you?

• What do you think the purpose of this email is?

• Is there anything in this email that is confusing to you, or might be confusing to others?

• Would you take any action after receiving this email? If so, why and what would you do?

• Do you recall if the subject line grabbed your attention? Do you feel the subject line was appropriate?

SUPPORT MATERIALS

Welcome Emails

This is an email you might automatically receive after inputting your authentication code, please take a minute to look it over.

• What do you think the purpose of this email is?

• Do these resources seem useful or not useful?

• What resources would you like to see before starting a new survey?

• Would you take any action after receiving this email? If so, why and what would you do?

• Do you recall if the subject line grabbed your attention? Do you feel the subject line was appropriate?

Thank you Emails

This is an email you might automatically receive after submitting your finished survey, please take a minute to look it over.

• What do you think the purpose of this email is?

• What information would you like to see after completing a survey?

• Would you take any action after receiving this email? If so, why and what would you do?

• Based on this email, what would you expect to happen next year?

• Do you recall if the subject line grabbed your attention? Do you feel the subject line was appropriate?

Content Canisters

This is a screenshot of the respondent portal; I have circled a box that contains some information. This box may automatically appear after you enter your authentication code. Please take a minute to look it over.

• What do you think the purpose of this box is?

• What information would you like to see here?

AIES Website

Let’s take a look at the AIES website.

• What are your first impressions?

• Does anything about this website stand out to you?

• What sort of information do you expect to be available on this website?

Overall Impressions

• In general, how likely are you to read these letters and emails when/if you receive them?

• What types of communications do you generally prefer (i.e., letters, emails, phone calls)?

Wrap-up

• That’s all the questions I have for you today! Do you have any other comments, questions, or suggestions for us?

Thank you so much for your time today.