Rapid Message Testing with Consumer Panel - FDA's Purple Book Website (CDER)

OMB Control No. 0910-0695

Expiration date: 2/28/2021

Purple Book

Cognitive Interview Guide

Color Key:

Purple – Section headings and titles

Black –Text for interviewer to read

Red – Interviewer instructions (not to be read aloud)

Green – Research questions (for interviewer’s information, not to be read aloud)

Introduction

Hello, my name is INTERVIEWER NAME. I work for Westat, a research company in Rockville, Maryland. Thank you for taking the time to be a part of this study.

Westat is working with the U.S. Food and Drug Administration, or FDA, to see what people think about FDA’s new Purple Book database, a searchable website that provides you with information on certain types of prescription medications. The FDA is interested in getting opinions about the information I am about to discuss with you.

In a moment, I will give control of the screen to you so you can look at a website. After you look at the website, I will ask you some questions about your thoughts and feelings about the information. Any ideas you have are okay.

IF PARTICIPANT JOINS BY PHONE ONLY, ASK THEM TO OPEN THE WEBEX MEETING ON THEIR COMPUTER. IF PARTICIPANT IS NOT ABLE TO VIEW THEIR COMPUTER SCREEN, RESCHEDULE INTERVIEW.

Informed Consent

Before we get started, there are a few things I need to tell you. This is a research project, and this interview is voluntary. That means that if you do not want to answer a specific question, just tell me and I’ll go to the next one. It is okay if you want to stop after we start. Just let me know.

All of your answers and everything you say will be kept secure to the extent permitted by law. This means that we will not share your name or information. Also, we will not link any information to your answers to figure out who you are. The interview will take about 45 minutes and you will receive $50 as a token of appreciation. We would like to record this conversation, if you are comfortable with that. The recording helps us make sure we hear everything you say correctly. Only the people who work on this research project will be able to listen to the recording and see our notes. Your name will not be linked to any of your responses, though we may include quotes that you provide in our reports. The recordings and our notes will be destroyed after we finish the project.

IF FDA STAFF ARE ON THE PHONE: I also want you to know that a couple other people from the FDA who work on this research project are listening to this interview to take notes as we talk.

Before we start, do you have any questions? Do you agree to be interviewed? Is it okay with you if I record the interview?

TURN ON RECORDER. The date and time is ____________. Now that I am recording, I want to ask again, is it okay if I record this interview?

Background Questions and Information Review

To start, I have a few questions.

I have in my notes that you have taken ____________ [RemicadeTM, InflectraTM, or RenflexisTM] in the past 8 weeks. Is that accurate? IF NO, INELIGIBLE.

Are you aware that ____________ [RemicadeTM, InflectraTM, or RenflexisTM] is a type of prescription medicine called a biological product, also known as a biologic?

What, if anything, do you know about biologics? IF NEEDED, What do you know about differences between biologics and other types of medicines?

How easy or difficult has it been for you to learn about [RemicadeTM, InflectraTM, or RenflexisTM]? Please respond on a scale of 1 to 5 where 1 is very difficult and 5 is very easy.

• What makes you say {PARTICIPANT’S SCALE NUMBER}?

How have you learned about this medicine or biologics in general? IF NEEDED, Specific websites, talking with your health care provider?

We are going to talk today about a database website. How familiar are you with databases? Please respond on a scale of 1 to 5 where 1 is very unfamiliar and 5 is very familiar.

• What makes you say {PARTICIPANT’S SCALE NUMBER}?

• IF 1 or 2, A database is used by organizations as a way of storing, managing and retrieving information. It's a sort of electronic filing cabinet.

The website we’ll discuss today is called the Purple Book, and it is a database of licensed biological products. What sorts of information would you expect to find on the Purple Book website?

• What functions or features might a website like this include to make it easy to learn about biologics? IF NEEDED, Search function, links, something else?

I am going to put a website on the screen now and hand control of the mouse to you. Please take a few minutes now to look at anything on this website that interests you. When you are done, I have some questions for you about the information.

CHECK IN AFTER 2 MINUTES. ALLOW 2-3 MORE MINUTES IF NEEDED.

CONCURRENT OBSERVATIONS/INSTRUCTIONS.

NOTE ANY OF THE FOLLOWING BEHAVIORS TO RECORD IN NOTES OR PROBE ON RETROSPECTIVELY:

• The webpages reviewed or actions performed.

• Any verbal reaction to statements in the information.

• Any verbal expressions of confusion, surprise, discomfort, offense. Note which statements evoked any of these reactions.

GENERAL PROBES

DIRECT PARTICIPANTS TO APPROPRIATE SCREENS AS NEEDED TO AID DISCUSSION.

RESEARCH QUESTIONS

How helpful is the Purple Book to participants in learning about their prescription biologic?

How do participants locate background information on the Purple Book?

How do participants use the search and the search results pages to locate information?

How do participants access definitions for product delivery icons?

How do participants locate full product information?

How do participants interact with the Drugs@FDA information page?

How do participants return to the Simple Search Results page from the FDA.gov information page?

How do participants approach and locate the social media buttons to share information about the Purple Book site?

What improvements do participants suggest for the Purple Book?

GENERAL IMPRESSIONS

IF NAVIGATED AWAY FROM FIRST PAGE: Please click [INSERT LINK] to return to the first page.

First, tell me your overall thoughts about what you see here.

What do you like about it?

What don’t you like?

How easy or difficult is it to understand the information on this first page?

Are there any words or phrases that confuse you or that you aren’t sure of?

Based on what you have seen, what is the goal of this website?

Who might use the website?

TASK 1: About the Purple Book

Next, I am going to ask you to find some information on this website.

• I will give you instructions before each activity.

• Please ask me if you have any questions that come up. I may not know the answers, but I need to know when questions arise.

• I would like you to think aloud and let me know what you are thinking as you go through this process.

We’ll start on the Purple Book Homepage [SCREENSHOT ABOVE]. [Randomize when this task is presented. Either as the first task or the last one.]

Figure 1 FDA Purple Book - About Page

Task 1) If you were interested in background information on the Purple Book, what would you do?

• Path(s):

• (L) Nav About

• FDA Search

• FAQ

Is this the information you expected to see?

• Can you tell me more about that?

What other information would you have liked to see?

• What else would be helpful to you here?

Are there other places on this website you might look for information or background on the Purple Book? Is so, show me what you would do.

• Path(s):

• (L) Nav About

• FDA Search

• FAQ

Do you have any other comments on what you see here?

IF THIS TASK IS LAST, Is there any information you have seen on these pages that you wish you had known when you first started using the Purple Book today?

• Can you tell me about that?

TASK 2: Searching for Biological Products

Figure 2 FDA Purple Book Landing Page

Suppose you are interested in learning more about [Remicade^TM, Inflectra^TM, or Renflexis^TM]. What would you do?

• Path(s):

  • Purple Book Search

Task Evaluation:

Independent completion - The participant completed the task without critical or non-critical errors

Completed with Non-Critical Errors - The participant completed after non-critical errors

Critical Error Committed - The participant was unable to reach the desired goal/complete the task

OBSERVE

• Did they try to use the FDA Search? Yes / No

• Did they type the full word or pick from the related terms? Type / Pick

How easy or difficult was the Purple Book’s Search to use? Please respond on a scale of 1 to 5 where 1 is very difficult and 5 is very easy.

• What makes you say {PARTICIPANT’S SCALE NUMBER}?

• Is there anything that would make the Search easier to use?

Figure 3 FDA Purple Book Simple Search Results

Is this the information you expected to see?

• Can you tell me more about that?

What other information would you have liked to see?

• What else would be helpful to you here?

Do you have any thoughts on the information you see here or how the page is structured?

Without touching your mouse or touchpad, please tell me what you think you could do on this page? [Record the participant’s responses. Do not read the list.]

• Click on the cards

• Hover over the delivery icons

• Say they could click on:

• Definitions for the icons

• Advanced Search

• New search

• Print

• Product labels

• The (i)s for more information

Tell me what you would expect to do first here, and why. [Record the participant’s responses. Do not read the list.]

• Click on the cards

• Hover over the delivery icons

• Say they could click on:

• Definitions for the icons

• Advanced Search

• New search

• Print

• Product labels

• The (i)s for more information

Do you have any other comments on this page?

TASK 3: Accessing Delivery Icon Definitions

Figure 4 FDA Purple Book Product Presentation Definitions

Figure 5 FDA Purple Book Administration Hover Over

Now let’s say you want to know how this product is administered. What would you do next?

• Path(s):

  • Click on the card to see the dosage information

  • Click on the link to see the legend

  • Click on the card

  • Product label link

How easy or difficult was it to find this information? Please respond on a scale of 1 to 5 where 1 is very difficult and 5 is very easy.

• What makes you say {PARTICIPANT’S SCALE NUMBER}?

Do you have enough information on this product to consider taking it?

• Can you tell me more about that?

What other information would you have liked to see?

• What else would be helpful to you here?

TASK 4: Product Details - Broader Product Information

Figure 6 Product Details

How can you learn more about one of these products?

• Path(s):

  • Link to the Product label

  • Click on the card

Task Evaluation:

Independent completion - The participant completed the task without critical or non-critical errors

Completed with Non-Critical Errors - The participant completed after non-critical errors

Critical Error Committed - The participant was unable to reach the desired goal/complete the task

If the participant does not go to this page on their own, please direct them to this page. Note whether or not you had to direct the participant to the page.

How easy or difficult was it to find this information? Please respond on a scale of 1 to 5 where 1 is very difficult and 5 is very easy.

• What makes you say {PARTICIPANT’S SCALE NUMBER}?

Is this the information you expected to see?

• Can you tell me more about that?

What other information would you have liked to see?

• What else would be helpful to you here?

TASK 5: Product Label on Drugs@FDA (Optional: Only if the participant goes to this page on their own)

Is this the information you expected to see?

• Can you tell me more about that?

What other information would you have liked to see?

• What else would be helpful to you here?

How would you return to Purple Book’s main page from here?

• Path(s):

  • Back button

  • Tab

Task Evaluation:

Independent completion - The participant completed the task without critical or non-critical errors

Completed with Non-Critical Errors - The participant completed after non-critical errors

Critical Error Committed - The participant was unable to reach the desired goal/complete the task

TASK 6: Returning to the Search Results page

How would you return to the Simple Search Results? [the page with the cards]

• Path(s):

  • Back button

Task Evaluation:

Independent completion - The participant completed the task without critical or non-critical errors

Completed with Non-Critical Errors - The participant completed after non-critical errors

Critical Error Committed - The participant was unable to reach the desired goal/complete the task

How easy or difficult was that? Please respond on a scale of 1 to 5 where 1 is very difficult and 5 is very easy.

• What makes you say {PARTICIPANT’S SCALE NUMBER}?

TASK 7: Social Media

Do you use Facebook or Twitter?

IF NO, GO TO WRAP-UP

IF YES: Would you post about or share the Purple Book using your personal social media platforms (Facebook and/or Twitter)?

• Can you tell me more about that?

If you wanted to post about or share the Purple Book on social media, how would you do that?

Path(s):

• Click “Facebook” or “Twitter” icons located under the Purple Book header on any/every page

Task Evaluation:

Independent completion - The participant completed the task without critical or non-critical errors

Completed with Non-Critical Errors - The participant completed after non-critical errors

Critical Error Committed - The participant was unable to reach the desired goal/complete the task

Would you connect with the Purple Book database through social media, in other words, if friends or family posted about the database on their social media platforms?

• Can you tell me more about that?

WRAP-UP

How helpful was this website in learning about [RemicadeTM, InflectraTM, or RenflexisTM]? Please respond on a scale of 1 to 5 where 1 is not at all helpful and 5 is extremely helpful.

• What makes you say {PARTICIPANT’S SCALE NUMBER}? IF NEEDED: What have you learned about your medication? Can you tell me more about that?

When you first saw the website, you said the website’s goal appears to be [INSERT]. Do you still think that is the goal or purpose of the website, or has your opinion changed?

• IF NEEDED: Now what do you think is the website’s goal?

When we started, you said this site might be used by [INSERT]. Now that you’ve used the website, would your answer stay the same?

• Is this a website you would return to on your own?

  • Can you tell me why [why not]?

• Are you aware of other online sources of information about your prescription medicines? Which ones?

• Do you like the Purple Book more or less compared those other online resources? Why?

• Do you think the Purple Book is any more or less trustworthy that those other online resources? Why?

• Is this a website that you would recommend to others?

  • Can you tell me why [why not]?

  • How would you describe the Purple Book to other people?

How would you describe your overall experience today working with the Purple Book website?

• Is there anything you expected to see on this website that you did not?

• Is there anything you expected to be able to do on this website that you could not?

Do you have any other suggestions for improving this website?

• Is there any other information that could be added? IF YES, What information?

• Is there anything about the website that could be changed? IF YES, What could be changed?

PROBE ON ANY OUTSTANDING ISSUES FROM OBSERVATION (INFORMATION THAT PARTICIPANT SEEMED CONFUSED ABOUT).

Closing

IF OBSERVERS ARE PRESENT, CHECK TO SEE IF THEY HAVE FURTHER QUESTIONS.

Those are all the questions I have for you. Is there anything we haven't talked about that you would like to tell me?

DISCUSS ANY PARTICIPANT COMMENTS.

Thank you for your time.

STOP TAPE RECORDER.