FAQ Web Pages and Alternative Designs Focus Group

Social Security FAQ Webpage Redesign Project

Usability Testing – Moderator’s Guide

1. Introduction/Welcome

Good evening. My name is ___________ and I will be moderating our session today. I am employed by Helix, a research firm located in Memphis, TN. We have contracted with the Social Security Administration (SSA) to moderate this and other usability sessions.

You have been invited here because SSA is interested in hearing your opinions about the Agency’s Frequently Asked Questions website and to observe your efforts to obtain information from the site. Our observations of your experience will help us improve this important SSA Internet page.

We will focus our efforts on three particular segments of the SSA website:

• Socialsecurity.gov home page,

• Frequently Asked Questions (FAQ) home page, and

• FAQ Answer pages.

I will ask you to find the SSA FAQ home page and to answer a test your ability to answer questions typically asked by the public about various Social Security programs and services. As you search the FAQ site for answers, I will ask you to provide feedback concerning what you are experiencing as you move about the site and what you are seeing.

Usability sessions will be conducted in the Baltimore/Washington DC and Memphis TN areas. SSA has requested these usability sessions in order to improve the FAQ service. Your opinion is very valuable to us.

When you came in today, you reviewed and signed an informed consent form for your participation in today’s usability testing session (moderator holds up a blank copy.) Do you have any questions about what you signed?

To help make our session run as smoothly as possible, I’d like to review some ground rules for our discussion:

1. We are audio taping (not videotaping) today’s session. This will be used only to ensure that the qualitative feedback and our observations of your experience are captured completely and accurately. Only your feedback and observations, as well as our observations of the session, will be shared with SSA. No personal information will be divulged.

2. In addition to your feedback, I will be observing how you move about and react to the FAQ website.

3. We will evaluate your user experience only for the purpose of gaining insights into how we can improve SSA’s FAQs. There are no right and wrong answers, no expected reaction to navigating around the FAQ pages. We simply want to identify areas for potential improvement. The purpose of a group session is for us to look, listen and learn.

4. There are some observers listening to and observing your session from a remote location. This is a small group from SSA and Helix who want to listen in on what you have to say and “from a distance” watch as you move about the FAQ website. Their observations will help us with our subsequent evaluations of this and other usability sessions. You should not feel constrained by their role. Everything you say or do here today is confidential. In addition, our observers have signed formal observer confidentiality forms.

5. If you need use the restroom, they are located (Moderator—specify). We can interrupt the session briefly, if needed.

6. Your participation is voluntary. Therefore, at any point you may leave the room.

7. If you have a cell phone, please turn it off or set to vibrate.

8. We estimate that our testing and discussion will last about an hour.

Do you have any questions before we get started?

Before we begin, I want to let you know that I’m not an expert on specific Social Security programs or services. My job as moderator is to assist the Administration in gathering user feedback that can be used to improve the SSA FAQ website. My job as a moderator is to:

1. Help guide the flow of your usability session

2. Facilitate our discussion in order to better understand your user experience

3. Make sure that your experiences, feedback and recommendations are accurately communicated to SSA

4. Ensure that we cover the necessary sections of the SSA website (i.e., SSA Home Page, SSA FAQ page, answers to FAQs, etc.)

Read Paperwork Reduction Act Statement:

Paperwork Reduction Act Statement – This focus group, cleared under OMB No. 0960-0788, meets the requirements of 44 U.S.C. § 3507, as amended by section 2 of the Paperwork Reduction Act of 1995. Participation in this focus group is voluntary. We estimate that it will take 1 hour to complete the focus group. If you would like to comment on our time estimate, please send your comments to: SSA, 6401 Security Blvd Baltimore, MD 21235-6401.

2. Self Introductions and Icebreaker

• Let’s begin with introductions. I will tell you a little about myself and then ask you to do the same.

• Please tell me if you have ever visited the SSA website (www.socialsecruity.gov) and, if so, you have ever used the FAQ section before.

• Were you able to find what you were looking for?

3. Frequently Asked Questions Website

I am going to ask you to navigate about the SSA website. I want to learn how you get to the FAQ section and then I want to observe your movements and actions while in the FAQ pages. I will ask you to come up with typical questions about Social Security and find an answer. I will also give you some questions to answer. We will stop periodically and talk about your experience and your assessment of various features. I will be interested in learning how you are moving about the page(s), which feature are clear and which are not. Are there aspects of FAQs that are confusing? Are there features that are missing, that you would expect to find at an FAQ site? We will talk about all aspects of your efforts to find answers to questions. Do you have any questions before we begin?

Social Security FAQ Webpage Redesign Project

Usability Discussion Questions & Session Guide

1. First impressions

• What is the very first thing you notice on the FAQ page?

• What are the first three areas or links that your eyes are drawn to?

• Based on what you see on this page, which areas or links do you think the Social Security Administration considers most important about its FAQ page?

• In general, what are your first impressions of the FAQ landing page?

2. Overall appearance of the FAQ page

• What is the initial reaction to the current FAQ page?

• Are there any concerns regarding the overall design of the current page?

• Does the FAQ page appear to be that of an official government site? Of the SSA home page?

• What do you think of the colors on the FAQ page?

• Does the FAQ site appear to meet likely task and informational needs?

• What changes regarding design and appearance of the FAQ pages would you recommend?

3. Content on the FAQ landing page

• Does one area of the FAQ page seem to have more useful content and information than another?

• Is the amount of information presented on the page too much, too little or about right?

• Is the information on the FAQ page what you would expect to find?

• Is there information you would expect to find on our FAQ page that is not there?

4. Navigation

• Are there problems with navigation on the FAQ page?

• Where were you just prior to navigating to the FAQ page?

• Which feature would you use to find the answer to your question (i.e., Search, Most Popular Answers or Most Popular Topics)?

• Are the descriptions of the links for “Most Popular Answers” (left column) and “Most Popular Topics” (right column) clear and understandable?

• Do you prefer one of the three navigation features over another?

5. Specific User Tasks

• Ask the user to find an answer to a Social Security question.

• After the user searches for answers to self-selected questions, ask the user to answer questions selected by the moderator:

• How many work credits do I need to qualify for Social Security Retirement benefits?

• What is the difference between Social Security disability and SSI disability payments?

• How old do you have to be to apply for retirement benefits?

• What is the address for your local Social Security office?

• Ask the user to verbalize their opinions about how easy it was to find the information they were seeking?

• Were the instructions clear?

• Were the answers received relevant to your question?

• Were the answers understandable and clear?

• Did the user receive too much or too little information?

• Did the user prefer searching or browsing by topic or categories?

• How did the user react when an answer was not found?

• How much or little did the user scroll?

Were the categories and labels intuitive to the user?

6. Recommendations

• Is there information on the FAQ home page or the answer pages that should be omitted?

• Is there information on the FAQ page that could be worded differently to make it more clear and understandable?

• Any other recommendations to improve the FAQ site?

Recording consent form

Thank you for participating in our usability research.

We will be recording your session to allow Helix Business Solutions and Social Security Administration team members who are unable to be here today to observe your session and benefit from your comments.

Please read the statement below and sign where indicated.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

I understand that my usability test session will be recorded.

I grant Helix Business Solutions and Social Security Administration permission to use this recording for internal use only, for the purpose of improving the designs being tested.

Signature: _______________________________

Print your name: _______________________________

Date: _______________________________

7