Form #4 Form #4 Discussion Groups Interview Guide for Users

Attachment E1: Interview Guide – Discussion Groups - Users

OMB No. XXXX-XXXX
Exp. Date XX/XX/20XX

Audience Discussion Group (Users) – Interview Guide

INTRODUCTION

Thank you all for agreeing to join us today. My name is [moderator’s name] and this is [Observer’s name, if applicable] and we work for [Westat/Mosaica Partners].

We are conducting this evaluation for the Agency for Healthcare Research and Quality (AHRQ). The purpose of today’s discussion is to explore the need for – and the actual use of – information and tools supporting the evaluation of health IT systems. We would like to explore:

• The kind of tools or aids you have actually used – or wished you had available – to support evaluation of health IT systems.

• Your use of surveys to evaluate health IT systems.

• Your opinion of how a specific tool – which we will show to you later in this session – could assist you in your role.

Because of the focus on the use of surveys in researching and implementing health IT systems, as we explained when we scheduled this interview, we are interviewing researchers and implementers with direct experience in this area.

Confirm that all participants have successfully signed on to the meeting software.

Next, we would like to confirm that you have successfully signed on to our web meeting room. Do you all see the Agency for Healthcare Research and Quality National Resource Center for Health IT homepage message on your screen?

We have 90 minutes scheduled for this call. Does that still work with your schedule?

Those who can’t dedicate at least an hour to the group discussion should be thanked and told we may call them to reschedule if there are any openings. If someone can make an hour but not the full 90 minutes ask them to drop off the call when they need to.

Public reporting burden for this collection of information is estimated to average 90 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (XXXX-XXXX) AHRQ, 540 Gaither Road, Room # 5036, Rockville, MD 20850.

We would like to begin by having each of you introduce yourself, and in a few sentences, describe your current role and involvement with health IT systems. I’ll now call on you, one by one, to introduce yourself.

Ask each participant to give his/her introduction.

Thank you. We certainly have an interesting group today. The results of this study will be provided to AHRQ staff to use in planning to enhance tools and aids they provide. The information you share with us today will help ensure that AHRQ continues to provide relevant and timely support to the health IT community.

Before beginning, we would like to remind you about how we will use the information you provide. Your responses will be kept confidential to the extent permitted by law, including AHRQ’s confidentiality statute, 42 USC 299c-3(c). We may include respondent comments in reports but will not attribute the comments to specific individuals or organizations. We also have a system to mark specific comments as off-limits for any reports that might be made public. If any of the information we cover today falls into this category, please let us know and we will mark the information as off-limits in our notes. Your participation in this study is voluntary. If at any time you do not feel comfortable answering a question, or do not know the answer, please let us know and we will move on. We would like to record this session to assist us in ensuring we accurately capture your comments.

Does anyone have any objections to our recording the session?

If Yes: We will simply take notes, but not record the interview.

If No: Start the recording feature of the meeting software.

Okay. During this group discussion, we will be asking you a series of questions related to your experiences in the research and implementation of health IT systems. We will show you portions of the AHRQ National Resource Center for Health IT Web site, and ask for your opinions about the contents and functionality of a specific tool available on the site.

Do you have any questions before we begin?

Q1. What kinds of tools and aids have you used to support your work in health IT research and/or implementation?

Q1.a. What sources did you use to locate those tools and aids?

Q1.b. How useful were those sources for finding relevant tools and aids?

Q2. What other kinds of tools and aids are you interested in using?

Q2.a. Why are you interested in those kinds of tools and aids?

Q2.b. How would you apply those to your work?

Q2.c. Have you been able to find these kinds of surveys – do they exist?

Q2.b.1. If so: What sources did you use to find those surveys?

Elicit Initial Reactions to Health IT Survey Compendium

Thank you. Now we will look at a specific tool that is currently available on the Agency for Healthcare Research and Quality (AHRQ) National Resource Center website. This is called the Health IT Survey Compendium and is a collection of surveys and other instruments.

Show the Health IT Survey Compendium webpage.

I will be asking for your candid feedback about this tool. All comments – both positive and negative – are welcome. I was not involved with creating the Survey Compendium, so negative comments will not hurt my feelings.

What is important to remember is that your feedback during today’s session will help AHRQ provide access to a collection of publicly-available tools related to the evaluation of health IT systems.

Q3. When enlisting you for this evaluation, you all identified yourselves as previous users of the Health IT Survey Compendium. When did you first access the Survey Compendium?

Q4. Do you recall your first impressions of the Health IT Survey Compendium?

Q4.a. What are your impressions now?

Q5. Why did you access it? What were you searching for? (What did you use the Health IT Survey Survey Compendium to find?)

Q6. Did you find what you needed?

Q6.a. If yes: How did you use what you found?

Q6.b. If some respondents did not find what they needed, ask: Where else did you search for these materials? Did you find what you were looking for there?

Q7. How useful was the Health IT Survey Compendium to you?

Q8. What are your thoughts about the types of audiences this site is intended for?

Q8.a. What types of people might find this Health IT Survey Compendium useful?

Category Meaning, Expectations, and Usability

Let’s look at the four categories that can be used to locate surveys.

Q9. What are your thoughts about these four categories provided to help find surveys? (Probe what they think the category headings mean to them – “Survey Type,” “Care Setting,” etc.)

Q9.a. What do you think about this first category – Survey Type? Are these types of interest to you or not? Why?

Q9.b. What about the second category – Care Setting? Are these settings of interest to you or not? Why?

Q9.c. What about the third category – Technology? Are these technology types of interest to you or not? Why?

Q9.d. What about the forth category – Respondent Type? Are these subgroups of interest to you or not? Why?

Q10. What other selection categories would you expect to see in this type of tool?

Q10.a. What additional categories, if any, would be helpful for narrowing your results?

Q11. What other ways do you expect to see or would like to have to efficiently and effectively locate surveys?

A description of survey types is shown below the four pull down menus that let you narrow your search for surveys.

Q12. What are your thoughts about that information and how it is presented?

Q12.a. What do you think about the descriptions provided for Survey Types?

Q12.b. Is the information helpful or not helpful to explain how these surveys might apply to your organization?

Q12.c. If not helpful: What other information, if any, should be added to improve these descriptions?

Display of Results

Let’s make a selection and view some results.

Select satisfaction or some other alternative from the “Survey Type” drop-down.

Q13. What are your reactions to the way results are displayed?

Q13.a. What do you think about the information (description, tagging, disclaimer, user level, URL) provided for the surveys?

Q13.b. What do you think about the number of results displayed?

Q13.c. What do you think about the order of the results?

Q14. Does the results display convey information that is relevant for you? Why or why not?

Q14.a. What, if any, additional information would you like to see to help you decide which survey(s) to view in more detail?

Relevance of Health IT Survey Compendium

Either:

1. Show lists of categories of surveys, or

2. Ask participants to suggest some selections within each category that are of interest, and allow participants a little time to scan the results to see the types of surveys yielded.

Q15. Do these lists of results provide surveys that seem relevant to you? Why or why not?

Q16. Would these surveys be useful for you? Why or why not?

Completeness of Health IT Survey Compendium

Q17. Are there other kinds of surveys that you expect to see, but are missing?

Q17.a. If so, What are they? What topics would they cover?

Q17.b. What kind of information would these types of surveys provide to you?

Ease of Use of the Health IT Survey Compendium

Q18. What, if anything, could be done to improve the method to locate surveys?

Q18.a. If responses are to retain as is: Why do you like the current approach?

Q18.b. If responses are to make changes: What would you suggest changing? What would you prefer?

Audience Interest and Needs

Q19. Now that you have seen the Health IT Survey Compendium, would you visit the site in the future? Why or why not?

Intended Use

Q20. How would you use these surveys in your current or future projects?

Q21. What could be done to make the Health IT Survey Compendium more relevant for you and your role within your organization?

Comparison with Other Sources

Q22. How does this Health IT Survey Compendium compare with other sources that you use or are aware of?

Wrap Up

Q23. Are there any issues related to health IT tools or aids that we have not discussed that you think would be appropriate for us to know?

As we mentioned in our telephone interview, you are eligible to receive an honorarium of $75 for your participation in this portion of the research study. You can expect this check to arrive within the next XX days.

Thank you all for your participation today and for helping us gather the kind of information that AHRQ can use to assess relevant tools to improve the efficiency and effectiveness of the evaluation of health IT systems.

Good bye.

11