Form 1 Medical group administrators: Initial, highest level adm

M edical Group Moderator Guide:

2016 AHRQ Measurement for Performance Improvement in Physician Practices

1. Introduction & Participant Instructions

Hi, and thank you for agreeing to speak with us today. My name is <FIRST and LAST NAME>, and I work for <Westat/RAND>, a research company based in <Rockville, MD/Santa Monica, CA>. Today I’d like to talk to you about measurement for performance improvement in your group practice.

Westat, together with RAND, is conducting an exploratory study funded by the Agency for Healthcare Research and Quality (AHRQ) to better understand the current landscape of measurement for internal performance improvement in physician practices in the United States. At this initial stage, we are looking to learn from you what groups like yours are doing to measure performance improvement and what about internal performance improvement is of greatest interest to you. Based on what we learn in this exploratory research, AHRQ hopes to identify implications, potential impacts, and future research opportunities for payers, regulators, and medical groups regarding internal measurements for performance improvement.

Your responses will be kept confidential. For our reports, the comments that you share will be grouped with comments from other people like you who are also taking part in this research. Your name and the name of your group practice will not be associated with anything you say.

Your participation in today’s session is completely voluntary. If you would like to stop at any time, please tell me and we will stop immediately. If there are any questions you do not want to answer, just tell me and we can skip them.

Other team members working on this project and AHRQ representatives may join us today as observers or listen to recordings as part of the research process. As a result, we would like to record this session. Recording will also ensure we don’t miss anything. Because we will be recording, please try to speak in a voice at least as loud as the one I’m using now so that we can make sure the tape is picking up our voices.

2. Consent

Interviewer Instructions:

• REVIEW CONSENT FORM WITH RESPONDENT

• ANSWER ANY QUESTIONS.

• OBTAIN CONSENT TO PROCEED AND TO RECORD.

• Explain FEATURE TO record, TURN IT ON, AND THEN…

• OBTAIN CONSENT AGAIN SO THAT CONSENT IS RECORDED.

Do you have any questions before we begin?

3. Interview Protocol

3.1 Background

1. We understand that you are the [title/position] in [physician group name]. Is that correct?

2. How long have you been working at [physician group name]?

3. How long have you been the [title/position]?

   1. Did you have another role here at [physician group name] before this one?

PROBE: Tell me more about that…

3.2 Measurement for Internal Performance Improvement

4. From your perspective, what is internal measurement for performance improvement?

PROBES:

• What does “performance improvement” mean to you?

• When you think of “internal measurement,” what do you mean?

  • Do you collect data other than those data required by external contracts or relationships?

  • Do you use or “internalize” data that you collect for external reporting or receive from external sources (publicly or privately) for needs specific to your group practice?

5. Talk me through your group practice’s processes for measuring performance internally.

PROBES:

• Who is involved in performance measurement within [physician group name]?

  • Plan developers/decision makers? Implementers? Observers/oversight?

• How does your group decide what to measure for internal performance improvement?

• What is measured? / What data are used?

  • Why did you choose these measures/this data?

    • Did you consider alternative measures? Tell me more about that…

• Why do you collect the measures/information you do or review/analyze the data you do?

  • What is your motivation for collecting/reviewing these data?

  • What is your primary goal for using this data/information?

• How has internal performance measurement evolved over time at [physician group name]?

  • Are there any internal performance measures that [physician group name] has used in the past that are no longer used?

  • Have you used other data internally in the past that you no longer do?

    • Why?

• How does the information you collect or use internally relate to your external measure reporting obligations, if at all? How about your external incentives, financial or otherwise? Tell me more about that…

  • [If applicable, based on how the group is organized] Are there any differences between internal performance measurement activities for different types of providers in your group (e.g., “affiliated” versus employed or “core group” physicians)?

• Tell me about your data collection/analysis process.

  • How do you collect/analyze/review data/information for internal performance improvement?

  • How has technology affected your ability to collect and use internal measurement for practice improvement? Has the technology available to you limited your ability to measure performance in any way?

  • Who has primary responsibility for data collection/data analysis/review?

  • What is the frequency of data collection/data review?

  • What are your data sources?

  • Can you estimate the costs associated with data collection/data review?

    • Have the costs of measurement affected which internal measures are chosen and how they are used?

  • What factors have made data collection/data review more or less difficult?

  • [If not clear:] Who has primary responsibility for performance improvement activities based on these internal performance measures?

INTERVIEWER NOTE: The goal of this question set is to understand what performance improvement in this group looks like and what activities are included. If the respondent needs more specific probing, ask about specific measures/data used and use probes above to engage respondent on these specific measures.

• What internally collected measures have been priorities for [physician group name] over the past 1 or 2 years?

6. How do you use the internal performance information that [physician group name] collects/reviews?

PROBES:

• How does group management and/or the governing board use this data/information?

• Are patients involved in the process? Do patients use the data or do you have a patient advisory council involved?

• How do clinicians use this data/information?

• Is internal performance measurement the basis for incentives within the group? Tell me more about that…

7. Do you have any examples you can share with me where performance improved on a measure you track internally? Tell me more about that…

PROBES:

• How do you determine/define improved performance?

• What’s worked and what hasn’t worked?

• In your opinion, what impact has internal performance measurement had on your practice?

  • What’s changed as a result of something you’ve measured?

  • Has internal performance measurement helped you provide better care to patients? Tell me more about that…

  • Has internal performance measurement been worth the investment?

• What are the effects of using internal performance measurement? Who is affected?

• Can you give me examples – walk me through some specific cases?

• Have there been any unintended results, both positive and/or less positive, of measuring internal performance improvement?

8. How do you report or communicate performance improvement within your group practice? What about outside your group practice? Tell me more about that…

PROBES:

• Have there been any unintended results of how you report or communicate performance improvement?

• How have physicians and other clinicians responded to performance improvement measurement and how this information is shared?

  • Do your clinicians see only their data? Everyone’s data (blinded or unblended)? Aggregate data?

• What are the implications for clinicians? Are they incentivized or penalized in any way based on internal measurement?

  • What is their motivation to buy-in?

INTERVIEWER NOTE: Throughout the course of the discussion take note of mention of specific measures. Request any artifacts that can give you details about specific measures used – checklists; measure specifications; de-identified copies of dashboards, reports, etc.

3.3 The Future

9. What internal performance measurement efforts are you planning, that you aren’t already doing?

   1. Are there any specific measures or processes that you are planning to add?

   2. Are there any specific measures or processes that you’d like to add, but may or may not be able to in the near future?

PROBE: Tell me more about that…

10. Are there any specific measures or elements of your internal performance improvement plan and/or process that you think you will discontinue in the next year or two?

PROBES:

• Why are you thinking about dropping them?

• How did you decide to discontinue them? Can you talk me through your decision making process?

3.4 Wrapping Up

11. What types of support or information is useful or would be useful to help you or groups like [physician group name] better use the information and data you gather from your internal measurement for performance improvement?

12. Have there been any major changes to your group or group organization over the last couple of years that have impacted internal performance improvement measurement that we have not yet discussed? Tell me more about that…

PROBES:

• How about any changes in group ownership, size, specialty mix, or risk bearing contracts/relationships?

13. Is there anything else we haven’t discussed that would be important for us to understand regarding internal measurement for performance improvement?

1. Anything that we haven’t asked that we should have asked?

14. Based on our discussions with [you/name of other group representative], we are also talking with [names of staff]. Is there anyone else we should be speaking to about internal measurement performance at this group?

Closing

Do you have any final comments that you would like to share before we finish?

On behalf of AHRQ and the Westat team, thank you very much for your time today and more importantly your insight. Again, our goal is to look to identify implications, potential impacts, and future research opportunities for payers, regulators, and medical groups regarding internal measurements for performance improvement. If you think of anything else you’d like to share with us after today, please don’t hesitate to reach out and send that information to me.

Thank you again!

REVISED as of 6/14/2016 7