Supporting Statement B - Care Coordination - 10.5.09

SUPPORTING STATEMENT

Part B

Coordinating Care across Primary Care and Specialty Care Practices

Version: October 5, 2009

Agency of Healthcare Research and Quality (AHRQ)

Table of contents

B. Collections of Information Employing Statistical Methods 3

1. Overview 3

2. Respondent universe and sampling methods 3

3. Information Collection Procedures 4

4. Methods to Maximize Response Rates 5

5. Tests of Procedures 6

6. Statistical Consultants 6

Section B. Collection of information employing statistical methods

Overview

This project consists of three main components:

1) Secondary analysis of electronic medical record (EMR) data related to the referral processes, which involves no actual data collection, although it does require the research team to extract information from the BMC EMR

2) Focus groups with providers, clinical staff, and administrative staff involved in the redesigned referral process; and

3) Project team intervention logs and meeting notes to document the processes by which the redesigned system is implemented.

Furthermore, in this protocol, the only data that will involve burden to subjects will be the focus group data and we will conduct no data collection from human subjects involving subject burden that will employ statistical methods. The EMR data will be analyzed with quantitative statistical methods, but these data are already in an EMR available to use electronically. The focus group data, implementation logs and meeting notes will be analyzed using qualitative data analysis and we will not employ statistical methods to conduct these analyses. However, we describe each data collection activity below, in addition to the descriptions available in Supporting Statement A.

B.1. Respondent universe and sampling methods

1) EMR Data

Separate samples will be abstracted for GI and OB referrals. Medical records are selected if they meet the following criteria: a) for the pre-implementation medical record reviews, patients must be over 18 years of age and have been the recipient of a referral to one of the specialty care clinics identified in the study in the 8 months prior to implementation; b) for the post-implementation medical record reviews, patients must be over 18 years of age and have been the recipient of a referral to one of the specialty care clinics identified in the study in the 8 months since the redesigned referral system was implemented.

Based on current referral levels, we estimate approximately a total of 2,185 GI referrals and 270 OB referrals for the diagnoses/procedures targeted in each 8 month review period across the three clinics. From this universe of referrals, we will review all OB referrals and will sample GI referrals stratified by primary care site to yield approximately 315 GI cases for review. Our sample will be selected using a random numbers table at each site and taking every 7^th referral. This data collection be performed by project staff and will not impose a time burden on the clinics.

2) Focus groups

The sample for the focus groups will include all of the providers, clinical staff, and administrative staff involved in the redesigned referral process at the five participating sites. Because of the limited number of potential participants, recruitment will be done informally by clinical leaders in that practice, who will make it clear that participation is voluntary and will use the script attached in B for recruitment.  

Inclusion criteria for the focus groups include being a medical provider, other clinical or administrative staff at the various primary care and specialty care sites, willingness to participate, and identification of role as important in implementation or carrying out of referral processes.

As we are including only clinicians at the five participating clinics, our focus groups are not representative of all health care clinics, although they are likely very similar to other clinics in urban settings serving poor and underserved populations. Therefore, although the results of the qualitative analysis of the focus group data cannot be generalized, they can provide important lessons learned that may be helpful to other clinics considering a redesign of their referral system.

3) Implementation logs

The implementation logs (see attachment D) are simply logs that will include documentation from the project team during the implementation process. There is no universe or sampling.

B.2. Information collection procedures

1) EMR Data

All data will be extracted from the EMR and entered into an SPSS database for analysis by the project Research Assistant. The pre-implementation data will be extracted and analyzed during the time period of months 0-2 before implementation (although it may run over, depending on how long the process takes), and, as described above, cover referrals made in the 8 months prior to implementation of the redesigned referral system. The post-implementation data will include all referrals during approximately the first 8-12 months of the implementation period (depending on our timeline and time available), and extraction and analysis will begin at the 10^th month after implementation begins. Activities involving abstraction and analysis will involve the research team only.

The following 11 outcome measures will be extracted from the electronic medical records:

Key Outcome	Data source to determine outcome
Referral information is sent from PCP site to specialty site.	Paper records and EMR
Of those referrals, what percent were complete?	Paper records and EMR
Percent of time an appointment scheduled for patient?	EMR
Time between referral made and initial specialty care appointment made	Paper records and EMR
Percent of time PCP was made aware appointment was scheduled?	EMR
Percent of time there is a note in medical record that referral received by specialist?	EMR
Percent of time PCP is notified if patient does not attend specialty appointment	EMR
Percent of time clinical information is available on referral form by the time of specialty appointment	Paper records and EMR
Percent of time specialist sends note to PCP after appointment?	EMR
Percent of time there is note in the medical record that PCP received communication back from specialist	EMR
Percent of time the specialist makes a note of follow-up plan and responsibility	Paper records and EMR

Standard empirical methods will be used to analyze the quantitative data collected from the EMR. All data will be entered into a database using SPSS 16.0. After data are collected, we will perform data checking on a 5%-10% sample of the data to check for data quality and missing data. Once the database is completed, we will perform standard statistical analyses, specifically descriptive statistics including frequencies, measures of central tendency, and measures of variability. Data will be stratified by site, by provider type (i.e., primary care provider versus specialist, MD versus other clinical staff, including RNs and PAs), by referral type requested, and potentially by other characteristics identified in the redesign process. We will examine change from pre-implementation to post-implementation using bi-variate analyses, such as t-tests and chi-square tests, as appropriate. If we have sufficient data, we will estimate regression models using the different performance measures as our outcome variables. Explanatory variables will include site, provider type and procedure/consultation type requested.

2) Focus groups

The focus groups will be conducted by two-person teams from the BUSPH evaluation team using semi-structured protocols (attached in B). The group sessions will pursue three general areas: the extent to which the new system is being used as intended; the perceived effectiveness of the new system as implemented; and the organization and culture of the clinical setting. By necessity, the details of the questions will be tailored to the emerging discussion of each focus group as influenced by group dynamics, the professional perspectives and sites of care of the participants.

We plan to conduct one focus group with clinicians and one focus group with administrative staff at each site, for a total of 10 focus groups. We expect each focus group to include 4-10 participants. All of the focus groups will be audio-taped with participants’ permission and professionally transcribed for analysis as word processing text files suitable for manipulation using qualitative software. Audio tapes will be transcribed verbatim and reviewed to confirm their accuracy.

Focus group and interview data will be coded using accepted qualitative methods. To ensure reliability and validity of the coding, two members of the BUSPH team will jointly examine two initial key informant interviews/transcripts. The two will then come together in order to develop a common set of content codes and formulate coding rules. They will then each independently code all remaining interviews/ transcripts and compare their coding to assess the reliability of the coding rules, with discrepancies resolved and coding rules modified through discussion. This process will be repeated until all interviews/ transcripts are coded. The goal of qualitative analysis, in this instance, is to establish a shared, coherent set of codes and coding rules through a process of critical review and consensus building.

3) Implementation logs

There will be no data collection. Each member of the project team will write notes in logs as appropriate.

B.3. Methods to maximize response rates.

1) EMR Data

Not applicable, as we are extracting data that exists in BMC’s EMR only and are not collecting any new data.

2) Focus groups

To maximize response rates, clinic staff will be invited by the project team member from their site to participate. They will also be told about the importance of the project and our goal of increasing the quality of the referral and transition of care process.

3) Implementation logs

Not applicable.

B.4. Tests of procedures

1) EMR Data.

Not applicable.

2) Focus groups

Not applicable.

3) Implementation logs

Not applicable.

.

The procedures used in the redesign of a patient referral system were not pretested.

B.5. Statistical consultants

1) EMR Data

We consulted Michael Schwartz, PhD, Richard D. Cohen Professor in Management and Professor of Health Care and Operations Management at the Boston University School of Management about the statistical aspects of design and quantitative analysis. Dr. Shwartz can be reached at 617-353-4243.

Mari-Lynn Drainoni, PhD, Co-Principal Investigator, Charles Williams, MD, Co-Investigator and Andrea Niederhauser, MPH will collect and analyze the EMR data. Dr. Drainoni can be reached at 617-414-1417; Dr. Williams can be reached at 617-414-4465; Ms. Neiderhauser can be reached at 617-414-1384.

2) Focus groups

Carol VanDeusen Lukas, Ed.D, Principal Investigator and Mari-Lynn Drainoni, PhD, Co-Principal Investigator have expertise in qualitative data collection and analysis. They will conduct the focus groups and direct the analysis. Andrea Neiderhauser, MPH will participate in data collection, coding and analysis.

Dr. VanDeusen Lukas can be reached at 857-364-5685; Dr. Drainoni can be reached at 617-414-1417; Ms. Niederhauser can be reached at 617-414-1384.

3) Implementation logs

Qualitative analysis of the implementation logs will be conducted by Carol VanDeusen Lukas, Ed.D, Principal Investigator; Mari-Lynn Drainoni, PhD, Co-Principal Investigator, Charles Williams, MD, Co-Investigator and Andrea Niederhauser, MPH.

Dr. VanDeusen Lukas can be reached at 857-364-5685; Dr. Drainoni can be reached at 617-414-1417; Dr. Williams can be reached at 617-414-4465; Ms. Niederhauser can be reached at 617-414-1384.

6