Supporting Statement Part B -- Standardizing Antibiotic Use in LTC Settings 4-18-2011

SUPPORTING STATEMENT

Part B

Standardizing Antibiotic Use in Long-term Care Setting
(AIR’s Study)

April 12, 2011

Agency of Healthcare Research and Quality (AHRQ)

Table of contents

B. Collections of Information Employing Statistical Methods 3

1. Respondent universe and sampling methods 3

2. Information Collection Procedures 5

3. Methods to Maximize Response Rates 6

4. Tests of Procedures 6

5. Statistical Consultants 7

B. Collections of Information Employing Statistical Methods

1. Respondent universe and sampling methods

The field test includes four populations: nursing homes, residents, staff (administers and nurses), and attending clinicians. We will purposively select 12 nursing homes based on their characteristics and willingness to participate and then include all eligible residents (i.e., those listed in the infection log) at each nursing home. We will obtain necessary information from publically available information (e.g., nursing home compare) to identify eligible nursing homes. Four nursing homes will be assigned to the control group, four to treatment group 1, which will receive follow-up re-training, and four to treatment group 2, which will not receive follow-up re-training. We will train all nursing staff and two clinicians at each nursing home.

Semi-structured interviews will be conducted with purposively selected facility personnel from each of the eight intervention sites (two champions -- likely the administrator, director of nursing and/or the assistant director of nursing, one line nurse, and one attending clinician). The administrator and the line nurse will be those who have served as the main champions of the project at each nursing home. The attending clinician will be the nursing home’s medical director.

Nursing Home Level. The field test will use a purposively selected sample of 12 Texas nursing homes that reflect the diversity of nursing home ownership and size (i.e., number of beds) in the U.S. In Texas, 87 percent of homes are Medicare and Medicaid-certified, and have an average occupancy rate of 72 percent. About 54 percent of the nursing homes in Texas have a bed size between 100 and 199 (U.S. average is 43%). Eighty percent of nursing homes are for-profit and 16 percent are non-profit. All others are government owned¹. Nationally, 54 percent of nursing homes are part of a chain². The medical records and infection log entries for all residents who are listed in the infection log as having a respiratory, urinary, or soft tissue infection will be examined. Data on the universe of infections and residents who meet these eligibility criteria will be collected. Data on the characteristics of the nursing home (e.g. monthly occupancy, number of beds) will also be collected.

The 12 study homes will be assigned to one of three groups. The first group of four homes will be the control group, which will not receive the intervention. Only secondary data abstracted from records by project research staff will be collected at the facilities in the control group. The second group of four homes will receive the intervention (training and use of the COF). The third group of four homes will receive the intervention training, the COF, and a follow-up training two months after the initial training. The reason for conducting this second training is because information from the TEP and the TMF’s experience is that nursing homes have tremendous turnover. This second training would train new staff and retrain any staff who need reinforcement. Primary and secondary data from the eight nursing homes in the two intervention groups will be collected.

To the extent feasible, the 12 nursing homes will be assigned to groups of three similar facilities, based on ownership and number of beds. Within each of these relatively similar triplets, one member will be assigned to each of the three study groups. Within each group, the goal is to have two homes operated by multi-facility chains, one national, and the other intra-state; one independent for-profit home and one not-for-profit home. This distribution approximates the national pattern of nursing home ownership. The objective will be to balance the three study groups on ownership and size to the extent feasible.

Resident-Infection Level. Based on preliminary testing in four nursing homes, we expect to identify 17 infections per month per nursing home or a total of 1,836 infections over the 9 months (3 months prior the intervention and 6 months post-intervention) across all 12 facilities. Also from this testing, we expect that 23% of residents will have multiple infections. Thus, we project that 1,430 residents will account for these 1,836 infections. The data to assess the impact on antibiotic usage at the infection and resident levels will come from all infection log entries, the Loeb Criteria COF for each infection log entry in the post-intervention period at the 8 treatment group facilities, and data abstracted by researchers from the infection log entry, MDS, and medical record of the resident for whom the infection log entry was made. Of these four data sources, only the Loeb Criteria COF imposes data collection burden on nursing home staff. The infection log, MDS, and medical record are clinical and administrative databases maintained by the nursing homes for resident care and to meet regulatory requirements and would be completed in any case as part of good administrative and clinical practice and regulatory obligation by the nursing home staff members. Of the projected 1,836 infections in our final dataset, we project that 816 will come from the eight intervention nursing homes during the 6-month post-intervention period (17 per month x 6 post-intervention months = 102 x 8 intervention facilities = 816 completed Loeb Criteria COFs).

The primary goal of the analysis is to compare the rate of correct use of antibiotics in the intervention groups with the comparison group. Assuming there will be 102 infections (17 x 6) in each home during the 6-month post-treatment period, and that there are four homes in the control group and eight homes in the treatment group. In preliminary testing, we observed 28% compliance with the Loeb criteria in three nursing homes, which we use as the assumed baseline rate we will observe in the field test. Using the formula from Hayes and Bennett³, the power for detecting a difference between 28% correct use of antibiotics in the comparison group and 55% in the intervention group (our improvement target), is plotted in Exhibit 4. Exhibit 4 assumes a one-sided test with 0.05 significance level and provides power estimates for a range of the values of k, the coefficient of variation between homes. The smaller the value of k, the less correlated the data are within each home, hence, the more power we will have for detecting an effect of the intervention.

In order to have at least 80% power to detect a difference between 28% and 55%, we need to have a k that is less than or equal to 0.374. The current estimate of k from four resident homes during the pre-treatment period is 0.56. However, according to our study design, we will consider the rate difference between post and pre-treatment periods for each home, and we expect the coefficient of variation based on the rate difference to be smaller than the coefficient of variation estimated from the pre-treatment rates alone.

Administrator, Nurse, and Clinician Levels. We will interview two champions (administrator, director of nursing and/or assistant director of nursing), one line nurse, and one clinician involved in the use of the COF at each of the eight treatment group homes or a total of 32 individuals. Each person will be interviewed three times over the course of the field test (the pre-implementation, post training and post-implementation semi-structured interviews). These interviews will be open-ended qualitative interviews and analyzed as qualitative data, so statistical power is not an issue.

Exhibit 4. Power Curve for Loeb Compliance Improvement from 28% to 55%

2. Information Collection Procedures

In all 12 homes involved in the field test, data will be collected retrospectively for the three-month period prior to initiation of the intervention and prospectively for the six months following the start of the intervention. Data collection will occur at four points. Pre-intervention data for all three months will be collected at the first visit, which for the eight intervention homes will include training on the intervention. Thereafter, researchers will return every two months to each home to collect data on the intervention period. At the four homes in intervention group 2, the researchers will re-train nursing home staff in the use of the COF during their second visit.

Exhibit 5 illustrates the data collection procedures that will be used for the treatment and control groups in both the pre-intervention and post-intervention periods.

All interviews will be audiotaped and attended by a notetaker. The trained and experienced qualitative interviewer will use semistructured interview protocols. Interviews will be conducted at three different points: 1) Prior to the intervention: 2) after the training and after one month of implementation; and 3) after the end of the intervention.

Exhibit 5. Data Collection Procedures by Stage and Group

	Group
Period and Activity	Treatment (8 Facilities)	Control (4 Facilities)
Pre-Intervention Medical Data Abstraction (3 months)	Abstracted from infection log, MDS, and medical record by researchers	Abstracted from infection log, MDS, and medical record by researchers
Post-intervention Medical Data Abstraction (6 months)	COF completed by facility nurses. MDS and medical record abstracted by researchers	Abstracted from infection log, MDS, and medical record by researchers
Pre intervention interviews	Interviews with 2 champions, doctor, and line nurse conducted by trained qualitative researchers.	No activities
Post training interviews	Interviews with 2 champions, doctor, and line nurse conducted by trained qualitative researchers.	No activities
Post intervention interviews	Interviews with 2 champions, doctor, and line nurse conducted by trained qualitative researchers.	No Activities

3. Methods to Maximize Response Rates

This project involves collecting data from nursing homes about residents, but not directly from residents. Nursing staff at eight of the participating nursing homes will be asked to complete COFs for each infection log entry. Three staff members and one attending clinician will be interviewed at each of the eight treatment group homes. All other data will be abstracted by researchers from nursing home records. There are no methods to maximize response rates for the semi-structured interviews.

4. Tests of Procedures

Usability testing (UT). Four members of the research staff employed by TMF Health Quality Institute (the Texas Quality Improvement Organization) were trained in the initial data collection protocols. They tested these protocols by gathering information on eight residents (two residents in each of four homes). The researchers completed a draft Loeb Criteria COF for each patient by abstracting data from the facility’s infection log. They also abstracted data from the residents’ Minimum Data Set (MDS) and medical records.

Cognitive Testing. Based on the findings from the UT and reactions from the Technical Expert Panel, we cognitively tested the Loeb Criteria COF with two LPNs and, based on the results, revised the form.

Small Scale Trial (SST): The SST is being conducted at four nursing homes. By the conclusion of the SST, TMF clinical researchers (nurses and therapists) will have visited each home four times. At the first visit, they trained two nursing home staff members in the use of the COF, abstracted data from the residents’ records and abstracted data from the infection log for the three months prior to the training. At subsequent visits, the researcher collects data from the infection log, Loeb Criteria COF, MDS, and medical record. At each home, the researcher will interview one administrator and one nurse about the intervention (8 persons in total). These interviews will focus on problems in the use of the intervention materials, any questions about the fidelity of the implementation to the intervention model, and staff attitudes about the intervention and protocols.

5. Statistical Consultants

Our statistical consultants include:

Charles Phillips, Principal Investigator, Texas A&M University’s School of Rural Public Health, (979) 845-2387.

Steven A. Garfinkel, Project Director, American Institutes for Research, (919) 918-2319

Hongwei Zhao, Co-Investigator, Department of Epidemiology and Biostatistics, Texas A&M University’s School of Rural Public Health, (979) 845-2387.

1 Centers for Medicare & Medicaid Services (2010). Nursing Home Compendium 2009.

2 Centers for Disease Control and Prevention (2006).National Nursing Home Surveys: Characteristics, staffing, and management. Retrieved November 2, 2010 from http://www.cdc.gov/nchs/data/nnhsd/nursinghomefacilities2006.pdf#01

3 Hayes, R. J. & Bennett, S. (1999). Simple sample size calculation for cluster-randomized trials. International Journal of Epidemiology, 28 (2), 319-326.

- 7 -