Survey of Hospital Quality Leaders OMB Statement B revised 12062016

The survey will sample from the universe of hospitals that publicly report HCAHPS patient experience data and have responded to the most recent iteration of the American Hospital Association (AHA) Annual Survey, which reports information on hospital characteristics such as total number of hospital beds. The sample frame will consist of a universe of more than 4,000 hospitals. We will pull a sample of 1,503 hospitals from this universe with the expectation of achieving 501 responses (assumes a conservative response rate of 33% based on prior work with hospital quality leaders; Blendon et al., 2004; Weissman et al., 2005). The sample design will stratify the sample by three factors: overall hospital quality performance, bed size, and census division, to allow for comparison of survey responses across these characteristics.

There are 4,625 hospitals in a dataset that contains 2013 HCAHPS data merged with 2011 AHA data. We will select a sample of 1,503 hospitals with the expectation of achieving approximately 501 responses. Of the hospitals contained in the merged data set, 369 have 300 or more completed surveys and were in the top decile of performance, scoring 9 or 10 out of 10 Hospital Compare reported HCAHPS measures. We will sample 100% of these high-performing hospitals. The remaining 1,134 hospitals will be sampled in equal numbers from 18 strata defined by cross tabulations of size (under 200 beds vs. 200 or more beds) with the nine census divisions (New England, Middle Atlantic, South Atlantic, East North Central, East South Central, West North Central, West South Central, Mountain and Pacific), such that 63 hospitals from each cell will be sampled, as depicted in Table 1. From the 1,503 total of sampled hospitals (369 high-performing and 1,134 non-high-performing distributed across size and census division), we anticipate receiving approximately 501 responses across all groups combined.

Table 1: Sample Allocation for 1,134 Non High-Performing Hospitals by Strata for Standardized Survey

Small (under 200 beds)

Large (200+ beds)

New England

63 sampled

(21 completes)

63 sampled

(21 completes)

Middle Atlantic

63 sampled

(21 completes)

63 sampled

(21 completes)

South Atlantic

63 sampled

(21 completes)

63 sampled

(21 completes)

East North Central

63 sampled

(21 completes)

63 sampled

(21 completes)

East South Central

63 sampled

(21 completes)

63 sampled

(21 completes)

West North Central

63 sampled

(21 completes)

63 sampled

(21 completes)

West South Central

63 sampled

(21 completes)

63 sampled

(21 completes)

Mountain

63 sampled

(21 completes)

63 sampled

(21 completes)

Pacific

63 sampled

(21 completes)

63 sampled

(21 completes)

Total of non-high-performing hospitals

567 sampled

(189 completes)

567 sampled

(189 completes)

Primary analyses will consist of descriptive statistics and comparisons between large vs. small hospitals, high performing vs. other hospitals, and comparisons of each census division to all other divisions combined. We will have 80% power (with 2-tailed tests and alpha=.05) to detect small to medium effect sizes. Specifically, for comparisons between large and small hospitals, we will have 80% power to detect changes of 8-13 percentage points for dichotomous variables (assuming that prevalences range from 10-90%) and a 0.25 standard deviation (Cohen’s d; small effect size) for continuous variables; for comparisons between high performing and other hospitals, we will have 80% power to detect changes of 9-15 percentage points for dichotomous variables and a 0.29 standard deviation (Cohen’s d; small-to-medium effect size) for continuous variables; and for comparisons between a single census division and all other divisions combined, we will have 80% power to detect changes of 12-20 percentage points for dichotomous variables and 0.40 standard deviation (Cohen’s d; small-to-medium effect size) for continuous variables. For descriptive statistics, 95% margins of error will be +/- 2.6-4.4% for dichotomous items and +/- 0.09 standard deviations for continuous measures.

B2. Data Collection Procedures

Data collection staff will contact each sampled hospital to confirm the name, job title, mailing address, email address, and telephone extension of the hospital quality leader using a standardized protocol. This will allow us to personalize survey invitations. All hospital leaders will receive an emailed invitation to participate in the survey via the web. After an initial email invitation is extended, a second email invitation will be extended after two weeks for those who have not responded. These invitations will contain sufficient information for informed consent as well as a hospital-specific PIN code that allows access to the web survey for that hospital. If no email address is available, then hard-copy invitations will be sent. Four weeks after the initial invitation to the web survey, we will begin phone follow-ups for non-responding hospital leaders. At six weeks after initial invitation, non-responding hospital leaders will receive an additional invitation (via email or letter). Eight weeks after initial invitation, non-responders will receive a paper version of the survey. At nine weeks after initial invitation we will conduct a second telephone prompt for survey completion. We anticipate closing the fielding period after 12 weeks of data collection..

Throughout data collection, we will track response and cooperation within each sample stratum and employ additional efforts or sample to achieve sufficient response in each stratum.

We anticipate the procedures outlined above will yield 501 completed surveys (33% response rate).

B3. Response Rates and Non-Response

Published surveys of hospital leaders conducted in the past 10 years report response rates as low as 20% and as high as 63% (Blendon et al., 2004; Weissman et al., 2005). In addition, surveys of organizations and/or individuals in leadership roles have experienced an overall decline in response rates similar to surveys of general populations (Cycota and Harrison, 2006; Baruch and Holtam, 2008). Similarly, surveys conducted in healthcare settings have seen a decline in response rates, increasing concern about non-response bias (Cook et al, 2009). Thus, we conservatively estimate a response rate of 33% and about 500 completed surveys. As described in Section B2 above, we plan to maximize response rates for the standardized survey through:

Careful identification of the appropriate respondent,
Use of personalization,
Multiple contact attempts,
Multiple modes of survey administration, and
Alternative modes for non-response contacts

We believe participants will be motivated to respond to the survey because it provides an opportunity to:

Share information about the projects their organization has prioritized and implemented
Contribute to an effort that will provide a national context for comparing their own hospital’s efforts to understand and improve patients’ experiences with other hospitals efforts in this area
Provide feedback about the ways in which HCAHPS is meeting (or not meeting) institutional needs

Using the techniques identified above will help to reduce non-response bias by encouraging participation among this group of busy professionals. It is important to minimize non-response bias to ensure that results are generalizable, interpretable and useful to hospital quality leaders and to AHRQ.

As described in section B2, we will track response rates across sampling strata and draw additional sample if needed to ensure sufficient response in each of the groups. These efforts will work in tandem with methods listed above to ensure that survey results can be generalized.

B4. Tests of Procedures or Methods

The survey was tested with fewer than nine entities. As part of the testing, a range of hospital types (size, quality performance, and region) were selected to capture variation in the expected range of responses. Testing informed the content of the survey, respondent identification procedures, and provided a basis for estimating administration burden. Relevant testing of another survey of the same population of respondents informed decisions about strategies to ensure sufficient survey response rates.

B5. Statistical and Data Collection Consultants

The survey, sampling approach, and data collection procedures were designed by the RAND Corporation under the leadership of:

Marc Elliot, PhD

RAND Corporation

1776 Main Street

Santa Monica, CA 90407

Kerry A. Reynolds, PhD

RAND Corporation

4570 Fifth Avenue

Suite 600

Pittsburgh, PA 15213

Rebecca Anhang Price, PhD

RAND Corporation

1200 South Hayes Street

Arlington, VA 22202

References

Blendon, R.J., Schoen, C., DesRoches, C.M., Osborn, R., Zapert, K., & Raleigh, E. (2004). Confronting Competing Demands to Improve Quality: A Five-Country Hospital Survey. Health Affairs, 23(3), 119-135.

Cook, J. V., Dickinson, H. O., & Eccles, M. P. (2009). Response rates in postal surveys of healthcare professionals between 1996 and 2005: an observational study. BMC health services research, 9(1), 1.

Cycota, C.S. & Harrison, D.A. (2006). What (Not) to Expect When Surveying Executives: A Meta-Analysis of Top Manager Response Rates and Techniques Over Time. Organizational Research Methods, 9, 133-160.

Baruch, Y. & Holton, B.C. (2008). Survey Response Rate Levels and Trends in Organizational Research. Human Relations, 61, 1139-1160.

National Compensation Survey: All United States December 2009 – January 2011. U.S. Department of Labor, Bureau of Labor Statistics.

Weissman, J. S., Annas, C. L., Epstein, A. M., Schneider, E. C., Clarridge, B., Kirle, L., Gatsonis, C., Feibelmann, S. & Ridley, N. (2005). Error reporting and disclosure systems: views from hospital leaders. JAMA, 293(11), 1359-1366.

File Type	application/msword
File Title	DRAFT Supporting Statement B for Survey of Hospital Quality Leaders
Subject	Supporting Statement B for Hospice Experience of Care Survey
Author	The RAND Corporation
Last Modified By	Windows User
File Modified	2016-12-06
File Created	2016-12-06