Supporting Statement Part B -- HSOPS DB Renewal_1-11-2019

SUPPORTING STATEMENT

Part B

Collection of Information for

Agency for Healthcare Research and Quality’s (AHRQ)

Hospital Survey on Patient Safety Culture Database

January 11, 2019

Agency of Healthcare Research and Quality (AHRQ)

Table of contents

B. Collections of Information Employing Statistical Methods 3

1. Respondent universe and sampling methods 3

2. Information Collection Procedures 5

3. Methods to Maximize Response Rates 7

4. Tests of Procedures 7

5. Statistical Consultants 8

B. Collections of Information Employing Statistical Methods

1. Respondent universe and sampling methods

The AHRQ Surveys on Patient Safety Culture™ (SOPS™) Hospital Database serves as a central U.S. repository for data from the survey. However, the database is comprised of data that are voluntarily submitted by hospitals that have administered the survey, and is not a statistically selected sample, nor is it a representative sample of all U.S. hospitals. Estimates based on this self-selected group may produce biased estimates of the population and it is not possible to compute estimates of precision from such a self-selected group.

AHRQ developed and pilot tested the Hospital SOPS (OMB NO. 0935-0115; Approved 2/4/2003). The original Hospital SOPS Database was last approved on September 30, 2016 (OMB NO. 0935-0162; expiration date 10/31/2019).

The first Hospital SOPS Database report was published in 2007. Results from 630 hospitals that participated in the 2018 database are available on the AHRQ web site.¹

According to the U.S. Census Bureau, the number of hospitals in the U.S. is estimated to be 5,795². However, the latest AHRQ Hospital SOPS Database Report consists of data from 630 hospitals which represents only 10.9% of the total estimated population of U.S. hospitals.

Table B-1 presents the 630 hospitals in the 2018 Database categorized by whether they had only submitted once or more than once.

Table B-1. Statistics for Nontrending and Trending Hospitals in 2018 Database

Overall Statistic	Nontrending (Submitted Once)	Trending (Submitted More Than Once)	Total 2018 Database
Number of hospitals	324	306	630
Number of individual survey respondents	154,404	228,430	382,834

Although the hospitals that voluntarily submitted data to the database do not constitute a statistically selected sample, the characteristics of these hospitals are fairly consistent with the distribution of hospitals registered with the American Hospital Association (AHA). The characteristics of database hospitals by bed size, teaching status, ownership, geographic region, and health care system status are presented in the following tables and are compared with the distribution of AHA-registered hospitals included in the 2015 AHA Annual Survey of Hospitals³.

Table B-2. Distribution of Database Hospitals and Respondents by Bed Size

(Compared to AHA-registered U.S. Hospitals)

Bed Size	AHA-registered U.S. Hospitals		2018 Database Hospitals		2018 Database Respondents
	Number	Percent	Number	Percent	Number	Percent
6-24 beds	770	12%	34	5%	2,558	1%
25-49 beds	1,445	23%	96	15%	13,360	3%
50-99 beds	1,237	20%	102	16%	23,967	86%
100-199 beds	1,263	20%	156	25%	79,082	21%
200-299 beds	646	10%	102	16%	75,910	20%
300-399 beds	468	6%	57	9%	50,639	13%
400-499 beds	201	3%	34	5%	41,249	11%
500 or more beds	3121	5%	49	8%	96,069	25%
TOTAL	6,251	100%	630	100%	382,834	100%

Table B-3. Distribution of Database Hospitals and Respondents by Teaching Status

(Compared to AHA-registered U.S. Hospitals)

Teaching Status	AHA-registered U.S. Hospitals		2018 Database Hospitals		2018 Database Respondents
	Number	Percent	Number	Percent	Number	Percent
Teaching	2,087	33%	226	36%	216,301	56%
Non-Teaching	4,164	67%	404	64%	166,533	44%
TOTAL	6,251	100%	630	100%	382,834	100%

Table B-4. Distribution of Database Hospitals and Respondents by Ownership and Control

(Compared to AHA-registered U.S. Hospitals)

Ownership and Control	AHA-registered U.S. Hospitals		2018 Database Hospitals		2018 Database Respondents
	Number	Percent	Number	Percent	Number	Percent
Government (Federal or non-Federal)	1,476	24%	77	12%	31,996	8%
Nongovernment not for profit	3,099	50%	489	78%	318,376	83%
Investor owned (for profit)	1,676	27%	64	10%	32,462	8%
TOTAL	6,251	100%	630	100%	382,834	100%

Table B-5. Distribution of Database Hospitals and Respondents by Geographic Region

(Compared to AHA-registered U.S. Hospitals)

Region	AHA-registered U.S. Hospitals			2018 Database Hospitals		2018 Database Respondents
	Number	Percent		Number	Percent	Number	Percent
New England	251	4%		33	5%	17,378	5%
Mid Atlantic	552	9%		61	10%	53,492	14%
South Atlantic/Associated Territories	1,009	16%		151	24%	107,584	28%
East North Central	912	15%		144	23%	86,680	23%
East South Central	501	8%		27	4%	15,304	4%
West North Central	789	13%		73	12%	30,230	8%
West South Central	1,066	17%		85	13%	33,861	9%
Mountain	521	8%		20	3%	13,436	4%
Pacific/Associated Territories	650	10%		36	6%	24,869	6%
TOTAL	6,251	100%		630	100%	382,834	100%
NOTE: States and territories are categorized into AHA-defined regions as follows:
New England: CT, MA ME, NH, RI, VT			West North Central: IA, KS, MN, MO, ND, NE, SD
Mid Atlantic: NJ, NY, PA			West South Central: AR, LA, OK, TX
South Atlantic/Associated territories: DC, DE, FL, GA, MD, NC, WV, VA, Puerto Rico, Virgin Islands			Mountain/Pacific/Associated Territories: AZ, AK, CA, CO, HI, ID, MT, NM, NV, OR, UT, WA, WY, American Samoa, Guam, Marshall Islands, Northern Mariana Islands
East North Central: IN, IL, MI, MI, WI
East South Central: AL, KY, MS, TN

Hospitals that submit data to the database receive a free, customized feedback report that displays their hospital’s results against the aggregated and de-identified results of other hospitals submitting data to the database. Hospitals that do not submit data to the database can still view the Hospital SOPS Database Report for aggregate results of those hospitals who voluntarily submitted their data to the Database. As part of a toolkit of support materials for the Hospital SOPS survey, hospitals can also use a Microsoft^® Excel-based Data Entry and Analysis Tool that is an Excel file with macros that will automatically produce graphs and charts of a hospital’s results once data are entered in a data sheet. Many hospitals use this tool to produce their results.

In the overall database report, hospitals are provided with a detailed description and explanation of the statistics that are presented and given examples and guidance on how to calculate their own scores for comparison.

2. Information Collection Procedures

Information collection for the AHRQ Hospital SOPS Database occurs in a periodic data collection cycle approximately every two years in June. The next submission period is June 2020. Information collection procedures for submitting and processing data are shown in Figure 1.

Figure 1. Hospital SOPS Database Data Submission

Step 1: Call for Data Submission. Announcements about the opening of data submission go out through various publicity sources. AHRQ’s patient safety and electronic newsletters target approximately 50,000 subscribers. In addition, the AHRQ Surveys on Patient Safety Culture listserv targets approximately 38,000 subscribers. An example of email announcements calling for data submission is shown in Attachment D, Email # 1 and # 3. Through these efforts, U.S. hospitals are made aware of and invited to submit their survey data to the database.

As the administrator of the database and under contract with AHRQ, Westat provides free technical assistance to submitting hospitals through a dedicated email address (DatabasesOnSafetyCulture@westat.com) and toll-free phone number (1-888-324-9790).

Step 2: Registration for Potential Participants. A secure data submission web site allows interested parties such as hospitals and health systems to register and submit data. Registration takes about 3 minutes to complete and asks for contact information and other basic information (see Attachment A). After registering, if registrants are deemed eligible to submit data, an automated email is sent to authenticate the account and update the user (see Attachment D, Email # 2).

Once users are registered and have a password, they can enter the main page menu of the web site. Information about eligibility requirements, data use agreements, and data file specifications regarding how to prepare their data for inclusion in the SOPS database is posted and can be reviewed.

Step 3: Enter Facility Information and Upload Questionnaire. At this step, users provide information about each of their hospitals, such as Medicare Provider ID, AHA ID, point of contact, method of survey administration, overall response rate, and--if they do not have an AHA ID--other facility characteristics (e.g., bed size, teaching status, and ownership) (see Attachment C). They also upload their survey questionnaire that they administered to enable us to determine whether any changes were made to the survey (see Attachment G, Figure 1).

Step 4: Submit Data Use Agreement (DUA). To protect the privacy of all participating hospitals, a duly authorized representative from the hospital must sign a Data Use Agreement (DUA) (see Attachment B). The DUA language was reviewed and approved by AHRQ’s general counsel and asserts that the hospital’s data will be handled in a secure manner using necessary administrative, technical and physical safeguards to limit access to it and maintain its confidentiality. In addition, the DUA explains that the data are used for the purposes of the database, that only aggregated results are reported, and that the hospital will not be identified by name. Data are not included in the database without this signed DUA. Users can fax, mail, or upload a copy of the signed agreement.

Step 5: Upload Data File(s). At this step, users are asked to upload their individual-level survey data for each hospital (see Attachment G, Figure 2). Data submitted through the secure data submission web site are encrypted to ensure secure transmission of the survey data. Data are accepted in Microsoft Excel^® format since this is the format preferred by hospitals. Users must upload one data file per hospital. If a user has multiple hospitals within a health system, users can upload one data file that identifies all of the hospitals in their system. The data file specifications (see Attachment E) are provided to data submitters to ensure that users submit standardized and consistent data in the way variables are named, coded, and formatted.

Once a data file is uploaded, a separate load program developed in Visual Basic (VB) reads the submitted files and loads them into the SQL database that stores the data. A data quality report is then produced and made available to the participant. This report displays item frequencies and flags out-of-range values and incorrectly reverse-coded items. If there are no problems with the data, an acknowledgement of data upload and acceptance will be granted during the user session. If data are improperly coded, the user is informed by having a message post on the screen indicating that the data file failed. Users are expected to fix any errors and resubmit their data file(s) for processing. Once there are no problems, the user is informed of the acceptance of data during the user session with an online message of acceptance.

Step 6: Approve Data Submission. Once all of the information required for submission has been submitted and approved, an email is sent to the hospital contact indicating that their data has received final acceptance.

3. Methods to Maximize Response Rates

AHRQ makes a number of toolkit materials available to assist hospitals with the SOPS surveys. The Hospital SOPS has a Survey User’s Guide that gives users guidance and tips about survey administration on the following topics: planning; selecting a sample; determining their data collection method; data collection procedures (including a section on web surveys); and analyzing data and producing reports⁴. The Survey User’s Guide also gives hospitals tips about how to increase response rates through publicity efforts, top management support, use of incentives, and following all steps of proper data collection protocols. Of the hospitals that voluntarily submitted their data for the 2018 Hospital SOPS Database, the average response rate was 54% across 630 hospitals.

The SOPS User Network promotes the database in a number of ways:

1. AHRQ and AHRQ SOPS email listservs;

2. Organizational partners and stakeholders that have national reach to hospitals;

3. Users that have contacted the SOPS technical assistance helpline about the hospital survey;

4. Other outlets such as webcasts and conferences.
   
   

As noted earlier in this document under Information Collection Procedures, Step 1 – Call for Data Submission, announcements about the opening of data submission go out through various publicity sources as a way to boost hospital participation in the database. AHRQ’s electronic newsletter targets approximately 50,000 subscribers. In addition, the AHRQ Surveys on Patient Safety Culture listserv targets approximately 38,000 subscribers. AHRQ, through its contractor Westat, provides free technical assistance to users through a dedicated email box and toll-free phone number. In addition, reminders are sent to database registrants to remind them of the deadline for data submission.

The 2018 Hospital SOPS Database had 630 hospitals which was considered a very successful submission cycle.

4. Tests of Procedures

Input and Feedback for the Development of the SOPS Database Submission System. Because the Surveys on Patient Safety Culture are public-use instruments, the SOPS program has generally modeled its data submission processes after those used by the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Database that has been in operation for many years. SOPS staff consulted with CAHPS Database staff and programmers to determine best practices for data submission. This information, as well as feedback obtained during the provision of technical assistance each year the database has been running, has been used to improve the SOPS online data submission system and process over time.

5. Statistical Consultants

Joann Sorra, PhD

Westat

1600 Research Blvd.

Rockville, MD 20878

joannsorra@westat.com

Naomi Yount, PhD

Westat

1600 Research Blvd.

Rockville, MD 20878

naomiyount@westat.com

1 SOPS Hospital Database. Content last reviewed May 2018. Agency for Healthcare Research and Quality, Rockville, MD. https://www.ahrq.gov/sops/quality-patient-safety/patientsafetyculture/hospital/hosp-reports.html Last accessed 12/3/2018.

2 U.S. Census Bureau, 2012 Statistical Abstract of the United States, 2012, Table 172. http://www.census.gov/library/publications/2011/compendia/statab/131ed/health-nutrition.html. Last accessed 12/10/2018.

3Data for U.S. and U.S. territory AHA-registered hospitals were obtained from the 2015 AHA Annual Survey of Hospitals Database, © 2015 Health Forum, LLC, an affiliate of the American Hospital Association. Hospitals not registered with AHA were asked to provide information on their hospital’s characteristics, such as bed size, teaching status, and ownership.

4 Hospital Survey on Patient Safety Culture. Content last reviewed August 2018. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/professionals/quality-patient-safety/patientsafetyculture/hospital/index.html Last accessed 12/10/2018.