Supporting Statement Part B -- Pretest of the Ambulatory Surgery Center SOPS

SUPPORTING STATEMENT

Part B

Pretest of the Ambulatory Surgery/Procedure Survey on Patient Safety Culture

Version: June 10, 2013

Agency for Healthcare Research and Quality (AHRQ)

Table of Contents

B. STATISTICAL METHODS 2

1. Potential Respondent Universe and Sample Selection Method 2

2. Information Collection Procedures 3

3. Methods to Maximize Response Rate 3

4. Tests of Procedures 4

5. Statistical Consultation and Independent Review 4

B. STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

Ambulatory Surgery/Procedure Survey on Patient Safety Culture

Cognitive Interviews. Cognitive interviews will be done on the draft Ambulatory Surgery SOPS before larger-scale pretesting is done. Fifteen cognitive interview participants will be selected from ambulatory surgical or procedure centers (ASCs) that will vary by type, size and geographic location. We will use participant recruitment methods such as sending research participation flyers to ASCs. Sites will be identified through Health Research & Educational Trust (HRET), recommendations by Technical Expert Panel members or their affiliated organizations, and local recruitment in the Rockville, MD region. We aim to conduct cognitive testing with 3 physicians, 6 nurses, two medical technicians, two administrative managers, and two administrative assistants.

Pretest Study Sample. The purpose of the larger scale pretest sample design are two-fold: 1) to obtain enough pretest data at the ambulatory surgery site level and individual respondent level to allow for sufficient sample size to examine the multilevel psychometric properties of the survey (item nonresponse, item response variability, factor structure, reliability and construct validity), and 2) to include a variety of ASCs that vary by specialty and size, as well as geographic region.

Since the goals for the data are to examine the psychometric properties of the survey, not to produce national estimates, purposive sampling will be used. Purposive sampling will ensure adequate variability on important ASC characteristics given the small number of sites in the pretest.

Anticipating that the survey will have about 50 items, for psychometric and factor analysis purposes we will need at least 13 respondents for every item included in the survey, or at least 529 total respondents. In addition, to conduct analyses at the site level, we need at least 40 sites per survey. The number of sites and number of individuals sampled were determined by these criteria.

Pretest Survey. We will need to recruit 40 ASCs to obtain enough data at the individual respondent level for analysis purposes (approximately 529 responses). Given the small number of staff within each ASC, a census of all clinical and non-clinical staff will be conducted within each facility. In addition, an ASC Information Form on facility background characteristics will be completed by an ASC manager at each site.

HRET will help recruit a range of ASC sites sizes by specialty (see Table 3). It is estimated that a total of approximately 756 individuals will have the opportunity to complete the pretest (see Table 4). With a response rate of 70%, the number of completed surveys will be 529. Because the purpose of the data collection is for psychometric analyses and not for parameter estimation, 529 responses across 40 sites will be sufficient.

Table 3. ASC by size and type: 40 pretest sites

ASC Size (# of Operating Rooms)	Type of ASC		TOTAL ASC SITES
	Freestanding	Affiliated with a hospital or larger health system
Small (1-2)	9	9	18
Medium (3-4)	6	6	12
Large (5+)	5	5	10
Total	20	20	40

Table 4. Individuals (managers, providers & staff) surveyed within 40 pretest sites

ASC Size (# of Operating Rooms)	Type of ASC		TOTAL INDIVIDUALS Administered Survey
	Freestanding	Affiliated with a hospital or larger health system
Small (1-2) assumes 12 staff	9 x 12 = 108	9 x 12 = 108	216
Medium (3-4) assumes 20 staff	6 x 20 = 120	6 x 20 = 120	240
Large (5+) assumes 30 staff	5 x 30 = 150	5 x 30 = 150	300
Total	378	378	756

* A 70% response rate = 529 completed surveys for analysis purposes

2. Information Collection Procedures

Survey. The pretest survey data collection for the Ambulatory Surgery SOPS will include these steps:

• Mailing the survey (which includes a cover letter) to the ASC office point of contact

• ASC point of contact distributes survey packets to staff

• A reminder notice distributed to nonrespondents

• Distribution of a second questionnaire to nonrespondents

3. Methods to Maximize Response Rate

Cognitive Interviews. Cognitive interview participants will receive cash remuneration ($200 for physicians and $100 cash for other staff). In addition, we are following up with a reminder notice and second questionnaire to nonrespondents in ASCs.

Survey. We are following up with a reminder notice and second questionnaire to nonrespondents in the ACSs.

4. Tests of Procedures

The procedures for this specific project have not been subjected to testing. However, both HRET and Westat have conducted many similar projects and are using well-established research methods with this project.

5. Statistical Consultation and Independent Review

Input from statistical analysts was used to develop the study design and plans for data analysis, including:

Joann Sorra, PhD, Senior Study Director, Westat, 301-294-3933

Scott Smith, PhD, Research Associate, Westat, 301-251-8288