Va 2

The purpose of this document is to address concerns raised by the OMB desk officer and Mr. Brian Harris about the VHA HCAHPS Pilot submission in a conference call on 10/18/07.

OMB specifically requested:

1. Provide a better description of the objectives of the study; e.g., what do we want to compare from the various arms? Response: A complete description of each of the study arms and what we hope to learn has been added. In addition, the information has been presented in tabular form (see below). We have simplified the study to include only 3 arms, dropping the one-column-two column formatting test (the formatting question has already been answered by an operational change from one column to two column format implemented temporarily in 2004). Scores went down up to 8 percentage points as a result of that change on the outpatient survey, and rebounded to pre-change level upon returning to one-column format.

Study Arm:	HCAHPS Short Form survey (existing SHEP sampling)	HCAHPS Short Form survey (HCAHPS sampling)	HCAHPS Long Form Survey (HCAHPS sampling)
Sample size:	2,700 outgo; 1,510 responses	2,700 outgo; 1,510 responses	2,700 outgo; 1,510 responses
Comparison 1:	Compare response rates, scores and demographics of the 2 sampling methods	Compare response rates, scores and demographics of the 2 sampling methods
Comparison 2:		Assess length effect	Assess length effect
Comparison 3:	Calibrate HCAHPS measure of healthcare satisfaction with SHEP overall quality (uses contemporaneous SHEP surveys as comparison)	Calibrate HCAHPS measure of healthcare satisfaction with SHEP overall quality (uses contemporaneous SHEP surveys as comparison)

2. What strategies do we have in mind to improve response rates? While we feel that the existing 50% response rate achieved by inpatient SHEP is in line with the expectations for HCAHPS, we realize that a second wave of mailing is part of the HCAHPS protocol. Therefore, we have added back the second wave of mailing for the Pilot, which we believe will result in a 56% response rate.

3. What was the difference in scores after we dropped the second wave of mailings in 2004? Response: Results of an 8-way pilot study conducted in 2004 (attached) indicate that scores went up 0.7 percentage points (from 74.1% to 74.8%) upon discontinuance of the second wave of mailing. However, response rates did suffer by 8.5 percentage points (14.7%). Scores for second mailing respondents were about 1 percentage point lower, compared with first mailing respondents.

4. More detailed sample size determination. Response: Sample size determination has been provided in detail

5. Provide details of an analysis plan. Response: a detailed analysis plan has been provided.

6. Describe sampling methods in more detail; i.e., define “SHEP sampling” and “HCAHPS sampling”. Response: These have been described in the justification, and are restated below (see below for description).

7. Provide copies of pre-notification letter, cover letter, and thank you /reminder postcard. Response: These are now included.

8. Standardize the race/ethnicity questions to be compliant with OMB requirements (American Indian or Alaska Native, Asian, Black or African American, Native Hawaiian or Other Pacific Islander, and White) Response: This has been done.

White

Black or African-American

Asian

Native Hawaiian or Other Pacific Islander

American Indian or Alaska Native

Existing SHEP sampling refers to sampling inpatients who were admitted to and discharged to home from the same bed section; medicine, surgery, psychiatry, neurology, rehab and SCI.

HCAHPS sampling: The survey will be administered to a random sample of live discharges that were 18 or older at admission, had an inpatient overnight stay, and had a non-psychiatric diagnosis as determined by Diagnosis Related Groups (DRG’s).

72 sampled per cell per each of 21 VISNs.