memo

To: Karen Matsuoko

From: Doris Lefkowitz

Cc: James Battles

Re: AHRQ responses to OMB’s Concerns on the Nursing Home and Medical Office Survey Safety Pilots

We have thoughtfully considered your concerns on both of these activities and are providing the following additional information to support and clarify our supporting statement for clearance.

Plans for evaluation of the instrument

Upon clearance of these ICRs, AHRQ will establish a committee who will be responsible for reviewing the analysis plan, and the results of the psychometric analysis. AHRQ will submit the committee’s review, along with the final instruments and report for feedback from OMB, allowing 30 days to receive feedback before proceeding with disseminating the instrument.

The committee will be chaired by Trena Ezzati-Rice, Ph.D who leads the Division of Statistics and Research Methodology in CFACT at AHRQ and James Battles, Ph.D. who is the project officer for this project. I will serve as an ex-officio member to insure any terms of clearance have been met and to forward the appropriate materials to OMB in a timely manner. In addition to AHRQ staff, the committee will include representatives from CMS, ASPE., NCVHS, and DoD (or the VA).

AHRQ Plans for dissemination and ongoing evaluation of the survey product.

AHRQ will prepare a Survey Guide for each instrument. These guides/tool kits will provide a description of the instrument, its intended use, how the instrument was developed and a brief report of the requests of pilot testing. The guide/tool kit will contain the final instrument along with suggestions for administration and ways use an interpret data generated by the survey. AHRQ intends to use the same format and approach that was used for the Hospital Survey on Patient Safety Culture AHRQ Publication No. 04-0041. This guide/tool kit can be downloaded from the AHRQ website (http://www.ahrq.gov/qual/hospculture/#toolkit).

AHRQ will create national bench marking databases for both of these new patient safety culture surveys molded after the Hospital Survey on Patient Safety Culture Comparative Database. Nursing homes and medical offices will be strongly encouraged to submit their data to the comparative database on an annual basis. AHRQ will publish a comparative database report each year on the shared results. The Hospital Survey on Patient Safety Culture 2007 Comparative Database Report serves an example of the type of report that is anticipated. This report can be found on AHRQ’s web site at http://www.ahrq.gov/qual/hospsurveydb/.

AHRQ will provide technical support to users of the Nursing Home Survey on Resident Safety and Medical Office Survey on Patient Safety as well as on going support of the Hospital Survey on Patient Safety Culture through the User Network Project for CAHPS and Patient Safety contract. AHRQ is currently seeking a qualified contractor to operate and manage this support contract with a Request for Proposal No. AHRQ-07-10024. The User Network Project for CAHPS and Patient Safety will be fully operational by the beginning of fiscal year 08, October 2007. The User Network Project for CAHPS and Patient Safety will be responsible for maintaining the comparative databases, technical support and providing further psychometric analysis of the Nursing Home Survey on Resident Safety and Medical Office Survey on Patient Safety based on data submitted to the comparative benchmarking databases. AHRQ will hold an annual users group meeting on patient safety culture assessment. AHRQ intends conduct continued research studies to explore the relationship of other quality measures such as the relation between Surveys on Patient Safety (SOPS) and patient’s perceptions of care as represented by CAHPS instruments, outcomes of care from Patient Safety Indicators (PSIs) from administrative data such as contained in HCUP through the AHRQ’s new User Network Project for CAHPS and Patient Safety contract.

AHRQ will continue to closely monitor the performance of the Nursing Home Survey on Resident Safety and Medical Office Survey on Patient Safety based on data submitted by users for a minimum of 24 months following pilot testing for psychometric consistency and reliability and validity of the instruments. Based on this continued review and analysis changes to the original instrument may be necessary.

Plan for remuneration and an “operations test”

One of the issues raised in our discussions has been how respondent remuneration is being used in this project, and especially in the case of the nursing home study, how offering individual remuneration might affect participation. AHRQ believes that offering remuneration is important for achieving the goals of the pretest in a timely way. We also recognize that some facilities may not offer remuneration for participation. While it is not possible to simulate the exact conditions of a facility conducting this type of survey with its employees, we propose a small test removing the individual incentive as an “operations test.”

Nursing Home Survey – addition of a no-remuneration “operations test”

Based on information received from consultations with researchers who have recently implemented surveys to nursing home staff, to ensure high response rates for our paper survey it appears necessary to provide individual nursing home staff members with a cash incentive to complete the survey.

We therefore plan to provide a $5.00 cash prepaid incentive to individuals with the first survey. However, we will conduct a small test of the effect of individual incentives on response rates by attempting to recruit 7 out of the total of 40 nursing homes that would be willing to conduct the survey with no individual incentive. Therefore, the plan is to recruit 33 sites where individuals will receive $5 prepaid and to recruit 7 sites where no individual incentives will be offered.

Because data collection will require the assistance of a nursing home point of contact (POC) to manage data collection at their nursing home (compile sample information, distribute surveys, promote survey response, conduct follow up, etc.), we plan to provide the POC with a one-time individual remuneration of $150 to offset the costs associated with these activities.

Medical Office Survey

Individual Incentive

The Medical Office Survey does not propose an individual level incentive.

Site-level Remuneration—Remains as originally proposed

Each of the 97 participating medical offices will receive $300 remuneration (payable to the medical office) upon conclusion of data collection to offset the costs associated with participation, which include: answering background questions about the characteristics of the medical office; putting together lists of office staff and staff positions; distribution of surveys and time to complete surveys; and conducting follow-up for non-response.

We hope that this information is helpful to your continuing review of this project and that we can move quickly towards clearing these requests for information collection.