Attachment B -- Examples

ATTACHMENT B

Current and Past

AHRQ Questionnaire and Data Collection Testing Surveys

OMB No. 0935-0124 - Project Officer: Mamatha Pancholi /CDOM

"Validation Pilot for the AHRQ Patient Safety Indicators"

The AHRQ Patient Safety Indicators (AHRQ PSI) were made publicly available in March, 2003 as an evidence-based tool for identifying potentially preventable adverse events based on readily available hospital discharge data. Continued refinement of the AHRQ PSI requires validation of the cases “flagged” in the administrative data through the collection of data from the medical record. The purpose of the validation pilot is to test medical record abstraction instruments, sampling methods, and data collection applications, in addition to testing a mechanism for conducting the validation through a voluntary collaborative of participating hospitals. The intent is to determine the feasibility of incorporating a validation component into the ongoing development process of the AHRQ PSI.

OMB No. 0935-0124 - Project Officer: Charles Darby /CQUIPs

"Voluntary Questionnaire and Data Collection Testing to Pretest Proposed Prescription Drug Plan Module Questions"

The prescription drug module that is being tested in this field test will become a module to add to a shortened version of the current Medicare Advantage and Medicare fee-for-service CAHPS surveys. When the prescription drug plan module is implemented the sampling units will be arrayed to permit aggregation of the data and presentation of CAHPS measures at levels representing MA and MA-PD plans, as well as stand-alone PDPs, and the original MFFS health plan.

OMB No. 0935-0124 - Project Officer: Charles Darby/CQUIPs

"Voluntary Questionnaire and Data Collection Testing to Pretest Proposed NCQA Questions and Methodology"

Testing conducted under this request will be done for the following purposes: (1) Analysis of revised item wording ----Assess candidate new wordings for selected items, (2) Mode Analysis-Evaluate the equivalence of items administered by mail and by telephone, (3) Case mix adjustment analysis-Evaluate variables that need to be considered to case mix adjust scores, (4) Psychometric Analysis-provide information for the revision and shortening of questionnaires based on the assessment of the reliability and validity of questionnaire items and composites.

OMB No. 0935-0124 - Project Officer: Patrik Johansson/CP3

"Voluntary Questionnaire and Data Collection Testing to Pretest Proposed American Indian CAHPS Questions and Methodology"

The information collected through preliminary research activities will be used to revise and test items, described in item 1 preparation for fielding of the American Indian CAHPs Survey by RAND Corporation and CNHS in calendar year 2005. The end result will be improvement in information collection instruments and the qualities of data collected, a reduction or minimize of respondent burden, increased agency efficiency, and improved responsiveness to the public.

OMB No. 0935-0124 – Project Officer: Doris Lefkowitz/CFACT

"Voluntary Questionnaire and Data Testing Pretest of New CAPI Software for the Household Component of the Medical Expenditure Panel Survey”

The testing conducted under this request will be done for the following purposes: a) Usability of the instrument by field interviewers. Because the new instrument is in large measure identical in content to the prior version, we anticipate the change should be transparent for the respondents. We do need to insure that the instrument is acceptable in the field for the interviewers however before launching the national effort next year. While the instrument has been reviewed extensively, AHRQ feels it is important to test this new version of the instrument. The field test will offer the opportunity to insure the proper functioning of the field management system with the new interview application, between round processing of interview data and the preparation of completed interview data for subsequent processing.

OMB No. 0935-0124 – Project Officer: Charles Darby/CQUIPS

"Voluntary Questionnaire and Data Collection Testing to Pilot Test Proposed Nursing Home Survey on Resident Safety”

The information collected in fall 2006 will be used to test draft survey items from the Nursing Home Survey on Resident Safety described in item #1. The end result will be improvement in the final survey instrument which will be made available to the public for use in nursing homes to assess resident safety culture. Researchers are also likely to use the final nursing home survey to assess the impact of resident safety culture improvement initiatives. This nursing home survey is an expansion of AHRQ's Hospital Survey on Patient Safety Culture (HSOPSC) which was released to the public in November 2004.The nursing home survey contains new and revised questions and composites that more accurately apply to the nursing home setting. Psychometric analysis will be conducted on the pilot data to assess missing data, item distributions, and the reliability and construct validity of the items included in the survey. The domain structure of the survey will be evaluated through factor analysis. During data analysis, problematic survey items will be eliminated resulting in a final version of the survey to be released for use in the public domain.

OMB No. 0935-0124 – Project Officer: Charles Darby/CQUIPS

“Voluntary Questionnaire and Data Collection Testing to Pretest Proposed Home Health Care CAHPS® Questions and Methodology”

This study is a one-time pilot field test involving home health care patients who receive skilled services from Medicare and/or Medicaid certified home health agencies. The field test to be conducted under this request will be done for the following purposes: a) To assess survey methodology – to determine how well a mail questionnaire with mail and telephone follow up performs with home health care patients of this type; b) Psychometric Analysis – to provide information for the revision and shortening of the questionnaire based on the assessment of the reliability and construct validity of questionnaire items and composites; c) Case mix adjustment analysis – Assess the variables that may be considered as case mix adjusters.

OMB No. 0935-0124 – Project Officer: Anna Caponiti/CQUIPS

“CAHPS Pretest of Proposed Health Literacy Questions and Methodology”

This study is a one-time pilot field test to be completed in the calendar years 2007 and 2008. The field test to be conducted under this request will be done for the following purposes:

1. Analysis of revised item wording – Assess candidate new wordings for selected items

2. Mode Analysis— Evaluate the equivalence of items administered by mail and by telephone and compare the characteristics of respondents who complete the survey by mail to respondents who complete the survey via telephone.

3. Case mix adjustment analysis – Evaluate variables that need to be considered for case mix adjustment of scores.

4. Psychometric Analysis—Provide information for the revision and shortening of questionnaires based on the assessment of the reliability and validity of survey items and composites.

The end result will be collection of the data related to the assessment of patients’ perspective on how well health information is communicated to them by health care professionals. The field testing will ensure that the future data collection yield high quality data and ensure a minimization of respondent burden, increase agency efficiency, and improve responsiveness to the public. The survey items will be added to currently available CAHPS ® surveys and will provide a venue to clinicians and practitioners to verify the quality of their services.

OMB No. 0935-0124 – Project Officer: Mamatha Pancholi/ CDOM

“Validation Pilot for the AHRQ Patient Safety Indicators Phase II”

The purpose and intent of Phase II is the same as Phase I: provide information on 1) the extent to which organizations were willing to participate on a volunteer basis; 2) the most effective protocols for deploying the data abstraction, sampling and data collection tools to these different organizations; and 3) whether the data collected would be sufficient to achieve the study aims. However, the Phase II study aims are broader and include the public release of the tools. Specifically, the pilot will study the feasibility of:

• Focusing on characterizing the cases mistakenly not flagged from the administrative data using data from the medical record. Identifying these “false negatives” is important for accurate comparative reporting, and requires more efficient sampling approaches.

• Selecting indicators that are relatively less common¹ and more clinical complex² than the indicators included in Phase I. These indicators require additional years of data for adequate sample sizes, and more intensive methods of training and monitoring.

• Including the public release of the data abstraction, sampling and data collection tools, which requires an assessment of reliability and regulatory compliance. Ultimately, validation of the AHRQ Patient Safety Indicators requires that these tools become publicly available in order that individual hospitals may use the tools to validate the results in their own institutions in a reliable manner. Public release recognizes that validation is not a static attribute of the indicators, but a dynamic and evolving one that requires continuous evaluation and assessment.

1 The less common indicators include Foreign Body Left in During Procedure, Postoperative Physiologic and Metabolic Derangement and Postoperative Wound Dehiscence.

2 The more clinically complex indicators include Postoperative Hemorrhage or Hematoma and Postoperative Respiratory Failure.