0990-0330_Appellant study_Section B Statistical Methods OMB 100611

Supporting Statement for

The Office of Medicare Hearings and Appeals (OMHA)

Appellant Climate Survey

and Supporting Regulations Contained in

42 or 45 CFR xxx.xxx

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

The respondent universe for this survey project is all appeals that have been “Closed” during a six month period. OMHA is focusing on a non-redundant appellant universe so as not to bias this study towards large entities that make up the vast majority of appeals, i.e. providers or suppliers file approximately 82% of the appeals and typically a single provider or supplier files multiple appeals throughout the year. Further, the survey takes into account the multiple hearings for a non-redundant appellant. Based on initial analysis of the population, there are roughly 3500 non-redundant appellants whose cases have been closed in the previous six months. Roughly 70% of these appellants – or 2500 – are individual beneficiaries. The remaining 1000 are providers or suppliers. The focus of this study is on appellant satisfaction with the existing process, and therefore it is important to treat each appellant – regardless of type – as an individual entity.

With a total population of 3500, a sample of roughly 400 is needed to achieve a level of 95% statistical significance. In order to ensure that the sample is representative of the larger population, OMHA randomly stratifies the sample by two factors: Hearing Format and Medicare Part.

There are three types of Hearing Formats:

• In-Person Hearings

• Telephone Hearings

• Video Teleconference (VTC) Hearings

Additionally, there are five types of Medicare Parts:

• Part A

• Part B

• Part C

• Part D

• Other – a combination of “IRMAA” and “Entitlement” Hearings

Given the percentage distribution of the entire population by these two factors, a final stratified random sample includes the number of completed surveys listed below.

Random, Stratified Sample per Year

Given the obtained response rate of 80.2%, the following number of appellants is contacted each year:

Total Number of Appellants OMHA will Contact, Given an 80% Response Rate

2. Procedures for the Collection of Information

OMHA derived its total sample based on three factors:

1. The number of unique (non-redundant) appellants who have had an appeal closed in the previous six months;

2. A 95% confidence level

3. A margin of error of +/- 5%.

Given the total respondent population (n=3500), an annual sample of 400 responses is needed.

In each survey cycle a new sample is used, but OMHA will sample without replacement, so as to ensure that no appellants are surveyed any more than once per year. A stratified random sample based on the two factors identified above – Hearing Format and Medicare Part – is used to obtain the most representative data possible.

The process of collecting data will occur in the following fashion:

1. At the beginning of each survey cycle, OMHA obtains a list of all unique appellants who have had an appeal closed in the previous applicable period.

2. OMHA provides this data to the contractor, Coray Gurnitz Consulting (CGC), who uses the stratified random sampling methodology outlined above to obtain a list of potential survey respondents.

3. CGC will continue to successfully use a telephone script for telephone respondents.

4. Using Don Dillman’s “Tailored Design Method” for maximizing survey response, OMHA has been successful in obtaining the appropriate level of response by sending out letters to all potential respondents announcing the survey. These letters contain official signatures and labels to verify the organization’s intent and professionalism.

5. At this time the CGC phone interviewers begin to contact potential respondents over the phone to inquire about their availability to complete the survey. If they are interested, a time for completion is scheduled.

6. Two weeks after the initial telephone calls have been made, reminder letters are distributed.

Given the time frame listed above, OMHA draws the sample and contacts each respondent three times in the first month of a given survey period: the first contact announces the survey, the second contact invites respondents to participate in the survey, and the third contact reminds respondents to complete the survey.

Telephone responses are stored electronically and downloaded into a Microsoft Excel or SPSS database.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Based on OMHA’s success in obtaining responses to this survey, the method has been effective in obtaining a response rate of approximately 80%. As such, a non-response bias is not an issue.

OMHA uses multiple strategies to maximize response rates and data quality, including:

• selecting a representative sample stratified to improve its power,

• providing Spanish-speaking telephone bank operators when needed,

• using official HHS OMHA letterhead signed by the Chief Administrative Law Judge, and

• maximizing cooperation through advance warning of the survey request and reminder letters.

The OMHA ACS telephone survey is designed to achieve the highest practical rates of response, commensurate with the importance of survey uses, time constraints, respondent burden and data collection costs.

4. Tests of Procedures or Methods to be Undertaken

Prior to the original fielding of the survey, OMHA tested by asking a small, random sample of potential respondents (n=9) to comment on the survey and its questions. The goal with this test was to improve question wording and identify data elements that should be collected and provided to the respondents in advance of their participation.

Since that time, the survey is regularly monitored during administration to track response rate and methodology, and make appropriate process revisions as necessary. For example, the internet version of the survey was eliminated since so few respondents utilized this form of collection.

The success of this survey and success of the process to administer the survey have led to efficiencies resulting in reductions of the cost to government by one-third of the original cost.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or

Analyzing Data

Two members of the OMHA staff will play a primary role in this project:

• Katheryn Michael, MAED, BSN, Training Program Manager, 703-235-0416, Katheryn.michael@hhs.gov

• Wanda Williams, Systems Analyst / Reports Specialist, 703-235-0673, wanda.williams@hhs.gov

This submission was prepared in consultation with contractors from Coray Gurnitz Consulting, specifically Ellen Wexler, M.P.A., (ellen.wexler@cgstrategy.com) and Kevin Coray, Ph.D. (kevin.coray@cgstrategy.com). Dr. Coray can be reached at 703-527-7001.

With respect to specific tasks, the following individuals were responsible for:

1. designing the data collection: Ms. Michael and Dr. Coray.

2. collecting the data: Coray Gurnitz Consulting staff.

3. analyzing the data: Coray Gurnitz Consulting staff.

5