AnnualAppellantClimateSurvey

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

The respondent universe for this survey project will be all appeals that have been “Closed” during a six month period. OMHA will focus on a non-redundant appellant universe so as not to bias this study towards large entities that make up the vast majority of appeals, i.e. providers or suppliers file approximately 82% of the appeals and typically a single provider or supplier files multiple appeals throughout the year. Further, the survey takes into account the multiple hearings for a non-redundant appellant. Based on initial analysis of the population, there will be roughly 3500 non-redundant appellants whose cases have been closed in the previous six months. Roughly 70% of these appellants – or 2500 – are individual beneficiaries. The remaining 1000 are providers or suppliers. The focus of this study is on appellant satisfaction with the existing process, and therefore it is important to treat each appellant – regardless of type – as an individual entity.

With a total population of 3500, a sample of roughly 400 will be needed to achieve a level of 95% statistical significance. In order to ensure that the sample is representative of the larger population, OMHA will randomly stratify the sample by two factors: Hearing Format and Medicare Part.

There are three types of Hearing Formats:

• In-Person Hearings

• Telephone Hearings

• Video Teleconference (VTC) Hearings

Additionally, there are five types of Medicare Parts:

• Part A

• Part B

• Part C

• Part D

• Other – a combination of “IRMAA” and “Entitlement” Hearings

Given the percentage distribution of the entire population by these two factors, a final stratified random sample would include the number of completed surveys listed below.

Random, Stratified Sample per Year

Given a response rate of 60%, the following number of appellants will need to be contacted each year:

Total Number of Appellants OMHA will Contact, Given a 60% Response Rate

2. Procedures for the Collection of Information

OMHA derived its total sample based on three factors:

1. The number of unique (non-redundant) appellants who have had an appeal closed in the previous six months;

2. A 95% confidence level

3. A margin of error of +/- 5%.

Given the total respondent population (n=3500), an annual sample of 400 responses is needed. In order to reduce the burden on OMHA staff and maintain a moving average of appellant satisfaction that can be viewed over time, responses will be obtained quarterly, with 100 responses generated in Q1, 100 in Q2, 100 in Q3, and 100 in Q4.

In each quarter a new sample will be used, but OMHA will sample without replacement, so as to ensure that no appellants are surveyed any more than once per year. A stratified random sample based on the two factors identified above – Hearing Format and Medicare Part – will be used to obtain the most representative data possible.

The process of collecting data will occur in the following fashion:

1. At the beginning of each quarter, OMHA will obtain a list of all unique appellants who have had an appeal closed in the previous six months. For the 1^st quarter of Fiscal Year 2009, this would include all appeals that have been closed between April 1^st and September 30^th, 2008.

2. OMHA will provide this data to the contractor, Coray Gurnitz Consulting (CGC), who will use the stratified random sampling methodology outlined above to obtain a list of potential survey respondents.

3. CGC will place the survey online so that respondents can access it electronically, and create a telephone script to be used for telephone respondents.

4. Using Don Dillman’s “Tailored Design Method” for maximizing survey response, OMHA will send out letters and e-mails to all potential respondents announcing the survey. These letters and e-mails will contain official signatures and labels to verify the organization’s intent and professionalism.

5. One week after these items have been distributed, OMHA will send out a second e-mail with the survey link, so that respondents are able to complete the survey online.

6. At this time the CGC phone interviewers will begin to contact potential respondents over the phone to inquire about their availability to complete the survey. If they are interested, a time for completion will be scheduled.

7. Two weeks after the initial e-mails and telephone calls have been made, reminder calls and e-mails will occur.

Given the time frame listed above, OMHA should be able to draw the sample and contact each respondent three times in the first month of a given quarter: the first contact announces the survey (one week into the quarter), the second contact invites respondents to participate in the survey (two weeks into the quarter), and the third contact reminds respondents to complete the survey (four weeks into the quarter).

Electronic survey responses will be stored electronically and downloaded into a Microsoft Excel or SPSS database. Telephone responses will be entered into one of these databases at the time of the call, and then stored in either format.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Based on OMHA’s review of other HHS surveys of a similar type, including surveys conducted agency-wide and ones developed for a specific OPDIV, a 60% response rate is expected. Although such a response rate is below the usual OMB requirement, the usage of Internet technology and the stratification of the sample should lead to minimal burden increases for OMHA’s staff and budget. Despite the expectation of a 60% response rate, several methods will be used to increase this rate of response.

First, OMHA will maximize response rates by using a mixed method survey approach. In the text “Survey Non-Response”, author and editor Don Dillman identifies several factors that will positively impact respondent cooperation, including:

• Using a notable agency for data collection:

All letters, surveys, and e-mail announcements will contain OMHA’s seal and logo, so as to improve the confidence respondents have with the survey.

• Advance Warning of the Survey Request:

As described above, each respondent will receive both an e-mail and standard mail survey announcement. Respondents will also receive a telephone invitation to schedule a time for survey completion, assuming they have not already completed the survey online.

• Follow-up Procedures:

After the initial invitations and survey links have been distributed, OMHA and its contractor will follow up with potential respondents via e-mail messages or telephone reminders.

Each of these methods will improve the overall response rate and minimize non-response. Furthermore, the initial reminder e-mails and phone calls will be complete within the first month of a given quarter, meaning that OMHA will have two months of focused follow-up to improve its response rate, if necessary. In a given quarter, OMHA will only need to collect 100 survey responses to achieve a 95% rate of confidence, giving the agency more than enough time to contact appellants about the survey.

4. Tests of Procedures or Methods to be Undertaken

OMHA anticipates two forms of testing prior to survey administration. First, a small, random sample of potential respondents (n=9) will be asked to comment on the survey and its questions. The goal with this test is to improve question wording and identify data elements that should be collected and provided to the respondents in advance of their participation.

A second test will be used to ensure that the Internet link created for this survey is totally accessible by all potential respondents. Because OMHA has four office locations, 1-2 internal staff members at each location can participate in this testing in an effort to ensure technical feasibility. This second testing procedure will not lead to content-specific changes in the survey. This testing will only be used to ensure that any technological system used to gather data will be fully functional at the time of survey announcement.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or

Analyzing Data

Three members of the OMHA staff will play a primary role in this project:

• Sharon Bailey – Attorney Advisor, 703-235-0144, Sharon.Bailey@hhs.gov

• Maria Price-Detherage – Director, Executive Office, 703-235-0689, Maria.Price-Detherage@hhs.gov

• Systems Analyst / Reports Specialist

This submission was prepared in consultation with contractors from Coray Gurnitz Consulting, specifically David Spak, M.P.P., (david.spak@cgstrategy.com) and Kevin Coray, Ph.D. (kevin.coray@cgstrategy.com). Dr. Coray can be reached at 703-527-7001.

With respect to specific tasks, the following individuals were responsible for:

1. designing the data collection: Ms. Bailey, Ms. Price-Detherage, and Dr. Coray.

2. collecting the data: Coray Gurnitz Consulting staff.

3. analyzing the data: Coray Gurnitz Consulting staff.

5