Description of FY Hearing Process Report Card Survey

DOCUMENTATION FOR THE GENERIC CLEARANCE

OF CUSTOMER SATISFACTION SURVEYS OMB No. 0960-0526

TITLE OF INFORMATION COLLECTION: Fiscal Year (FY) 2017 Hearing Process Report Card (HPRC) Survey

SSA SUB-NUMBER: D-01

BACKGROUND

Social Security disability benefit applicants who do not agree with the Social Security Administration’s (SSA) initial decision on their claim are legally entitled to appeal. As part of the appeals process, claimants can file a request for a hearing before an administrative law judge (ALJ) who decides whether SSA should award benefits.

In FY 2008, we began conducting a satisfaction survey with two groups of applicants who recently participated in an ALJ hearing. The first group consists of applicants who received a favorable decision from the ALJ; the second group includes applicants who received an unfavorable decision. The survey assesses overall satisfaction with the hearing, and addresses various aspects of service that can have a bearing on satisfaction with the process. We are conducting the survey again in FY 2017 to track changes in customer perceptions.

SURVEY

Description of Survey

We developed two questionnaires: one for awarded and one for denied hearing applicants. The two surveys are identical with the exception of one question related to notice clarity, which we tailored to address the different outcomes experienced by these two groups. We have successfully administered all the questions in the survey with these populations.

The survey covers a variety of topics related to the service SSA provides during the hearing process, including the following:

• The ease of finding information about how to file a hearing request;

• The quality of information from SSA explaining the hearing process;

• Ease of contacting SSA and the helpfulness of the information provided about the status of the hearing request;

• Whether the person chose to have a representative handle the hearing;

• Whether the person had a hearing with a judge face-to-face or by video conference;

• Satisfaction with the location of the office where the hearing is held;

• Satisfaction with the judge’s performance; including clarity of explanations; preparedness to discuss the case; courtesy; and time spent with the claimant;

• Satisfaction with the length of the various stages of the hearing process: from date hearing requested to date hearing held; from date hearing held to date decision received; and the overall period from date hearing requested to date decision received;

• Satisfaction with notice clarity; including the explanation of the judge’s decision; the explanation of the amount of benefits (for those awarded benefits); and the explanation of how to appeal the judge’s decision (for those denied benefits);

• Satisfaction with SSA staff including their helpfulness, courtesy, and job knowledge;

• Overall ratings of the hearing experience and SSA’s service.

Statistical Information

Sample Selection

In FY 2017, SSA expects to issue just over 589,000 hearing decisions on disability claims, consisting of approximately 55 percent favorable (award) and 45 percent unfavorable (denial) decisions. The following table displays the estimated volume of hearing decisions for each of SSA’s ten regions and the National Hearing Center (NHC) for FY 2017.

Region	Estimated Yearly Hearing Awards	Estimated Yearly Hearing Denials
Boston	12,200	10,000
New York	23,700	19,400
Philadelphia	31,500	25,800
Atlanta	84,900	69,500
Chicago	49,400	40,400
Dallas	41,600	34,000
Kansas City	14,900	12,200
Denver	9,100	7,400
San Francisco	33,400	27,300
Seattle	10,300	8,500
NHC	13,000	10,600
Total	324,000	265,100

Because of the relatively small size of these universes, we select decisions at three intervals over the fiscal year in order to stratify the sample by outcome (award/denial) and geographic area.

At each interval, we select a separate sample of hearing decisions from each of SSA’s 10 regions and the NHC. We select 900 cases from each region and 1,000 cases from the NHC, with an equal distribution of awards and denials. The total sample sizes for the year equal:

Geographic Area	Awards	Denials
Each Region (x 10)	1,350	1,350
NHC	1,500	1,500
Total	15,000	15,000

We weight the results to arrive at a national overall satisfaction rating that represents the universe of hearings decisions processed by SSA during the fiscal year.

Methodology

An SSA-approved contractor conducts the survey by mail using a scannable questionnaire. We mail a pre-notification postcard to all sampled individuals advising them to expect the survey questionnaire. Less than one week later, we mail a questionnaire with a cover letter explaining the purpose of the survey. We include both the Paperwork Reduction Act and the Privacy Act statements in the initial cover letter. We make two additional contacts to solicit participation; we send a follow-up postcard and a duplicate survey package (questionnaire and follow-up cover letter).

Response Rate

SSA takes the following steps to maximize the response rate for this survey:

• We mail out a pre-notification postcard to inform sampled individuals about the upcoming survey, and to be on the lookout for the envelope containing the actual questionnaire. Using a postcard format allows the sampled individuals to see that SSA sanctions the survey. Additionally, the postcard identifies the contractor who is conducting the survey for SSA, which should increase the likelihood that sampled individuals will open the envelope when they receive the questionnaire.

• Within a week after we send the initial postcard, we mail the questionnaire along with a cover letter encouraging sampled individuals to respond by emphasizing the importance of the survey.

• One week later, we mail a reminder postcard to all sampled individuals.

• Two weeks after we send the reminder postcard, we mail a follow-up letter and another copy of the questionnaire to sampled individuals who have not responded as of that point.

• The questionnaire is short (one page - front and back), has an eye‑catching title, and is easy to read and complete. The scannable versions design is for ease of use by the disabled population, e.g., font sizes are large, difficult fill-in bubbles are not used.

• We send a Spanish version of the survey to those sampled individuals identified in the sample selection process as preferring to do business in Spanish.

• We provide SSA’s national toll-free 800 number so sampled individuals can call if they have any questions.

In the FY 2013 HPRC survey, we achieved a combined response rate of 45 percent at the national level. Among the regions, the combined response rates ranged from 42 percent to 47 percent. Considering our prior experience, we anticipate that we will see equally good response rates in the FY 2017 survey. (We are referencing the FY 2013 survey results because we have not yet published the final data for the FY 2015 survey.) Note that SSA routinely conducts a non-responder analysis to identify any significant differences between the responder and non-responder populations and their potential impact on the survey results.

Sampling Variability

The key variable for this survey is overall satisfaction with SSA’s service. We define satisfaction as a combined rating of excellent, very good or good (E/VG/G). The table below displays the most recent E/VG/G ratings for the nation. Assuming a 50 percent response rate and similar ratings in the FY 2017 survey, our proposed sample sizes are large enough to provide a sampling variability at the 95‑percent confidence level acceptable for the intended purpose of the survey.

Stratum	Sample Size	Key Variable (E/VG/G Rating)	Sampling Variability
Awards	15,000	85%	+/- .9
Denials	15,000	40%	+/- .8

The table below shows the range of the E/VG/G overall service ratings among the regions in the last published survey. Based on an anticipated response rate of 50 percent from the regional samples of awarded and denied cases, we estimate that the sampling variability will fall in the ranges shown. These ranges are acceptable given the intended purpose of the survey.

Stratum	Regional Key Variable (E/VG/G Rating)	Sampling Variability
Awards	77% to 88%	+/- 3.7 to +/- 2.5
Denials	32% to 43%	+/- 4.4 to +/- 5.1

SSA’s Office of Quality Review and Office of Quality Improvement (OQI) are responsible for sampling and data analysis. Douglas Turnbull, Director, Office of Modeling, OQI, SSA, provides statistical support. OMB can reach Mr. Turnbull at (410) 966-2595.

IF FOCUS GROUP MEMBERS WILL RECEIVE A PAYMENT, INDICATE AMOUNT:

We will not compensate participants for this survey.

USE OF SURVEY RESULTS:

We first conducted the HPRC Survey in FY 2008 to establish a baseline measure of customer satisfaction with the hearing process. SSA uses the survey to track any changes in perceptions that may result from agency initiatives to reduce the hearing backlog and improve efficiency.

BURDEN HOUR COMPUTATION (Number of responses (X) estimated response time (/60) = annual burden hours):

Number of Responses: 30,000

Estimated Response Time: 5 minutes

Annual Burden Hours: 2,500 hours