eRPC OMB Questions-art comments

OMB Questions concerning the electronic Request for Program Consultation (eRPC) tool

Question 1

1. The supporting statement says that this has been piloted on 9 DDSs. What have DDSs been using in the past for appealing regional quality assurance determinations?

Under the current “Rebuttal Process”, the Disability Determination Services (DDS)’s follow a multi-step process. In Step 1, they prepare memoranda on (DDS) agency letterhead to rebut deficiencies cited by a regional Office of Quality and Performance (OQP). A memorandum typically contains the reasons that the DDS feels will support its determination of the case including references to the Program Operation Manual System (POMS) sites supporting DDS’s point of view. It is generally 2 to 3 pages in length. It must also be supported by a separate medical narrative and, if applicable a vocational narrative as well. A Step 1 rebuttal can not be proposed unless all requested development (in the error citation) has been obtained. Copies of the memorandum are sent to the regional OQP and the regional Center for Disability Programs (CDP).

If OQP agrees with DDS’s arguments, it will remove the deficiency citation from the record. If OQP does not agree with DDS, it sends a narrative memorandum (of 1 to 3 pages) to CDP and DDS reviewing why OQP does not accept DDS’s position. OQP makes the case available to CDP to review.

If DDS persists in its position, it can request a second level rebuttal through an additional narrative memorandum to CDP, or CDP can request a second level rebuttal on its own initiative by forwarding a memorandum to OQP. At the second level, the regional OQA forwards the case to OQP at Social Security’s Central Office. That office makes the final determination and forwards a memorandum to DDS, CDP, and the regional OQP office.

Question 2

2. What were some of the lessons learned from the pilot and what where the ensuing changes made to this ICR?

The pilot worked extremely well. All components involved (DDSs, OQP and CDPs) were pleased with the ease of the process. A complete evaluation of the RPC pilot was released in 12-06. The previous rebuttal process took from several weeks to several months for each case. The eRPC process is required to attain a 7 calendar day turn-around time. So far, after 250 RPCs, the average turnaround time is 3 days. The eRPC process is totally electronic and eliminates handling paper in the rebuttal process. Claims in paper folders will continue to follow the older rebuttal process. In the clearance package submitted to you earlier, the Office of Disability Programs included a proposal for DDS to limit its input to the name and social security number of the claimant, a reference to support its position, and a brief narrative explaining why the deficiency should be removed. We propose to modify the eRPC tool to accommodate this change.

On the 10 pages following, we are including a section of the RPC evaluation that addresses your ‘lessons learned’ question.

Question 3

3. The supporting statement says there is a 3333 hour increase, but ROCIS says 2250 please clarify.

We inadvertently entered the incorrect increase in burden hours in item 15 of the supporting statement. The figures (4,500 respondents and 2,250 burden hours) in item 12 of the supporting statement and in ROCIS are the correct figures.

Pilot Findings:

1. Assess the effectiveness of the new RPC business process:

1. Does it provide faster turnaround times than the current rebuttal process?

The average time to review and respond to a request for Program Consultation (RPC) submission from the DDS was 3 days during the pilot. This compares to a 30 day turnaround time for rebuttals in the DQB during the same period in 2005.

Comment:

“The RPC response time provides far superior customer service.”

2. Does it result in quick identification and referral of needed policy changes, additional training and business process issues?

Despite the relatively short duration of the pilot, 16 unique issues were identified by the RPC’s submitted to ODP, and one POMS change has already been made in addition to 4 more referrals to policy components to review existing policy for clarity. Training has already been held with the DDQO on mental retardation cases, and an improvement has already been noticed in that area in the number of errors cited that cause disagreement. Comments:

“Policy units are notified when particular areas of policy or body systems are generating a disproportionate number of conflicts in the RPC process.”

“It provides a consistent message to all components regarding policy issues and the search function on the eRPC site assists in understanding policy issues before deficiencies are cited.”

A number of changes to the business process have been recommended and discussed among the components. One issue of particular interest is whether or not to change the definition of a cited error if the original citation was not correct but an error of a different type does exist. ODP is collecting comments on proposed changes to the business process and will create a draft POMS for IRD as appropriate.

3. Is it user friendly?

The surveys of users show the system to be very user friendly:

• 11 of the 13 survey participants from DDS and ODP reported the eRPC system is user friendly.

• 14 of 18 of all participants reported no observed eRPC performance problems.

4. Is it used more frequently than the current rebuttal process?

During the pilot, there was a 75% increase in the number of RPC’s filed over the number of rebuttals filed in the same period of 2005 (35 vs. 20). The rate of filing as a percent of total error returns to the DDS was 4.5% in the pilot compared to 3.4% in 2005, a rate increase of 34%.

5. Does it help to foster more dialogue between components on policy issues?

Over half of the responses (13 of 20) indicated a noticeable increase in dialogue among ODD/DDS, ODP, and OQP.

Comments:

“It is very gratifying to see the dialogue and interaction between the components. It is great to see that the DDSs have a venue to express their concerns and questions.”

“The RPC process removes the layers of the rebuttal process and allows the DQBs and DDS an opportunity to resolve differences.”

“With OQP as active participants in the RPC staffing, we are seeing a cooperative attitude between ODP and OQP.”

6. Does it create any bottlenecks in any component?

14 of 18 of all participants reported no observed eRPC performance problems. There were no reports of backlogs due to either the business process or system.

7. Does it require more or less resources than the existing rebuttal process in each component?

Only one of the seven OQP respondents said it took more staff time than the old rebuttal process. ODD/DDS surveys showed an even distribution among responses with two indicating it took more time; two saying it took more time; and two reporting it took the same amount of time.

Comments from the DDS:

“Although large amounts of medical consultant and vocational specialist time is not required, it does require more policy research on each case. In addition, because a wider range of issues can be addressed, we are doing more RPC's than we did rebuttals. However, we think this is a good thing.”

“Actual DDS staff time is about the same. Staff still review the file, confer, and prepare the RPC essentially the same way.”

8. What is a reasonable productivity level for RPC reviewers under the new business process?

Each RPC submission required an average of 3 hours and 25 minutes of hands on task time by an RPC reviewer in ODP. Each reviewer spends approximately 2/3 of his/her available time on RPC reviews, the other 1/3 on other projects within ODP. Based on the average task time, it is estimated that each ODP reviewer can handle about 340 RPC submissions a year

2. Assess the effectiveness of the new data collection tool:

   1. Is it easy for DDS to use to submit an RPC?

4 of 6 said eRPC was user-friendly, and the comments indicated only minor problems which were resolved as the pilot progressed.

2. Is the template an effective and efficient way to capture the policy issue in question?

100% of DDS survey responders said that the eRPC template was an effective and efficient way to capture the policy issue in question.

Comments:

“It is simple and straight forward. It requires us to look at policy NOT whether or not we agree with the DQB review. It takes much less time because we do not present the claim, just the summary of our argument.”

“They find the process far superior to the old rebuttal process. It forces the DDS to focus on the facts and avoid posturing and extraneous details resulting in a better DDS argument/product.”

3. Does the tracking and control system work well for ODP?

5 of the 7 ODP survey responses indicated that the tracking and control system works well for them.

4. Is the database easily accessible to DDS, OQP and ODP through the search function?

14 of 20 survey responders said they had used the Search function in eRPC. Of these, 79% (11) said that it had been helpful in researching policy questions. Comments were made that the function would become more useful as more cases are loaded in the database.

5. Are reports accurate and useful for all components?

Only 5 of 12 responders form DDS and OQP had used the MI reports as of the time of the survey. The MI reports became available in November, and management in ODP, OQP and ODD are still working out the questions of who should have access to which information. Decisions need to be made on whether to provide case level data to users in addition to the standard reports available on the eRPC web site. Requests to be able to drill down to this level of data have been made by the Regions in the pilot in order to help them use the data from RPC as a tool for individual learning and improvement. This is an area needing further discussion between all of the components

3. Assess the effectiveness of training and instructions

   1. Was training adequate to begin using data collection tool?

Five ODD/DDS management and six ODP participants were asked if they had received the necessary training on using the eRPC program. Eleven of the 13 said their training was sufficient.

2. Were business process instructions clear?

11 of 13 responders stated they had received the necessary training on the RPC Business Process. Also, 12 of the 13 reported that the instructions and user’s guide were clear.

4. Assess the effectiveness of reducing errors in adjudication and case reviews:

   1. Was the new RPC process integrated into the DDS and DQB quality process?

Both the ODD/DDS and OQP surveys asked about the improvement of and the value added by the new process compared to the rebuttal system. Twelve of 13 responses indicated it was a better process. All six of the ODD/DDS surveys reported that eRPC process provides better quality decisions and establishes a clear feedback loop.

Comments from OQP:

“RPC staffing include representation from various components (i.e. policy, quality, ODD) resulting in a more comprehensive evaluation of the pertinent issues of the case.”

“RPC process has opened up dialogue between ODP, OQP, ODD and the DDSs. RPC prevents the claimant from getting caught in the politics of SSA.”

“We have initiated regular training sessions for the reviewer staff and RMCS and notify/discuss every RPC submission with those directly involved.”

Comments from DDS:

“We meet with the staff involved in the claim and discuss everything pertinent to the RPC. We provide refresher training based on the RPC responses as warranted – one on one, small group, etc.”

“We have already used our feedback for training and re-educating our entire staff in looking at policy in relation to adjudication. Our interpretation of the application of policy over the last 25 years has been very narrow. We are attempting to bring about a culture change within the DDS.”

“It is useful for training as well as individual feedback, to keep everyone on the same page.”

2. Was the new RPC process successful in reducing the number of cited errors and increasing consistency of policy application?

From an objective standpoint, using data from Massachusetts can give us some insight into the potential of RPC for error elimination. Massachusetts was most aggressive of all the pilot states, submitting an RPC on over 13% of their error returns from DQB. During the pilot, Massachusetts was able to almost triple the reduction in their error rate due to rescinded errors over the baseline period in 2005.

A significant number of the RPC’s submitted by Massachusetts were on cases where they agreed with the OQP error return, but had other policy questions, such as how to apply the policy correctly in order to correct the error. This use of the RPC process, while not resulting in a rescinded error on a particular case, could very well eliminate future error returns as the DDS learned how to use and apply the policy in a consistent and appropriate way.

Massachusetts Error Rates in RPC Pilot and Baseline

	Gross Error Rate	Net Error Rate	% Change
Baseline 9/05 – 11/05	14.19%	13.99%	- 1.40%
Pilot 9/06 – 11/06	13.41%	12.77%	- 4.77%
% change in error rates during Pilot	- 5.49%	- 8.72%

From a subjective standpoint, some comments from the surveys:

From OQP:

“It provides a consistent message to all components regarding policy issues and the search function on the eRPC site assists in understanding policy issues before deficiencies are cited.”

“We believe it will provide more consistent decisions and that those decisions will reflect national policy direction. The feedback available on eRPC is excellent.”

“We are not a DDS but we believe that the RPC responses will promote consistency in policy interpretation.”

From DDS:

“We believe this will help deliver a National program rather than a state or regional program.”

“It is the best thing that has happened in the SSA quality component in decades. We really do believe that the program will benefit from this process. It is our DDS’s view that we are already seeing greater consistency among the components. We have also learned more from this process in its short existence than we have from many years of the old rebuttal responses, which frequently were unclear, uninformative, and/or inconsistent with previous returns. I have found my staff more receptive to differing policy interpretations due to a general sense of “fairness” and a clear recognition that it really is all about the policy. eRPC is even better than had been anticipated when it was in the talking and early formation stages. SSA really got it right this time!”

“This process, if rolled out and sufficiently staffed to ensure continued prompt turnaround and quality of response, will go a long way toward the goal of national consistency. It does require a culture change. For example, one of my supervisors is disappointed in the process because he has not "won" the RPCs he has done. The culture change is that it is not about winning or losing, but about learning and developing national consistency and policy clarity.”