CMS-R-305, OMB Approval #0938-0786 MATHEMATICA POLICY RESEARCH
Added title page: “CMS External Quality Review (EQR) Protocols” with PRA disclaimer
Consolidated the Introduction, all Protocols, and appendices in a single document to facilitate cross-referencing and integration of activities
Removed individual Tables of Contents within the Introduction and each Protocol; added a combined Table of Contents following the title page
Restructured and simplified the Introduction to improve narrative flow and usability, beginning with (1) background, (2) an overview of the EQR protocols, (3) issues to consider before conducting the EQR and EQR-related activities, (4) tips for drafting EQR technical reports, (5) tips for getting started on the protocols, and (6) a list of protocols for mandatory and optional EQR-related activities with a brief description of each protocol (including a hyperlink to the corresponding protocol). Specifically:
Added call-out box of key definitions (managed care plan, external quality review, external quality review organization, and EQR-related activities)
Defined and summarized federal regulations related to Medicaid and CHIP managed care quality; updated text and citations based on 2016 Medicaid and CHIP managed care final rule; added more information about EQR requirements in 42 C.F.R. Part 438
Added Figure 1 to show the scope and evolution of EQR in Medicaid and CHIP between 1997 and 2018
Added Figure 2 to show the relationship between EQR, state quality strategy, and QAPI, as well as text summarizing the interrelated quality requirements that apply to Medicaid managed care
Added Figure 3 to clarify the EQR process to the reader, and the difference between EQRO, EQR, and EQR-related activities
Added text distinguishing between mandatory and optional EQR-related activities; and a call-out box listing mandatory and optional EQR-related activities
Added Table 1 to show federal financial participation match rates by managed care plan type and protocol (MCO, PIHP, PAHP, and PCCM entity)
Added Figure 4 linking EQR protocols to the applicable mandatory or optional EQR-related activities
Added text about the applicability of HIPPA throughout all EQR-related activities and the EQR technical report process
Revised text to provide: (1) steps to prepare for EQR-related activities, (2) description of who may conduct EQR-related activities, and (3) a call-out box with step-by-step instructions from selecting an entity to conduct the EQR-related activity(ies) to confirming responsibilities, regulations, and a timeline with the entity conducting EQR-related activity(ies) and EQR participants
Added section on non-duplication for mandatory EQR-related activities to reduce the administrative burden on MCPs and states while still ensuring relevant information is available to EQROs for the annual EQR
Added guidance for drafting EQR technical reports that are actionable, clear, and concise; that highlight substantive findings; and that contain actionable recommendations. Additionally added a call-out box of step-by-step tips for drafting an effective EQR report
Added a “TIP” figure instructing the reader to use the “go now!” buttons to navigate to EQR protocols and appendices, as well as a link to the CMS managed care quality TA inbox to submit questions
Added an overview of the mandatory and optional EQR-related protocols, including a summary of each protocol, followed by a “go now!” button that hyperlinks directly to the protocol within the full document
Reordered protocols to follow the order referenced in the May 2016 Medicaid and CHIP managed care final rule
Restructured each protocol to follow the same design and layout. This includes (1) an up-front header of whether the protocol is a mandatory or optional EQR-related activity, (2) the purpose of the EQR-related activity and background including the regulatory underpinnings of the EQR-related activity, (3) a figure illustrating the protocol activities and steps, followed by links to supplemental resources (such as worksheets or appendices) to help complete the steps/activities, (4) a step-by-step description of the activity and step(s) (if applicable) within the protocol, including data sources and data collection activities to promote data accuracy, validity, and reliability; and proposed method(s) for analyzing and interpreting the data; and (5) instructions and guidelines that may be used in implementing the protocol. Also included call-out “TIP” box hyperlinks to relevant worksheets and/or appendices, and call-out boxes describing these resources. Introduced call-out boxes, tables, and figures to orient the reader and increase readability, and aligned the narrative with the applicable worksheets
Updated all protocols to align with the May 2016 Medicaid and CHIP managed care final rule
Edited all headings and sub-headings from passive to active voice
Revised all worksheets for consistent format and readability
For each protocol with accompanying worksheets, edited and moved text from 2012 protocols to worksheets; revised worksheets to make them more actionable and avoid duplicative text in the protocol, included instructions on how the worksheets may be used, followed by a crosswalk of each worksheet to the applicable activity and step in the protocol. Also simplified language across worksheets and added an “overall assessment” field at the end of each worksheet to summarize recommendations for improvement.
For protocols 1 and 2, developed two reporting frameworks to facilitate consistency across reports:
Protocol 1 Worksheet 1.11. Framework for Summarizing Information about Performance Improvement Projects (PIPs). This reporting framework provides guidance to (1) summarize general information about the PIP, (2) describe the PIP improvement strategies or interventions, (3) document performance measures and results, and (4) record validation information, including EQRO recommendations to improve the PIP, and
Protocol 2 Worksheet 2.14. Framework for Summarizing Information about Performance Measures. This reporting framework provides guidance to summarize the results for each performance measure validated for each MCP, including (1) an overview of each performance measure, (2) performance measure results, and (3) validation status. Additionally included an example of a completed Worksheet 2.14.
Activity 1, Step 1. Review the Selected Study Topic
Added reference to the National Quality Strategy and CMS priorities and initiatives when developing PIP study topics
Added suggestion that states review performance on child and adult Core Set performance measures to identify opportunities to improve performance through a managed care PIP
Activity 1, Step 2. Review the Study Question
Added a call-out Q & A to help the reader identify if a PIP study question is clear, concise, and answerable
Edited and moved text about “good” and “poor” study questions into a table (Table 1.1)
Activity 1, Step 4. Review the Sampling Method
Added text to explain the importance of appropriate sampling methods and directed the reader to Appendix B, Sampling Approaches for EQR Data Collection Activities, which provides an overview of sampling methodologies applicable to PIPs
Activity 1, Step 5. Review the Selected Study Variables and Performance Measures
Added two call-out boxes, “What is a study variable?” and “Tips for choosing study variables”
Added text suggesting the user consider data availability when selecting variables and performance measures for a PIP
Edited and moved text for types of variables for PIPs to a table (Table 1.2)
Edited and moved text for types of measurement scales for PIPs to a table (Table 1.3)
Added figure to provide guidance when selecting PIP outcome measures (Figure 1.2)
Activity 1, Step 6. Review the Data Collection Procedures
Added text describing the difference between administrative data collection and medical record review
Activity 1, Step 7. Data Analysis and Interpretation of Study Results
Added paragraph on importance of accurate data analysis to inform any changes based on results
Added text directing the reader to analytic reports of PIP results prepared by the MCP as the primary source for assessment
Activity 1, Step 8. Assess the Improvement Strategies
Added paragraph about the Plan Do Study Act (PDSA) cycle and the Institute for Healthcare Improvement Model for Improvement as a tool to guide improvement work
Added call-out box summarizing the Model for Improvement and PDSA cycle
Activity 1, Step 9. Assess the Likelihood that Significant and Sustained Improvement Occurred
Consolidated Steps 9 and 10 from the 2012 version into a single step about measuring improvement
Activity 1, Step 3. Conduct Detailed Review of Measures
Edited and moved text to call-out box, “Resources for Detailed Review of Measures”
Added call-out box about using HEDIS® measures calculated by HEDIS®-certified software
Activity 1, Step 5. Prepare for the MCP Onsite Visit
Edited and moved text to call-out box, “Potential Onsite Participants”
Activity 2. Conduct Onsite Visit Activities
Edited and moved text to call-out box, “Purpose of the Onsite Visit”
Added applicable Medicaid and CHIP regulation citations to Worksheet 3.1. Compliance Review and Worksheet 3.4. Compliance Interview Questions
Background
Added regulatory references applicable to compliance review, additional areas for potential compliance review, and frequency of compliance review and reporting
Activity 2. Perform the Preliminary Review and Activity 3. Conduct MCP Site Visit
Clarified pre-onsite visit activities and onsite activities conducted in Activities 2 and 3
Inserted placeholder because protocol is not yet available
Added Worksheet 5.7. Suggested Format for Reporting Encounter Data Information in the EQR Technical report. This reporting framework is intended to help the EQRO report findings from the encounter data validation activities by MCP.
Background
Added definition of encounter data, and the relationship between encounter data, the Medical Statistical Information System (MSIS), and the Transformed Medicaid Statistical Information System (T-MSIS)
Added call-out box, “State Uses of Encounter Data”
Added federal regulations applicable to encounter data
Added reference to the CMS Encounter Data Toolkit
Activity 1. Review State Requirements
Revised text to specify that the state should provide the EQRO with the following information: (1) specific requirements regarding the MCPs’ collection and submission of encounters, (2) requirements regarding the types of encounters that must be validated, (3) standards for the submitted data, (4) state standards for encounter data completeness and accuracy, (5) data dictionary, (6) description of the information flow from the MCP to the state, (7) a list and description of automated edits or checks performed on the data, (8) timeliness requirements for data submissions, (9) any EQRO validation reports from previous years, and (10) any other information relevant to encounter data validation
Activity 2, Step 1. Review the MCP’s ISCA
Added a step-by-step list of MCP capabilities the EQRO should understand based on findings from the ISCA
Added a call-out box, “Potential Causes of Encounter Data Errors by MCPs”
Activity 3, Step 1. Develop a Data Quality Test Plan
Revised text to provide step-by-step suggestions to develop the data quality test plan
Activity 3, Step 2. Encounter Data Macro-Analysis—Verification of Data Integrity
Revised text to provide step-by-step suggestions to verify data integrity
Activity 3, Step 3. Encounter Data Micro-Analysis—Generate and Review Analytic Reports
Revised text to provide step-by-step suggestions when reviewing analytic reports for broader data quality issues
Activity 3, Step 4. Compare Findings to State-Identified Benchmarks
Revised text to provide a list of benchmarks to compare encounter data to
Activity 4. Review Medical Records
Added a list of questions to consider when reviewing medical records
Added call-out box, “Sampling Guidance for Medical Record Review”
Background
Revised introductory paragraph to provide justification for surveys as a resource for assessing the experience of managed care enrollees and providers, and the protocol’s goal to provide guidance about designing and conducting surveys that produce valid and reliable results
Reorganized protocol to clarify activities under Section I (administering the survey) and Section II (validating the survey)
Activity I.2. Develop a Work Plan
Added a new activity (develop a work plan) because the work plan is essential to guiding the implementation of the survey, including the project management plan, schedule, and reporting requirements
Activity I.3. Select the Survey Instrument
Updated text clarifying the options for selecting the survey instrument (use an existing validated survey instrument; adapt an existing survey instrument with additional state-specific questions; or develop a new survey instrument)
Updated and moved 2012 text to table 6.1, which provides examples of existing survey instruments and hyperlinks to these instruments
Added call-out box defining validity and reliability
Added text suggesting that when adapting a questionnaire, the EQRO should consult with an expert in survey design
Added call-out box of best practices in questionnaire design for a state or EQRO interested in developing a new survey instrument when the survey purpose requires answers to questions not answered by existing instruments
Added text defining face and content validity
Activity I.4. Develop the Sampling Plan
Added two new steps: (1) determining the number of units to sample (Step 3) and (2) selecting the sample (Step 4)
Activity I.5. Develop a Strategy to Maximize Response
Added guidelines for identifying the specific data needed to locate sample members before survey launch
Added guidelines for designing a data collection strategy that maximizes response, including an advance letter, multiple and varied call attempts, multi-mode surveys, and multiple languages
Added text about tailoring data collection strategies to the survey population
Added call-out box with guidance on integrating web-based outreach in data collection
Added two new steps: (1) specifying the method used to calculate the response rate (Step 3), and (2) including a plan for a non-response analysis (Step 4)
Activity I.6. Develop a Quality Assurance Plan
Added call-out box of tips for quality assurance checks
Activity I.7. Implement the Survey According to the Work Plan
Added Table 6.2. Sample data collection schedules by week to complement the work plan
Activity I.8. Prepare and Analyze Survey Data and Present Results in Final Report
Combined 2012 activities 7 and 8, and added the following steps: (1) implement post-processing procedures, (2) calculate the sampling weights, (3) conduct a non-response analysis, (4) analyze survey data, and (5) prepare and submit a final report
Activity II. Validating a Survey
Updated to follow applicable changes to the activities and steps associated with administering a survey
Added a section on understanding potential sources of survey error and a figure (Figure 6.2) illustrating what those errors are and when they may occur
Added Table 6.3, which maps common sources of survey error to data collection activities, and remedies to minimize error
Updated to reflect all applicable changes to Protocol 2 (Validation of Performance Measures)
Updated to reflect all applicable changes to Protocol 1 (Validation of Performance Improvement Projects)
Added Table 8.1, which crosswalks Protocol 8 implementation activities 1 – 9 with Protocol 1 validation activity 1 steps
Added call-out box, “How does a focus study differ from a PIP?” to accompany updated text
Revised text to reflect updates in Protocol 1 relevant to the study topic, study question, study variable(s), data collection, and analyzing and interpreting study results
Added guidance for selecting focus study topics and call-out box, “Factors to consider when selecting a focus study topic”
Inserted placeholder because protocol is not yet available
Appendices
Added Figure A.1 to clarify provider, MCP, and EQR data activities under the ISCA
Added Figure A.2 to provide an overview of ISCA activities
Updated text about T-MSIS, value-based purchasing, alternative payment models, integrated care models, and the adoption and meaningful use of certified EHRs that are relevant to information systems assessments
Revised Worksheet A.2, Information System Review Worksheet and Interview Guide for the EQRO to prompt as needed on any issues identified in Worksheet A.1 (Information System Capabilities Tool)
Updated based on changes in the Introduction and Protocols
Updated based on changes in the Introduction and Protocols
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Michaella Morzuch |
| File Modified | 0000-00-00 |
| File Created | 2021-01-15 |