PCMH Items Demonstration_OMB Statement A_05152017_FINAL_TC

Supporting Statement

Part A

CAHPS PCMH Items Demonstration Study

Version May 15, 2017

Agency for Healthcare Research and Quality (AHRQ)

TABLE OF ContentS

SUPPORTING STATEMENT: CAHPS PCMH Items Demonstration Study 1

Introduction 1

A. Justification 1

A1. Necessity of Information Collection 1

A2. Purpose and Use of Information 3

A3. Use of Information Technology 3

A4. Identifying Duplication 3

A5. Impact on Small Businesses 3

A6. Consequences of Less Frequent Data Collection 4

A7. Special Circumstances 4

A8. CMS Federal Register Notice 4

A9. Respondent Payments or Gifts 4

A10. Assurance of Confidentiality 5

A11. Sensitive Questions 7

A12. Burden of Information Collection 7

13. Capital Costs 9

A15. Program Changes or Adjustments to Annual Burden 10

A16. Tabulation and Publication of Results 10

A17. Display of OMB Expiration Date 11

A18. Exceptions to the Certification Statement 11

References 12

List of Attachments 12

SUPPORTING STATEMENT:

CAHPS PCMH Items Demonstration Study

Introduction

The Agency for Healthcare Research and Quality requests clearance from the Office of Management and Budget (OMB) under the Paperwork Reduction Act of 1995 to collect information on the, “Use of CAHPS information during PCMH Transformation: A CAHPS PCMH Items Demonstration Study.”

A. Justification

A1. Necessity of Information Collection

The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see http://www.ahrq.gov/hrqa99.pdf), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:

1. Research that develops and presents scientific evidence regarding all aspects of health care;

2. The synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and

3. Initiatives to advance private and public efforts to improve health care quality.

Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.

The patient-centered medical home (PCMH) is a model for delivering primary care that is patient-centered, comprehensive, coordinated, accessible, and continuously improved through a systems-based approach to quality and safety.

As primary care practices across the United States seek National Committee for Quality Assurance (NCQA) recognition as patient-centered medical homes (PCMH), they can choose to administer the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Clinician and Group (CG-CAHPS) survey with or without the CAHPS PCMH supplemental item set (AHRQ, 2010; Hays et al, 2014; Ng et al., 2016; Scholle et al., 2012). NCQA offers a special patient experience distinction to practices that opt to use the PCMH CAHPS items set in their CG-CAHPS survey tool. While over 11,000 practices, representing an estimated 15-18% of primary care physicians, are currently recognized for PCMH by NCQA (NCQA, 2015), fewer than 3% of them submit patient experience surveys to NCQA when applying for recognition under NCQA's PCMH recognition program. Of the over 11,000 practices recognized for PCMH by NCQA, 300 hold the special patient experience distinction and are administering the CAHPS PCMH supplemental item set¹. The special patient experience distinction is designated for a one-year period, and 242 additional practices have previously achieved the special patient experience distinction but no longer have it.

Despite the rapid movement toward PCMH primary care transformation and the increasing use of PCMH CAHPS items, little is known about why practices choose to administer the CAHPS PCMH supplemental item set and the ways in which practices are using these CAHPS data and the CAHPS PCMH supplemental item information (about access, comprehensiveness, self-management, shared decision making, coordination of care, and information about care and appointments) to understand and improve their patients’ experiences during PCMH transformation. The CAHPS PCMH Items Demonstration Study will investigate:

• How practices assemble and select items for inclusion in their patient experience surveys (e.g. core, PCMH, other supplemental, and homegrown items),

• How practices across the U.S. use CAHPS and the PCMH item set during PCMH transformation,

• Primary care practice leaders’ perspectives on NCQA PCMH Recognition and CAHPS Patient Experience Distinction,

• Effects of changes made during PCMH transformation on patient experiences reported on CAHPS surveys and any CAHPS PCMH items, and

• Associations between PCMH transformation and patient experience scores

To achieve the goals of this project the following data collections will be implemented:

1) Office Manager Interview administered via phone about practice characteristics to describe the type of practices in the study, understand whether and how these practice characteristics influence PCMH transformation and patient experience, and to examine potential response bias (Attachment A and B).

2) Physician/Clinical Leader Interviews administered via phone with the lead PCMH clinical expert about the details, decisions and processes of designing their patient experience survey, PCMH transformation, NCQA PCMH Recognition and CAHPS Patient Experience Distinction and their use of patient of patient experience data during the transformation process (Attachment C - G).

3) PCMH-A Assessment Tool to be completed by the lead PCMH clinical expert via fax or email (before or after the interview on the standardized form) to collect validated metrics on the “PCMH-ness” of the practice (Attachment H).

4) CAHPS Patient Experience Data Files, which are patient-level de-identified CAHPS patient experience data covering the period of PCMH transformation for the participating practice. These data are collected independently of this study by the practice (or network) via their current vendor. With the PCMH clinical expert (or a person they designate who handles their data) in each of the participating practices to securing the required data use agreements to assure secure transmittal of these CAHPS data files to RAND. These data will be used to understand practices’ CAHPS patient experience trends and associations with PCMH implementation during practices’ PCMH journey.

This study is being conducted by the RAND Corporation, a grantee under the AHRQ CAHPS cooperative agreement, pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).

A2. Purpose and Use of Information

Characterizing primary care practices’ use of CAHPS and CAHPS PCMH items will provide important insight into the activities practices conduct during PCMH transformation to improve patient experience scores. This information may be useful in supporting practices that lag behind their peers, learning from practices with outstanding records of patient experience, and providing recommendations that may be used to refine the content of the CAHPS survey items.

A3. Use of Information Technology

The CAHPS PCMH Items Demonstration Study will employ some information technology for data collection purposes. The primary mode of data collection will be phone—all the primary care practices in the sample will be contacted first by phone. After the initial phone call, participation and follow-up information will be sent by email, fax or Fed Ex, according to the providers’ preferences. Participation includes Office Manager Interview (2 minutes) collected via phone during the recruitment of practices, Physician/PCMH Leader Interviews collected via phone (40 minutes), completion of a PCMH-A Assessment Tool (10-15 minutes) either on paper (returned by Fed Ex or fax) or electronically via a fillable PDF file (returned by email), and obtaining de-identified CAHPS Patient Experience Data Files for those practices that administer the CAHPS survey.

A4. Identifying Duplication

The components of this data collection effort are designed to gather the data necessary to assess primary care practices’ uses of CAHPS patient experience data during PCMH transformation. No similar data collection is currently being conducted. The proposed information collection does not duplicate any other effort and the information cannot be obtained from any other source.

A5. Impact on Small Businesses

No small businesses or other small entities will be significantly impacted by this information collection.

A6. Consequences of Less Frequent Data Collection

This Supporting Statement requests clearance for a one-time data collection.

A7. Special Circumstances

There are no special circumstances associated with this information collection request.

A8. CMS Federal Register Notice

The 60-day Federal Register notice Vol. 81, No. 149, page 51199 (Attachment I) was published on August 3^rd, 2016.

A9. Respondent Payments or Gifts

A range of non-monetary incentives will be offered to study participants.  These non-monetary benefits include:

         Opportunity to participate in a PCMH patient experience project

         Opportunity to participate in a project funded by AHRQ

         Potential to improve the collection of patient experience data during PCMH transformation process

         For those practices that submit CAHPS patient experience data files: a report comparing the participating practice’s CAHPS patient experience trend data to all the other sites who submitted CAHPS patient experience data participating in the study

In addition, we request post-paid honoraria. We are requesting an honorarium of $75 for each interviewee’s time to complete the Physician/PCMH Leader Interview (Attachment G) (n=176 for a total of $13,200). Pre-testing of similar efforts with primary care physicians indicated that such an honorarium is necessary to obtain participation. There is a long history supporting the use and effectiveness of honorarium and incentives to promote response/participation (Hogan and LaForce, 2008; James et al 2011), and the honoraria is one of several tools the team will employ to maximize the participation rate. The team’s experience from pretesting of these exact protocols is that practices and clinicians will be motivated to participate because of their interest in improving the collection of patient experience data during the PCMH transformation process but providing a $75 honorarium per interview to those who, in addition to performing their jobs, take the time to be interviewed (total time of 40 minutes) will yield greater participation and net a more diverse representation of practices. Research to test the effect of incentives on physician response rates (James et al 2011) and a review article specific to incentives influence on physician response rates (Kellerman et al 2001) demonstrate that offering an incentive to physicians can increase response rates by about 10%.

A10. Assurance of Confidentiality

All persons who participate in this data collection will be assured that the information they provide will be kept private to the fullest extent allowed by law. Informed consent from participants will be obtained to ensure that they understand the nature of the research being conducted and their rights as participants. Participants who have questions about the consent statement or other aspects of the study will be instructed to call the RAND principal investigator and/or the administrator of RAND’s Institutional Review Board (IRB). Participants will receive informed consent and confidentiality information via the recruitment and interview process as found in Attachments A-H. The confidentiality paragraph used in the Physician Interview is in the box on the next page. Data use agreements will be signed for secure transfer of CAHPS data files.

Confidentiality Script for Interview

Go through informed consent script. (2 minutes)

Explain general purpose and format of the interview, and RAND’s role in this effort:

• We are researchers from RAND, a nonprofit research institution.

• We are doing a research project funded by Agency for Health Care Research and

Policy (AHRQ), which is part of the federal government’s Department of Health

And Human Services

• Project is intended to learn about your clinic’s experiences becoming a PCMH

and how that may influence patients’ experiences with health care.

• FOR THOSE PARTICIPATING IN THE CAHPS DISTINCTION ONLY:

We are interested in your experience obtaining the NCQA Patient Experience

Distinction and administering CG CAHPS surveys plus CAHPS PCMH items.

* Each participating practice will receive a report of the practice’s mean case

mix adjusted CG-CAHPS scores and CAHPS PCMH items (if relevant)

compared to mean scores of the participating practices with current CAHPS

distinction, past CAHPS distinction, and Recognition only practices

in the study. No practices will be identified in these reports.

• We want to learn from you and your experiences. Our discussion will take about

40 minutes. If you need to take a break at any time, please let me know.

• Data will be reported so that neither you nor your organization can be identified.

• Your participation is voluntary, and you can decline to discuss any topic that we

raise. We will not report your participation to anyone outside the research team.

• You will receive $75 honorarium for completing the interview today

We would like to record this discussion for note-taking purpose only. We will

destroy the tape as soon as notes have been completed. You can still participate

if you do not want to be taped

May we record this discussion?

[With permission, turn on the audio recorder.]

If you have any specific questions about this project, you may contact:

Ron Hays, Ph.D. Principal Investigator- CAHPS RAND 1776 Main Street Santa Monica, CA 90407 Telephone: 310-794-2294 FAX: 310-794-0732 Email: hays@rand.org

Jim Tebow Human Subjects Protection Committee RAND P.O. Box 2138 Santa Monica CA 90407-2138 Telephone: 310-393-0411 x7173 FAX: 310-393-4818 Email: tebow@rand.org

Turn on recorder. Do you have any questions before we begin?

The study has been approved by IRB and has an approved Data Safeguarding Plan to further ensure the privacy of the information that is collected. RAND will assign a data identifier (ID) to each respondent. All electronic files directly related to the administration of the study will be stored on a restricted drive of a secure local area network. Access to data is limited to those employees working on the project. Additionally, files containing interview, assessment, or CAHPS data and information revealing sample members’ individual identities will not be stored together on the network. The information is stored together within the Survey Research Group during project administration and until the Principal Investigator requests destruction of direct identifiers. No single file will contain both a member’s response data and his or her contact information. RAND staff will destroy participant contact information once all data are collected and the associated data files are reviewed and finalized by the project team. Files containing contact information for requirement and data collection may be stored on staff computers or in staff offices following the procedures that have been reviewed and approved by RAND’s IRB (as of March 16, 2016).

A11. Sensitive Questions

The Office Manager recruitment script (Attachment A) and Office Manager Interview (Attachment B), Physician recruitment script (Attachment C), cover letter (Attachment D.1 and D.2), AHRQ endorsement letter (Attachment E) and info sheet (Attachment F), the Physician/PCMH Leader Interview protocol (Attachment G), or PCMH-A Assessment Tool (Attachment H) do not include any questions of a sensitive nature.

A12. Burden of Information Collection

Table 1 shows the estimated annualized burden and cost for the respondents' time to participate in this data collection. These burden estimates are based on tests of data collection conducted on nine or fewer entities. As indicated below, the annual total burden hours are estimated to be 179.8 hours. The annual total cost associated with the annual total burden hours is estimated to be $16,004.05.

Table 1 shows the estimated annualized burden for the respondents' time to participate in this data collection. The CAHPS PCMH Items Demonstration Study will recruit 150 practices including the participating practices’ office managers and one physician/lead PCMH clinical expert. We will recruit and administer the Office Manager Interview by phone to all office managers who are reached by phone. We estimate that we will be able to reach roughly half of the practices with current and past CAHPS distinction and only need to contact 150 practices with only PMCH recognition to meet our targeted sample. This yields interviews with 450 office mangers (Attachment A and Attachment B), recruiting all sampled physicians/PCMH leaders (Attachment C) sending them a recruitment packet that includes a cover letter (Attachment D), an AHRQ endorsement letter (Attachment E), and an info sheet (Attachment F) and then administer the Physician/PCMH Leader Interview protocol questions (Attachment G) by phone to 176 physicians/PCMH leaders, and 150 physicians/PCMH leaders will self-administer the PCMH-A Assessment Tool (Attachment H).

We have calculated our burden estimate for Office Manager Interview asked during physician/PCMH leader recruitment using an estimate of 3-5 questions a minute as the Office Manager Interview questions are closed-ended survey questions. The Office Manager Interview contains 10 questions and is estimated to require an average of 5 minutes (script + interview questions); this estimate is supported by the information gathered during a pilot of these questions. For the Physician/PCMH Leader Interview, we have calculated the burden estimate to require an average of 40 minutes per interview. For the PCMH-A Assessment Tool, we calculated our burden using a conservative estimate of 4.5 items per minute. Prior work suggests that 3-5 items on an assessment tool can typically be completed per minute, depending on item complexity and respondent characteristics (Berry, 2009; Hays & Reeve, 2010). The PCMH-A Assessment tool contains 36 items and is estimated to require an average completion time of 10 minutes.

Participating practices will be asked to submit any available CAHPS Patient Experience data files (e.g. submission of de-identified data including a data dictionary via encrypted transfer) for the period of time covering their NCQA PCMH Recognition history. Each practice will have an average estimate of 3 CAHPS Patient Experience data files to submit per one submission, which we based on the average number of years of PCMH history of the sample. In addition, we conservatively estimate that half of the control practices (25/50) administer CG-CAHPS data, as this percentage is unknown; while 90% of the participating current and past CAHPS practices (90/100) will submit CAHPS data, yielding 115 submissions of CAHPS patient experience data files. As indicated below, the annual total burden is estimated to be 179.8 hours.

The total annual cost burden for the project is estimated to be $16,004.05.

Exhibit 1.  Estimated annualized burden hours

Data Collection Task	Number of respondents	Number of responses per respondent	Hours per response	Total burden hours
Office Manger Interview (Attachment A plus B)	450	1	5/60	37.5
Physician/PCMH Leader Interview (Attachment C plus G)	176	1	40/60	117.3
PCMH-A Assessment Tool (Attachment H)	150 (Same Physicians as above)	1 (same person as above)	10/60	25
Total	776	NA	NA	179.8

+ The same respondent completes the Physician Interview and PCMH-A Assessment Tool and submits the CAHPS Patient Experience Data Files.

Exhibit 2. Estimated annualized cost burden

Data Collection Task	Number of requests	Total burden hours	Average hourly wage rate*	Total cost burden
Office Manger Interview	450	37.5	$57.44a	$2,154.00
Physician/PCMH Leader Interview	176	117.3	$97.33b	$11,416.80
PCMH-A Assessment Tool	150 (Same Physicians as above)	25	$97.33b	$2,433.25
Total	776	179.8	NA	$16,004.05

+ The same respondent completes the Physician/PCMH Leader Interview and PCMH-A Assessment Tool..

*Occupational Employment Statistics, May 2015 National Occupational Employment and Wage Estimates United States, U.S. Department of Labor, Bureau of Labor Statistics. http://www.bls.gov/oes/current/oes_nat.htm

^aBased on the mean wages for General and Operations Managers, 11-1021 within Healthcare Support Occupations, the occupational group most likely tasked with completing the Office Manager Questions.

^aBased on the mean wages for Physicians and Surgeons, 29-1060, the occupational group most likely tasked with completing the Physician Interview, PCMH-A Assessment Tool, and submitting the CAHPS Patient Experience Data Files.

13. Capital Costs

There are no capital costs.

14. Estimates of Annualized Cost to the Government

The total cost of this data collection to the government is $179,000 total; $179,000 in already funded-grant costs and $0 in government personnel costs. The data collection is a one-time collection. Exhibit 3 shows a breakdown of the total cost and annualized cost for the data collection and data processing and analysis led by the grantee. Exhibit 4 shows a breakdown of the government personnel costs related to this data collection effort.

Exhibit 3.  Estimated Total and Annualized Cost

Cost Component	Total Cost	Annualized Cost
Project Development	NA	NA
Data Collection Activities	$82,000	$82,000
Data Processing and Analysis	$50000	$50000
Publication of Results	$5,000	$5,000
Project Management	$42,000	$42,000
Overhead	0	0
Total	$179,000	$179,000

Exhibit 3b. Federal Government Personnel Cost

Project Officer GS- 15 Step 5	5%	$7,466
Health Scientist Administrator GS 13 Step 5	5%	$5,371
Total		$12,837

Annual salaries based on 2017 OPM Pay Schedule for Washington/DC area:

https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2017/DCB.pdf

A15. Program Changes or Adjustments to Annual Burden

This is a new information collection request.

A16. Tabulation and Publication of Results

For planning purposes, we anticipate data collection will begin in late May 2017 and conclude in September 2017. Analyses of these data will occur during October and November 2017 to contribute to drafting a summary article anticipated to be completed in December 2017, revised and submitted to a peer-reviewed journal by January 2018.

Analysis will be primarily descriptive, indicating the types of activities primary care practices engage in to make use of CAHPS, CAHPS PCMH Items and other patient experience data.

Table 2: Timeline of Tasks and Publication Dates

Activity	Proposed Timing of Activity
Prepare field materials	October 2016–December 2016
Recruit and schedule Interviews	May 2017–September 2017
Field Office Manager Interviews	May 2017–September 2017
Field PCMH-A Assessment Tool	May 2017–September 2017
Analyze data	October 2017–November 2017
Draft paper summarizing findings	December 2017
Revise and submit paper to peer-reviewed journal	January 2018

In addition to summarizing the overall findings of the CAHPS PCMH Items Demonstration Study, RAND will work with AHRQ to develop timelines for developing other products for broad dissemination of the results that may include additional peer-reviewed publications, webinars, podcasts, and other products. Such products will increase the impact of this work by exposing the results to a broader audience of primary care practice leaders and policymakers. The publication of the overall findings also may result in additional dissemination products such as press releases, open door calls, and other events.

A17. Display of OMB Expiration Date

The expiration date for OMB approval of this information collection will be displayed on the study documents.

A18. Exceptions to the Certification Statement

There are no exceptions to the certification statement identified in item 19 of OMB Form 83-I associated with this data collection effort.

References

Agency for Healthcare Research and Quality (AHRQ). (2010). CAHPS PCMH Fact Sheet. September 2010. AHRQ Publication No. 10-CAHPS003.

Berry, S. (2009) How To Estimate Questionnaire Administration Time Before Pretesting: An Interactive Spreadsheet Approach. Survey Practice, Vol 2(3).

Hays R.D., Berman L.J., Kanter M.H., Hugh M., Oglesby R.R., Kim C.Y., Cui M., Brown J. (2014). Evaluating the psychometric properties of the CAHPS Patient-centered Medical Home survey. Clin Ther. May; 36(5): 689-696.e1. doi: 10.1016/j.clinthera.2014.04.004. Epub 2014 May 5.

Hays, R. D., & Reeve, B. B. (2010). Measurement and modeling of health-related quality of life. In J. Killewo, H. K. Heggenhougen & S. R. Quah (eds.), Epidemiology and Demography in Public Health (pp. 195-205). Elsevier.

Hogan, S.O. & LaForce, M. (2008). Incentives in Physician Surveys: An Experiment Using Gift Cards and Checks. ASA Proceedings, Survey Research Methods section. 4179-4184. Accessed on June 14 2016 at: http://www.amstat.org/sections/srms/proceedings/y2008/files/hogan.pdf

James, K.M., Ziegenfuss, J.Y., Tilburt, J.C., Harris, A.M., Beebe, T.J. (2011). Getting Physicians to Respond: The impact of incentive type and timing on physician survey response rates. Health Serv Res. Feb; 46(1 Pt 1): 232-242. doi:  10.1111/j.1475-6773.2010.01181.x

Kellerman, S.E. & Herold, J. (2001). Physician Response to Surveys: A review of the literature. American Journal of Preventative Medicine. 61-67.

NCQA.  (2015). Internal data from NCQA's Monthly Progress Reports.  Recognition Programs Operations.  Washington, DC: NCQA.  July 2015

Ng, J.H., Henry, E., Oberlander, T., Shi, P., Scholle, S.H. (2016). Shortening a Patient Experiences Survey for Medical Homes. Healthcare, 4, 1.

Scholle, S.H., Vuong, O., Ding, L., Fry, S., Gallagher, P., Brown, J.A., Hays, R.D., & Cleary, P.D. (2012). Development of and field-test results for the CAHPS PCMH survey. Medical Care, 50, S2-10.

List of Attachments

Attachment A: Script for Office Manager

Attachment B: Office Manager Interview Practice Questions

Attachment C: Script for Physician/PCMH Leader

Attachment D: Physician/PCMH Leader Cover Letter

Attachment E: AHRQ Endorsement Letter

Attachment F: Information Sheet Sent with Cover Letter

Attachment G: Physician/PCMH Leader Interview Protocol

Attachment H: PCMH-A assessment

Attachment I : Federal Register Notice

1 As of 2015, the most recent year of information that is available.