Fast Track Supporting Statement

Request for Approval under the “Generic Clearance for the Collection of Routine Customer Feedback” (OMB Control Number: 0935-0179)

T ITLE OF INFORMATION COLLECTION: Provider Interview Guide

PURPOSE: Following the completion of app usage by 10 patients at each site, semi-structured interviews will be scheduled and conducted by telephone with one provider at each of the 18 practice sites. During this interview, providers will be shown the usage statistics from the process worksheets completed for their practice. Topics of the interview will address their experience with using the PRO data and will include: 1) how they actually used PRO data to inform clinical decision making, 2) reasons for use or non- use of PRO data, 3) how can the process be optimized, and 4) general satisfaction, preferences, and barriers.

DESCRIPTION OF RESPONDENTS: We will recruit one provider from each pilot testing practice site for a total of 18 providers for this data collection. Each provider will have seen patients who used the app prior to a clinical encounter.

TYPE OF COLLECTION: (Check one)

[ ] Customer Comment Card/Complaint Form [ ] Customer Satisfaction Survey

[ ] Usability Testing (e.g., Website or Software) [ ] Small Discussion Group

[ ] Focus Group [ X] Other: _Interview____

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name:___Alexandra Burn_____________________________________

To assist review, please provide answers to the following question:

Personally Identifiable Information:

1. Is personally identifiable information (PII) collected? [ ] Yes [X ] No

2. If Yes, will any information that is collected be included in records that are subject to the Privacy Act of 1974? [ ] Yes [ ] No

3. If Yes, has an up-to-date System of Records Notice (SORN) been published? [ ] Yes [ ] No

Gifts or Payments:

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [ ] Yes [ X ] No

BURDEN HOURS

Category of Respondent	No. of Respondents	Participation Time	Burden
Individuals	18	30/60	9
Totals			9

FEDERAL COST: The estimated annual cost to the Federal government is $928.98_

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents

1. Do you have a customer list or something similar that defines the universe of potential respondents and do you have a sampling plan for selecting from this universe? [X ] Yes [ ] No

If the answer is yes, please provide a description of both below (or attach the sampling plan)? If the answer is no, please provide a description of how you plan to identify your potential group of respondents and how you will select them?

Pilot testing will occur across 18 different clinical sites. Participating test sites include a mix of MedStar practices, and practices from the Capital Area Primary Care Research Network (CAPRICORN) Practice Based Research Network, a network of 100 primary care providers across the Washington DC metro area committed to practice based research.

Administration of the Instrument

1. How will you collect the information? (Check all that apply)

[ ] Web-based or other forms of Social Media

[ ] Telephone

[ X ] In-person

[ ] Mail

[ ] Other, Explain:

2. Will interviewers or facilitators be used? [ ] Yes [X ] No

Please make sure that all instruments, instructions, and scripts are submitted with the request.

A

Form Approved
OMB No. 0935-0179
Exp. Date 11/30/2020

ttachment A. Provider Interview Guide

We are working on a study to learn more about your experience with the PRO app. We want to get some feedback from you on the things you like and dislike, and the things that can be improved.

Office Location:
Physician Name:
Physician Specialty:

Demographics Questions

1. What is your gender?

1. Male

2. Female

3. (Open field)

1. What is your primary specialty?

2. How many years have you been in practice, including residency?

   1. <5

   2. 5-10

   3. 11-15

   4. 16-20

   5. > 21

Interview Moderator Guide

PRO Data Usage

4. How many times have you reviewed PRO data since the beginning of this pilot?

   1. # of Patients

   2. # of instances (e.g., # of times per patient)

5. Are you currently using any other patient reported outcomes (PRO) data such as these to help inform your clinical care and decisions? Why/Why not?

6. How do the data generated from this PRO app compare to other patient reported outcomes data (and visualizations) that you use?

Process

7. Did you receive any training for the PRO survey?

   1. Was the training effective?

   2. How could the PRO survey training be improved?

8. Would you like any job aids or reference materials available to you for the PRO survey?

9. How did you access the PRO data?

1. Did you understand how to access the survey data? Was there any confusion/uncertainty? Did you require assistance?

10. Were you alerted when a patient sent data?

1. If so, how were you alerted?

11. How long does it take for you to access and review the survey data?

Public reporting burden for this collection of information is estimated to average 30 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-0179) AHRQ, 5600 Fishers Lane, # 07W41A, Rockville, MD 20857.

12. Do you have any concerns about the amount of time it took you to access and review patient data?

13. Do you have any concerns about the amount of time it would take you to access and review patient data during high patient volume times?

14. Have you experienced any issues while trying to access or review survey data (crashing, freezing, unavailable data)?

15. If you have questions about the PRO survey data, who do you ask?

16. Thinking about the overall process for collecting PRO data (i.e., patient obtains the tablet from HCP upon arrival to appointment, fill out the survey, return the tablet to the HCP before appointment, survey data pushed to the EHR), is there anything about the process that works well?

1. What are the challenges with this process?

2. How does the PRO process (data access & review) affect the clinical schedule/flow?

3. Did the data impact (positively or negatively) your clinical decision making?

4. Is there anything about the process that could be improved?

17. If there was the option for patients to complete the PRO survey on their phone or tablet at home prior to coming to the hospital/clinic before the visit, would you prefer patients to fill out the survey in the hospital/clinic (before the appointment) or at home?

    1. Do you foresee any problems if patients fill out the survey at home (compliance issues, workload demands placed on front desk staff to send out email links, potential lack of access to internet/technology to be able to complete the survey at home, etc.)?

18. Have you received any feedback from the front desk staff regarding the PRO survey process?

19. How long would you estimate it takes the front desk staff to initiate the PRO data collection process?

20. Have you received any feedback from patients regarding the PRO survey process?

    1. Did patients need assistance?

    2. Did patients comprehend survey content?

    3. Did patients understand how to access, fill out, and submit the survey?

    4. Were patients able to easily read the survey content (e.g., font, font size, layout)

    5. Did patients have any technology issues?

21. If you had to speculate, what kind of impact do the PRO surveys have on the patient experience (positive, negative, no impact)

22. How often did you or other practice staff need to assist patients with completing the PRO survey on the tablet?

Content

23. Why are the PRO data important? What is the purpose of the PRO data?

24. What do you do with the PRO data?

25. How do you feel about the content of the survey?

1. Were the questions applicable to your patients?

2. Were there any questions /data missing that you would have liked to have reported?

26. How easy or difficult was it for you to understand the content/questions in the survey?

1. Did you have any questions or confusion about the content/questions in the survey?

1. How did you resolve your confusion?

2. Did the survey provide any support or information that helped to resolve your question/confusion?

3. Is there any additional information that should be added to help someone resolve a question or any confusion (e.g., help text, customer service number for questions/issues)?

27. Did you find the survey data to be useful?

1. Did the data have any impact on the clinical care provided?

28. Are the data presented in a useful way?

29. Did the survey have any impact (good or bad) on the conversation between you and your patient during your appointment?

1. Did the conversation improve or worsen?

2. Was there any change in efficiency for the conversation?

3. Did the survey facilitate any unplanned conversation about your patient’s condition (e.g., did the survey data alert you to questions or concerns that you wanted to talk about with your patient that you were not previously anticipating)?

Overall

1. Is there anything else we should know about the PRO app or the process?

6