SUPPORTING STATEMENT
Part B
Patient-Reported Health Information Technology and Workflow
January 31March 28, 2013
Agency of Healthcare Research and Quality (AHRQ)
B. Collections of Information Employing Statistical Methods 3
1. Respondent Universe and Sampling Methods 3
2. Information Collection Procedures 6
3. Methods to Maximize Response Rates 8
B. Collections of Information Employing Statistical Methods 3
1. Respondent Universe and Sampling Methods 3
2. Information Collection Procedures 5
3. Methods to Maximize Response Rates 6
4. Tests of Procedures 7
5. Statistical Consultants 7
A purposive sample of six study sites has been selected for this case-study research project. This sample size is consistent with case study methodology, which typically employs small sample sizes, ranging from 4 to 10 cases (Creswell & Plano Clark, 2011; Morse, 1994). The six study sites will be selected from three ambulatory care networks: the Alabama Practice Based Research Network (APBRN), the Dean Health System outpatient clinics in Wisconsin, and the Wisconsin Research Education Network (WREN). The APBRN is a network comprised of 90 physicians across 75 practices, who collaborate to implement practice-based studies. The Dean Health System includes 63 physician practices throughout Wisconsin. The WREN network is comprised of 40 clinics in 38 Wisconsin communities that participate in innovative research and quality improvement projects. All three networks have extensive experience with multi-site studies. The physician practices within the three networks vary in size, in the use of health information technology (IT), and in the types of technology used to elicit information directly from patients.
This project will examine three types of health IT (e-formpatient portals, secure messaging, and e-formspatient portal) in two different sizes of physician practices (small and medium). A total of six study sites will allow exploration of a range of workflows associated with the three types of patient-reported information technology as well as a range of workflow problems barriers and facilitators that can be linked to the sociotechnical context of each site (Morse, 1994; Sandelowski, 1995).
All study sites are small to medium-sized ambulatory care practices with no more than 10 physicians per practice. Study sites that use different health IT applications to elicit and use information from patients were selected, to demonstrate a variety of technologies and workflows for soliciting, receiving, processing, displaying, and acting upon patient-reported information. The types of information, and the technologies used to collect and manage this information, will be factors in the analysis. Study sites were intentionally selected to vary based on size and on the technologies and types of patient information in use, in order to understand common themes and differences in workflow. Final site selection balanced key attributes, including:
Size (small vs. medium)
Demographics of the patient population served by the practice (race, gender, income)
Geography; at least two distinct regions of the country with different population demographics and health care markets, to understand any impact of these environmental factors. Alabama and Wisconsin were selected for their differences on these parameters; Alabama has a large minority and low-income population and modest managed care penetration; Wisconsin has a more homogenous, predominantly Caucasian population and has a high managed care penetration. In addition, health IT is generally more pervasive in Wisconsin than in Alabama.
Types of technology employed to capture patient-reported information
At least two sites that use a patient portal (sometimes referred to as [electronic] personal health records or PHRs; allows patients to view portions of their medical records [e.g., laboratory test results] and supports other health-related tasks such as making appointments or requesting medication refills. Some patient portal applications exist as stand-alone Web sites; other portal applications are integrated into an existing electronic health record [EHR] system)
At least two sites that use secure messaging with patients (use of secure e-mail between patients and clinicians, typically using the secure messaging functionality in the EHR and/or patient portal)(email between patients and clinicians)
two sites that use a patient portal, in which patients can enter information (which may or may not link directly into their medical records)
At least two sites that useing e-forms (surveys that are administered using computerized media [e.g., tablets, laptops] to collect information from patients using pre-formatted forms before or during patient visits)computerized media (e.g., tablets, laptops) to collect information from patients using pre-formatted forms before or during outpatient visits
Study sites were selected using a 3 (types of health IT) x 2 (small and medium-sized practices) x 2 (Alabama and Wisconsin) design, striving to recruit small and medium-sized practices that use different types of health IT for patient-reported information, and are either in Alabama or in Wisconsin. Sites may have more than one type of health IT. These parameters yielded the study sites depicted in Exhibit 5 below.
Exhibit 5. Study Site Attributes
Practice # |
Health IT for patient-reported information |
Size |
State |
1 |
e-forms |
Small |
AL |
2 |
Patient portal, secure messaging, e-forms |
Small |
AL |
31 |
Patient portal |
Medium |
AL |
42 |
Patient portal |
Medium |
AL |
5 |
Patient portal, secure messaging, e-forms |
Medium |
WI |
653 |
Secure messaging |
Small |
WI |
Types of respondents at each site will include:
A Practice Manager, the individual in each practice who manages day-to-day operations
A Physician Leader, who is most knowledgeable about health IT and health IT implementation.
Clinicians (which include physicians, nurse practitioners, physicians assistants, nurses)
Office staff
Patients
These respondents will be involved in the following types of data collection:
Each practice has one Practice Manager; this individual will be asked to complete a Pre-Visit Questionnaire, will be interviewed about contextual information about the practice and use of health IT, and will participate in an initial walk-through or tour of the practice (Practice Tour); there will be no sampling.
Each study site will be asked to identify one Physician Leader, who is most knowledgeable about health IT and health IT implementation. The Physician Leader will be interviewed about the sociotechnical context of the practice, and, in particular, the specific health IT system as it is used by clinicians in the practice.
At each study site the clinicians and office staff who are involved in capturing or using patient-reported information will be asked to participate in a direct observation during which researchers watch how health IT is used, followed by an individual interview. Since all study sites are small – to medium-sized practices, all clinicians and office staff who are involved in capturing or using patient-reported information will be included. In the unlikely event that there are more individuals at a study site than can be observed and interviewed during the 3-to 4 day site visit, priority will be given to those with the most responsibility for working with patient-reported information.
All clinicians and office staff in the practice will be asked to complete surveys; there will be no sampling. Respondents will be offered the option of web-based or paper surveys, and they will be asked to complete the surveys during or within one week after the site visit at their practice.
In each practice, five to seven patients who report information using health IT will be interviewed. The patients selected for interviews will be a convenience sample of individuals who meet the following selection criteria:
Have appointments scheduled with a study site clinician during the three to four day site visit
Have experience reporting information to their clinical care team, using one or more of the relevant technologies (see Exhibit 5 above)
Vary by age and education (to ensure that not all are young, educated, and advanced computer-users)
Staff at each study site will be asked to review their scheduled appointments for the three to four days during which the data will be collected onsite, and recommend patients who meet these criteria. The first five patients who keep their appointments and agree to be interviewed will be selected.
After each site visit, a Workflow Process Map(s) will be prepared and the Practice Manager and Physician Leader will be asked to review the accuracy of the map(s) for their practice.
There will be one case study conducted at each of the six study sites; no repeated data collection is planned.
Each study site will be asked to identify their Practice Manager, who will help to schedule the case study. Each case study will include the following:
Pre-Visit Questionnaire: The Practice Manager will be asked to complete a brief questionnaire, two weeks prior to the site visit (Attachment B). The paper and pencil questionnaire will be mailed to the Practice Manager and asked that it be returned prior to the site visit. The information collected will be useful for planning the site visit.
Interviews with Practice Manager and Physician Leader: At each study site, the Practice Manager and Physician Leader will each be asked to participate in a one hour, in-person interview during the site visit (Attachments D and E, respectively).
Observations of Clinicians and Office Staff: Observations of both clinicians and office staff will be recorded on the Observation Form (Attachment F). The observations will be used to create a detailed Workflow Process Map(s). Patients will also be asked if they may be observed during their visit with the clinician. They will be provided with an information sheet that explains the purpose of this activity, and will be assured of confidentiality.
Interviews with Clinicians and Office Staff: Following observations, each clinician who was observed will be interviewed in person for up to one hour, and each office staff person who was observed will be interviewed in person for up to one hour. The same interview guide will be used for both clinician and office staff interviews (see Attachment G).
Survey of Clinicians and Office Staff: All staff in each study site will be asked to complete a survey (Attachments H and I). Surveys will be distributed online using staff emails supplied by the Practice Manager. If not all potential respondents have individual e-mail addresses at work, a paper-and-pencil version of the survey will be distributed during the site visit. If paper-and-pencil surveys are used, these will be individually distributed to each of the clinicians and office staff. This distribution mechanism has achieved high response rates in previous research (Hoonakker et al., 2011, Hoonakker et al., 2008). Because some clinicians or office staff may not be available (e.g., on vacation or sick) during the site visit, if necessary, surveys and stamped envelopes will be left for those absent respondents to complete and mail back.
Patient Interviews: Five patients will be interviewed in person at each small practice and up to seven at each medium-sized practice. Patients will be identified by practice staff, and approached by researchers before or immediately after their office visits. Patients will be invited to participate in an interview and will receive an invitation sheet that explains the project and the interview to be conducted (Attachments R and S). Patients will receive a verbal explanation of the interview that is being requested, and an explanation of how their experiences with reporting information using health IT will help to improve this process in the future. Patients will be assured of confidentiality and will have an opportunity to have their questions answered.
Post-Visit Follow-up to Review the Workflow Process Map(s): The Practice Manager and Physician Leader from each study site will each be asked to participate in a one hour, phone interview (Attachment L) following the site visit. The Practice Manager and Physician Leader will be contacted via email prior to the interview (Attachment K) and will be sent the workflow process map(s) developed following site visit at each practice for review during the interview. Hand-written notes of the interview will be taken.
All in-person interviews will be audio-recorded if agreed upon by the study participant. If the study participant does not agree to be audio-recorded, hand-written notes of the interview will be taken. The audiorecordings or notes will then be transcribed by a professional transcriptionist and uploaded into NVivo for analysis. Survey respondents will be offered the option of online or paper surveys to complete, and will be encouraged to do so during the week of the site visit. Once completed, the surveys will be uploaded to a database to be analyzed.
The study sites will be subcontractors and their participation in the research will be a condition of their subcontracts. The in-person interviews and observations will be approved by the study site leadership in advance, and all of their staff will be informed of the leadership’s decision to serve as a study site.
Patients will receive a $10 gift card after completing an interview.
Survey response will be maximized by offering options for online or paper surveys, repeated email reminders, and encouragement by their practice leadership.
The surveys will bewere pretested with up to nine respondents, to refine wording of questions, and ensure that answer categories are exhaustive and mutually-exclusive, and to minimize any ambiguity in interpretation of questions or answer categories.
The workflow observation forms have been used in previous studies and are sufficiently flexible to be suitable for any ambulatory practice; they do not require additional pre-testing. The Pre-Visit Questionnaire and interview guides are modeled based on those used in previous studies and do not require additional pre-testing.
The Clinician and Office Staff Survey was pre-tested with three respondents: a physician, a medical social worker, and a medical office receptionist. The survey was pre-tested to refine wording of questions, ensure that answer categories are exhaustive and mutually-exclusive, and minimize ambiguity in interpretation of questions or answer categories.
Results of pre-testing showed that respondents had difficulty understanding the health information technology terms, such as EHR, HIE, and patient portal, used in the survey. Based on these results, additional background information, examples, and/or definitions were added to the survey introduction, and in several survey questions, and in the introduction for Section F. Further, the order of sections C, D, and E were rearranged so that questions related to patient portals are presented first, then questions about secure messaging, and finally questions about e-forms. Pre-testing also showed that the question about health IT training was too broad, and it was revised to be specific to the types of health IT that practices may use. Finally, wording changes were made in some of the questions and/or instructions. In Section A, the titles of certain individuals were changed (e.g., Primary Care Physician was replaced with Physician). In Section B, response categories were reversed to make them more consistent with the rest of the questionnaire (i.e. response categories are now arranged from negative to positive). Specific answer choices were added for Question 5. Questions with the word “data” were replaced with the word “information” and the word “retrieve” was replaced with the word “receive”. In the Section about secure messaging, the instructions were changed to include instructions for Question 2 (“If you provide direct care (physicians, physician assistants, nurse practitioners, nurses, and medical assistants) please fill out Question 2 and the questions in Table A below”). Results of pre-testing also showed that not all questions in Sections C, D, and E were applicable to non-clinicians. Therefore, a response category for, “Not applicable,” was added. Question 1 and Question 2 in Section G were changed to be asked at the practice level instead of at the personal level (i.e. “How satisfied are you with the care provided at your practice?”).
This study does not rely on statistical sampling, analysis of response bias, or similar methods. Analysis of observation and interviews will be entirely qualitative. Survey data will be pooled across the six study sites and analyzed at the level of respondents’ job category (e.g., physician, nurse, patient).
The specific data collection methods to be used in this study have been successfully used in multiple studies of health IT workflow by the Principal Investigator, Pascale Carayon, Ph.D. and her colleagues Peter Hoonakker, Ph.D. and Randi Cartmill, MS. Based on prior successful research using these methods, no additional statistical consultation is considered necessary.
AHRQ’s contractor, Abt Associates and Abt’s partner organization, the University of Wisconsin-Madison, will be responsible for overseeing the recruitment of participants, conducting all of the data collection, and analyzing and reporting the findings. The project director is Andrea Hassol. Ms. Hassol can be reached by phone at 617-349-2488
Andrea or by email at hassol@abtassoc.com. Dr. Pascale Carayon will oversee the data collection and analysis processes. Dr. Carayon can be reached by phone at 608-265-0503 and by email at carayon@engr.wisc.edu.
References
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and Conducting Mixed Methods Research (2nd ed.). Los Angeles, CA: Sage Publications.
Hoonakker, P. L. T., Carayon, P., Douglas, S., Schulz, K., Walker, J., & Wetterneck, T. B. (2008). Communication in intensive care units. In L. L. Snelwar, F. L. Mascia & U. B. Meontedo (Eds.), Organizational Design And Management-IX (pp. 715-721). Santa Monica, CA: IEA Press.
Hoonakker, P.L.T., Carayon, P., McGuire, K., Khunlertkit, A., Wiegmann, D., Alyousef, B. Xie, A. & Wood, K. (In press) Motivation and Job Satisfaction of Tele-ICU Nurses. Accepted for publication by the Journal of Critical Care.
Hoonakker, P. L. T., Carayon, P., Khunlertkit, A., Mcguire, K., & Wiegmann, D. (2011). Case Study Research: An example from the tele-ICU. In M. Goebel, C. J. Christie, S. Zschernack, A. I. Todd & M. Mattison (Eds.), Organizational Design And Management (ODAM) X (Vol. I, pp. 121–127). Grahamstown, South Africa: IEA Press.
Morse, J. M. (1994). Designing funded qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research. Thousand Oaks, CA: Sage.
Sandelowski, M. (1995). Sample size in qualitative research. [10.1002/nur.4770180211]. Research in Nursing & Health, 18(2), 179-183.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Andrea Hassol |
| File Modified | 0000-00-00 |
| File Created | 2021-01-29 |