Supporting Statement Part B 2-26-2008

Justification for OMB Clearance for Paperwork Reduction Act

Part B

Feasibility of Secure Messaging for Pediatric Patients with Chronic Disease: Pilot Implementation in Pediatric Respiratory Medicine

February 21, 2008

Task Order Officer:

Robert Mayes

Senior Advisor

Center for Primary Care, Prevention & Clinical Partnerships

Agency for Healthcare Research & Quality

540 Gaither Road, Room 6131

Rockville, MD 20850

Tel. 301-427-1492

Table of contents

Collections of Information Employing Statistical Methods 3

1. Respondent universe and sampling methods 3

2. Information Collection Procedures 3

3. Methods to Maximize Response Rates 4

4. Tests of Procedures 4

5. Statistical Consultants 4

Part B: Collections of Information Employing Statistical Methods

B.1 Respondent Universe and Sampling Methods

Respondent Universe: The Yale University School of Medicine Pediatric Respiratory Medicine Clinic at Yale-New Haven Children’s Hospital has approximately 2300 visits per year by 900 patients. The clinic is staffed by 5 attending pediatric pulmonologists, 1 nurse-practitioner, 3 pediatric pulmonary fellows, and 2 nurses. Two-thirds of the visits are for patients with asthma. Eighty of the patients are followed for cystic fibrosis; these patients come to clinic frequently, sometimes weekly, usually every 1-3 months. Other patients with chronic diseases include 50 children with bronchopulmonary dysplasia and 100 children with sleep disorders. The remainder of the visits are for patients referred for secondary evaluations (e.g. for cough, wheeze, or pneumonia).

Patients and their guardians/parents will be invited to participate if they seek care at the Pediatric Respiratory Medicine Clinic. There are approximately 900 patients who seek care on a regular basis at the clinic on an annual basis. The clinic serves patients with a wide range of respiratory problems ranging from asthma, cystic fibrosis, and other pulmonary problems. Generally, the clinic serves patients from birth to 21 years of age. Of the approximately 900 patients who seek care at the Clinic annually, we anticipate approximately 300 will be enrolled during the study period at this single clinical site.

Recruitment Procedures: All patients receiving care from the Pediatric Respiratory Medicine Clinic will be offered enrollment to use the new secure messaging system. During enrollment, they will be asked whether or not they would like to participate in the study by completing a survey; their enrollment will not be impacted by their decision to participate or not in this study. In fact, families/patients will be sent a letter (see appendix) alerting them to the new secure messaging feature of the clinic. When patients come to the clinic for their appointments, clinicians will provide the patients and their families with a brochure that describes how to use the product and will discuss with them the possibilities for using the messaging system; they can also review this information online at a provided website address. Exam rooms will also have posters announcing the availability of clinical messaging. A computer kiosk in the waiting room will also be available for patients and their families to use a demonstration version of the system and electronically signup as desired, obtaining their username and password after enrollment.

Patients will also receive an information sheet explaining our desire to study how well secure messaging is adopted by patients, their families, and the providers of healthcare at the clinic. They will be invited to participate in anonymous surveys; enrollment is not contingent on whether or not they elect to participate in the study. A small select group of patients will be identified (as below in inclusion/exclusion criteria) for qualitative interviews. Signed consent after discussion of risks and benefits will be obtained from this small group (~20) of patients and families prior to their participation in the qualitative interviews.

Staff working at the Pediatric Respiratory Medicine Clinic will also be invited to participate at a regularly scheduled staff meeting. Participation of staff is important as their opinions and experiences with the secure messaging system will be invaluable. However, participation will be strictly voluntary and consent will be obtained prior to enrolling staff. Participation (or lack thereof) will not reflect upon any individual’s job performance, nor will those who chose to participate receive preferential treatment.

Inclusion/Exclusion Criteria: All patients receiving care at the Pediatric Respiratory Medicine Clinic are eligible for inclusion. There are no other inclusion or exclusion criteria; all patients are eligible to be enrolled to complete the surveys. Similarly, all staff working in the Pediatric Respiratory Medicine clinic are eligible to be recruited to the study.

Sampling Methods: For the e-messaging component of the study, 300 patients will be included using a convenience sample. For the patient/family qualitative interviews, a small subset of the 300 patients will be chosen by purposeful sampling (i.e. random selection with the goal to identify representative patients of different genders, ages, and disease types) and included only with signed consent.

For simplicity, a table on the number of entities in the sampling universe is provided.

Data Collection Activity	Patient/Family Sampling Universe	Number of Expected Respondents
Demographic Survey	900	300
E-Messaging	900	300
Qualitative Survey	900	30
Data Collection Activity	Provider Sampling Universe	Number of Expected Respondents
E-Messaging	10	10
Qualitative Survey	10	10
Pre-Intervention Provider Log	10	10
Post-Intervention provider Log	10	10

The data collection information for the demographic survey (see Attachment D) uses OMB approved questions for determining race/ethnicity and other similar variables.

B.2 Information Collection procedures

Purpose and Main Research Questions

Developing health information technology (HIT) techniques for communicating with patients is essential in the coming years both to take advantage of the growing technologies as well as to increase the efficiency of communication with patients. Secure messaging is one such technology. However, the impact of secure messaging on the patient-provider dyad has not been fully evaluated. In particular, when these systems are implemented for children and adolescent patients, there are special issues of safety that need evaluation. The goals of this project are to further the state of knowledge about secure messaging with providers of children with chronic illness.

Project Goal: To implement and to evaluate the impact of the secure messaging system called ClinicalMessenger^TM in the Yale University School of Medicine Pediatric Respiratory Medicine Clinic. This system has been purchased and is ready to be implemented.

Specific Aims of Evaluation:

Aim 1: To understand the content of what children, adolescents, and their parents will send as a secure message to their provider.

Information will be catalogued regarding the content of the messages, which can be used to develop a triage system for the messages. Qualitative interviews will be conducted with a purposeful sample of patients and their families to assess their attitudes towards messages.

Aim 2: To evaluate the impact of secure messaging with regard to provider-time spent, ED-utilization for medication refills, and qualitative satisfaction by the patients and clinicians.

Measures will be:

• E-mails and phone calls

• To assess provider time spent, a comparison will be made between the time providers spend on emails and phone calls to patients and their families pre-intervention versus time providers spend secure-messaging with patients and their families post-intervention.

• To assess ED-utilization for medication refills, a review of ED-utilization for patients in the clinic pre and post-intervention will be done.

• To assess satisfaction by the patients, qualitative interviews with patients and separately clinicians will be conducted to elicit feedback about the process and degree of satisfaction.

Statistical Methodology

This is a qualitative study rather than a quantitative one, as such; it is not possible to make any power calculations with the exception of the assessment of ED utilization for medication refills. It is planed to enroll approximately 300 patients at minimum over the course of the year and survey their satisfaction with use of the secure instant messaging compared to historical contact (phone calls) and contact initiated via other methods by the enrolled group and non-enrolled patients belonging to the clinic. All of the above mentioned providers will also be asked to voluntarily participate in the qualitative evaluation of the project.

Demographic Survey Forms: A short (14-item) survey that elicits information on race, income, medical condition, internet access and current provider contact habits. A copy of this questionnaire can be found in Attachment D.

Content Analysis of e-messages: A full audit trail of number, type of messages, and message participants is saved on the Kryptiq SQL database. Queries can be easily run off this database to generate reports of statistical usage of the clinical messaging system. A report will be written to pull this audit information together with the actual message content information that is saved directly in the EMR (Centricity) Oracle database. Using this tool, the actual message contents and audit trails for all communication between patients and providers during the period of March – June 2008 will be reviewed.

Qualitative Interviews: Qualitative interviews with the patients and their families will be conducted to assess their attitudes towards electronic messaging. The interviews will be performed by an investigator, who will not be involved in the medical care of the patients. Interviews will take place either at the clinic, on the phone, or at a place preferred by the patient. The interviews will use a standardized set of open-ended questions with probes to elicit detail and will be tape-recorded and transcribed by an independent transcriptionist. (See Attachment E).

Provider Logs: To assess provider-time spent, the project research assistant will compare the time providers spend on emails, phone calls, and related follow-up with patients and their families pre-intervention versus time providers spend on secure-messaging, phone calls, and related follow-up with patients and their families post-intervention.

Chart Reviews: To assess ED-utilization for medication refills, ED-utilization for patients in the clinic pre and post-intervention will be reviewed. The full-list of all patients who are registered patients of the clinic will be obtained from clinic records as well as from the administrative database. Patients will be defined as patients of the clinic if they have had 2 or more visits in the past 2 years.

Pre-intervention assessment: The clinic’s administrative database will be utilized to determine which of those patients have had ED visits in the past year. Those medical records will be reviewed to identify the reason for all ED visits by patients who are registered with the clinic and identify which of those are for medication refill only.

Post-intervention analysis: Patients that had ED visits for the 5- month period of March-June 2009 will be identified; those records will be reviewed to identify which were for medication refills.

The rate of patients registered for the clinic who came to the ED for one or more refills pre- versus post-intervention will be compared. The metric for comparison will be the number of patients who came to ED per 1000 registered patients/per year pre-intervention versus post-intervention. Because the rates will be dependent, a McNemar’s test or other test adjusted for dependent data will be used to assess the comparison statistically. The assessment will be stratified based on whether patients had used the messaging system.

Patient and Provider assessment of satisfaction: To assess satisfaction by the patients and providers, qualitative interviews with patients and separately with clinicians will be conducted to elicit feedback about the process and degree of satisfaction. Interviews with the patients and their families will be part of the qualitative assessment performed in Section B.1 above. A parallel methodology to conduct qualitative interviews with all of the providers who care for patients in the clinic will be utilized.

Security Measures:

Responses from patients participating in qualitative interviews will recorded and stored. After interview transcription from the recorded interviews, the results will be stored onto a study-specific folder on the Hospital Secure Server. AES-128 Encryption will be applied to the folder containing the information and accessible only by the three primary investigators and two listed study personnel. Study data will not be stored in unencrypted moveable media. Maintenance on a secure server (accessible only from within the Hospital network) in an encrypted folder will obviate the need for moveable media. Wherever possible, patient information from the qualitative interviews will be de-identified and stored as above on a Hospital Secure Server with AES-128 bit encryption. Very limited access to the secure server will be given to project staff as outlined above (Dr. Hsiao, Dr. Bazzy-Asaad, Dr. Benin; Project Manager Tina Tolomeo, MSN, APRN and Research Assistant Diana Edmonds. The primary investigator will also review access and use of data on a regular basis.

After completion and analysis of the study data, the data will be stored for five years on the secure, encrypted, server for referral and analysis. At conclusion of that period, with the aid of the Hospital Information Systems & Technology Security Officer, the data will be securely erased, likely using a zeroing tool. While the AHRQ is funding this study and will be monitoring the progress of data collection, they will not have any access to study data and will not be given access to the secure servers.

Quality control procedures for field work and or data coding or preparation.

Instrument and Consent form development: Instrument and consent form development was based on the use of existing, well-tested design principles. In addition, sound methodological principles of questionnaire design were emphasized in all phases of the questionnaire development process. For example, these included: the extensive use of direct, properly time referenced/contexted, single idea, closed questions throughout the questionnaire; use of simple, short skip (filter) questions when required; the placement of non-threatening questions at the beginning of the questionnaire, with the most relevant questions near the beginning; straightforward instructions; and, thorough pre-testing of the complete questionnaire package, which encompassed criteria on content (language, information level, relevancy), flow of questions, timeliness, and quality of response.

Survey Personnel: The data collector for the project is highly experienced in data collection activities and will receive appropriate training in tracking respondents, responding to enquiries from respondents, coding responses to open-ended questions, data validation, and quality control, as well as confidentiality and data security procedures.

Activities After Data Collection: All completed surveys and interviews will be manually checked, edited and coded to ensure that codes were used in a uniform way before data entry. The finalized data will be checked several times to reduce the possibility of error. The machine-readable data will also be checked by automated validation and correlation. Any errors/inconsistencies will be manually rectified by the project team. Any editing, coding, and other changes to the original collected data were carefully documented and analyzed for consistency, completeness, and relevancy in terms of the subsequent results.

B. 3 Methods to maximize Response Rates

The investigators will use a number of proven methods to maximize participation rates in the study. First, the instructions for the demographic survey are straightforward, and there are only 8 questions in the survey, which can be completed in a very short time. Second, the use of the e-messaging system is expected to result in a high level of participation. A computer kiosk in the waiting room will be available for patients and their families to use a demonstration version of the system and electronically obtain their username and password if desired. The inquiries and questions submitted by the study subjects over the e-messaging system will be completed by the respondents at their own convenience. Furthermore, the close monitoring of the messages will encourage additional participation by the respondents. Finally, the qualitative interviews will be short (how long?) and user-friendly. Respondents will be allowed to spend as much time or as little time as they want to answer the questions.

B.4 Tests of Procedures

Response: Not Applicable

B. 5 Statistical Consultants

The statistical consultant for the project is Jeph Herrin, Ph.D. His telephone number is (434) 295-5342. The project’s research assistant, Diana Edmonds will arrange and perform the interviews. The project manager, Tina Tolomeo, MSN, APRN, AE-C as well as Drs. Hsiao, Benin and Bazzy-Asaad will perform the qualitative analysis of the interviews.