SUPPORTING STATEMENT
Part B
Comparative Effectiveness Research – Continuing Education
Version: April 7th, 2011
Resubmission: July 27, 2011
Agency of Healthcare Research and Quality (AHRQ)
Table of contents
B. Collections of Information Employing Statistical Methods 3
1. Respondent universe and sampling methods 3
2. Information Collection Procedures 3
3. Methods to Maximize Response Rates 4
4. Tests of Procedures 4
5. Statistical Consultants 5
All health care professionals who participate in any of the 39 CME/CE/CEU online modules funded by AHRQ (See description in Part A) will be considered candidates in the respondent universe. AHRQ’s contractor, PRIME, Inc., is creating a total of 45 CME/CE/CEU modules, and 6 of those modules are being delivered by Academic Detailing and will be included in a separate OMB Clearance. The 60-day, 6-month and 1-year survey instruments will be electronically sent to all participants. The respondent universe will be determined by the number of health care professionals who voluntarily complete the surveys, in accordance with standard practices in continuing medical education.
Since the first AHRQ CER modules were released in March 31, 2011, PRIME has issued 1,608 educational certificates of completion for 4 modules, averaging 402 certificates per course in 60 days. We calculated the total number of certificates to be issued by multiplying the number of certificates (402) issued in this first 60 days x 6 (365 divided by 60) to estimate the number of certificates that would be issued per year.
Projected over the course of accreditation, we estimate that an average of 2,412 certificates will be issued for each course.
Based on PRIME’s experience conducting interval-based surveys among all disciplines, we expect survey completions for the 60 day and 6 month intervals to be 35% of 2,412 or 844. A 20 % completion rate or 2,412 or 484 at the end of one year for a single module is anticipated.
Those who have completed the module will receive surveys at each measurement interval; completion of each emailed survey may be done for any/all time intervals by all participants. As such, participants may complete 1, 2, or all 3 surveys, or combinations thereof. Failure to respond to a survey does not eliminate their receipt of subsequent surveys.
Respondents will include physicians, physician assistants, nurse practitioners, nurses, pharmacists, case managers, psychologists, medical assistants, health educators, and other allied health professionals.
Surveys will be automatically generated via the LMS. The 60-day survey will be sent 60 days after each participant of a module completes the module and accesses his/her certificate of completion. After 844 surveys have been returned, no additional surveys will be sent. The 6-month survey will be sent 6 months after each participant of a module completes the module and accesses his/her certificate of completion (regardless of whether or not they responded to the 60-day survey). After 844 surveys have been returned, no additional surveys will be sent. The 1 year survey will be sent 1 year after each participant of a module completes the module and accesses his/her certificate of completion (regardless of whether or not they responded to the 60-day or 6-month surveys). After 484 responses are received, no more surveys will be sent out.
Surveys will be sent to the email address used by the participant at the time of program registration. Surveys will be imbedded in the sent email to enhance convenience and efficiency for the respondent answering the survey. In accordance with FCC regulations, respondents will have the ability to opt-out of future email communications regarding the surveys.
Surveys will be completed online and returned electronically by respondents. Data is automatically stored in a secure database. Each participant’s answers are stored with their unique identifier ensuring that each survey answer is appropriately linked to each question answered by the participant.
Data Quality Control (DQC) processes are incorporated into the LMS where data calculations are weighted and scored. DQC is composed of 3 components:
1) verification of what is received from others;
2) verification of what is provided to others;
3) internal quality control peer review procedures.
Evaluation data is rendered to provide percentages of respondents’ answers to each multiple choice question. This data is initially reviewed by AHRQ’s contractor, PRIME, Inc., specifically the Education Manager, who transfers data into templated metrics reports for review by Laurence Greene, PhD, and Sharyn Lee, RN, MS, DHL, CCMEP. Dr. Greene and Ms. Lee are adult learning experts who consistently review program evaluation data with a goal to provide interpretation and analysis, with crossover validation of each other’s work. Initial compilation of the data by the LMS system provides consistent methods of data measurement, scoring.
Before reports are released for use, they are reviewed by Lynn Goldenberg, RN, BSN, Director of Accreditation and Compliance, in a systematic process which allows for the ability to check the integrity of the data and minimize problems affecting interpretation. The DQC process is consistently monitored by the Director of Accreditation and Compliance in accordance with CME/CE/CEU criteria established by accreditation bodies.
Dissemination of the 39 CME/CE/CEU online modules will be enhanced through PRIME’s outreach and relationships with several national health care organizations and member societies. These organizations have agreed to provide links to the online modules from their member-based websites. As part of this relationship, they have also agreed to encourage their members to participate in the followup surveys, recognizing that participation in the surveys can assist with learning retention and positive reinforcement that may ultimately improve patient care. Through reminder encouragements from member organizations, we anticipate that participants will be more inclined to complete the surveys. The survey instruments will require no more than 3 minutes to complete. The brevity of these surveys will improve the likelihood of participants completing them. The convenience of online surveys imbedded in the participant’s email will reinforce the likelihood of completion, particularly because the initial learning occurred in an online environment, maximizing the likelihood that respondents are Internet savvy and have provided an accurate email address.
PRIME adheres to an
internal Quality Assurance Policy and Procedures for each CME/CE
module released. This policy and processes will be systematically
applied to all of the AHRQ modules.
Each program module undergoes review, approval, and sign-off from the external expert faculty, as well as at least 3 external expert peer reviewers. At that time, it is posted to a beta site. The beta site is a local copy of the entire website. The program is viewed on the beta by the Director of Operations and the Project Director, and any necessary edits/changes are applied by the IT department personnel. Once changes are approved, the content modifications or additions are posted to the live website by the IT Department via secure means. Comments posted by module participants are retrospectively monitored each day by the Scientific Program Director, and the entire website is monitored daily by the Lead Website Developer.
No statistical consultants will be used for this project.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Statement Part B |
| Subject | <p>(Supporting Statement Template and Instructions; instructions are in italics) SUPPORTING STATEMENT Part B |
| Author | Heather Nalls |
| File Modified | 0000-00-00 |
| File Created | 2021-02-01 |