SUPPORTING STATEMENT
Part A
Eisenberg Center Voluntary Customer Survey Generic Clearance
for the Agency for Healthcare Research and Quality
Version: August 12, 2016
Agency of Healthcare Research and Quality (AHRQ)
Table of contents
A. Justification 3
1. Circumstances that make the collection of information necessary 3
2. Purpose and use of information 6
3. Use of Improved Information Technology 7
4. Efforts to Identify Duplication 7
5. Involvement of Small Entities 7
6. Consequences if Information Collected Less Frequently 7
7. Special Circumstances 8
8. Consultation outside the Agency 8
9. Payments/Gifts to Respondents 8
10. Assurance of Confidentiality 10
11. Questions of a Sensitive Nature 10
12. Estimates of Annualized Burden Hours and Costs 10
13. Estimates of Annualized Respondent Capital and Maintenance Costs 13
14. Estimates of Annualized Cost to the Government 14
15. Changes in Hour Burden 14
16. Time Schedule, Publication and Analysis Plans 15
17. Exemption for Display of Expiration Date 16
List of Attachments 16
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see http://www.ahrq.gov/hrqa99.pdf), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
1. research that develops and presents scientific evidence regarding all aspects of health care; and
2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policymakers, and educators; and
3. initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
The Agency for Healthcare Research and Quality (AHRQ) requests that the Office of Management and Budget (OMB) renew under the Paperwork Reduction Act of 1995 AHRQ’s Generic Clearance to collect information from users of work products and services initiated by AHRQ’s John M. Eisenberg Center for Clinical Decisions and Communications Science (Eisenberg Center). The Eisenberg Center is an innovative effort aimed at improving communication of findings to a variety of audiences (“customers”), including consumers, clinicians, and health care policymakers. The Eisenberg Center compiles research results into a variety of useful formats for customer stakeholders.
This research has the following goals:
1) Conduct research into effective communication of research findings in order to improve the usability and rapid incorporation of findings into medical practice and decision making.
2) Conduct research into effective strategies for disseminating evidence-based products, tools, and resources to consumers, clinicians, and other health care professionals, and policymakers.
3) Evaluate outcomes reported by clinicians and other healthcare professionals resulting from participation in continuing medical education (CME) initiatives and activities.
4) Conduct research into factors associated with successful collaboration between AHRQ and partnering institutions and organizations in synthesizing, translating, and disseminating evidence-based research.
Clearance is being requested to cover a three-year period in which differing numbers of products and research activities may be conducted during each year. The collections proposed include activities to assist in the development of materials to be disseminated through the Eisenberg Center and to provide feedback to AHRQ on the extent to which these products meet customer needs. These materials include summary documents that summarize and translate the findings of research reports for various decision-making audiences, such as consumers, clinicians, or policymakers. The summaries are designed to help these decision makers use research evidence to maximize the benefits of health care, minimize harm, and optimize the use of health care resources. In addition, each year, a unique research project will be undertaken to study successful approaches to disseminating AHRQ products in various health care settings and clinical environments. Also, each year the Eisenberg Center will develop one interactive decision aid for clinical problems identified from selected research reports. The intent is for the decision aid to increase the customer’s knowledge of the health condition, options, and risk/benefits; lead to greater assurance in making a decision; increase the congruence between values and choices; and enhance involvement in the decision making process. Information collections conducted under this generic clearance are not required by regulation and will not be used to regulate or sanction customers. Data collections will be entirely voluntary, and information provided by respondents will be combined and summarized so that no individually identifiable information will be released.
The data collections listed below will be implemented to achieve project goals. Note: Assessments such as interviews and surveys are here denoted formative if conducted prior to product development or determination of dissemination channels; usability testing or pretesting if conducted while reviewing a draft product, proposed dissemination approach, or other proposed content/strategy; and evaluation if conducted for summative evaluation or to assess satisfaction after the product has been in use or the dissemination campaign, learning activity, or other initiative undertaken.
Data collections will include the following:
1) Interviews for Product and Decision Aid Development, Testing, and Use. Individual interviews will be conducted with clinical professionals, patients, or other health care consumers, or health policymakers. In some cases focus groups may be substituted for patient interviews. These formative and pretesting/cognitive interviews will allow for (1) collecting input from target audiences regarding the development of summary products and decision aids; (2) determining if intended information and messages are being delivered effectively through products that are developed and disseminated through the Eisenberg Center; (3) assessing whether changes in topical knowledge levels can be identified following exposure to Eisenberg Center informational or instructional products or aids; (4) identifying product strengths and weaknesses to facilitate improvements that are practical and feasible; and (5) assessing decision support from the perspective of each audience. In addition, the Eisenberg Center will conduct a new research project annually to inform the enhancement of existing health information products, beyond what is currently being provided. The accompanying assessments will likely consist of interviews conducted with target audience members and may be integrated into the existing product interviews discussed above. See Attachments A – F for assessment formats to be used in product development and testing of summary products, and Attachments G – H for development and testing of decision aids.
2) Interviews for Dissemination Activities. Interviews will be conducted with leadership and staff of health systems, hospitals, and/or clinics in which dissemination activities are conducted to explore, prior to initiating the project, those pathways holding the greatest potential for successful uptake of the AHRQ materials. Interviews will be conducted again after project conclusion with administrators and product users (e.g., consumers, clinicians) to assess success of dissemination efforts, perceptions around product access, challenges that arose, and strategies to facilitate future successful dissemination initiatives.
3) Survey for Decision Aids. Following delivery of the decision aid, a user survey will be completed to explore subjects’ impressions of the tool, including ease of use, clarity of presentation, length, balance of information, rating of interactive features, and overall satisfaction. Both clinicians and patients/consumers will be surveyed. For patients, the customer satisfaction survey may include decisional outcome measures (e.g., decisional conflict, desire for involvement in decision-making), measures of attitudes and self-efficacy, and indicators of choice intention or actual choice made. If the aid is evaluated within a clinical context, measures of physician-patient interaction will also be considered. Additionally, clinicians may be interviewed about the impact of the aid on decision making, clinical flow, and patient outcomes.
4) Survey for Summary Products (initial, follow up). Very brief surveys will be offered to health care professionals, consumers, and policymakers that use the online summaries. Immediately upon accessing the summaries, visitors will be asked to complete a brief survey assessing for whom they were seeking information, how the product might be used, and an email address for a follow-up survey. Respondents will subsequently be sent an email asking them to complete a follow-up online survey assessing how the information has been used, whether it influenced health care practices, and any barriers to use or suggestions for improvement. See Attachment K.
5) Survey of Patient and Consumer Advocacy Organizations. Each project year, representatives from consumer and patient advocacy organizations will be invited to attend a meeting and participate in ongoing activities to facilitate engagement in AHRQ systematic review, translation, and dissemination activities. Surveys by phone or online questionnaire will be used to assess the quality of the in-person meeting and ongoing activities, the impact and value of engaging with AHRQ, the value of research and translation products for the target audiences, how partners and their constituents are using the products, and ways to make the products and partnerships with AHRQ more useful for partners and have a broader reach.
6) Survey of AHRQ Partners. AHRQ, through the Evidence-based Practice Center (EPC) Program and Eisenberg Center, partners with organizations when developing, translating, and/or disseminating research reports and related products. AHRQ partners include developers of clinical practice guidelines, payers, other Government agencies, private companies, consumer and patient advocacy groups, and health care systems. Surveys by phone or online questionnaire, followed by targeted interviews, will be used to assess the impact and value of AHRQ research products for the target audiences, determine how partners are using the products, and identify ways to make the products and partnerships more useful for partners and have a broader reach.
7) CME Outcomes Survey. AHRQ through the Eisenberg Center will offer AMA PRA Category 1 continuing medical education (CME) credit for certain products that it develops. Clinicians wishing to claim credit must complete an outcomes assessment survey delivered online two months after completing the activity. Attachment L illustrates the assessment content and format of these surveys.
8) Interviews and Surveys for Dissemination Research Project. Each project year the Eisenberg Center will propose and conduct a unique research project aimed at disseminating products. As part of that project, formative interviews and potentially cognitive testing will be conducted with consumers, clinicians, and administrators from participating health systems, hospitals, and/or clinics for purposes of assessing current dissemination initiatives, similar products available to their consumers, ways to optimize dissemination, and other indicators as determined by the project aims. These three audiences may also be asked to complete follow-up surveys and/or participate in interviews to document project outcomes and lessons learned from the study.
This study is being conducted by AHRQ through its contractor, Baylor College of Medicine, pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).
In accordance with OMB guidelines for generic clearances for voluntary customer surveys and Executive Order 12862, AHRQ has established an independent review process to assure the development, implementation, and analysis of high quality customer surveys within AHRQ. Specifically, AHRQ understands that each activity conducted must be submitted to OMB with a supporting statement and accompanying instruments. Information collection may not proceed until approved by OMB.
The information obtained using the data collection strategies described above will be used to develop, improve and/or maintain high quality health care informational products and services to lay public and health care professionals. Each product previously developed by the Eisenberg Center was proposed, drafted, tested, and revised with heavy reliance on data collected in a manner similar to those approaches described in this clearance. This includes data collected at the formative stage when ideas for the product and its information parameters are being developed, through draft testing and revisions, and finally to product implementation and evaluation of its usefulness in practice. Work on implementing and evaluating dissemination strategies and approaches will complement the development activities in optimizing delivery to the targeted audiences.
Information technology will be used for data collections (e.g., online surveys) whenever possible to reduce the burden on the public. In some instances, however, the most appropriate methodology will involve written or oral responses to brief questionnaires and interview questions. Individuals may also be asked to use or interact with computer and/or Internet technologies to assess functionality and ease of use of electronic materials, tools and and/or systems. In some cases, respondents will be asked to review and rate and/or comment on materials prepared using text and graphics to deliver messages about one or more therapies of interest.
Each survey or other data collection instrument will be designed to reflect the specific information needs, decision support opportunities, and dissemination pathways appropriate to the customer population served. During the development of these voluntary instruments and other data gathering tools, groups within and outside of AHRQ will be consulted as needed. Plans to conduct information gathering will be reviewed prior to implementation, and any potential duplication will be identified in the review and approval process.
The survey instruments and procedures for completing the instruments will be designed to minimize the burden on all respondents and will not have a significant impact on small businesses or other small entities. Questionnaires will be brief, yet of a sufficient length to collect the necessary data. The burden is entirely voluntary.
The proposed information collections are appropriate vehicles to examine customer experiences and perceptions with regard to products and services developed by the Eisenberg Center and its ability to communicate effectively with a variety of audiences. Collection of data on a less frequent basis would reduce the practical utility of the information and would inhibit the Eisenberg Center’s ability to: (1) determine how well its products and services are meeting customers’ current and anticipated needs; (2) identify problem areas with existing products and services and determine what improvements should be made to improve these products and services; and (3) identify and develop new products and services. The importance of frequent interaction and data collection with targeted audiences was reflected in the request for applications issued in the contracting process, which specifically mandated formative, operational, and assessment data collection efforts related to the Eisenberg Center and its products.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), notice was published in the Federal Register on May 27, 2016, on page 33675 for 60 days (see Attachment X). No substantive comments were received.
The Eisenberg Center will consult as needed with AHRQ’s in-house statistical staff, Evidence-based Practice Centers (EPCs), other Federal agencies, and other organizations who may also be involved in similar or related efforts to identify areas of interest and concern to customers. As appropriate, panels of outside experts may be established to assist in design and implementation of the surveys.
No remuneration to respondents for written, telephone, web, or other forms of surveys or interviews will be given, except as follows. On a case-by-case basis, consideration will be given for modest remuneration for participants in focus groups and semi-structured interviews. This remuneration is meant to reimburse the participants for their time and expenses including potential travel to and from the testing facility. Maximum honoraria amounts of $50 will be paid to consumers participating in interviews or focus groups, while honoraria amounts of up to $250 will be paid to expert physicians, policymakers, and health system managers and administrators for performing complex testing processes. These individuals have limited time availability and are accustomed to receiving similar levels of recompense for their valuable input, which is essential to effective product development and testing. To ensure an equitable process, identical financial incentives will be offered to all individuals participating in a specific activity, and participants will be allowed to drop out of the study at any time without loss of incentive. Factors influencing the rate of remuneration will include: (1) the participant’s qualifications/demonstrated expertise; (2) projected time allocation and any preparations required for the activity; (3) urgency of gathering data and the implications with regard to incentivizing participants; and (4) projected likelihood of loss of income or other costs to participants (e.g., child care costs, travel expenses) that are associated with participation.
Remuneration for interviews and other activities demanding participant time is a recognized standard industry practice, without which it would be difficult to achieve appropriate and adequate participation. Although the published literature is somewhat mixed in terms of quantifying the extent to which financial remuneration motivates participation in surveys or interviews as well as the incentives required to obtain high response rates, a meta-analysis conducted by Eleanor Singer and colleagues on face-to-face and telephone surveys found that paying incentives produced a positive and statistically significant effect.1 The effect was linear (i.e., modeling a curvilinear term failed to produce significant results), leading the authors to conclude (p. 223), “Within the limits of incentives and response rates occurring in these experiments, more money results in higher response rates.” Increasing the participation burden on respondents resulted in even stronger effects for monetary motivation (which suggests that a disproportionately greater compensation rate may be required for the longer interviews typically conducted by the Eisenberg Center). Further, the authors found that the data from some, but not all, included studies produced results consistent with the conclusion: “paying an incentive may be useful in obtaining higher numbers of respondents in demographic categories that might otherwise tend to be underrepresented in sample surveys (e.g., low income or nonwhite race).”
The maximum payment ranges specified here have been determined in light of both the empirically-defined principles summarized above and the responsiveness observed among those invited to participate in identical work conducted by the Eisenberg Center during the prior 5-year contract period. Lesser payments were offered in some instances and were not found to produce the responsiveness desired, nor to motivate highly qualified clinicians, senior-level administrators, and various content experts to integrate these types of assessment activities into their already overtaxed schedules. The maximum amounts proposed here are meant to compensate for their efforts (e.g., a priori review of products to be discussed during an interview, along with the accompanying literature base as necessary) using a rate equal to or slightly exceeding their typical hourly rates for work performed. If significantly lower payments were substituted, recruits would likely consist of highly motivated participants, and hence the potential for volunteerism bias to result in perspectives not generalizable to the targeted populations at large. With respect to consumers, in some cases the Eisenberg Center wishes to identify and form a “panel” that would be queried on multiple occasions to provide feedback (e.g., in reaction to ongoing product modifications). If no or minimal incentives are offered, consumers will be less likely to commit to ongoing participation in these assessments unless predisposed to volunteer. In addition, some Eisenberg Center work requires accessing and recruiting from exceptionally unique subpopulations, often with a compressed lead-time. This process would be hampered if minimal compensation amounts were offered.
Nor do the participants and circumstances proposed under this clearance fall into the categories defined by Ruth Grant as potentially causing undue influence or coercion.2 Specific proposed remuneration will be included in the supporting statement for activities specified under this generic clearance.
Individuals and organizations will be assured of the confidentiality of their replies under Sections 944(c) and 308(d) of the Public Health Service Act [42 U.S.C. 299c-3(c) and 42 U.S.C. 242m(d)]. They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.
Respondents will be advised that surveys and/or other data collection activities in which they may be asked to participate are entirely voluntary, and that any information they provide will be combined and summarized with information provided by others and no individually identifiable information will be released. In instances where respondent identifiers are needed (e.g., continuing medical education [CME] follow-up), information collection will fully comply with all respects of the Privacy Act.
No questions of a sensitive nature are anticipated under this generic clearance.
Exhibit 1 shows the estimated total burden for the respondents' time to participate in this research. These estimates assume a maximum of 141 Summary products over 3 years with separate products developed for clinicians, policymakers, and consumers.
Formative interviews, and in some cases focus groups, will be used to conduct needs assessment and will be held with clinicians and consumers for development of the products and decision aids, and additionally with policymakers for those products in which policy recommendations are applicable. Interviews will be conducted with no more than 2,115 persons for product development, 180 persons for decision aid development, and 180 persons for development of dissemination initiatives over 3 years, and each will last about 60 minutes.
Once the products are developed they will be subjected to in-person or telephone interviews for purposes of usability and product testing with clinicians, policymakers and consumers. In-person/telephone interviews will be conducted with about 2,115 persons for products and 180 persons for decision aids over 3 years and will take about 60 minutes on average. A second round of interviews will be conducted only occasionally with one or more of the targeted populations if necessary due to substantial product revisions. These interviews may also be used to inform product enhancements in relation to the annual enhancement study. Because these specifications cannot be determined in advance, clearance is being requested for two testing rounds with every product and every audience.
Evaluation surveys will be conducted with approximately 6,000 representatives across the targeted audiences (i.e., consumer, clinician, policymaker) for the health information products and 2,400 persons who have used the decision aids over the 3-year period. The product surveys will take about 5 minutes to complete, and the decision aid surveys about 10 minutes. A follow-up survey will be completed for the product evaluations, which will also last about 5 minutes, while a subset of 180 of those having used the decision aids will be asked to participate in a follow-up evaluation interview lasting an hour.
Those involved in or targeted by the dissemination initiatives will be asked to participate in evaluation interviews, which will include up to 480 persons completing interviews across the 3 project years. Note: Because the timing of interviews with persons at the 6 total partner organizations has not yet been finalized, AHRQ is requesting that all dissemination-related interviews be approved for the first project year. For simplicity, the interviews are presented as annualized in Exhibits 1 and 2.
The unique dissemination research project to be proposed and completed annually will include 135 formative interviews with consumers, clinicians, and administrators, with each lasting 1 hour. Follow-up evaluation surveys and interviews will be conducted with 360 and 180 persons, respectively.
AHRQ partners will be asked to complete surveys and interviews in relation to their prior or ongoing collaborative work with AHRQ. These will include 150 persons completing surveys and 60 follow-up interviews. Similar types of surveys designed with the goal of improving products and expanding their research will be completed by 90 representatives of advocacy organizations across the 3 years, with each survey lasting about 10 minutes.
Clinicians that have completed CME accrediting requirements and are requesting CME credit will be asked to complete a follow-up outcomes survey two months following completion of the online activity. These will be completed by no more than 27,000 clinicians over 3 years and will require 5 minutes to complete.
The total burden hours are estimated to be 13,875 annually or 41,625 over 3 years.
Exhibit 1. Estimated annualized burden hours
Form Name |
Number of respondents |
Number of responses per respondent |
Hours per response |
Total burden hours |
Product Formative Interviews |
705 |
1 |
1 |
705 |
Product Pretesting Interviews |
705 |
2 |
1 |
1,410 |
Product Evaluation Surveys |
2,000 |
2 |
5/60 |
333 |
Dissemination Formative Interviews |
40 |
1 |
1 |
40 |
Dissemination Evaluation Interviews |
120 |
1 |
1 |
120 |
Decision Aid Formative Interviews |
60 |
1 |
1 |
60 |
Decision Aid Pretesting Interviews |
60 |
1 |
1 |
60 |
Decision Aid Evaluation Interviews |
60 |
1 |
1 |
60 |
Decision Aid Evaluation Surveys |
800 |
1 |
10/60 |
133 |
Research Project Formative Interviews |
45 |
1 |
1 |
45 |
Research Project Evaluation Surveys |
120 |
1 |
10/60 |
20 |
Research Project Evaluation Interviews |
60 |
1 |
1 |
60 |
Partnership Evaluation Surveys |
50 |
1 |
10/60 |
8 |
Partnership Evaluation Interviews |
20 |
1 |
1 |
20 |
Advocacy Meeting Evaluation Surveys |
30 |
1 |
10/60 |
5 |
CME Outcomes Surveys |
9,000 |
1 |
5/60 |
750 |
Total |
13,875 |
na |
na |
3,830 |
* For the 3-year contract period, product formative interviews and product testing interviews will each comprise 300 consumers, 300 clinicians, and 105 policymakers; product evaluation surveys will include 800 consumers, 800 clinicians, and 400 policymakers; dissemination-related formative interviews will include 40 health system/hospital/clinic administrators; dissemination-related evaluation interviews will include 40 consumers, 40 clinicians, and 40 administrators; formative interviews, pretesting interviews, and evaluation interviews for the decision aids will each include 30 consumers and 30 clinicians; evaluation surveys for the decision aids will include 400 consumers and 400 clinicians; formative interviews for the annual dissemination research project will include 15 consumers, 15 clinicians, and 15 administrators; evaluation surveys for the research project will include 50 consumers, 50 clinicians, and 20 administrators; evaluation interviews for the research project will include 20 consumers, 20 clinicians, and 20 administrators; the AHRQ partner surveys will include 50 partners; the AHRQ partner evaluation interviews will include 20 partners; the health advocates surveys will include 30 participants; and CME outcomes surveys will include 500 clinicians for each of 18 CME activities.
Exhibit 2. Estimated annualized cost burden
Form Name |
Number of respondents |
Total burden hours |
Average hourly wage rate* |
Total cost burden |
Product Formative Interviews |
705 |
705 |
$54.81a |
$38,641 |
Product Pretesting Interviews |
705 |
1,410 |
$54.81a |
$77,282 |
Product Evaluation Surveys |
2,000 |
333 |
$54.00a |
17,982 |
Dissemination Formative Interviews |
40 |
40 |
$49.84a |
$1,994 |
Dissemination Evaluation Interviews |
120 |
120 |
$54.74a |
$6,569 |
Decision Aid Formative Interviews |
60 |
60 |
$57.19a |
$3,431 |
Decision Aid Pretesting Interviews |
60 |
60 |
$57.19a |
$3,431 |
Decision Aid Evaluation Interviews |
60 |
60 |
$57.19a |
$3,431 |
Decision Aid Evaluation Surveys |
800 |
133 |
$57.19a |
$7,606 |
Research Project Formative Interviews |
45 |
45 |
$54.74b |
$2,463 |
Research Project Evaluation Surveys |
120 |
20 |
$55.96b |
$1,119 |
Research Project Evaluation Interviews |
60 |
60 |
$54.74b |
$3,284 |
AHRQ Partner Evaluation Surveys |
50 |
8 |
$54.50c |
$436 |
AHRQ Partner Evaluation Interviews |
20 |
20 |
$54.50c |
$1,090 |
Advocacy Meeting Evaluation Surveys |
30 |
5 |
$21.21d |
$106 |
CME Outcomes Surveys |
9,000 |
750 |
$91.66e |
$68,745 |
Total |
13,875 |
3,830 |
na |
$237,610 |
* National Compensation Survey: Occupational wages in the United States May 2014, “U.S. Department of Labor, Bureau of Labor Statistics.”
a Rate based on the mean and/or weighted mean wages for various combinations of consumers (00-0000 all occupations), clinicians (29-1060 physicians and surgeons, 29-1062 family and general practitioners), and health policymakers (11-0000 management occupations, 11-3111 compensation & benefits managers, 13-1141 compensation, benefits & job analysis specialists, 11-9111 medical and health service managers, 13-2053 insurance underwriters and 15-2011 actuaries).
b Rate based on the mean and/or weighted mean wages for various combinations of consumers (00-0000 all occupations), clinicians (29-1060 physicians and surgeons, 29-1062 family and general practitioners), and health system/hospital/clinic administrators (11-9111 medical and health services managers).
c Rate based on the mean wages for AHRQ partners (25-1071 health specialties teachers, postsecondary, 11-1021 general and operations managers, 21-0091 health educators, 21-1093 social and human service assistants, 11-9111 medical and health services managers).
d Rate based on the mean wages for health advocacy organizations (21-1093 social and human service assistants [social advocacy organizations], 21-0091 health educators).
e Rate based on the mean wages for clinicians (29-1060 physicians and surgeons, 29-1062 family and general practitioners).
Exhibit 2 depicts the estimated total cost burden associated with the respondents' time to participate in this research. The cost burden is estimated to be $237,610 annually.
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. If none write: There are no direct costs to respondents other than their time to participate in the study.
There are no direct costs to respondents other than their time to participate in the study.
Exhibit 3. Estimated Total and Annualized Cost
Cost Component |
Annualized Cost |
Total Cost |
Project Development |
$87,102 |
$261,307 |
Data Collection Activities |
$865,813 |
$2,597,440 |
Data Processing and Analysis |
$192,233 |
$576,700 |
Publication of Results |
$117,444 |
$352,331 |
Project Management |
$148,378 |
$445,134 |
Overhead |
$6,300 |
$18,900 |
Total |
$1,417,271 |
$4,251,812 |
Exhibit 3b. Federal Government Personnel Cost
Activity |
Federal Personnel |
Hourly Rate |
Estimated Hours |
Cost |
Data Collection Oversight |
Health Scientist Administrator |
53.91 |
75 |
$4,043.25 |
Review of Results |
Health Scientist Administrator |
53.91 |
75 |
$4,043.25 |
Total |
$8,086.50 |
|||
Annual salaries based on 2016 OPM Pay Schedule for Washington/DC area: http://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2015/DCB.pdf
Although AHRQ has requested Clearances from the Office of Management and Budget in the past, the present burden hour request is based on a new contract that includes both (a) product development, dissemination, research, and evaluation tasks identical to those submitted previously yet with greater numbers of products developed, and (b) new projects that have not been reviewed and approved by the Office of Management and Budget. In addition, the proposed work significantly expands the number of products for which CME credit is offered to clinicians, both in this country and abroad, with up to 500 CME participants completing an outcomes assessment survey per CME activity (54 activities across the 3 years). However, the burden associated with each respondent is modest: about 5 minutes to complete the brief survey.
Additionally, the current proposal assumes up to two rounds of data collection associated with the usability testing of 47 products per year. Although this significantly increases the total burden hour estimate, in reality there will be very few circumstances in which a second round of usability testing interviews will be required.
The purpose of the activities described is to gather both quantitative and qualitative information on Eisenberg Center products and delivery formats and vehicles, including information on their value and utility to the audiences for whom they are intended. Information from analyses of data gathered during formative phases of product development will be used to revise and refine the products and delivery systems prior to dissemination to targeted audiences. Information from analyses of data gathered from users of the information distributed via the Web and through other means (e.g., EHR systems, presentations at professional meetings) will be used to: (a) determine if further revisions or refinements would enhance value or utility of materials currently available; and (b) characterize the quality and appropriateness of efforts to use resources effectively in supporting delivery of high quality health care. This latter function will be of special importance in guiding decisions by AHRQ regarding future efforts related to the Eisenberg Center and the nature and scope of support committed to it. Data collected from patient advocacy groups and other organizations and associations partnering or considering partnering with AHRQ will be analyzed to evaluate dimensions related to the partnership experience; early indicators of any barriers or challenges, particularly with regards to product dissemination; and feedback received from the partners’ constituents to allow for tracking progress and ensuring effectiveness of the products and activities being developed for end users.
The analyses will be descriptive, and it is unlikely that results can be generalized to the larger populations. The results of these findings are primarily for internal use but may be shared with key government policy and management officials, AHRQ staff, public and private health providers, and members of the general public.
In-person/telephone interviews and focus groups: Interviews to aid in product development and early discovery of problems with draft information products will be conducted in person or remotely via telephone or the Web. The analyses will be qualitative and consist mostly of narrative summaries and thematic analysis of the discussions. When feasible, assessments will be done in controlled environments for purposes of evaluating systematically different products (e.g., research reports, key messages) in order to refine and enhance their readability, comprehension, and usefulness. Characteristics of respondents needed for product testing will be specified. Participants typically come to a designated location where they are presented with instructions and are then exposed to the study material and asked to respond to a series of questions that will allow the investigators to assess product features or conditions. These technology-facilitated assessments allow the investigator to vary features of reports and decision aids in a systematic way to facilitate evaluation of those features and their use under varying conditions. Some interviews involving content experts or leadership in AHRQ organizational partners may be conducted remotely by phone to allow for analyzing data representing individuals or organizations that are geographically disbursed and/or who may have limited time allowance for face-to-face or on-site meetings. By conducting one-on-one interviews, individuals are afforded the privacy and flexibility needed to allow them to discuss reactions, reveal interpretations, and communicate how they might use the material and other resources in decision making.
Customer surveys: Electronic technology may be used for this type of information collection. The surveys will be accessible through the Effective Health Care Web site and perhaps information systems specific to partnering organizations and institutions. In addition to summarizing item responses by frequency counts, percentages, and other descriptive statistics, basic demographic information will be collected and summarized. On occasion, similar information may be collected over the phone from customers who do not use the Web frequently to allow for their inclusion in the assessments. Use of brief surveys may be particularly useful in conducting the unique dissemination projects to be conducted annually. These mechanisms would allow for efficient and timely collection of data from various target audience representatives that would be of interest in a project involving a large health organization or health system.
AHRQ does not seek this exemption.
List of Attachments:
Attachment A: Topic Development_Clinician Script.docx
Attachment B: Pretesting_Clinician Script.docx
Attachment C: Topic Development_Consumer Script.docx
Attachment D: Pretesting_Consumer Script.docx
Attachment E: Topic Development_Policymaker Script.docx
Attachment F: Pretesting_Policymaker Script.docx
Attachment G: Cognitive Interview Script_Decision Aid.docx
Attachment H: Pretesting Interview Script_Decision Aid.docx
Attachment I: Online Survey Form A_Decision Aid.pdf
Attachment J: Online Survey Form B_Decision Aid.pdf
Attachment K: Online Surveys_Summary Product.docx
Attachment L: Outcomes Survey_CME Activities.docx
Attachment M: 60-Day Federal Register Notice
1 Singer E, Van Hoewyk J, Gebler N, Raghunathan T, McGonagle K. The effect of incentives on response rates in interviewer-mediated surveys. Journal of Official Statistics. 1999;15(2):217–230.
2 Grant, RW. Rethinking the ethics of incentives. Journal of Economic Methodology. 2015;22(3): 354-372.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Statement Part A Template |
| Subject | <p>(Supporting Statement Template and Instructions; instructions are in italics and should be deleted when complete) |
| Author | Heather Nalls |
| File Modified | 0000-00-00 |
| File Created | 2021-01-23 |