SUPPORTING STATEMENT
Part A
Online Submission Form for Supplemental Evidence and Data for Systematic Reviews for the Evidence-based Practice Center Program
Version: (December 13, 2018)
Agency of Healthcare Research and Quality (AHRQ)
Table of contents
A. Justification 3
1. Circumstances that make the collection of information necessary 3
2. Purpose and use of information 5
3. Use of Improved Information Technology 6
4. Efforts to Identify Duplication 7
5. Involvement of Small Entities 8
6. Consequences if Information Collected Less Frequently 8
7. Special Circumstances 8
8. Consultation outside the Agency 8
9. Payments/Gifts to Respondents 9
10. Assurance of Confidentiality 9
11. Questions of a Sensitive Nature 9
12. Estimates of Annualized Burden Hours and Costs 9
13. Estimates of Annualized Respondent Capital and Maintenance Costs 10
14. Estimates of Annualized Cost to the Government 10
15. Changes in Hour Burden 11
16. Time Schedule, Publication and Analysis Plans 11
17. Exemption for Display of Expiration Date 12
List of Attachments 12
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see http://www.ahrq.gov/hrqa99.pdf), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote healthcare quality improvement by conducting and supporting:
1. research that develops and presents scientific evidence regarding all aspects of health care;
2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. initiatives to advance private and public efforts to improve healthcare quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special healthcare needs, including individuals with disabilities and individuals who need chronic care or end-of-life healthcare.
The AHRQ Evidence-based Practice Center (EPC) Program develops evidence reports and technology assessments that summarize evidence for federal and other partners on topics relevant to clinical and other health care organization and delivery issues—specifically those that are common, expensive, and/or significant for the Medicare and Medicaid populations. Better understanding and use of evidence in practice, policy, and delivery of care improves the quality of health care.
These reports, reviews, and technology assessments are based on rigorous, comprehensive syntheses and analyses of the scientific literature on topics. EPC reports and assessments emphasize explicit and detailed documentation of methods, rationale, and assumptions. EPC reports are conducted in accordance with an established policy on financial and nonfinancial interests.
This research has the following goals:
Use research methods to gather knowledge on the effectiveness or comparative effectiveness of treatments, screening, diagnostic, management or healthcare delivery strategies for specific medical conditions, both published and unpublished, to evaluate the quality of research studies and the evidence from these studies.
Promote the use of evidence in healthcare decision making to improve healthcare and health
Identify research gaps to inform future research investments
The Institute of Medicine standards for quality systematic reviews include an assessment of publication bias through the identification of unpublished studies. This is an important source for bias which could affect the nature and direction of research findings. Identifying and including the results of these additional unpublished studies may provide a more complete and accurate assessment of an intervention’s effect on outcomes. An important way to identify unpublished studies is through requests to medical device manufacturers, pharmaceutical companies, and other intervention developers.
The proposed project involves sending a notification via an email listserv and via Federal Register notice as needed of the opportunity to submit information on unpublished studies or other scientific information to the EPC Program website, with one request per systematic review topic. Because research on each topic must be completed in a timely manner in order for it to be useful, the collections are never ongoing—there is one request and collection per topic. Investigators in the EPC Program will review the information and assess potential risk of bias from both published and unpublished studies and its impact on the EPC Program’s findings.
To achieve the goals of this project the following data collections will be implemented:
Online Submission Form Instrument. This information is collected for the purposes of providing supplemental evidence and data for systematic reviews (SEADS). The online submission form (OSF) collects data from respondents on their name and the information packet. This happens following notification of opportunity to submit via email listserv and/or Federal Register notice as needed, with one request per topic. For the purposes of meta-analyses, trial summary data from missing and unidentified studies are sought. For the purposes of constructing evidence tables and quality ratings (e.g. on public reporting of cost measures or health information exchange), data can vary (e.g., URLs, study designs, and consumer-mediated exchange forms). Submitters are informed of the types of information that would be most helpful to include in the information packet which include a list of all sponsored but unpublished studies (both completed and ongoing), as well as comment on the completeness of information provided.
The EPC Program currently uses broad- based email announcement via email listserv and a Federal Register notice as needed to allow the public to know about each topic, and the opportunity to submit scientific information.
The proposed project does not duplicate other available sources of this information. Available study registries and databases may not be complete to sufficiently inform the Program’s research.
The purpose of SEADS requests is not to collect generalizable data, but to supplement the published and grey literature searches EPC investigators are conducting.
The EPC Program currently uses broad- based email announcement via email listserv and a Federal Register notice as needed to allow the public to know about each topic, and the opportunity to submit scientific information. The EPC Program does not anticipate more than 15 topics per year with SEADS requests.
The Effective Health Care website houses information and documents specific to the EPC Program. Through this website, documents are shared with the public, and give stakeholders the opportunity to comment on interim documents, such as the proposed scope of a product and a draft report. The Effective Health Care website would also serve as a gateway for the electronic submission of information and materials (SEADS), allowing access to an online submission form (OSF; see Attachment B for an outline) upon the finalization the research scope for the individual topics. Users of the OSF website will be industry stakeholders and investigators involved in the sponsoring of studies on interventions and healthcare strategies related to the topics investigated by the Program. The responses and submissions are intended to be included in statistical analyses to evaluate the different treatment options for patients suffering from the conditions under study.
The information can be uploaded as a MS Word document, PDF, or as a ZIP file, which potentially reduces the burden on the submitter. A portal will be open for at least four weeks for each topic. If the interventions under study include devices or other intervention types not requiring the ingestion of any substances, this period will coincide with the Federal Register Notice. The OSF is not a questionnaire.
There is only one required field in the OSF (the submitter’s name), and one required information packet to be uploaded. Submitters may choose to include additional details, such as a description of the information, medical condition, drug intervention, and e-mail address. Submitters are informed of the types of information that would be most helpful to include in the information packet, which includes a list of all sponsored but unpublished studies (both completed and ongoing), as well as comment on the completeness of information provided. It states that this is a voluntary submission. Submitters are informed that the contents of all submissions will be made available to the public upon request. All SEADS are reviewed by the EPC investigator team.
In addition to electronic submission of SEADS through the Effective Health Care Program website, respondents are also provided with an e-mail address for the EPC Program to which they may email their files directly or contact for a way to send materials through the mail.
The EPC Program uses notifications via email listserv and may use the Federal Register notice as needed to let the public to know about ongoing topics and the opportunity to submit scientific information. While the Program has worked with representatives from the Food and Drug Administration (FDA) when part of a stakeholder panel, and attempted to obtain publicly available information from relevant FDA resources, because the information submitted to the FDA is proprietary information, it may be heavily redacted and limit its usefulness. Moreover, the Electronic Freedom of Information Act (eFOIA) of 1996 means that FDA materials like drug approval packages are readily available only after 1996. Thus, a standard FOIA is required for those studies completed up to 1996. However, FOIA requests are described on FOIA.gov to take about a month for simple requests and much longer for more complicated requests. Since the systematic reviews conducted by EPCs are on a short schedule to ensure their prompt use in healthcare settings, additional time for FOIAs are likely not practical.
Additional factors limiting the usefulness of FDA resources are that the FDA only conducts approvals for pre-marketing studies with specific labeling most reliably available for primary efficacy outcomes. This leaves out information on post-marketing studies, off-label uses, and many secondary efficacy outcomes. For these data, ClinicalTrials.gov is an important resource. However, it is only recently that results are required to be uploaded in addition to the trials being registered on ClinicalTrials.gov. Furthermore, studies subject to regulation by the FDA, such as investigational device exemptions, are not required to be registered on ClinicalTrials.gov; and if these studies fail regulatory testing, such as futility analyses, the FDA will not make their outcomes or circumstances available to the public on their website since the device has likely not been approved.
The passing of Section 801 of the Food and Drug Administration Amendments Act (FDAAA 801) in September of 2007 means that results of trials conducted before this date are not required to be posted on ClinicalTrials.gov. Thus, identified trials on ClinicalTrials.gov older than this date without results would likely require FOIAs as well and, in reference to the statement two paragraphs above, this is not a highly viable option due to time constraints.
This activity does not intend to intentionally involve nor exclude or impact any small entities. The process used to collect data is designed to minimize the burden on all respondents. The OSF for SEADS includes one required field and allows for the submission of any scientific material. The required field is the submitter’s name. This is the minimum required information.
This is a one-time collection for each topic. If this collection is not conducted, it will negatively impact the scientific rigor and comprehensiveness of the research. Moreover, this research is intended to inform clinician and patient decision making in healthcare, and guidance in clinical practice. An incomplete assessment of the evidence due to the absence of SEADS runs the risk of biasing these decisions, and negatively impacting health outcomes for individuals and future research investments by researchers and research funders.
A particular manufacturer may develop an intervention that is used for multiple topics, or related topics. If this arises, an effort will be made to check previous submissions on related topics.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), a notice was published in the Federal Register on April 1, 2019 on page 12254for 60 days (see Attachment C ).
The EPC Program has previously consulted with outside consultants on general and specific areas of the OSF that remain unchanged from 2015. The consultants previously consulted include:
Harlan Krumholz, MD (Yale School of Medicine);
Kay Dickersin, PhD (Johns Hopkins Bloomberg School of Public Health); and
Steven Goodman, MD, PhD (Stanford School of Medicine).
No payments or gifts to respondents will be given.
Section 944 (c) of the Public Health Service Act [42 U.S.C. 299c 3(c)]requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied unless they consent to the use of the information for another purpose.
This activity does not entail questions of a sensitive nature.
Exhibit 1 presents estimates of the reporting burden hours for the data collection efforts. Time estimates are based on pilot testing of materials and what can reasonably be requested of respondents. The number of respondents listed in “Number of respondents per SEADS request” of Exhibit 1 reflects a projected 33% response rate with approximately 1-2 responses per request and assumes about 15 SEADS requests per year
Online Submission Form: A form for submitting scientific evidence and data related to medical interventions sponsored by organizations and individuals such as pharmaceutical companies and independent researchers. Other than upload of an information packet, the form has only one required field for the submitter’s name. Suggested items to include in the information packet include a list of completed and in progress studies and comment on if the list is complete.
Exhibit 1. Estimated annualized burden hours
Form Name |
Number of SEADS requests |
Number of SEADS request that receive response |
Number of responses per SEADS request |
Annual number of SEADS responses |
Hours per response |
Total burden hours per annum |
Online Submission Form (OSF) |
15 |
5 |
1.5 |
7.5 |
15/60 |
Exhibit 2. Estimated annualized cost burden
Form Name |
Number of SEADS requests |
Total burden hours per SEADS |
Average hourly wage rate* |
Total cost burden |
OSF |
15 |
1.87 |
$61.39a |
$ 115.10
|
aBased on the mean wages for Public Relations and Fundraising Managers, 11-2031, the occupational group most likely tasked with completing the OSF.
There are no direct costs to respondents other than their time to participate by contributing data voluntarily.
The total cost of this data collection to the government is $1,167.75 per year. The data collection is a one-time collection per topic. Exhibit 3 shows a breakdown of the government personnel costs related to this data collection effort.
Exhibit 3. Federal Government Personnel Cost
Activity |
Federal Personnel |
Hourly Rate |
Estimated Hours per topic |
Number of topics per year |
Cost |
Data Collection Oversight |
GS-14 |
$51.90 |
1.5 |
15 |
$1,167.75 |
Annual salaries based on 2018 OPM Pay Schedule for Washington/DC area: https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2018/DCB_h.pdf
Each SEADS request is a new collection. The burden is lower as we expect fewer responses. The nature of systematic reviews is to secure comparable evidence on the efficacy and effectiveness of numerous treatments for health related diseases and disorders. These reviews aim to inform healthcare decision making by clinicians and consumers, and inform guidance on clinical practice. The findings of these reviews are intended to help clinicians and consumers make the best decisions in their particular circumstances. In general, the goal for these reports is to be completed within a year. The steps that go into each review therefore are on a tight schedule and are not ongoing in order to fulfill their purpose. Thus, there are no ongoing collections of information from study sponsors and industry stakeholders for the same topic.
Exhibit 4 Idealized Data Collection Timeline for Each SEADS
|
|
Description (in chronological order) |
Due Date
|
Request/receive list of intervention sponsors (contacts) from EPC investigators |
Roughly 1 month following EPC award date of systematic review |
|
|
Final protocol of research review |
Roughly two months after contact list received |
Open SEADS submission portal |
Within 3 days of final protocol |
Send notification of opportunity to submit via email listserv |
Concurrent with portal opening |
Close SEADS submission portal |
4 weeks after letters sent |
Alert EPC investigator team of portal closure |
Within 2 days of portal closure |
Data analysis |
4-6 months after portal closure |
Final report (AHRQ publication) |
6-9 months after portal closure |
|
|
Publication Plan:
Research review results will be
disseminated through AHRQ publication under the auspices of the AHRQ
EPC and EHC Programs.
Analysis
Plan:
Provided any data submitted by intervention sponsors is not redundant and is useful for the purposes of either meta-analysis or evidence tables, the EPC investigator team will include it in the research review.
Exhibit 6. SEADS Collection and Analysis Plans
Instrument |
When administered and to whom |
Analysis sub-goal |
Analysis plan |
Notification of opportunity to submit SEADS email (Attachment A) |
|
None |
None |
Online Submission Form (Attachment B) |
|
Tabulate the responses to assess their impact on the systematic review. |
|
Data Entry Form (Not Required) |
|
Tabulate the responses from sponsors to assess their impact on the systematic review. |
|
AHRQ does not seek this exemption.
List of Attachments:
Attachment A -- Opportunity to submit Scientific Information email
Attachment B -- Website portal for Submission of Supplemental Evidence and Data for Systematic Reviews
Attachment C -- Federal Register Notice
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Statement Part A Template |
| Subject | <p>(Supporting Statement Template and Instructions; instructions are in italics and should be deleted when complete) |
| Author | Heather Nalls |
| File Modified | 0000-00-00 |
| File Created | 2021-01-16 |