Supporting Statement A for
National Institute of Mental Health (NIMH)
Recruitment Milestone Reporting (RMR) System
Revised March 2016
OMB No. 0925-0697
Name: Melba Rojas
Address: Office of Science Policy, Planning and Communications, NIMH
6001 Executive Blvd, MSC 9667
Bethesda, Maryland 20892
Telephone: 301-443-4335
Email: melba.rojas@nih.gov
Table of contents
A. JUSTIFICATION
A.1 Circumstances Making the Collection of Information Necessary
A.2. Purpose and Use of the Information Collection
A.3 Use of Information Technology and Burden Reduction
A.4 Efforts to Identify Duplication and Use of Similar Information
A.5 Impact on Small Businesses or Other Small Entities
A.6 Consequences of Collecting the Information Less Frequently
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency
A.9 Explanation of Any Payment of Gift to Respondents
A.10 Assurance of Confidentiality Provided to Respondents
A.11 Justification for Sensitive Questions
A.12 Estimates of Hour Burden Including Annualized Hourly Costs
A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record
keepers
A.14 Annualized Cost to the Federal Government
A.15 Explanation for Program Changes or Adjustments
A.16 Plans for Tabulation and Publication and Project Time Schedule
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions
List of Attachments:
Attachment 1: Screenshots of Recruitment Milestone Reporting (RMR) Database
Attachment 2: Summary of Public Comments Received
Attachment 3: Published 30-day Notice
Attachment 4: NIMH Policy for the Recruitment of Participants in Clinical Research
A. Justification
This revision request confirms a change in the Recruitment Milestones Reporting System (RMR) reporting requirements. The National Institute of Mental Health (NIMH) Recruitment Milestones Reporting System (RMR) is a tool developed by the NIMH to enable automated reporting and approval of enrollment milestones for clinical research studies, and to ensure NIMH-sponsored investigators develop realistic recruitment milestones and that these targets are met throughout the course of the clinical research. The system is currently approved for use under OMB No. 0925-0697, Expiration Date 05/31/2017. The system is designed to monitor the recruitment of participants in all NIMH-sponsored clinical trials, regardless of size, as well as other clinical research studies that plan to enroll 150 or more human subjects in a single study, and allows NIMH program staff to monitor more effectively the recruitment of participants.
Clinical studies can have difficulty recruiting, and accurate and timely reporting is the best way to ensure recruitment goals are met within the expected timeframe. Reaching recruitment goals enables studies to answer scientific questions with sufficient power (i.e., by having data from an adequate number of study participants), generalize results to the appropriate target population, and disseminate those results to providers and the general public. Ultimately, timely completion of recruitment ensures that scientific advances (e.g. treatments) are available to the public as quickly as possible.
A.1 Circumstances Making the Collection of Information Necessary
In order for NIMH clinical research to meet its overall objectives it is critical that each study meet agreed upon recruitment goals. In 2010, the Institute of Medicine (IOM) published a report on the National Cancer Institute’s (NCI) approach to clinical trials, A National Cancer Clinical Trials System for the 21st Century: Reinvigorating the NCI Cooperative Group Program. This report noted the slow pace of completion of NCI clinical trials as a concern. A 2013 report in the British Medical Journal showed that fewer than half of NIH-funded trials resulted in papers published in peer-reviewed journals within 30 months of trial completion, and a third still had no published results after a median of 51 months.1 Problems with slow completion and publication of results apply to NIMH clinical trials as well. For NIMH clinical trials, the median time to enroll the first subject is approximately 10 months after the Notice of Grant Award, when funding is released. For trials that began in FY 2013, about 30% of trials did not report enrolling subjects in their first year, which is associated with delayed completion of trials. Of the trials completed in 2014, 83% were completed while in a no cost extension (NCE), meaning that most trials were not completed in the original funding period. Even though no additional public funds are awarded during the NCE, the delay in completing a clinical trial means a delay in providing results to the public.2 Clinical research studies have difficulty recruiting participants for many reasons. An analysis of FY 2015 NIMH studies with recruitment challenges found lack of available participants, low response rates, higher than expected screening failures, recruitment site issues, institution transfers, IRB approval delays, changes to the study design and assessments, and staffing issues as the primary reasons for recruitment delays. These problems affect both large and small trials; an NIMH analysis of FY 2014 trials found no significant differences in meeting recruitment goals between large and small studies. As smaller trials provide information about safety, efficacy, and feasibility of performing a larger study, it is critical that these trials have sufficient participants to inform later studies and determine if investment in a larger trial is appropriate.
The NIMH Policy for the Recruitment of Participants in Clinical Research (see Attachment 4) allows NIMH Program Officials and other NIMH staff to monitor the recruitment of participants in all extramural NIMH-sponsored clinical trials, regardless of size, as well as other clinical research studies that plan to enroll 150 or more human subjects in a single study. The RMR system allows investigators to develop a recruitment plan that includes tri-yearly milestones for recruitment of the total study population, and for recruitment of racial and ethnic minority participants as required by NIMH policy. Once recruitment is scheduled to begin, investigators report actual progress on recruitment milestones three times per year (by April 1, August 1, and December 1), and investigators who fail to meet their milestones may be requested to submit interim monthly reports. The NIMH Program Official is responsible for the programmatic, scientific, and/or technical aspects of a grant, contract, or cooperative agreement. The Program Official evaluates study progress and provides oversight of study recruitment as part of his/her duties. Regular monitoring of recruitment allows the Program Official to determine if the study is meeting its goals, if intervention is needed, and if further funding is warranted. In calendar year 2014, Program Officials requested monthly monitoring for 43 studies that were not meeting recruitment goals. As of November 2015, over half of these studies met recruitment goals and/or increased the rate of recruitment of participants into the study.
Reporting of recruitment milestones in the RMR system applies to participants in all extramural NIMH-sponsored clinical trials, regardless of size, as well as other NIMH extramural-funded clinical research studies that plan to enroll 150 or more human subjects in a single study. Investigators who fail to meet their recruitment milestones may be requested to submit interim monthly reports. Generally, NIMH will consider studies to have failed to meet recruitment milestones if less than 50% of total cumulative target milestones are achieved at any point beyond the first year of recruitment, or if less than 75% of total cumulative recruitment milestones are achieved with less than 18 months remaining in the proposed project period. NIMH program staff may waive the requirement for monthly reporting at their discretion (e.g. for smaller studies subject to more variability in the percent target achieved). To ensure studies remain on track, monthly reporting will be required for four (4) months after the study shows evidence that recruitment is achieving 85% of their reported milestones, or until study completion (whichever occurs first).
The primary use of the information collected is to ensure that realistic recruitment milestones are established from the onset of a project, and that these milestones are met throughout the course of research. By ensuring timely recruitment into clinical research studies, NIMH can reduce the need to extend timelines or supplement funds in order to complete the research project, potentially increasing efficiency in the funding process and expediting the availability of treatments for mental illnesses. NIMH program staff use the information collected by the RMR system to evaluate the feasibility of the recruitment milestones and timeline, the appropriateness of the distribution of subjects to the research question, and the acceptability of actual recruitment progress. When an investigator submits the proposed recruitment milestone targets, the Program Official uses professional judgement to evaluate the plan for feasibility and acceptability as described above in Section A.1. If the plan is both feasible and acceptable, the Program Official approves the plan. If the proposed recruitment plan does not appear to be feasible or appropriate, the NIMH Program Official contacts the investigator prior to award to discuss potential modifications, allowing issues to be addressed as early as possible. The Program Official evaluates recruitment progress tri-annually. When progress is acceptable, the Program Official approves the progress. At any time, if recruitment falls significantly below the milestones projected by the PI/recipient and agreed to by NIMH, NIMH will determine viable options depending on the severity and duration of the recruitment shortfalls. Generally, the Program Official will first advise the PI/recipient to correct the deficiencies. When studies are identified as being at high risk of not completing (currently less than 5% of NIMH studies with 150 or more participants are at high risk), monthly reports allow NIMH staff to evaluate if changes to the recruitment plan are effective and if completion of the study is feasible. If studies identified as high risk achieve their total recruitment milestones, monthly reporting will no longer be required beyond four months after achieving at least 85% of their total recruitment milestones. If the PI/recipient fails to correct the deficiencies, NIMH will take further action, in accordance with Public Health Service policy3 (i.e., suspension, termination, or withholding of support). Staff in the Grants Management Branch or the Contracts Management Branch will ensure the information collected is included in official files and will work with program staff to determine the terms and conditions of the award. Staff in the NIMH Office of the Director and other appropriate NIMH offices will use the information to identify studies or research areas in which recruitment is a potential concern, and monitor the effectiveness of recruitment milestone reporting.
All information is submitted via the internet and collected in the electronic RMR software application. This method of collection is considered to be the least burdensome to the investigator. A Privacy Impact Assessment was completed for the database.
NIMH staff examined NIH and NIMH data sources to determine if existing information was available. NIH investigators currently provide an overall recruitment target, along with a breakdown by sex/gender, race, and ethnicity, as part of the application/proposal for a grant, cooperative agreement, or contract. This information is provided using the SF 424 or PHS 398 Targeted/Planned Enrollment Table. In addition, investigators report their actual recruitment, including the sex/gender, race, and ethnicity of subjects, in annual progress reports (PHS 2590).
NIMH staff determined that currently collected information is insufficient to meet recruitment monitoring needs, due to the lack of a stated timeline for recruitment and the infrequency of reporting. In order to identify recruitment challenges and recommend early corrective action, NIMH staff must know the proposed timeline for recruitment and receive frequent reports on progress towards recruitment milestones.
Principal Investigators of NIMH clinical trials, regardless of size, and NIMH clinical research studies planning to recruit 150 or more subjects will be required to submit recruitment milestones and tri-annual recruitment milestone progress reports. While most of these investigators are at universities and large research organizations, a small number of these investigators are at small businesses or other small entities. To minimize burden on investigators, including those at small businesses and other small entities, the information requested is the minimum required for the purpose of monitoring recruitment.
Recruitment Milestone Reporting requires NIMH investigators to submit planned recruitment milestones prior to funding, as well as tri-annual reports of actual recruitment progress for the duration of recruitment. Investigators who fail to meet their milestones may be requested to submit interim monthly reports. NIMH staff determined that tri-annual reporting is the minimum required to ensure effective recruitment monitoring, and studies with identified recruitment challenges require a minimum monthly reporting frequency for effective monitoring. Less frequent reporting would delay identification of studies with recruitment challenges, as well as delay technical assistance and corrective action. Delaying corrective action could result in the inability of the study to meet its objectives within the expected timeframe, including lack of statistical power to answer the research question, and lack of generalizability of the results to the target population. Ultimately, failure of the study to meet its objectives within the expected timeframe could result in the delayed availability of treatments to the public.
There are no special circumstances relating to the guidelines of 5 CFR 1320.5.
A Federal Register Notice for this revision request was published on June 1, 2015, Vol 80 FR 31053 (https://www.federalregister.gov/articles/2015/06/01/2015-13112/proposed-collection-60-day-comment-request-national-institute-of-mental-health-recruitment-and). Four (4) public comments were received – see Attachment 2 for a summary of the comments.
NIMH carefully considered all comments received. Comments include concerns about the clarity of the announcement, the utility of the information collected, the frequency of tri-annual and monthly reporting, and the need for educational materials and technical support.
To address the clarity of the announcement, NIMH has revised the language in this notice to confirm that the proposed collection includes all NIMH-funded clinical trials, regardless of size. To address concerns about the utility of the information collected, NIMH has included additional information about the circumstances requiring reporting, as well as the purpose and use of the information, in Sections A.1 and A.2 above. Additional explanation of the utility of reporting for small trials is in Section A.1 above. Both large and small trials are required to maintain records of enrollment due to Institutional Review Board and NIH requirements. Therefore, it is not expected that the burden of reporting for smaller trials will be greater than the burden for larger trials. NIMH Program Officials may waive the requirement for monthly reporting at their discretion, as explained in Section A.2 above.
To address concerns about the frequency of reporting, NIMH confirms tri-annual reporting to be the minimum necessary to provide effective recruitment monitoring. When studies fall significantly behind their recruitment goals, monthly reporting is necessary in order to ensure the study can be completed within the proposed budget and timeframe. Without intervention, studies may fail to recruit enough subjects to answer the scientific questions, resulting in delays in the availability of knowledge and treatments to the public; lack of generalizability of the information to the appropriate population; inappropriate exposure to risk for study participants; and inefficient use of resources. NIMH has also clarified the criteria that will be used to determine if a study requires monthly reporting. As explained in Section A.2, studies will require monthly reporting if less than 50% of target milestones are achieved at any point beyond the first year of recruitment, or if less than 75% of recruitment milestones are achieved with less than 18 months remaining in the proposed project period. Because studies generally take time to achieve a steady rate of recruitment, monthly reporting will not be required in the first year of a study, with the exception of short duration studies that have less than 18 months remaining when they fail to meet recruitment goals. Based on current study performance of clinical research studies with 150 or more participants, NIMH expects less than 5% of respondents will require monthly reporting. To address concerns about the need for educational materials and support, NIMH has developed a “Points to Consider about Recruitment and Retention While Preparing a Clinical Research Study” document which is available on the NIMH website (http://www.nimh.nih.gov/funding/grant-writing-and-application-process/recruitment-points-to-consider-6-1-05_34848.pdf). In addition, technical assistance to aid investigators with developing a recruitment plan is available from NIMH Program Officers.
Revisions to the RMR system were an internal NIMH policy change; therefore, there was no consultation with an outside agency.
No payment or gifts will be made to respondents.
The legal authority to collect this information is granted under 42 U.S.C Sections 232, 281 and 285 of the Public Health Service Act. These records will be maintained in accordance with the Privacy Act System of Record Notice 09-25-0036, (http://oma.od.nih.gov/public/ms/privacy/pafiles/0036.htm) covering “Extramural Awards and Chartered Advisory Committees (IMPAC 2), Contract Information (DCIS), and Cooperative Agreement Information, HHS/NIH.” The NIH System of Record Notice was previously published in the Federal register on September 26, 2002, Volume 67, No 187, page 60742.
No sensitive questions will be asked and no Personally Identifiable Information (PII) will be collected.
There is a slight decrease in the total annual burden hours for this revision request due to a typographical error on the burden table in the initial RMR application filed in 2014 (which lists 75/60 as the average burden per response, intended to mean 75% of 1 hour). The average burden per response is 45 minutes.
Estimated Annualized Burden Hours
|
|||||
Form Name |
Type of Respondent |
Number of Respondents |
Number of Responses per Respondent |
Average burden per response (in hours) |
Total Annual Burden Hours |
Tri-yearly NIMH Recruitment Milestone Reporting (RMR) |
NIMH Principal Investigators |
900 |
3 |
45/60 |
2025 |
Monthly NIMH Recruitment Milestone Reporting (RMR) |
NIMH Principal Investigators |
40 |
9 |
45/60 |
270 |
Total |
|
940 |
3,060 |
45/60 |
2295 |
The United States Department of Labor Bureau of Labor Statistics (May 2014) National Occupational Employment and Wages by Major Occupational Group is being referenced (Life, Physical and Social Science Occupations) – see http://www.bls.gov/oes/current/oes_nat.htm#19-0000.
Annualized Cost to the Respondents
|
||||
Type of Respondent |
Number of Respondents |
Average Burden per Response (in hours) |
Hourly Wage Rate* |
Respondent Cost |
NIMH Principal Investigators |
940 |
45/60 |
$33.69 |
$23,751 |
Staff |
Grade/Step |
Salary |
% of Effort |
Fringe (if applicable) |
Total Cost to Gov’t |
Federal Oversight |
|
|
|
|
|
RMR Administrator |
GS-13-1 |
$90,823 |
20% |
|
$18,165 |
RMR IT Project Manager |
GS-14-10 |
$139,523 |
.5% |
|
$698 |
|
|
|
|
|
|
Contractor Cost |
|
|
|
|
|
RMR IT Developer |
N/A |
$160,919 |
15% |
|
$2,413 |
|
|
|
|
|
|
Travel |
|
|
|
|
|
Other Cost |
|
|
|
|
|
|
|
|
|
|
|
Total Annualized Cost |
|
|
|
|
$21,276 |
This program change constitutes a revision request in RMR requirements. Currently, NIMH-sponsored clinical trials with less than 150 enrolled human subjects are not required to report in the RMR system. This revision of recruitment milestones reporting in the RMR system applies to participants in all extramural NIMH-sponsored clinical trials, regardless of size, as well as other clinical research studies that plan to enroll 150 or more human subjects in a single study. Investigators who fail to meet their milestones may be requested to submit interim monthly reports. Generally, NIMH will consider studies to have failed to meet recruitment milestones if less than 50% of target milestones are achieved at any point beyond the first year of recruitment, or if less than 75% of recruitment milestones are achieved with less than 18 months remaining in the proposed project period. NIMH program staff may waive the requirement for monthly reporting at their discretion (e.g. for smaller studies subject to more variability in the percent target achieved). To ensure studies remain on track, monthly reporting will be required for four (4) months following the achievement of at least 85% of total recruitment milestones or until study completion (whichever occurs first). A table showing the reporting timeline is below.
NIMH Recruitment Milestone Reporting Timeline
|
|
Activity |
Time Schedule |
Program Officer requests submission of recruitment milestones as part of just-in-time requests of grant, cooperative agreement, or contract |
2-3 months prior to award |
Deadline for submission of recruitment milestones of grant, cooperative agreement, or contract |
Prior to award date |
Investigator submits tri-annual recruitment progress reports for the duration of recruitment cycle |
April 1, August 1, and December 1 |
Tri-annual recruitment milestone reports provided to NIMH Director and NIMH Division Directors |
Mid-April, mid-August, and mid-December |
Investigators failing to meet recruitment milestones submit interim reports |
Monthly |
There is no specific plan to publish the data collected from this form. These data are for internal monitoring purposes.
The OMB expiration date currently displayed in the system is accurate.
These exceptions do not apply to this revision request.
1 5. Ross JS et al. Publication of NIH funded trials registered in ClinicalTrials.gov: Cross sectional analysis . BMJ. 2012 Jan 3; 344: d7292.
2 Gogtay N and Insel T. NIMH Clinical Trials: Portfolio, Progress to Date, and the Road Forward. National Institute of Mental Health. 2015.
3 See NIH Grants Policy Statement Part 2, Subpart A, 8.5.2 at http://grants.nih.gov/grants/policy/nihgps_2013/nihgps_ch8.htm and Federal Aquisition Regulations Part 49 at https://www.acquisition.gov/sites/default/files/current/far/html/FARTOCP49.html.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Statement 'A' Preparation - 04/05/2011 |
| Subject | Supporting Statement 'A' Preparation - 04/05/2011 |
| Author | OD/USER |
| File Modified | 0000-00-00 |
| File Created | 2021-01-24 |