Supporting Statement A for
Early Career Reviewer Program Application and Vetting System (EAVS)
National Institutes of Health
Center for Scientific Review (CSR)
OMB: 0925-0695
Expiration Date: 05/31/2020
Date: March 27, 2020
Check off which applies:
New
X Revision
Reinstatement with Change
Reinstatement without Change
Extension
Emergency
Existing w/o OMB approval
Name: Hope Cummings, Ph.D.
Address: 6701 Rockledge Drive, Room 4134, Bethesda, MD, 20892-7776
Telephone: 301-402-4706
Email: hope.cummings.nih.gov
Table of Contents
A. ABSTRACT 4
A.1 Circumstances Making the Collection of Information Necessary 4
A.2. Purpose and Use of the Information Collection 5
A.3 Use of Information Technology and Burden Reduction 6
A.4 Efforts to Identify Duplication and Use of Similar Information 7
A.5 Impact on Small Businesses or Other Small Entities 7
A.6 Consequences of Collecting the Information Less Frequently 7
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 8
A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside 8
A.9 Explanation of Any Payment of Gift to Respondents 9
A.10 Assurance of Confidentiality Provided to Respondents 9
A.11 Justification for Sensitive Questions 11
A.12 Estimates of Hour Burden Including Annualized Hourly Costs 11
A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record
keepers 13
A.14 Annualized Cost to the Federal Government 13
A.15 Explanation for Program Changes or Adjustments 13
A.16 Plans for Tabulation and Publication and Project Time Schedule 14
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate 14
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions 14
List of Attachments:
Attachment 1: Screen Shots of Online Application
Attachment 2: Sample Letters to Applicants
Attachment 3: ECR Website - www.csr.nih.gov/ECR
Attachment 4: Privacy Impact Assessment (PIA)
Attachment 5: Privacy Act Memo
Attachment 6: Notice of NIH’s Interest in Diversity (NOT-OD-031)
Abstract
The Center for Scientific Review (CSR) is the portal for NIH research grant applications and their review for scientific merit. Our mission is to see that NIH grant applications receive fair, independent, expert, and timely reviews – free from inappropriate influences – so NIH can fund the most promising research. To accomplish this goal, Scientific Review Officers (SRO) form study sections consisting of scientists who have the technical and scientific expertise to evaluate the merit of grant applications. The CSR Early Career Reviewer (ECR) program was developed to identify and train qualified scientists who are early in their scientific careers and who have not had prior CSR review experience. Currently, the online application software, the ECR Application and Vetting System (EAVS, see Attachment 1), is accessed by ECR program applicants who provide their names, contact information, a description of their areas of expertise, and their professional CV. The information is transmitted to SROs who vet the applications to determine if reviewers are an appropriate fit for their review panels. This information collection request (ICR) is to revise the EAVS software by including additional questions and to make it more user-friendly.
A.1 Circumstances Making the Collection of Information Necessary
NIH Center for Scientific Review: Overview
This is a request for a revision of an Office of Management and Budget (OMB) approval, under the Paperwork Reduction Act of 1995 (OMB No. 0925-0695, Expiration Date: 05/31/2020). This clearance approves an information collection request (ICR) to use the EAVS software to process applications for the ECR program.
The Center for Scientific Review (CSR) is the portal for NIH grant applications and their review for scientific merit. We receive all research grant applications sent to NIH and handle the review of more than 75% of those by organizing peer review groups (referred to as study sections or review panels) to evaluate research grant applications. Our mission is to see that NIH grant applications receive fair, independent, expert, and timely reviews – free from inappropriate influences – so NIH can fund the most promising research.
To accomplish this goal, Scientific Review Officers (SRO) form study sections consisting of scientists who have the technical and scientific expertise to evaluate the merit of grant applications. Potential study section members are identified by SROs by canvassing the scientific literature, outreach at national scientific meetings, review of successfully funded applications, and by referral from other scientists. The legislative authority to collect information from potential reviewers to determine their appropriateness to serve on grant application review panels is 42 USC Section 241: Research and Investigations General.
Early Career Reviewer Program
The NIH Center for Scientific Review (CSR) Early Career Reviewer (ECR) program was developed to (1) identify and train qualified scientists who are early in their scientific careers and who have not had prior CSR review experience to become effective reviewers, (2) help emerging researchers advance their careers by exposing them to peer review, and (3) enrich the existing pool of NIH reviewers by including scientists from less research-intensive institutions as well as those from traditionally research-intensive institutions. This PRA clearance request is to revise our online application software for ECR program applicants.
ECR Application Process
To apply for inclusion in the ECR Program, applicants provide their Curriculum Vitae (CV) or professional resumes and provide some identifying information, areas of expertise, and selection of preferred study sections via an online application. They are vetted for inclusion in the ECR program primarily based on the information provided in their CVs. CVs provide substantial information regarding current employment, publication history, grants received, and other professional achievements, all of which are very useful in evaluating applicants’ eligibility to serve on study sections as reviewers. A customary practice of SROs in the course of their duties at CSR is to review the CVs of potential reviewers.
To facilitate vetting of ECR applicants, CVs and information regarding their areas of scientific and technical expertise are forwarded through an interface from the online application to SROs of the study sections selected by applicants. SROs evaluate applicants’ eligibility and respond electronically to the ECR program staff regarding the vetting outcome. Letters are sent via email to applicants informing them of their acceptance in the ECR program or providing them with feedback as to why they did not qualify (see Attachment 2: Sample Letters to Applicants). The latter group is encouraged to strengthen their CV and reapply at a later date.
Since the initiation of the OMB approved Early Career Reviewer Application and Vetting System (EAVS), we have received more than 6,595 registrations to use the system and more than 6,401 applications. We currently average approximately 101 applications per month. The vetting component of the system refers applications to Scientific Review Officers who review approximately 98 applications per month.
In summary, ECR applicants’ contact information, CV, and areas of expertise must be collected to appropriately evaluate them for inclusion in the ECR program. An online application system is more cost-effective and efficient than transmitting information between applicants and ECR staff by mail. The software program (EAVS) has been developed to assist with this effort. A revision of our OMB clearance is being sought to continue our use of the software with some revisions that include minor aesthetic enhancements and additional questions to capture applicant’s program eligibility as well as applicants’ socio-demographic and socio-economic characteristics to support NIH’s renewed Interest in Diversity (NOT-OD-20-031).
ECR Application and Vetting System (EAVS) Application
This is a Revision application. The primary purpose of this ICR is to continue to gather information from ECR program applicants via an online application in a systematic way that reduces burden on the applicants and governmental costs associated with processing. ECR applicant information is provided for the purpose of evaluating qualifications to serve on review panels at CSR. Information gathered to determine eligibility will include the ECR applicants’ names and institutions, areas of expertise, training credentials, professional accomplishments, email addresses, ORCID and NIH Commons names, most of which is provided in their CVs (see Attachment 1: Screen Shots of Online Application Format). Applicants will continue to be asked to select up to four study sections that match their areas of expertise. This information will be used to help SROs identify applicants that are appropriate matches for their review panels. The provision of a Commons ID and the ORCID allows staff members to more easily add selected ECRs to study section rosters via NIH data systems (IMPACII). The ECR website (Attachment 3) provides a link to the EAVS online application. The application software was developed by the CSR’s Information Management Branch (IMB) in collaboration with CSR ECR staff for use by ECR applicants and for CSR SROs.
The request to revise the application to add sensitive questions are in line with NIH’s renewed Interest in Diversity (i.e. NOT-OD-20-031). These sensitive questions include applicants’ race, ethnicity, sex, disability and disadvantage background. Applicants can choose if they would like to answer these sensitive questions (i.e. optional). Periodic summaries of the information collected (e.g. number of applicants by institution or state, applicants’ socio-demographic characteristics) will be generated and used to help assess interest and acceptance in the ECR program as it relates to applicants’ characteristics. These summaries will help with program evaluation and may be used to improve current processes (e.g. outreach efforts). Information will continue to be reported in aggregate form without individual identifiers. The ECR program is not a systematic investigation and the data gathered will not produce any generalizable knowledge.
Other revisions include aesthetic enhancements to the online interface and additional questions to assess applicant’s program eligibility. If an applicant is ineligible for participation in the program based on the general qualifications (e.g. prior NIH reviewer), the system immediately informs the applicant the reasons for disqualification.
ECR Application and Vetting System (EAVS) - Burden for Applicants
EAVS allows applicants to access an application form through our ECR website and provide all the information needed for vetting the applicant for inclusion in the ECR program. This has reduced burden on the applicant and is more efficient and cost-effective for CSR. This online system reduces staff time to process applications and increases the speed of transmission of applications for the ECR program to SROs who are in the best position to evaluate applicants’ appropriateness to serve as reviewers. It also allows for seamless provision of feedback to applicants regarding their eligibility and places those accepted into a pool of applicants who will be invited to participate in future study sections. Attachment 1 provides screenshots of EAVS. Completion of the online application should take no longer than 25 minutes per applicant.
Privacy Impact Assessment
A Privacy Impact Assessment was completed for the NIH CSR Early Career Reviewers Database. This is renewed annually. (see Attachment 4) The most recent PIA is for the continued maintenance of the database of applicants for the ECR program. The online application software directs information into the NIH CSR Early Career Reviewers Database. It was determined by the CSR Privacy Officer that a Confidentiality Agreement would not be necessary for this proposed project. The existing protections afforded to all reviewers who serve on committees apply to applicants for the ECR program. See Attachment 5: Privacy Memo.
Most of the information that is gathered through the EAVS application is not available from any other source or agency. Some of the eligibility information gleaned from applicants’ CVs might also be available through university websites, IMPACII, social media sites, professional association websites or via telephone or email contact with the applicant. These sources are used inconsistently by applicants and may include outdated or incomplete information, all of which introduce the potential for inconsistency in the manner in which applicants are evaluated.
Researchers from small organizations are eligible to apply for the ECR program using EAVS. The time estimated for completion of the online application is the same for all applicants, including those from small businesses.
Frequency of Application
Applicants for the ECR program complete the application and upload their CVs only once after selecting the “Apply for the ECR Program” option in EAVS. Applicants who do not meet the minimum requirements for inclusion in the ECR program are provided with feedback on why they were not accepted. All of those not accepted have the option to reapply after they have had time to strengthen their qualifications. Reapplication can occur at any time and consists of repeating the original application steps. Applicants can apply more than twice if needed. It is possible that those accepted into the ECR program will change jobs and/or contact information during their time of participation in the ECR program. These individuals can update their information using the same online application by selecting the “update your information” option (see Attachment 1). Annual queries will be made to those who are eligible, but not yet recruited to a panel, to determine continued interest in the ECR program. If the response is in the negative or no response is received, their information will be archived or removed from the database.
Consequence of not collecting applicant information
As described in Section A.4, some of the eligibility information gleaned from applicants’ CVs might also be available through other sources, but these sources are used inconsistently by applicants and may include outdated or incomplete information— all of which introduce the potential for inconsistency in the manner in which applicants are evaluated. Completing the online application and uploading a CV standardizes the process and helps ensure that all applicants are evaluated based on the same criteria. The online application and CV also greatly reduce staff time that would be needed to conduct detailed online searches for information on each applicant. Given the high volume of applicants, it is more cost-effective to ask applicants to provide their CVs and other information (e.g. degrees, training, study section preferences) than to hire staff to conduct individualized online searches to verify the qualifications of each applicant. It also prevents the problem of misidentifying information online as belonging to applicants when it may belong to another person with a similar name. Finally, providing all information at the time of application reduces the number of times that CSR staff will have to contact the applicant to verify or request additional information.
The ECR Application and Vetting System (EAVS) fully complies with all guidelines stated in 5 CFR 1320.5.
January 31, 2020, Volume 85, Number 21, Pages 5677-5678. There were no public comments in response to this Federal Register Notice.
Consultation with advisors regarding the program
The plan to continue to use EAVS was discussed with and approved by the Director of CSR, Noni Byrnes; Associate Director for Diversity and Workforce Development, Gabriel Fosu; Senior Advisor to the Director, Christine Melchior, who also directs the ECR program; and the staff of the Information Management Branch of CSR who developed the online application software. Feedback received from Scientific Review Officers at CSR have led to minor improvements in EAVS that have either simplified or added greater clarity to instructions. In addition, as the qualifications for acceptance to the ECR program have been updated, corresponding changes have been made in EAVS.
Feedback from persons outside the agency
This application is not for a research project. Eight hundred ECRs were surveyed in May 2016 regarding the ECR program and suggestions were solicited for improvements in the program. No recommendations were received to discontinue or to modify EAVS.
Internal Testing of EAVS
The ECR program staff have engaged in continuous quality control of all processes related to the application and vetting of applicants with EAVS. They have worked with IMB staff to make minor adjustments to the software to facilitate all processes. Since the original OMB approval of EAVS, the requirements for participation in the ECR program have slightly changed. Corresponding changes have been made in EAVS and tested internally by ECR Program staff and CSR review staff.
NIH Agency consultation
Scientific Review Officers at the Center for Scientific Review with experience using EAVS have also provided feedback on EAVS. They described the use of EAVS for processing applications to the ECR program as easy to use, straightforward and essential to the successful implementation of the ECR program.
Participation in the ECR program is voluntary. No compensation is provided for applying to the program. Those selected to be part of the ECR program have the opportunity to serve on review panels. All reviewers on study sections are paid an honorarium for their reviews. No additional compensation is paid to ECRs.
System of Record
The IT system in which information is stored is titled the NIH CSR Early Career Reviewers Database. The system was assessed with a Privacy Impact Assessment. The date of the completion of the PIA was 12/16/16. The Privacy Act applies to this activity. The number of the Privacy Act Systems of Record Notice that covers the information collection is 09-25-0036 “NIH Extramural Awards and Chartered Advisory Committee (IMPAC II), Contract Information (DCIS), and Cooperative Agreement Information, HHS/NIH.” The date of publication in the Federal Register was September 26, 2002 (see Attachment 5).
Privacy
It was determined by the CSR Privacy Officer that a Confidentiality Agreement would not be necessary for this project. The existing protections afforded to all reviewers who serve on committees apply to applicants for the ECR program and all information will be private to the extent permitted by law. Personally identifiable information (PII) will continue to be collected (see A.2) Also, applicants will maintain control of the location from which they complete the online application. This allows applicants to maintain control over the privacy of their data during the application process.
Disclosures to Applicants
The ECR website (see Attachment 3) currently includes information regarding the nature of the ECR program, benefits of participation in the ECR program, eligibility requirements, and what can be expected to occur following application. Details have been included regarding the online application process, a clear statement that participation is voluntary, and confidentiality assurances. Applicants who do not wish to or are unable to complete an online application can contact the ECR office by email to apply for the program.
This revision includes capturing sensitive information from applicants. The request to revise the application to add sensitive questions are in line with NIH’s renewed Interest in Diversity (i.e. NOT-OD-20-031). These sensitive questions include applicants’ race, ethnicity, sex, disability and disadvantage background. Applicants can choose if they would like to answer these sensitive questions (i.e. optional). Providing this information
is not required for a complete application, acceptance into the program, or for recruitment as a reviewer. Applicants will be told that this sensitive information is optional and will be given a web link to learn more about NIH’s Interest in Diversity Notice (see Attachment 6). Periodic summaries of the sensitive information will be generated and used to help assess interest in and acceptance into the ECR program. These summaries will help with program evaluation and may be used to improve current program processes (e.g. outreach efforts). Summaries will be reported in aggregate form, without individual identifiers, and will not produce any generalizable knowledge.
Also, applicants’ CVs will continue to be collected as part of the online application. While the CV typically contains only academic achievements and credentials, it is possible for applicants to include sensitive information in their CVs. All CVs are stored in a password protected shared filed that is only accessible by ECR Program Staff and SROs at CSR.
Annual Hour Burden
The current flow of applicants is approximately 101 per month. Assuming a constant rate, the ECR program should receive approximately 1212 applications per year. Using an estimated response time of 25 minutes per application, the annual burden for applicants is 505 hours.
A.12 - 1 ESTIMATES OF ANNUALIZED BURDEN HOURS
|
||||
Type of Respondents |
Number of Respondents |
Number of Responses per Respondent |
Average Burden Per Response (in hours) |
Total Annual Burden Hours |
Research scientists |
1212 |
1 |
25/60 |
505 |
Totals |
|
1212 |
|
505 |
A.12.2 Annualized Cost to Respondents
A survey of 171,487 full-time faculty from 847 institutions published in the 2019 College And University Professional Association for Human Resources (CUPA-HR) (https://www.higheredjobs.com/salary/salaryDisplay.cfm?SurveyID=51) showed that assistant professors across disciplines are earning an average of $70,028 per year for 4 year public and private schools based on a 9-month work year. This can be extrapolated to an average of $93,372 per year based on a 12-month work year and $48.63 per hour based on a 40-hour work week. The vast majority of New Investigators are Assistant Professors. Therefore, this number was used to calculate the annualized costs to respondents.
Based on an estimate of 1,212 applications per year, each taking 25 minutes to complete, the anticipated average annual respondent cost of the ECR online application is $24,558.15.
Type of Respondents |
Total Annual Burden Hours |
Hourly Respondent Wage Rate* |
Respondent Cost |
Research Scientists |
505 |
$48.63 |
$24,558.15 |
Totals |
505 |
|
$24,558.15 |
* - Source of estimate: (https://www.higheredjobs.com/salary/salaryDisplay.cfm?SurveyID=51
As described in Section A.3 on Burden Reduction, NIH/CSR staff members are involved in maintaining the software application and in the processing of applications for the ECR program. EAVS greatly reduces the staffing resources needed to process applications, provide feedback to applicants, and manage application data. ECR personnel costs related to the processing of applications includes 2% Program Director time and 15% Software Developer time. The estimated annual cost to the federal government for is $25,256.00.
Cost Descriptions |
Grade/Step |
Salary |
% of Effort |
Fringe (if applicable) |
|
Total Cost to Gov’t |
Federal Oversight |
|
|
|
|
|
|
Program Director |
15/8 |
170,800 |
2% |
|
|
3,416.00 |
Contractor Cost |
|
|
|
|
|
|
Senior Software Developer |
|
130,000 |
15% |
12% |
|
21,840.00 |
Travel |
|
|
|
|
|
|
Other Cost |
|
|
|
|
|
|
|
|
|
|
|
|
|
Total |
|
|
|
|
|
25,256.00 |
Minor adjustments have been made in the ECR program requirements and to the EAVS application that correspond to these changes. Additional changes to EAVS include 1) minor aesthetic enhancements to the online interface, 2) sensitive questions that capture applicants socio-demographic and background characteristics (see A.11 for explanation) and 3) an additional question to check applicant’s eligibility for the program and immediate system feedback with reasons for disqualification from the program.
We are not requesting approval to not display the expiration date for OMB approval. Therefore, Section A.17 is not applicable.
We are not requesting an exception to certification for the Paperwork Reduction Act. Therefore, Section A.18 is not applicable.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Statement 'A' Preparation - 04/05/2011 |
| Subject | Supporting Statement 'A' Preparation - 04/05/2011 |
| Author | OD/USER |
| File Modified | 0000-00-00 |
| File Created | 2021-01-14 |