Supporting Statement A for
Prevention Research Expertise Survey
National Institutes of Health, Office of Disease Prevention (ODP)
OMB# 0925-0728, Expiration date 12/31/2018
Date: 09/06/2017
Check off which applies:
New
Revision
Reinstatement with Change
Reinstatement without Change
Extension
Emergency
Existing
Federal Government Employee Information:
Name: David Murray, Ph.D.
Address: 6100 Executive Blvd, Suite 2B03, MSC 7523, Bethesda MD 20892
Telephone: (301) 827–5561
Fax: (301) 480–7660
Email: david.murray2@nih.gov
Table of contents
A. JUSTIFICATION
A.1 Circumstances Making the Collection of Information Necessary
A.2. Purpose and Use of the Information COLLECTION
A.3 Use of Information Technology and Burden Reduction
A.4 Efforts to Identify Duplication and Use of Similar Information
A.5 Impact on Small Businesses or Other Small Entities
A.6 Consequences of Collecting the Information Less Frequently
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency
A.9 Explanation of Any Payment of Gift to Respondents
A.10 Assurance of Confidentiality Provided to Respondents
A.11 Justification for Sensitive Questions
A.12 Estimates of Hour Burden Including Annualized Hourly Costs
A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record keepers
A.14 Annualized Cost to the Federal Government
A.15 Explanation for Program Changes or Adjustments
A.16 Plans for Tabulation and Publication and Project Time Schedule
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions
List of Attachments
Attachment 1: Screenshots of Prevention Research Expertise Survey
Attachment 2: PRES Consultants and Advisors
Attachment 3: PIA Memo
Attachment 4: Participant Emails
Attachment 5: Privacy Act Memo
A. Justification
To identify experts in prevention science methods, we worked with our contractor, IQ Solutions, Inc., to develop online software which will allow us to collect researchers’ names, contact information, and resumes, as well as to have those researchers identify their level of expertise in a variety of prevention science methods and content areas. The data collected with this software was used to support a web-based Electronic Directory that SROs can use to identify researchers with expertise in specific prevention science methods and content areas for invitation to serve on one of the NIH review panels. This system will also be shared with review staff in the other Institutes and Centers at NIH, as well as other DHHS agencies. Given our plans to create an automated system for reviewer information collection, we are now seeking OMB approval for a revision to our data collection plan (see Attachment 1) for 3 additional years.
A.1 Circumstances Making the Collection of Information Necessary
The NIH Office of Disease Prevention (ODP) Prevention Research Expertise Survey (PRES) program was developed to (1) identify methodologists with expertise in content areas related to prevention science, (2) identify mid- and senior- level researchers who may have an interest in serving on review panels, and (3) in its initial phase, work with the NIH Center for Scientific Review (CSR) to enrich the existing pool of NIH reviewers by including researchers with methodological and prevention science expertise that review prevention applications.
Researchers interested in providing their information for the PRES program will submit some identifying information, content and methodological areas of expertise, Curriculum Vitae (CV) or professional resumes, and willingness to serve on a review panel. They are included in the PRES program based on their self-reported level of expertise in methodological and prevention science content areas, as well as, the information provided in their CVs. CVs are commonly used documents for recording work history and accomplishments. Creation and updating of a CV is a common exercise of researchers and other professionals. A customary practice of Study Review Officers (SROs) in the course of their duties is to review the CVs of potential reviewers. CVs provide substantial information regarding current employment, publication history, grants received, and other professional achievements, all of which are very useful in evaluating applicants’ eligibility to serve on review panels as reviewers. The legislative authority to collect information from potential reviewers to determine their appropriateness to serve on grant application review panels is 42 USC Section 241: Research and Investigations General.
Study Review Officers will continue to have access to researchers’ CVs and self-reported assessment of methodological and prevention science content expertise through the PRES and Electronic Directory. PRES program staff will continue to maintain the PRES and Electronic Directory and these data will be available to SROs as they recruit members of their review sections. In summary, PRES participants’ contact information, CV, and methodological and prevention science content areas of expertise must be collected for inclusion in the PRES program.
Since our previously approved OMB request, we have collected data from roughly 2,700 investigators and approximately 30 NIH Scientific Review Officers (SROs) are using the database. The SROs informed us during their piloting and use of the database that there were additional topics they wanted to be included to help them better identify investigators to serve on review panels. Thus, the primary purpose of this OMB clearance request is to collect additional data not collected in the previously deployed online survey including additional study design topics (4 new, 1 deleted), research methods topics (9 new, 1 deleted), content topics (12 new, 1 deleted), populations (6 new, 2 deleted), as well as a new geographic region of research category with 10 new topics and the income category of the region/country in which the research is performed with 4 new topics. This will be done via the already developed and launched PRES online survey in a systematic way that reduces burden on the participants and governmental costs associated with processing (see Attachment 1 to view the updated survey with new topics marked as “New”).
A.3 Use of Information Technology and Burden Reduction
Survey Completion via Email - Burden for Prevention Researchers. Those interested in joining the PRES program will be directed to the ODP website to complete the survey. An online survey is more cost-effective and efficient than transmitting information between scientists and PRES program staff by mail. The software program has already been developed and is currently in use. The website will direct users to complete a user profile and assess their expertise in three areas: methodological, content area, and population focus. Based on usability testing, the total estimated time burden for users is approximately 25 minutes.
Privacy Impact Assessment. In January, 2015, the Privacy Impact Assessment (PIA) Form was created for the NIH ODP Prevention Research Expertise Survey program and submitted to the NIH Senior Official for Privacy (see Attachment 3). The PIA included the development of a database of respondents for the PRES program. As indicated above, the database includes applicants’ names, email address, academic institution and position, areas of expertise, and NIH eRA Commons ID. The online survey software will direct information into the PRES Electronic Directory.
The information that will be gathered through the PRES is not available from any other source or agency.
Researchers from small organizations will be eligible to complete the online survey. The information being requested of all prevention researchers has been held to the minimum required for evaluation of appropriateness for inclusion in the PRES program and NIH review panels.
Frequency of Survey Completion. Prevention researchers will complete the survey and upload their CVs only once. If those researchers gain more expertise, change jobs, and/or contact information, after completing the survey, they can update their information using the same online mechanism by selecting the “update your information” option (https://prevention.nih.gov/prevention-research/expertise-survey).
Consequence of not collecting information. As described in Section A.4, some of the information gleaned from researchers’ CVs might also be available through university websites, IMPACII, social media sites, professional association websites or via telephone or email contact with the applicant. These sources are used inconsistently, may include outdated or incomplete information, and present the potential for inconsistency in the manner in which possible reviewers are evaluated. Completing the online survey and uploading a CV standardizes the process and provides SROs with one place to find mid- and senior- level methodological experts for review panels.
The online survey and CV also greatly reduces staff time that would be needed to conduct detailed online searches for information on each applicant. Given the high volume of prevention researchers, it is more cost-effective to ask PRES program participants to provide their CVs and information regarding their degree, training, methodological and content area expertise, and willingness to serve on a NIH study section, than to hire staff to conduct individualized online searches to verify the qualifications of each participant. It also prevents the problem of misidentifying information online as pertaining to a participant when it may belong to another person with a similar name. Finally, providing all information at the time of survey submission reduces the number of times that ODP PRES program staff or SROs contact the PRES participant to verify or to request additional information.
The PRES software and survey project fully complies with all guidelines stated in 5 CFR 1320.5. No special circumstances
This proposed information collection was previously published in the Federal Register on July 6, 2017, Volume 82, Number 128, pages 31337-31338, and allowed 60 days for public comment. No public comments were received.
Consultation with advisors regarding the program. The plan to develop PRES was discussed with and approved by the Associate Director for Prevention and Director of the Office of Disease Prevention, Dr. David M. Murray; Director of CSR, Dr. Richard Nakamura; Director of the CSR Division of AIDS, Behavioral and Population Sciences, Dr. Karyl Swartz; the staff of the Information Management Branch of CSR who developed the Early Career Reviewer Application and Vetting System (EAVS) online application software; and the ODP Prevention Research Coordinating Committee. Their recommendations were incorporated into the development plans for the online survey software.
Feedback from outside consultants. This application is not for a research project. Comments were solicited from persons outside of NIH regarding the planned online survey system. These individuals are senior level investigators with expertise in methodology (See Attachment 2 for a list of “Outside Consultants.
Internal Advisors. IQ Solutions began internal testing of NIH advisors on September 15, 2014. This allowed for identification and resolution of programming errors. On October 8, 2014, IQ Solutions, Inc. staff provided PRES program staff with test cases as part of their internal Usability Acceptance Testing. This included provision of scenarios that PRES participants would be likely to encounter. This allowed PRES staff to pre-test the responsiveness of the program to common scenarios encountered when completing the survey such as not having an NIH eRA Commons ID. In addition, PRES program staff carefully examined each screen seen by program participants and provided feedback to IQ Solutions, Inc. staff regarding cosmetic changes and programming problems. Internal testing of the survey has been completed and PRES was formally launched after OMB approval in January 2016.
In addition to PRES program staff, senior staff at ODP with previous experience as Program Officers volunteered to assist with Usability Acceptance Testing of the survey to vet completion ease and time. Feedback was provided during the pilot testing process. Some feedback included ambiguity with terms used to describe methods and indicating interest in participating in a review panel, which have been addressed. Other problems included awkward placement of “user profile” buttons and lack of white space on background page. In all cases, the feedback provided was incorporated into refinements of survey (see Attachment 1).
NIH Agency consultation. The development of plan for creating the PRES and Electronic Directory was part of a collaborative effort between the ODP program staff, IQ Solutions (IQS), and the Director of ODP. A draft of the plan and the beta version of the software designed to collect information online from applicants/participants and the flow of information from PRES participant to the Electronic Directory was developed by ODP Staff and IQ Solutions and approved by Dr. David Murray, Director of the ODP. Paris Watson and Dr. Jocelyn Lee, the PRES Coordinators, worked with Rich Panzer and Jason Hamrick, IT software developers from IQS, to develop the PRES program development plan that detailed the information to be collected using the online survey, the flow of data into a secure database (called the “Electronic Directory”), how the information will be resourced and accessed by the SROs.
The beta version of the Electronic Directory was reviewed and analyzed for usability by internal advisors made up of NIH Center for Scientific Review (CSR) SROs (see Attachment 3). After their feedback was incorporated into the plan for developing the Electronic Directory, the PRES development plan was reviewed again by Dr. David Murray, Director of ODP. Watson, Panzer, and Hamrick, and Lee presented the plan to Dr. Murray. The PRES development plan and budget for production were approved.
One of the main purposes for developing the PRES is to help the SROs identify experts in prevention science methods to include on their review panels. The data collected with this software was used to create the Electronic Directory that NIH review staff can use to identify researchers with expertise in specific prevention science methods and content areas for invitation to serve on one of the CSR review panels. The Electronic Directory was developed in parallel with the online survey because the survey’s database will serve to populate the Electronic Directory with potential reviewers. In coordination with Dr. Karyl Swartz (Director of the CSR Division of Aids, Behavioral and Population Sciences), a group of CSR SROs were selected for a 30-day beta-testing of the Electronic Directory. Due to the timeline for planning of the September/October 2015 CSR study sections, we convened PRES Electronic Directory beta training session in late June 2015. The training session for the Electronic Directory was led by Panzer and Hamrick from IQ Solutions in coordination with Watson and Lee from ODP.
In order to capture SRO user feedback and any problems or questions related to the Electronic Directory, we created a dedicated email address (odpsrorolodex@mail.nih.gov) that was reviewed daily and maintained by ODP PRES staff. In addition, after the 30-day beta-testing of the Electronic Directory, (June 2015), a Usability Focus Group was convened in early August to capture final feedback that will be incorporated into the Electronic Directory. The requested revisions to the data collection for PRES are based on feedback from the SROs and CSR after they have used the tool for the past year (2016) (See Attachment 2 for list of NIH Advisors).
A.9 Explanation of Any Payment of Gift to Respondents
The opportunity to participate in the PRES program is voluntary. No compensation is provided for completion of the PRES survey.
A.10 Assurance of Confidentiality Provided to Respondents
System of Record. The number of the Privacy Act Systems of Record Notice that covers the information collection is 09-25-0036, titled, Extramural Awards and Chartered Advisory Committees (IMPAC 2), Contract Information (DCIS), and Cooperative Agreement Information, HHS/NIH published September 26, 2002 vol. 67 pgs. 60742-60784 (See Attachment 5 Privacy Act Memo).
To maximize the protection of their personal information, investigators will be asked to complete a user profile prior to completing the PRES program survey that will include creation of a username and password. Applicants’ CVs and additional identifying data will be saved on a secure website hosted by the NIH Office of Information Technology (OIT). This data will be accessible only by select DHHS staff with PIV cards that have been granted permission by ODP PRES program staff. ODP PRES staff will work with OIT to grant select SROs access to the Electronic Directory and will delete permissions for access if an employee leaves their position as a Study Review Officer. SROs must login to the database with their PIV cards.
Since PRES was approved for use by OMB in late 2015, the website has included a statement regarding privacy as part of the introduction to completing the survey. That statement indicates that the information provided will be kept private to the extent allowed by law and not disclosed to anyone but the staff of ODP PRES program and DHHS staff who evaluate submissions except as otherwise required by law. We also included in the introduction to the survey a statement that clearly states that 42 USC Section 241: Research and Investigations General provides legislative authority to collect information for the PRES program, that participation in the program is strictly voluntary, and that no consequence exists for choosing not to participate.
Prevention Research Expertise Survey and Electronic Directory. The information to be retained in the searchable Electronic Directory includes investigators’ names, job title, email address and institutions. We are not collecting date of birth, social security number, home address, race, ethnicity, or gender. We have chosen not to collect race and ethnicity data two reasons: (1) the information provided PRES program participants will be made available to SROs for the purpose of potential inclusion in study review panels. We want to avoid potential bias as well as the appearance of bias based on race/ethnicity in their selection process. (2) When investigators create NIH eRA Commons accounts, they have the option to choose whether or not to disclose information regarding race and ethnicity. Access to those data is limited to protect the applicant. However, when data summaries by race/ethnicity are needed for program evaluation purposes, only approved personnel with specific responsibility for data summaries can access those data. PRES program staff will not have access to sensitive data from individual PRES participants.
The information to be collected via the online survey is the same information used by SROs to vet all other potential review panel members. In the course of daily job duties, SROs gather information on potential reviewers including education and job title, professional accomplishments, publications, and any other information that represents the expertise of the potential reviewer.
The revision to the online survey will allow for collection of identical types of information from each PRES participant in addition to their self-assessed expertise in methodological, content area, population focus, geographic region of research and the income category of that region as area fields (see Attachment A, Revisions to PRES Survey). Summaries are periodically tallied in an aggregate form for administrative use regarding the institutions at which investigators are employed. Names of individual investigators are not included in these summaries. The information from participants is not used for research or survey purposes.
A.11 Justification for Sensitive Questions
Sensitive information is not needed to determine each researcher’s fit with a prevention-focused review panel and will therefore, not be collected.
A.12.1 Estimates of Hour Burden Including Annualized Hourly Costs
Annual Hour Burden. Based on data and analytics tracking PRES participation over FY2016, it appears that the flow of participants will be approximately 60 per week for a new investigator completing the survey for the first time. Assuming a constant rate, the PRES program should receive approximately 3,120 new investigator survey completions per year. Using an estimated response time of 25 minutes per survey, the annual burden for participants is 1300 hours. Section A.3 provides additional details regarding this estimate. Returning investigators (those who have completed previous versions of the survey) will also be invited to return and update their information based on the additional topics added (discussed in sections A2 and A15). Using an estimated response time of 15 minutes per returning investigator to update their responses to the new version, the annual burden for those participants is 250 hours. Please see Table 12-1 below for details.
Table 12-1 Estimated Annualized Burden Hours
Type of Respondents |
Number of Respondents |
Number of Responses per Respondent |
Average Burden Per Response (in hours) |
Total Annual Burden Hours |
Investigators |
3120 |
1 |
25/60 |
1300 |
Returning Investigators to update information |
1000 |
1 |
15/60 |
250 |
Total |
4120 |
4120 |
|
1550 |
A.12-2 Annual Cost to respondent
A survey of college professors published in April 2016 in the Annual Report on the Economic Status of the Profession, 2015-16 (https://www.aaup.org/report/higher-education-crossroads-annual-report-economic-status-profession-2015-16) showed that assistant professors across disciplines are earning an average of $83,805 per year based on a 12-month work year and $40.16 per hour based on a 40-hour work week (2,087 hours/year). The vast majority of investigators completing the ESP survey will be Assistant Professors. Therefore, this number was used to calculate the annualized costs to respondents. Based on an estimate of 4,120 participants per year, and a total annual burden of 1550 hours, we estimate the total respondent cost to be $62,248.
Table 12-2 Annualized Cost to Respondents
Type of Respondents |
Number of Respondents |
Total Annual Burden Hours |
Hourly Respondent Wage Rate* |
Respondent Cost |
Investigators |
4120 |
1550 |
$40.16 |
$62,248 |
Total |
|
|
|
$62,248 |
A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers
There are no additional cost burdens to respondents other than those described in section A.
Annual Cost to Federal government is $118,931. As with any computer software, Federal personnel time will be needed annually to update and maintain the online survey software, including adding this revision to the already developed PRES online survey (one-time development).
Cost Descriptions |
Grade/Step |
Salary |
% of Effort |
Fringe (if applicable) |
Total Cost to Gov’t |
Federal Oversight |
|
|
|
|
|
PRES Administrator/Support |
GS13-01 |
$94,796 |
25% |
|
$23,699 |
|
|
|
|
|
|
|
|
|
|
|
|
Contractor Cost |
|
|
|
|
|
PRES Contract Total |
|
|
|
|
$95,232 |
Travel |
|
|
|
|
|
Other Cost |
|
|
|
|
|
|
|
|
|
|
|
Total |
|
|
|
|
$ 118,931 |
A.15 Explanation for Program Changes or Adjustments
This is a revision to the previous PRES Survey. There are no plans to publish any of the information collected from participants. Periodic summaries of the information collected (e.g., number of participants by institution, state, or region) will be for internal use only. In those cases, information will be reported in aggregate form without individual identifiers. Since our previously approved OMB request, we have collected data from roughly 2,700 investigators and approximately 30 NIH Scientific Review Officers (SROs) are using the database. The SROs informed us during their piloting and use of the database that there were additional topics they wanted to be included to help them better identify investigators to serve on review panels. Thus, the primary purpose of this OMB clearance request is to collect additional data not collected in the previously deployed online survey including additional study design topics (4 new, 1 deleted), research methods topics (9 new, 1 deleted), content topics (12 new, 1 deleted), populations (6 new, 2 deleted), as well as a new geographic region of research category with 10 new topics and the income category of the region/country in which the research is performed with 4 new topics. This will be done via the already developed and launched PRES online survey in a systematic way that reduces burden on the participants and governmental costs associated with processing (see Attachment 1 for screenshots, and below for more details on the revisions).
Revisions and additions to current online survey include the addition and deletion of topics and addition of new categories listed below:
Study Design Topic:
4 New topics – Dissemination and Implementation; Individually randomized group-treatment trials (IRGTs) or Partially Nested Designs; Multiple baseline designs; Stepped-wedge design
1 Deleted topics – Effectiveness and implementation studies
Research Methods:
9 New topics – Biomarker and/or risk factor validation; Electronic medical/health records; Imaging (fMRI, MRI, CT, Ultrasound, EEG, MEG, ECG, PET, X-ray, etc); Machine Learning; Migration analysis (attrition, emigration, etc.); Mobile Health (mHealth); Natural language processing; Power/sample size analysis; Sex and gender analysis
1 Deleted topics – Statistical power analysis
Content Topics:
12 New topics – Alzheimer’s disease; Growth and development; Health services research; Immunology; Influenza; Maternal/Paternal/Child Health (label change, same meaning); Metabolomics; Microbiome; Patient-centered outcomes; Pneumonia; Sex as a biological variable; Vaccines
1 Deleted topics – Health care delivery
Populations:
6 New topics – Adolescents; American Indian/Alaska Native; Children; Infants and neonates; Racial/ethnic minority populations; Sexual and Gender Minorities (LGBTI)
2 Deleted topics – Infants, children, adolescents; LGBTI
Geographic Region of Research (1 completely new category):
10 New topics –United States; Canada; Central America; South America; East Asia and Pacific; Europe and EU; Middle East and North Africa; Russia, Eurasia, and Arctic; South Asia; Sub-Saharan Africa
Income Category of the Region/Country in which the research is performed (1 completely new category):
4 New topics –Low Income Countries; Lower-middle Income Countries; Upper-middle Income Countries; High Income Countries
The following are estimated dates for completion of the PRES software revision.
1. Fully functional PRES site ready for internal testing – 09/04/2017
2. Fully functional PRES site ready for pilot testing – 09/14/2017
3. Fully functional PRES site ready for production after OMB Approval – 09/21/2017
A.16 - 1 Project Time Schedule |
|
Activity |
Time Schedule |
Letters sent to respondents |
1 week after OMB approval |
Field questionnaire |
Entire duration of OMB approval for project |
Completed field work |
Ongoing |
Validation |
Ongoing |
Analyses |
Ongoing |
Publication |
N/A |
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate
Not applicable, as we are not requesting approval to not display the expiration date for OMB approval.
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions
Not applicable, as we are not requesting an exception to certification for the Paperwork Reduction Act.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Subject | Supporting Statement A |
| Author | Lopez, Maria (NIH/NICHD) [E] |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |