Supporting Statement A for ODP PRES

The Office of Disease Prevention (ODP) is the lead Office at the National Institutes of Health (NIH) responsible for assessing, facilitating, and stimulating research in disease prevention and health promotion, and disseminating the results of this research to improve public health. Prevention is preferable to treatment, and research on disease prevention is an important part of the NIH’s mission. The knowledge gained from this research leads to stronger clinical practice, health policy, and community health programs. ODP collaborates with NIH, other Department of Health and Human Services (DHHS) agencies, and other public and private partners to achieve the Office’s mission and goals. One of our priorities is to promote the use of the best available methods in prevention research and support the development of better methods. One of our strategies is to help NIH Scientific Review Officers (SROs) identify experts in prevention science methods to include on their review panels. This will strengthen the panels and improve the quality of the prevention research supported by NIH. To identify experts in prevention science methods, we worked with our contractor, IQ Solutions, Inc., to develop online software which will allow us to collect researchers’ names, contact information, and resumes, as well as to have those researchers identify their level of expertise in a variety of prevention science methods and content areas. The data collected with this software will be used to support a web-based Electronic Directory that SROs can use to identify researchers with expertise in specific prevention science methods and content areas for invitation to serve on one of the NIH review panels. If the initial rollout with the Center for Scientific Review (CSR) is successful, this system will also be shared with review staff in the other Institutes and Centers at NIH, as well as other DHHS agencies, to use in a similar same way. Given our plans to create an automated system for reviewer information collection, we are now seeking OMB approval.

A.1 Circumstances Making the Collection of Information Necessary

NIH Office of Disease Prevention: Overview. The Office of Disease Prevention (ODP) is the lead Office at the National Institutes of Health (NIH) responsible for assessing, facilitating, and stimulating research in disease prevention and health promotion, and disseminating the results of this research to improve public health. Prevention is preferable to treatment, and research on disease prevention is an important part of the NIH’s mission. The knowledge gained from this research leads to stronger clinical practice, health policy, and community health programs.

The mission of the ODP is to improve the public health by increasing the scope, quality, dissemination, and impact of prevention research supported by the NIH. The ODP will fulfill this mission by providing leadership for the development, coordination, and implementation of prevention research in collaboration with NIH Institutes and Centers and other partners.

Prevention Research Expertise Survey Program. The NIH Office of Disease Prevention (ODP) Prevention Research Expertise Survey (PRES) program was developed to (1) identify methodologists with expertise in content areas related to prevention science, (2) identify mid- and senior- level researchers who may have an interest in serving on review panels, and (3) in its initial phase, work with the NIH Center for Scientific Review (CSR) to enrich the existing pool of NIH reviewers by including researchers with methodological and prevention science expertise that review prevention applications. This OMB clearance request is to deploy our new online survey for the PRES program.

PRES Process. Researchers interested in providing their information for the PRES program will submit some identifying information, content and methodological areas of expertise, Curriculum Vitae (CV) or professional resumes, and willingness to serve on a review panel. They are included in the PRES program based on their self-reported level of expertise in methodological and prevention science content areas, as well as, the information provided in their CVs. CVs are commonly used documents for recording work history and accomplishments. Creation and updating of a CV is a common exercise of researchers and other professionals. A customary practice of Study Review Officers (SROs) in the course of their duties is to review the CVs of potential reviewers. CVs provide substantial information regarding current employment, publication history, grants received, and other professional achievements, all of which are very useful in evaluating applicants’ eligibility to serve on review panels as reviewers. The legislative authority to collect information from potential reviewers to determine their appropriateness to serve on grant application review panels is 42 USC Section 241: Research and Investigations General.

Study Review Officers will have access to researchers’ CVs and self-reported assessment of methodological and prevention science content expertise through the PRES and Electronic Directory (See Attachment 2). PRES program staff will maintain the PRES and Electronic Directory and these data will be available to SROs as they recruit members of their review sections.

In summary, PRES participants’ contact information, CV, and methodological and prevention science content areas of expertise must be collected for inclusion in the PRES program. An online survey is more cost-effective and efficient than transmitting information between scientists and PRES program staff by mail. The software program has been developed. OMB clearance is being sought to deploy the software.

A.2 Purpose and Use of the Information Collection

The PRES is not a research project. This is a new information collection request. There are no plans to publish any of the information collected from participants. Periodic summaries of the information collected (e.g., number of participants by institution, state or region) will be for internal use only. In those cases, information will be reported in aggregate form without individual identifiers. The primary purpose of this PRA clearance request is to gather this information from PRES program participants via an online survey in a systematic way that reduces burden on the participants and governmental costs associated with processing.

Proposed PRES and Electronic Directory. PRES participant information is provided for the purpose of evaluating qualifications to serve on review panels. This OMB request is to gather information via an online survey for the purpose of evaluating PRES participants’ eligibility to serve on review panels. Scientific Review Officers will access PRES data through a secure Electronic Directory. The Electronic Directory will include the PRES participants’ names and institutions, methodological and content areas of expertise, training credentials, professional accomplishments, email addresses, and NIH commons IDs, most of which will be provided in their CVs (see Attachment 1: Screen Shots of Online Survey Format). Prevention researchers will provide a voluntary self-assessment of their level of expertise in methodological, content area, and population focus area fields to better direct applicants to SROs responsible for prevention science review panels. The provision of commons IDs allows SROs to more easily add appropriate PRES program participants to prevention science review committee rosters via NIH data systems (e.g., IMPACII).

PRES has been developed to gather this information from prevention researchers via an online survey. This will reduce burden on the PRES program participants and be more efficient and cost-effective for the ODP PRES program staff and NIH review staff. An online system will reduce ODP staff time to process prevention researchers’ survey data and increase the speed of transmission of data to SROs who are in the best position to select a researcher that best fits their needs. It will also allow for seamless provision of feedback to PRES program staff on SROs’ experience with the accuracy of researchers’ self-assessment and outdated or incomplete data.

The ODP website will provide a link to the online survey. Attachment 1 includes screenshots of PRES. The software has been developed by IQ Solutions, Inc. in collaboration with NIH ODP staff for the sole use by ODP and SROs in forming scientific review panels to review applications for research support. If the initial rollout with CRS is successful, this system will also be shared with review staff in the other Institutes and Centers at NIH, as well as other DHHS agencies, to use in a similar way

A.3 Use of Information Technology and Burden Reduction

Privacy Impact Assessment. In January, 2015, the Privacy Impact Assessment (PIA) Form was created for the NIH ODP Prevention Research Expertise Survey program and submitted to the NIH Senior Official for Privacy. The PIA included the development of a database of respondents for the PRES program. As indicated above, the database includes applicants’ names, email address, academic institution and position, areas of expertise, and NIH eRA Commons ID. The online survey software will direct information into the PRES Electronic Directory. Once PRES has been approved by OMB, the PIA will be updated to reflect its approval.

Survey Completion via Email - Burden for Prevention Researchers. Those interested in joining the PRES program will be directed to the ODP website (See Attachment 2: PRES Website). The website will direct users to complete a user profile and assess their expertise in three areas: methodological, content area, and population focus. Based on usability testing, the total estimated time burden for users is approximately 20 minutes.

A.4 Efforts to Identify Duplication and Use of Similar Information

The information that will be gathered through the PRES is not available from any other source or agency.

A.5 Impact on Small Businesses or Other Small Entities

Researchers from small organizations will be eligible to complete the online survey. The information being requested of all prevention researchers has been held to the minimum required for evaluation of appropriateness for inclusion in the PRES program and NIH review panels (see Section A.3).

A.6 Consequences of Collecting the Information Less Frequently

Frequency of Survey Completion. Prevention researchers will complete the survey and upload their CVs only once. If those researchers gain more expertise, change jobs, and/or contact information, after completing the survey, they can update their information using the same online mechanism by selecting the “update your information” option (see Attachment 1).

Consequence of not collecting information. As described in Section A.4, some of the information gleaned from researchers’ CVs might also be available through university websites, IMPACII, social media sites, professional association websites or via telephone or email contact with the applicant. These sources are used inconsistently, may include outdated or incomplete information, and present the potential for inconsistency in the manner in which possible reviewers are evaluated. Completing the online survey and uploading a CV standardizes the process and provides SROs with one place to find mid- and senior- level methodological experts for review panels.

The online survey and CV also greatly reduces staff time that would be needed to conduct detailed online searches for information on each applicant. Given the high volume of prevention researchers, it is more cost-effective to ask PRES program participants to provide their CVs and information regarding their degree, training, methodological and content area expertise, and willingness to serve on a NIH study section, than to hire staff to conduct individualized online searches to verify the qualifications of each participant. It also prevents the problem of misidentifying information online as pertaining to a participant when it may belong to another person with a similar name. Finally, providing all information at the time of survey submission reduces the number of times that ODP PRES program staff or SROs contact the PRES participant to verify or to request additional information.

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

The proposed PRES software will fully comply with all guidelines stated in 5 CFR 1320.5.

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

Published 60-day Federal Register notice. April 7, 2015, Volume 80, Number 66, Pages 18641 - 18642. There were no public comments regarding the proposed work in response to this Federal Register Notice.

Consultation with advisors regarding the program. The plan to develop PRES was discussed with and approved by the Associate Director for Prevention and Director of the Office of Disease Prevention, Dr. David M. Murray; Director of CSR, Dr. Richard Nakamura; Director of the CSR Division of Aids, Behavioral and Population Sciences, Dr. Karyl Swartz; the staff of the Information Management Branch of CSR who developed the Early Career Reviewer Application and Vetting System (EAVS) online application software; and the ODP Prevention Research Coordinating Committee. Their recommendations were incorporated into the development plans for the online survey software.

Feedback from outside consultants. This application is not for a research project. Comments were solicited from persons outside of NIH regarding the planned online survey system. These individuals are senior level investigators with expertise in methodology (see Attachment 3).

Internal Advisors. IQ Solutions began internal testing of NIH advisors (see Attachment 3) on September 15, 2014. This allowed for identification and resolution of programming errors. On October 8, 2014, IQ Solutions, Inc. staff provided PRES program staff with test cases as part of their internal Usability Acceptance Testing. This included provision of scenarios that PRES participants would be likely to encounter. This allowed PRES staff to pre-test the responsiveness of the program to common scenarios encountered when completing the survey such as not having an NIH eRA Commons ID. In addition, PRES program staff carefully examined each screen seen by program participants and provided feedback to IQ Solutions, Inc. staff regarding cosmetic changes and programming problems. Internal testing of the survey has been completed.

In addition to PRES program staff, senior staff at ODP with previous experience as Program Officers volunteered to assist with Usability Acceptance Testing of the survey to vet completion ease and time. Feedback was provided during the pilot testing process. Some feedback included ambiguity with terms used to describe methods and indicating interest in participating in a review panel, which have been addressed. Other problems included awkward placement of “user profile” buttons and lack of white space on background page. In all cases, the feedback provided was incorporated into refinements of survey.

NIH Agency consultation. The development of plan for creating the PRES and Electronic Directory was part of a collaborative effort between the ODP program staff, IQ Solutions (IQS), and the Director of ODP. A draft of the plan and the beta version of the software designed to collect information online from applicants/participants and the flow of information from PRES participant to the Electronic Directory was developed by ODP Staff and IQ Solutions and approved by Dr. David Murray, Director of the ODP. Paris Watson and Dr. Jocelyn Lee, the PRES Coordinators, worked with Rich Panzer and Jason Hamrick, IT software developers from IQS, to develop the PRES program development plan that detailed the information to be collected using the online survey, the flow of data into a secure database (called the “Electronic Directory”), how the information will be resourced and accessed by the SROs.

The beta version of the Electronic Directory was reviewed and analyzed for usability by internal advisors made up of NIH Center for Scientific Review (CSR) SROs (see Attachment 3). After their feedback was incorporated into the plan for developing the Electronic Directory, the PRES development plan was reviewed again by Dr. David Murray, Director of ODP. Watson, Panzer, and Hamrick, and Lee presented the plan to Dr. Murray. The PRES development plan and budget for production were approved.

One of the main purposes for developing the PRES is to help the SROs identify experts in prevention science methods to include on their review panels. The data collected with this software will be used to create the Electronic Directory that NIH review staff can use to identify researchers with expertise in specific prevention science methods and content areas for invitation to serve on one of the CSR review panels. The Electronic Directory was developed in parallel with the online survey because the survey’s database will serve to populate the Electronic Directory with potential reviewers. In coordination with Dr. Karyl Swartz (Director of the CSR Division of Aids, Behavioral and Population Sciences), a group of CSR SROs were selected for a 30-day beta-testing of the Electronic Directory. Due to the timeline for planning of the September/October 2015 CSR study sections, we will convene PRES Electronic Directory beta training session in late June 2015. The 30-day beta-testing of the Electronic Directory will take place July 1-31, 2015. The training session for the Electronic Directory will be led by Panzer and Hamrick from IQ Solutions in coordination with Watson and Lee from ODP.

In order to capture SRO user feedback and any problems or questions related to the Electronic Directory, we created a dedicated email address (odpsrorolodex@mail.nih.gov) that will be reviewed daily and maintained by ODP PRES staff. In addition, after the 30-day beta-testing of the Electronic Directory, (July 1-31, 2015), a Usability Focus Group will be convened in early August to capture final feedback that will be incorporated into the Electronic Directory.

A.9 Explanation of Any Payment of Gift to Respondents

The opportunity to participate in the PRES program is voluntary. No compensation is provided for completion of the PRES survey.

A.10 Assurance of Personal Information Provided to Respondents

System of Record. The IT system in which information is stored is entitled the NIH Prevention Research Expertise Survey. The system was assessed with a Privacy Impact Assessment. The date of the completion of the PIA was 01/29/2015. The Privacy Act applies to this activity. The number of the Privacy Act Systems of Record Notice that covers the information collection is 09-25-0036. This application for Paperwork Reduction Act Clearance has been completed in consultation with the CSR Privacy Officer, Karen M. Plá.

Privacy. Personal information is protected in NIH ODP PRES through compliance with laws, regulations and other mandates (e.g. Privacy Act, OMB Guidance). Additionally, ODP has in place operational safeguards such as training, education and awareness for PII protection. ODP has also implemented multiple technical security controls to limit access to data, monitor events that affect personal information and has an incident response plan to handle breaches to this data. Annual Privacy Impact Assessments (PIA) will be held to review and mitigate privacy risks.

To maximize the protection of their personal information, investigators will be asked to complete a user profile prior to completing the PRES program survey that will include creation of a username and password. Applicants’ CVs and additional identifying data will be saved on a secure website hosted by the NIH Office of Information Technology (OIT). This data will be accessible only by select DHHS staff with PIV cards that have been granted permission by ODP PRES program staff. ODP PRES staff will work with OIT to grant select SROs access to the Electronic Directory and will delete permissions for access if an employee leaves their position as a Study Review Officer. SROs must login to the database with their PIV cards.

When PRES has been approved for use by OMB, the website will include a statement regarding privacy as part of the introduction to completing the survey. It will indicate that the information provided will be kept private to the extent allowed by law and not disclosed to anyone but the staff of ODP PRES program and DHHS staff who evaluate submissions except as otherwise required by law. We will also include in the introduction to the survey a statement that clearly states that 42 USC Section 241: Research and Investigations General provides legislative authority to collect information for the PRES program, that participation in the program is strictly voluntary, and that no consequence exists for choosing not to participate.

Prevention Research Expertise Survey and Electronic Directory. The information to be retained in the searchable Electronic Directory includes investigators’ names, job title, email address and institutions. We are not collecting date of birth, social security number, home address, race, ethnicity, or gender. We have chosen not to collect race and ethnicity data two reasons: (1) the information provided PRES program participants will be made available to SROs for the purpose of potential inclusion in study review panels. We want to avoid potential bias as well as the appearance of bias based on race/ethnicity in their selection process. (2) When investigators create NIH eRA Commons accounts, they have the option to choose whether or not to disclose information regarding race and ethnicity. Access to those data is limited to protect the applicant. However, when data summaries by race/ethnicity are needed for program evaluation purposes, only approved personnel with specific responsibility for data summaries can access those data. PRES program staff will not have access to sensitive data from individual PRES participants.

The information to be collected via the online survey is the same information used by SROs to vet all other potential review panel members. In the course of daily job duties, SROs gather information on potential reviewers including education and job title, professional accomplishments, publications, and any other information that represents the expertise of the potential reviewer.

The online survey will allow for collection of identical types of information from each PRES participant in addition to their self-assessed expertise in methodological, content area, and population focus area fields. Summaries are periodically tallied in an aggregate form for administrative use regarding the institutions at which investigators are employed. Names of individual investigators are not included in these summaries. The information from participants is not used for research or survey purposes.

A.11 Justification for Sensitive Questions

Information to be collected and its purpose. Investigators’ names, degree, email addresses, job titles, and employers will be collected as part of the online survey. As indicated in section A.10, sensitive information will not be collected. Email addresses are collected as the primary means of communication between the participants, PRES program staff, and SROs. Telephone numbers and addresses are not specifically requested. However, it is not uncommon for investigators to include contact information in their CVs. All CVs are stored in a password protected shared file that is only accessible to ODP PRES program staff and SROs.

Collection of information on educational attainment, job title, and employer are required to allow SROs to determine a researcher’s fit with a prevention-focused review panel. The CV contains information regarding publication history, research activities, and other accomplishments, all of which are needed to determine the eligibility to be included in a pool of potential review panel reviewers.

Sensitive information is not needed to determine each researcher’s fit with a prevention-focused review panel and will therefore, not be collected.

Privacy. In addition to the protections described in section A.10, investigators will maintain control of the location from which they complete the online survey. This allows investigators to maintain control over the privacy of their data during the submission process.

Disclosures to Investigators The ODP website (see Attachment 2) will include information regarding the nature of the PRES program, benefits of completing the survey, and what can be expected to occur following submission. Details will be added to the website regarding the online submission process, a clear statement that participation is voluntary, and assurances that personal information is secure to the extent required under the Privacy Act as described in A.10. Investigators who do not wish to or are unable to complete an online submission will be able to contact the ODP PRES program staff by email to include their information.

A.12 Estimates of Hour Burden Including Annualized Hourly Costs

Annual Hour Burden. Based on recent emails from professional societies indicating an interest in participating in the ESP program, it appears that the flow of participants will be approximately 60 per week. Assuming a constant rate, the ESP program should receive approximately 3,120 survey completions per year. Using an estimated response time of 20 minutes per survey, the annual burden for participants is 1040 hours. Section A.3 provides additional details regarding this estimate.

A.12 - 1 ESTIMATES OF ANNUALIZED BURDEN HOURS
Type of Respondents	Number of Respondents	Frequency of Response	Average Time per Response	Annual Hour Burden
Investigators	3,120	1	20/60	1040
Totals	3,120			1040

Annualized Cost to Respondents. A survey of 1251 colleges published in April 2012 in the Chronicle of Higher Education (http://chronicle.com/article/faculty-salaries-barely-budge-2012/131432) showed that assistant professors across disciplines are earning an average of $66,564 per year based on a 9 month work year. This can be extrapolated to $88,752 per year based on a 12 month work year and $42.53 per hour based on a 40 hour work week (2,087 hours/year). The vast majority of investigators completing the ESP survey will be Assistant Professors. Therefore, this number was used to calculate the annualized costs to respondents.

Based on an estimate of 3,120 participants per year, each taking 20 minutes to complete, the anticipated average annual respondent cost of the PRES survey is $43,806. According to the report described above, the range in annualized salaries for assistant professors is quite large ($150,000/12 months to $66,666/ 12 months). Therefore, the potential range in annualized cost to respondents is $32,898 at the lower end to $74,026 on the higher end.

A.12 - 2 ESTIMATED ANNUALIZED COST TO RESPONDENTS
Type of Respondents	Number of Respondents	Frequency of Response	Average Time per Respondents (in hours)	Hourly Wage Rate*	Respondent Cost
Investigators	3,120	1	20/60	$42.53	$43,806
Totals	3,120				$43,806

* Source of estimate: http://chronicle.com/article/faculty-salaries-barely-budge-2012/131432

A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record

There are no additional cost burdens to respondents other than those described in section A. 12.

A.14 Annualized Cost to the Federal Government

Total annualized Costs to the Federal Government $106,428/year

Staff	Grade/Salary (Percent FTE or effort)	ODP PRES Hourly Rate	Annualized Cost
PRES Staff
PRES Administrator	GS14-05 (10%)	$58.28	$12,164
PRES Support	GS12-01 (20%)	$36.30	$15,276
PRES Coordinator	GS13-01 (35%)	$43.52	$31,788

PRES Contractor Costs
PRES IT Developer	208 hours/year	$120	$24,960
PRES Development Costs (annualized over 3 years)			$22,240

Total Annualized Cost			$106,428

Online Survey Development Costs. IQ Solutions, Inc. has estimated that the development of the online software for the PRES program has taken 556 hours to complete at a rate of $120 per hour. This is a one-time development cost that has been annualized over three years. Therefore, the cost for development is estimated at $22,240 per year for three years for a total of $66,720. Attachment 4 (Estimated Level of Effort to Build the Online Survey) provides a summary of the estimated level of effort required at each stage of development to build the PRES online survey.

As with any computer software, personnel time will be needed annually to update and maintain the online survey software. The estimated time needed will be 208 hours per year at a rate of $120 per hour (contractor rate) for a total annual cost of $24,960.

A.15 Explanation for Program Changes or Adjustments

A.16 Plans for Tabulation and Publication and Project Time Schedule

The estimated number of hours to complete the online survey software is presented in Attachment 4 (Estimated Level of Effort to Build the PRES Online Survey). The following are estimated dates for completion of the overall software development project.

Non-functional portal site available for review (no workflow, just the data entry / front end site only) by the ODP staff – 11/01/14
Fully functional PRES site ready for internal testing – 12/01/14
Fully functional PRES site ready for pilot testing – 09/01/15
Fully functional PRES site ready for production after OMB Approval – 09/15/15

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

We are not requesting approval to not display the expiration date for OMB approval. Therefore, section A.17 is not applicable.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

We are not requesting an exception to certification for the Paperwork Reduction Act. Therefore, Section A.18 is not applicable.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Watson, Paris (NIH/OD) [E]
File Modified	0000-00-00
File Created	2021-01-24

Supporting Statement A for ODP PRES