CSR SupportingStatementA 12 05 2014

Attachment 1 – Example of Previously Approved Applicant Survey Instrument – Focus Group Moderators’ Guide
Attachment 2 – Example of Previously Approved Applicant Survey Instrument

SUPPORTING STATEMENT

Generic Clearance for Satisfaction Surveys of Customer,

Center for Scientific Review (CSR), National Institutes of Health (NIH)

Justification

A.1 CIRCUMSTANCES MAKING THE COLLECTION OF INFORMATION NECESSARY

This is a request for reinstatement without change of an Office of Management and Budget (OMB) approval, under the Paperwork Reduction Act of 1995, for a generic clearance (OMB No. 0925-0474). This clearance approves voluntary Satisfaction Surveys of Customers for the Center for Scientific Review (CSR), National Institutes of Health (NIH) to implement Executive Order 12862 within the agency.

Executive Order 12862 directs agencies that "provide significant services directly to the public" to "survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services". CSR/NIH provides significant services directly to the public. The mission of the National Institutes of Health (NIH) is to improve human health through biomedical and behavioral research. The NIH implements this mission by conducting intramural research in its own laboratories; supporting extramural research by scientists at universities, medical schools, hospitals, and research institutes; training basic and clinical research investigators; and fostering and supporting biomedical communication as well as public health information dissemination.

The Center for Scientific Review (CSR) is responsible for providing support to the Office of the Director, National Institutes of Health. This support is the receipt, referral, scientific, and technical peer review of applications for: grants that support the advancement of fundamental scientific knowledge to meet the NIH mission of extending healthy life and reducing the burdens of illness and disability, grant opportunities that support research-related training and career development, Small Business Innovation Research (SBIR) grants, and a variety of other awards. CSR plays a pivotal role in assuring NIH funds only the most meritorious biomedical research. Each year, CSR receives nearly 86,000 applications and reviews roughly 70% of them by recruiting roughly 16,000 experts to its study sections^¹. Additional scientists serve on other NIH advisory councils, which provide a second level of peer review and make funding recommendations based on priorities set by Congress, DHHS, and the public. For nearly 60 years, this peer review system has enabled NIH to fund cutting-edge research.

The Center for Scientific Review of the National Institutes of Health requests the Office of Management and Budget (OMB) to reinstate generic clearance OMB No. 0925-0474 for surveys of our customers for the period of three years. The OMB approval expires 10/31/2014. We have ensured these survey activities, which were and are designed to gather and measure customer satisfaction with the CSR's processes and operations, satisfy the requirements and the spirit of Executive Order (EO) 12862. CSR staff has reviewed the OMB “Resource Manual for Customer Surveys.” We are confident all our survey activities meet the requirements and follow the guidelines of this manual. Further, the CSR agrees to adhere to the guidelines set forth in both EO 12862 and in the OMB Manual.

CSR recognizes the generic clearance covers only voluntary surveys of our partners and customers. Participation in these surveys will be strictly voluntary. Personal identifiers will be excluded from all surveys, and additional measures will be taken to ensure anonymity of respondents. Our surveys will be both qualitative and quantitative. The generic clearance is limited to collection of information from current and immediate past customers of CSR.

CSR strives to maintain its leadership as an international model of impartiality, rigor, fairness, and excellence in the peer review of state-of-the-art biomedical research. Input from our customers is an essential component of our efforts to continuously improve our processes and operations. Results gathered in these surveys will be used to help us improve and refine our peer review organization, to refocus the strategies that we use to accomplish our objectives, and to identify areas in need of improvement.

Quality management principles have been integrated into CSR's culture and these surveys will be at the core of CSR's continuing assessment of peer review operations. The data collected from these surveys will assess our customers' satisfaction with CSR's reorganization of grant review study groups, operations, and processes. CSR will present data and outcomes from these surveys to the NIH Advisory Committee and based on feedback from this committee formulate improvement plans and action steps when necessary. To move our quality initiative forward, CSR needs input from its customers.

A.2 PURPOSE AND USE OF THE INFORMATION COLLECTION

This is a reinstatement without change of an already approved clearance. The primary use for information gathered through voluntary customer surveys is to identify strengths and weaknesses in current services provided by CSR/NIH and to make improvements that are practical and feasible. Obtaining information about the satisfaction of customers is consistent with EO 12862, as well as a good business practice. If CSR does not collect this information, vital feedback regarding customer and partner satisfaction or dissatisfaction with various aspects of revisions to the peer review services will be unavailable. This would inhibit CSR’s ability to develop, implement and refine programs, products, and services in a manner most consistent with the needs of our customers. The information collected in these surveys will be used by CSR personnel: 1) to assess the quality of the changed operations and processes used by CSR to review grant applications; 2) to assess the quality of service provided to our customers; 3) to assess the quality and relevancy of the reorganized scientific review groups; 4) to assist with the design of modifications of these operations, processes, and services, based on customer input; 5) to develop new modes of operation based on customer need; and 6) to obtain customer feedback about the efficacy of implemented modifications. These surveys will almost certainly lead to quality improvement activities that will enhance and/or streamline CSR's operations. The satisfaction surveys are tailored to the specific changes CSR/NIH is considering implementing. CSR will submit specific survey instruments as they become available and will report the results of completed surveys to DHHS and OMB on a yearly basis.

These satisfaction surveys have been used to implement multiple enhancements to the NIH peer review system. A sample of the applicant survey instrument that has been fielded is included in Attachment 1.

Previously, our satisfaction surveys were used to inform our use of bi-coastal meetings, discounted airfares for participants, use of our electronic and video assisted review platforms, orientation sessions for new study section Chairs, a feasibility study of telework hoteling space, Integrated Review Group (IRG) evaluations, and surveys of early stage and new investigators. This initiative resulted in significant cost savings and allows the Center for Scientific Review to enable the participation of those scientists who are unable to travel to peer review meetings, promote telework for Scientific Review Officers (SRO), and provide insights into the peer review process by key stakeholders (See Attachments 1 and 2 enclosed, OMB approved May 8, 2014). The knowledge gained from these surveys directed our efforts to improve new review platforms, provide additional office space for SRO teleworkers, provide better guidance to new study section Chairs, and make the peer review process more user friendly to key stakeholders even as we increased the number of grant applications reviewed.

A.3 USE OF INFORMATION TECHNOLOGY AND BURDEN REDUCTION

As appropriate, automated information technology will be used to collect and process information for these surveys. In many instances the most appropriate methodology will involve Internet administration of surveys but some may be written.

A.4 EFFORTS TO IDENTIFY DUPLICATION AND USE OF SIMILAR INFORMATION

Each survey will be designed to reflect the specifics of the review related change being assessed. Any potential duplication will be identified in the internal CSR review and approval process. Information about plans for customer surveys will also be shared among CSR components at an early stage so activities can be coordinated.

A.5 IMPACT ON SMALL BUSINESSES OR OTHER SMALL ENTITIES

It is not expected that small businesses will be involved in these customer surveys.

A.6 CONSEQUENCES OF COLLECTING THE INFORMATION LESS FREQUENTLY

Surveys will be conducted only at intervals that are considered appropriate to measure the impact of changes implemented as a result of initial satisfaction surveys and to monitor the continued level of performance. In most instances, a satisfaction survey is likely to be conducted on an annual or biennial basis after establishment of a baseline. Collection on a less frequent basis would reduce the practical utility of the information and inhibit the program's ability to monitor changes.

A.7 SPECIAL CIRCUMSTANCES RELATING TO THE GUIDELINES OF 5 CFR 1320.5

These surveys will be implemented in a manner fully consistent with 5 CFR 1320.5

A.8 COMMENTS IN RESPONSE TO THE FEDERAL REGISTER NOTICE AND EFFORTS TO CONSULT OUTSIDE AGENCY

This proposed information collection was published in the Federal Register on August 21, 2014 (p. 49523, Vol. 79). Not a single response was received in response to the 60 day notice.

CSR may use focus groups and other qualitative information collection activities to identify areas of interest and concern to customers and then build the design and content of its quantitative surveys upon this qualitative input. CSR will call upon its in-house statistical staff and contractors in developing survey plans.

A.9 EXPLANATION OF ANY PAYMENT OF GIFT TO RESPONDENTS

There will be a need for nominal remuneration to focus group participants who are asked to leave their usual location and travel to a central location to compensate them for the time and inconvenience required. The level of remuneration will depend on the amount of respondent time and expense projected for each focus group.

A.10 ASSURANCE OF CONFIDENTIALITY PROVIDED TO RESPONDENTS

A sample of reviewers and applicants from each fiscal year will be asked to participate in one of the surveys. Those records are kept by CSR. However, the responses to the questionnaire surveys are entirely anonymous and have no identifiers to link them to individual respondents. Thus, their privacy is ensured by the anonymity. This will be explained to respondents. For surveys contacted by an independent contractor, respondents will be given the Internet address and a unique link to the survey web site. The system is programmed so only a respondent who has a unique link can access a survey form. The link will be automatically deactivated after the form is completed and submitted. The survey will not contain any identifying information.

Even the email address, which could possibly identify individual respondents, will be separated from the survey responses as they are entered into the data set. As the contractor's computer server receives the completed surveys, the identity of the responding computer is also eliminated. The electronic responses will be downloaded automatically into the data set. Contractors will perform the analyses. CSR staff will see only aggregate data. No individual information will be able to be extracted. Additionally, respondents will be informed that their participation is voluntary and that no consequences will be associated with not responding or with responding. This procedure will be explained to respondents. Individuals contacted in the course of these surveys will be assured of the confidentiality of their replies under 42 USC 1306, 20 CFR 401 and 422, 5 USC 552 (Freedom of Information Act), 5 USC 552a (Privacy Act of 1974), Privacy Act System of Records Notice: 09-25-036, and OMB Circular No.A-130.

For surveys conducted within CSR, three specific key technical staff must have access to data with personal identifiers to deploy surveys to the appropriate respondents and to build the data base. End users will never have access to personally identifiable data. At the end of each NIH Council Round, all personally identifiable data will be stripped away and deleted from the survey platform, leaving only de-identified archival data. Archived data will never have personally identifiable data.

A.11 JUSTIFICATION FOR SENSITIVE QUESTIONS

These voluntary customer surveys will not involve personal information of a sensitive nature.

A.12 ESTIMATES OF HOUR BURDEN INCLUDING ANNUALIZED HOURLY COSTS

Respondents are expected to be a mix of adult scientists connected to the research community. It is estimated that e-mail/mail surveys will take 30 minutes to complete. The annual hour burden is, therefore, estimated to be 3,963 hours for approximately 7925 respondents in each of the 3 fiscal years. Focus groups: It is projected that in each year of this approval approximately 20 focus groups will be convened, primarily for the purpose of customer input into the design of satisfaction surveys. Each focus group is expected to require 1.5 hours and will include approximately 12 members. The focus groups are expected to take 1.5 hours to complete. The annual burden for the 20 annual focus groups is estimated to be 360 hours for each of 3 fiscal years. Estimated costs to the respondents consist entirely of their time. Costs for time were estimated using a rate of $50.00 per hour for adult science professionals. The estimated annual cost burden for respondents for each year for which the generic clearance is requested is $72,050 for FY 2015, $72,050 for FY 20016, and $72,050 for FY 2017. Thus, the total combined FY 2015 to FY 2017 potential hour burden on the respondents is estimated to be 12,969 hours for 24,495 respondents for all surveys that would be conducted under this generic clearance. If all planned surveys are conducted, the total three year cost is estimated to be 216,150.

TABLE A12-1 ESTIMATES OF HOUR BURDEN

Type of Respondent	Number of Respondents	Frequency of Response	Average Time per Response (Hr)	Total Annual Hour Burden
Adult scientific professionals (via Mail/Te1438hone/Internet)	7925	1	30/60	3963
Adult scientific professional (via focus groups)	240	1	90/60	360
Total	8165	1		4323

TABLE A12-2 ESTIMATES OF ANNUALIZED COST TO RESPONDENTS

Type of Respondents	Number of Respondents	Frequency of Response	Average Burden Per Response	Hourly Wage Rate	Respondent Cost
Adult science professionals - Focus groups	240	1	90/60	$50.00*	$18,000.00
Adult science professionals -mail/telephone/ e-mail surveys	7925	1	30/60	$50.00*	$198,125.00
Total…………………					$216,125.00

*Source of hourly wage rate estimates: A survey of 1,156 colleges published in April 2014 in the Chronicle of Higher Education http://chronicle.com/article/2013-14-AAUP-Faculty-Salary/145679?cid=megamenu#id=table showed that full professors across disciplines are earning an annual mean wage ranging from $215,500 to $73,600 per year based on a 9 month work year. The vast majority of respondents will be full professors. Based on this report, the range in annualized salaries extrapolated for a 12 month work year is quite large ($287,333 to $98,133) with an annual hourly wage rate of $138/hour to $47/hour, based on a 40 hour work week.

A.13 ESTIMATE OF OTHER TOTAL ANNUAL COST BURDEN TO RESPONDENTS OR RECORD KEEPERS

Focus group participants will be reimbursed for any travel or incidental costs associated with traveling to a central location for interview. Except for focus groups, costs to respondents will be limited to their time to provide the requested information.

A.14 ANNUALIZED COST TO THE FEDERAL GOVERNMENT

The annualized cost to the Federal Government for the proposed data collection effort is estimated to be approximately $632,500.

Item	Salary	%Effort	Annualized Cost
2 NIH Project Oversight Officers – GS14/15	260,000 (130,000 x 2)	100%*	260,000
1 NIH Program Analyst GS13/14	100,000	100%*	100,000
Contractor Staff – cost for each survey	8,000-10,500 (10 surveys x 9,250)		92,500
Contractor Staff – cost for each focus group	9,000 (20 focus groups x 9,000)		180,000
Total			$632,500

*Assumes 1.75 weeks per project and 30 projects = 52.5 weeks or just over 1 year.

A.15 EXPLANATION FOR PROGRAM CHANGES OR ADJUSTMENTS

This is the same satisfaction survey of Center for Scientific Review's customers approved by OMB in 2011 under OMB No. 0925-0474. CSR is requesting a reinstatement without change of No. 0925-0474 to continue the activity of assessing customer satisfaction with the organizational and process changes made by CSR. A 30-day notice was published on October 28, 2014 in a timely manner. However, the OMB package was inadvertently not submitted. The 30-day notice is being re-published. NIH policy changes since 2011 and enhancements in technical support have increased the need for more frequent customer satisfaction surveys with a broad stakeholder constituency. The 2011 OMB approval was for 5075 responses and 1438 burden hours. However, there are more initiatives being undertaken by CSR each year which results in additional respondents. In May of 2014 we had to request 1000 additional burden hours to complete a new focusing on a study of new investigators and their submission experiences. The initiative survey lengths will be comparable. For the e-mail surveys, the anticipated annual hour burden for FY 2015 through FY2017 is 4,323 hours for approximately 8,165 respondents in each of the 3 fiscal years. Focus groups will be convened, primarily for the purpose of customer input into the design of satisfaction surveys. Each focus group is expected to require 1.5 hours per person and will include approximately 12 members. CSR expects to have 20 focus groups per year. The focus groups are expected to take 1.5 hours per respondent to complete. The annual burden for focus groups is estimated to be 360 hours for each of 3 fiscal years.

A.16 PLANS FOR TABULATION AND PUBLICATION AND PROJECT TIME SCHEDULE

The satisfaction surveys that will be collected under the generic clearance will be conducted only once per review cycle. The projected survey schedule is: 5-10 surveys per year. Topics over the 3 years will include: scientific review using differing review formats such as internet assisted review, virtual and telepresence reviews, scientific review utilizing Editorial Boards, score calibration and score compression, automated referral of grant applications, modes of review for orphan applications, policy changes on continuous submission and change in A1 status, etc.

The surveys will be distributed electronically. Information collected from the surveys will be analyzed within two months of receipt. Analysis plans will be specific to the goals and designs of the individual surveys. For all types of surveys, the analyses will be mostly descriptive, rather than inferential. The results of any analysis will be disseminated to key management officials at CSR, NIH management, CSR members and chairs, NIH principle investigators, and CSR employees within six months of the survey's completion. Relational databases will be designed and analysis-ready data inputted. Analyses will include cross tabulation of the questions that indicate consistency and validity of responses. Analysis result tables will be designed and produced to present the information in an aggregate format in order to maintain confidentiality. Content analysis will be performed on the narrative responses. A narrative report with accompanying charts will be provided to CSR management.

TABLE 16.1 TIMETABLE SUMMARY

Month 1	Construct relational databases and produce coded survey instruments.
Month 2	Construct survey collection Internet website
Month 3	Post survey instrument and send email invitations to participate
Month 4	Post survey instrument and send email invitations to participate
Month 5	Download survey responses and input data
Month 6&7	Analyze coded and narrative data and write reports for CSR management
Month 8	CSR management study report and decide if changes need to be implemented.
Month 9	CSR disseminates results and any changes that are to be made

A17 REASON(S) DISPLAY OF OMB EXPIRATION DATE IS INAPPROPRIATE

No exemption is being requested.

A.18 EXCEPTIONS TO CERTIFICATION FOR REDUCTION ACT SUBMISSIONS

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.

1 * Data from 2012 and rounded to the nearest 1,000.

File Type	application/msword
File Title	Supporting Statement A for
Author	Andrea Kopstein
Last Modified By	Guadagma
File Modified	2014-12-17
File Created	2014-12-10