Supporting Statement A - DIR Review Assessment Survey

SUPPORTING STATEMENT A

DIR Objective Review Assessment Survey

Terms of Clearance: None

A. Justification

1. Circumstances of Information Collection

The Health Resources and Services Administration (HRSA) currently has approval under the generic clearance, Office of Management and Budget (OMB) Control No. 0915-0212, to conduct customer satisfaction surveys and focus groups. This collection of information helps fulfill the requirements of:

1. Executive Order 12862, “Setting Customer Service Standards,” which directs Agencies to continually reform their management practices and operations to provide service to the public that matches or exceeds the best service available in the private sector.

This is a request for approval by the Health Resources and Services Administration (HRSA), Office of Federal Assistance Management’s (OFAM), Division of Independent Review (DIR) to continue using the Application Review Assessment Survey. This Survey Assessment is used to evaluate the HRSA objective review process for competing opportunities. The purpose of the Survey Assessment is to optimize the review process and administrative functions associated with managing annual competitive grant application reviews.

HRSA’s Division of Independent Review (DIR) is responsible for administering the review of eligible applications submitted for grants and cooperative agreements under HRSA competitive announcements. DIR ensures that the independent review process is objective, effective, economical, and complies with applicable statutes, regulations and policies. The review of applications is performed by review participants that are experts knowledgeable in the field for which funding support is requested.

The DIR process is in accordance with the U.S. Department of Health and Human Services' (DHHS) Uniform Administrative Requirements, Cost Principles, and Audit Requirements for HHS Awards 45 CFR 75, the HHS Grants Policy Statement, and the Public Health Service (PHS) Act, Sections 799(f) and 806(e).

Executive Order 12862 directs agencies that "provide significant services directly to the public" to "survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services".

The objective of surveying HRSA Application Review participants is to provide insight to management regarding their opinions, experiences and perceptions of the notice of funding opportunities and the conduct of the overall HRSA objective review process.

2. Purpose and Use of the Information

DIR is requesting approval to continue using the Survey Assessment of two (2) similar electronic forms, where each survey is slightly tailored to review duties, expectations, and experience of the:

• Reviewers and

• The panel Chair.

The Survey Assessment electronic forms will be used in DIR’s Field Reviews and Web-assisted Teleconferences. The outcome of both types of reviews are similar producing a ranking and summary statements. Review participant experience may differ depending on the level of interaction among reviewers.

Reviewers and Chairs, for all DIR reviews, are selected from the HRSA Reviewer Recruitment Module (RRM) using self-reported professional expertise. Reviewers evaluate the merit of competitive grant and cooperative agreement applications for possible Agency funding.

3. Use of Improved Information Technology

DIR has operated as the HRSA central review function since 2003, coordinating objective review activities between participants (Reviewers and Chairs), HRSA staff, and the review contractor. DIR routinely uses other web-based tools such as Reviewer Recruitment Module (RRM), and the Survey Monkey electronic platform specifically for data collection of review participant information. Use of an electronic format (like Survey Monkey) greatly enhances the rapid collection, analysis, and long term archival of review related comments in a highly reproducible and standardized format for trend analyses.

Where possible, the DIR Review Assessment form(s) uses simple drop-down menus, checkboxes and radio buttons to simplify the data collection process, reduce the respondent time, and burden. Attached are screen shots of the proposed Survey Monkey forms for Chairs (Attachment 1), and Reviewers (Attachment 2).

Sensitive information such as birthdates and social security numbers are not collected.

3. Efforts to Avoid Duplication

HRSA DIR does not utilize any other web-based vehicle or electronic format for objective review participants to submit review assessment information in a standardized fashion. DIR deems it necessary to collect review participant comments and experience to implement a measurable quality assurance program to assess the HRSA objective review process effectiveness. The web-based review assessment form mirrors DIR’s review operational changes and enhances the on-line review process and participant database. Consistency in the use of web-based systems and access, data capture, analysis tools, and ease of use were all taken into consideration for use of this particular assessment system. The Reviewer and Chair survey are unique to this activity and the information is not found elsewhere.

3. Involvement of Small Entities

This assessment survey does not have an impact on small businesses or other small entities. Individuals who serve as HRSA grant review participants may be affiliated with small entities. The information requested is the very minimum needed to assess objective review functions and is less burdensome to reviewer participants than the paper model.

6. Consequences of Collecting the Information Frequently

Each review session is unique to the program grant Notice of Funding Opportunity (NOFO), and could include one to18 panels depending on the number of eligible applications received by HRSA. Each review participant may have a slightly different experience based on their professional qualifications, prior review experience, and type of applications assigned to evaluate. Survey data collected from each review participant within a review session will be compared as a review “group”, and review-to-review comparisons will be made for overall trends.

HRSA on average hosts up to 85 objective review sessions per year, with an average of 5 panels per review, and up to 10 panel members (reviewers and chairs). This calculates up to a possible 4,250 survey encounters per year. Sustained use of an electronic platform with this inherent analysis capability will continue to enhance timely reporting and reduce assessment time.

There are no legal obstacles to reduce the burden.

7. Consistency With the Guidelines of 5 CFR 1320.5

This Reviewer and Chair surveys will be implemented in a manner fully consistent with 5 CFR 1320.5.

7. Consultation Outside the Agency

In accordance with 5 CFR 1320.8(d), on December 15, 2020, a 60 day notice was published in the Federal Register for HRSA’s generic clearance, OMB Control No. 0915-0212 (Vol. 85, No.241, pages 81210). No public comments were received.

9. Remuneration of Respondents

There will be no payment to respondents for submitting an assessment form. Review participant services are paid as honoraria to reviewers and chairs.

9. Assurance of Confidentiality

This HRSA assessment review participant survey does not collect personally identifiable information (PII) from review participants. Participation is voluntary and responses are anonymous. Participation or non-participation does not affect their selection for future reviews.

Information and data are maintained securely through the Survey Monkey Platform using a user name and password under a “locked” HRSA account. Survey downloads are stored in a HRSA database, and analyses are conducted using HRSA internal programs behind firewalls. Review specific assessment forms are identified by the unique HRSA review and panel number distributed to only the review participants, not publically accessible, and does not capture any PII from its users.

11. Questions of a Sensitive Nature

The surveys do not contain questions of a sensitive nature.

12. Estimates of Annualized Hours

Grant Recruitment Form	Number of respondents	Responses per Participant	Total responses	Hours per response	Total burden hours
Reviewer	2000	1	2000	0.25	500
Chair	300	1	300	0.25	75
Total	2300		2300		575

Note: This is a voluntary survey and DIR cannot be certain how many reviewers and chairs may or may not take the survey.

13. Estimates of Annualized Cost Burden to Respondents

There are no capital or startup costs and no operation and maintenance of service costs to respondents associated with this application.

13. Annualized Cost to the Government

The use of a web-based database form for the collection and organization self-reported reviewer information produces economic and business process efficiencies. An estimated 80 FTE hours will be utilized for system administrative activities (form creation to data analysis and reporting) using existing DIR staff and resources. DIR staff time dedicated to system management is 15% of a GS-13 step 2 (2021 Office of Personnel Management Salary Table) FTE (approximately $16,071.9).

13. Explanation for Program Changes or Adjustments

Not applicable. This is a continuation of a previously approved activity under HRSA’s generic clearance and will be included in the total burden currently approved by OMB under OMB Control No. 0915-0212.

13. Plans for Analysis and Timetable of Key Activities

The Division of Independent Review continues to perform analyses of data from each individual review for process and quality improvement in the objective review experience. Narrative information from the survey Assessment is summarized and examined using descriptive analysis. Findings are only used for internal service improvement and the information collected is not published outside of the Agency.

17. Exemption for display of Expiration Date

No exemption is requested. The expiration date will be displayed.

18. Certification

This information collection activity complies with the requirements in 5 CFR 1320.9.

Attachments

1. HRSA Survey Assessment of Objective Peer Review –Reviewer Form 1 (9 screen shots)

2. HRSA Survey Assessment of Objective Peer Review – Chair Form 2 (4 screen shots)

3. HRSA Survey Assessment of Objective Field Review- Field Review (7 Screenshots)

5