National Learning Community for HIV CBO Leadership Evaluation
OMB No.0920-New
October 27, 2021
Contact:
Program Official/Project Officer
Name: Grace “Chela” Hall, PhD, MPH, MPP
Title: Health Scientist
CIO: Centers for Disease Control and Prevention,
National Center for HIV, Viral Hepatitis, STD, and TB
Division: Division of HIV/AIDS Prevention
Branch: Capacity Building Branch
Address: 1600 Clifton Road, MS E40, Atlanta, GA 30333
Phone: 404-639-5224
Fax: 404-639-0915
Email: glh6@cdc.gov
Table of Contents
Section
A. Justification
1. Circumstances Making the Collection of Information Necessary
2. Purpose and Use of the Information Collection
3. Use of Improved Information Technology and Burden Reduction
4. Efforts to Identify Duplication and Use of Similar Information
5. Impact on Small Businesses or Other Small Entities
6. Consequences of Collecting the Information Less frequently
7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
9. Explanation of Any Payment or Gift to Respondents
10. Protection of the Privacy and Confidentiality of Information Provided by Respondents
11. Institutional Review Board (IRB) and Justification for Sensitive Questions
12. Estimates of Annualized Burden Hours and Costs
13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers
14. Annualized Cost to the Government
15. Explanation for Program Changes or Adjustments
16. Plans for Tabulation and Publication and Project Time Schedule
17. Reason(s) Display of OMB Expiration Date is Inappropriate
18. Exceptions to Certification for Paperwork Reduction Act Submissions
Exhibits
Table 12A Estimates of Annualized Burden Hours
Table 12B Estimated Annualized Burden Costs to Respondents
Table 16 Project Time Schedule
LIST OF ATTACHMENTS
1: Authorizing Legislation
2: 60-Day FRN
2a: Public Comments
3: Email Invitation for National Learning Community for HIV CBO Leadership Registration
4: Word version of National Learning Community for HIV CBO Leadership Registration Form
5: Screenshots of National Learning Community for HIV CBO Leadership Registration Form
6: Email reminder for National Learning Community for HIV CBO Leadership Registration Form
7: Email invitation for National Learning Community for HIV CBO Leadership Post-Participation Survey
8: Word version of National Learning Community for HIV CBO Leadership Post-Participation Survey
9: Screenshots of National Learning Community for HIV CBO Leadership Post-Participation Survey
10: Email reminder for National Learning Community for HIV CBO Leadership Post-Participation Survey
11: Email invitation for National Learning Community for HIV CBO Leadership Semi-Structured Interview
12: National Learning Community for HIV CBO Leadership Semi-Structured Telephone Interview
13: Email reminder for National Learning Community for HIV CBO Leadership Semi-Structured Interview
14: Privacy Impact Assessment (PIA)
15: Security Controls
16: IRB Project Determination
Section A. JUSTIFICATION
1. Circumstances Making the Collection of Information Necessary
Background
The Centers for Disease Control and Prevention requests a 3-year approval for a new information collection entitled, “National Learning Community for HIV CBO Leadership Evaluation”.
This information collection is authorized by Section 301 of the Public Health Service Act (42 U.S.C. Sec. 792[295k] (a) (Attachment 1: Authorizing Legislation).
The circumstances making this collection necessary arises out of the CDC’s need to assess the National Learning Community’s usefulness of training materials used by mid- and senior-level Community Based Organization (CBO) leaders’ management of people, programs, and organizations. Applying the Kirkpatrick four-level of training evaluation we are focused on getting National Learning Community participants reactions to the overall training gaining insight to whether the training fulfilled learning objectives and insight to the learners’ experience. We are also focus on two other levels of the Kirkpatrick evaluation model in which we are seeking information to determine if we built skills and knowledge during the training, and for those who complete the intensive if they are able to apply their new knowledge/skills to their work. This information collection fits into CDC’s broader research agenda in that it falls under the essential public health service of 1) informing, educating, and empowering people about health issues; 2) mobilizing community partnerships to identify and solve health problems; 3) linking people to needed personal health services and assure the provision of health care when otherwise unavailable; 4) assuring a competent public health and personal health care workforce.
The Asian & Pacific Islander American Health Forum (APIAHF) program offers the distance-based National Learning Community for HIV CBO Leadership (Learning Community) — designed to provide evidence-based tools they can use to engage staff. This Learning Community builds on an existing body of CDC CBA Provider Network (CPN) sustainability, quality improvement, and program manager trainings and resources. It also leverages APIAHF’s 32-year history of providing CBA to CBOs — including 25 CDC-funded HIV CBOs — and its expertise applying a Human-Centered Design (HCD) approach to meet CBO quality improvement and sustainability challenges.
2. Purpose and Use of the Information Collection
For the purposes of program improvement, data will be collected from participants of the National Learning Community for HIV CBO Leadership eLearning short courses and a 6-week synchronous creative problem-solving intensive. Data will be used to improve outreach efforts; to improve the short courses and the creative problem-solving intensive sessions which are not rated highly by program participants; and to improve access to technology platforms used to deliver these services throughout the program’s duration. The Post-Participation Survey (att 8 and 9) will collect information once from 270 respondents and the Semi-Structured Telephone Interview will collect information one time as well from 135 respondents. In the end, the data collected by the funded recipient, Asian & Pacific Islander American Health Forum (APIAHF), will be used to evaluate the overall effectiveness of the National Learning Community for HIV CBO Leadership.
At the end of each program year, data will be analyzed, reviewed, and used to improve the program in the following ways:
#1 – How participants found out about the National Learning Community for HIV CBO Leadership program will be used to identify which outreach strategies have been most successful and further emphasize those methods.
#2 – Evaluate the enrollment trends beyond the foundational course offerings (i.e., supplemental courses, coaching, and National Learning Community creative problem-solving Intensive).
#3 – Evaluate the enrollment trends in the intensive part of the program.
#4 – Evaluate the enrollment in the individual courses, to determine the patterns of usage.
The findings and recommendations will be discussed by APIAHF and CDC to determine if changes are necessary for the next program year’s iteration. The program improvement cycle will repeat annually throughout the National Learning Community for HIV CBO Leadership cooperative agreement funding period.
The
information will be collected from CBO leaders who support HIV
prevention, treatment, and care services. The participants will
mostly be from organizations that are funded directly or indirectly
by CDC. Learning Community participants’ positions may include:
Program Coordinator/Manager/Supervisor/Director;
Prevention
Coordinator/Manager/Supervisor/Director;
Testing
Coordinator/Manager/Supervisor/Director;
Linkage and Navigation
Coordinator/Manager/Supervisor/Director;
Outreach
Coordinator/Manager/Supervisor/Director.
The APIAHF is committed to continuous quality improvement of its CBA services and products. The proposed information collection will provide the APIAHF with necessary information to improve program processes and operations. In the absence of this evaluation, APIAHF’s ability to make timely and essential mid-course corrections, if needed, to better meet the needs of its consumers will be greatly impaired.
APIAHF will disseminate the summarized information through reports to CBB and its funding recipients, and possibly publications or presentations. All data will be shared in the aggregate. If results are shared with the public via presentations or publications, results will be shared in the aggregate, and any information that may identify an agency or individual will be masked. As individuals participating in the Learning Community will be representing their organizations, there will be no impact on their personal privacy.
The information collection system consists of three instruments administered to Learning Community participants.
Before participants are granted access to the Learning Community, they will receive an email invitation (Attachment 3) to an online Learning Community Registration Form (Attachments 4 and 5). One week after the email invitation for the Registration Form is sent out, a reminder email will be sent to participants who have not completed it (Attachment 6).
Three months after a Learning Community participant registers, they will receive an email invitation (Attachment 7) to complete an online Post-Participation Survey (Attachments 8 and 9). One week after the invitation email for the Post-Participation Survey is sent out, a reminder email will be sent to participants who have not completed it (Attachment 10).
Participants of the Creative Problem-Solving Intensive will receive an invitation email (Attachment 11) to schedule in a semi-structured phone interview (Attachment 12) upon completion of the six-week Creative Problem-Solving Intensive. One week after the invitation email is sent, a reminder email will be sent to participants who have not scheduled an interview (Attachment 13). Those who respond to the email and schedule an interview will be contacted by evaluation staff from the Asian & Pacific Islander American Health Forum for the interview. The interview will be conducted by phone only via Zoom, using a privacy code. It will last 15 minutes. Participants will be asked if they agree for the interview to be recorded in order to note their responses correctly. If they agree, the recording will be conducted through Zoom. If they do not agree to the interview being recorded, they will be asked if they agree to the interviewer taking written notes during the interview. If they do not agree to either option, the interview will not take place.
3. Use of Improved Information Technology and Burden Reduction
Online versions of the quantitative data collection tools will reduce the overall burden on respondents by allowing them to submit their responses electronically. These information collection instruments were designed to collect the minimum information necessary for the purposes of this project.
4. Efforts to Identify Duplication and Use of Similar Information
The information being collected is specific to the National Learning Community for HIV CBO Leadership. This data collection will assess how well the Learning Community meets the knowledge and skills needed of its participants to improve management practices of HIV prevention service delivery, enhance organizational capacity, and enhance agency sustainability over time.
5. Impact on Small Businesses or Other Small Entities
This data collection will not involve small businesses or other small entities.
6. Consequences of Collecting the Information Less Frequently
This request is for a one-time information collection for registration into the Learning Community, three months after registering for the Learning Community, and after completion of the Creative Problem-Solving Intensive. Each participant will receive the registration survey and the post-participation survey only once. Those who participate in the Creative Problem-Solving Intensive will have the opportunity to participate in a one-time phone interview. Each participant will therefore respond to a minimum of two and a maximum of three data collections total. There are no legal obstacles to reduce the burden.
Not collecting this information would hinder C4H’s ability to:
Assess the quality of services delivered by C4H
Identify and address the technical assistance needs of community-based organizations, health departments, and healthcare organizations
Identify programmatic areas of improvement
Respond promptly to emerging problems identified through feedback from consumers
7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
This request fully complies with the regulation 5 CFR 1320.5.
8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
The 60-day federal register notice was published on 04/26/2021, Volume 86, Number 78, Pages 22053-55 (Attachment 2). CDC received one public comment identifying several issues for consideration. No response was sent to the responder and no changes were made to the information collection (Attachment 2a).
9. Explanation of Any Payment or Gift to Respondents
CDC will not provide payments or gifts to respondents.
10. Protection of the Privacy and Confidentiality of Information Provided by Respondents
The Privacy Officer for CDC / ATSDR has assessed this package for applicability of 5 U.S.C. § 552a. The Privacy Act is applicable because PII is being collected under this CDC funded activity. Employees of community-based organizations (CBOs) will be speaking from their official roles. Participation in the information collection activities is voluntary for the respondents.
Of the three instruments, the Registration Form is the only tool which collects categories of information in identifiable format from individual respondents such as: business street address, work phone, work email address, and job title. These identifiable Registration Form data are needed to complete registration. The identifiable data from the Registration Form is entered, stored, and transmitted in a separate database from all the other data collected. A unique identifier will be used on all data collection instruments to enable the linking of data from multiple data collection tools. The unique identifier consists of the first two letters of the first name, the first two letters of the last name, the month of birth, and the day of birth.
There are several safeguards in place to handle data. Data will be securely transferred, stored, and accessed according to existing protocols and data security and confidentiality standards at APIAHF (which are written to adhere to all standards of the CDC/NCHHSTP data security and confidentiality guidelines) (Attachment 16). To minimize the risks to participants, data will be securely stored on APIAHF servers and access to the data will be limited to authorized study staff meeting security training requirements. No data will be sent to CDC.
The system security plan is included in the OCISO C&A process and the contingency (or backup) plan for this information collection, (as mandated by OCISO), is to manage this information from a pre-determined OCISO approved off-site location.
http://www.cdc.gov/about/leadership/leaders/seligman.htm.
Files are backed up and stored in accordance with CDC standards and OCISO guidelines. All users’ access is “role based” and reflects a “need to know” policy established by CDC.
(http://www.cdc.gov/about/leadership/leaders/seligman.htm” (http://aops-mas-iis.od.cdc.gov/Policy/Doc/policy449.htm)
A non-research determination was made and therefore, IRB review is not needed (Attachment 16). This data collection is not considered research based on the description and justification and based on the definition of research as defined by the federal policy for the protection of human subjects (45 CFR 46)(Attachment 16).
11. Institutional Review Board (IRB) and Justification for Sensitive
IRB
Since this is associated with programmatic improvement, it was determined that this did not require (IRB) oversight. We received for a project determination approval from the NCHHSTP (Attachment 16).
Sensitive Questions
No information will be collected that are of personal or sensitive nature. Respondents are participating in their official capacity as health professionals in CBOs, health departments, and healthcare organizations.
12. Estimates of Annualized Burden Hours and Costs
The average time to complete the instruments including time for reviewing instructions, gathering needed information and completing the instrument, was approximately three minutes for the Registration Form (Attachments 4 and 5), 9 minutes for the Post-Participation Survey (Attachments 8 and 9), and 15 minutes for the Semi-Structured Interview (Attachment 12).
The number of respondents is calculated based on the number of health professionals receiving CDC funded CBA services during the years 2019 to 2023. We estimate that during that timeframe, 270 health professionals will respond to the Registration Form, 270 health professionals will respond to the Post-Participation Survey, and 135 health professionals will participate in the Semi-Structured Interview.
Based on these results, the estimated time range for actual respondents to complete the instruments is 3 minutes for the 270 respondents who will complete the Registration Form and 9 minutes for the Post-Participation Survey, plus an additional 15 minutes for the 135 respondents who will also participate in the semi-structured interview. For the purposes of estimating burden hours, the upper limit of this range (i.e., 30 minutes) is used. The total burden hours are 89. The following table shows estimated burden.
Table 12A: Estimates of Annualized Burden Hours
Type of Respondent |
Form Name |
No. of Respondents |
No. Responses per Respondent |
Average Burden per Response
|
Total Burden Hours |
CBO Managers |
Registration Form (att #4 & 5) |
270 |
1 |
3/60 |
14 |
CBO Managers |
Post Participation Survey (att #8 & 9) |
270 |
1 |
9/60 |
41 |
CBO Managers |
Semi-Structured Phone Interview (att #12) |
135 |
1 |
15/60 |
34 |
Total |
|
|
|
|
89 |
Estimates for the average hourly wage for respondents are based on the Department of Labor (DOL) National Compensation Survey estimate for management occupations – Social & Community Services Managers (https://www.bls.gov/oes/current/oes119151.htm March 2020, accessed June 2020). Based on DOL data, an average hourly wage of $35.05 is estimated for the respondents. The following table shows estimated burden and cost information.
Table 12B: Estimated Annualized Burden Costs to Respondents
Type of Respondent |
Form Name |
Total Burden Hours |
Hourly Wage Rate |
Total Respondent Costs |
CBO Managers |
Registration Form (att #4 & 5) |
14 |
$35.05 |
$491 |
CBO Managers |
Post Participation Survey (att #8 & 9) |
41 |
$35.05 |
$1,437 |
CBO Managers |
Semi-Structured Phone Interview (att #12) |
34 |
$35.05 |
$1,192 |
Total |
|
|
|
$3,120 |
13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers
There will be no direct costs to the respondents other than their time to participate in each information collection.
14. Annualized Cost to the Government
The annualized cost to the government is estimated to be $4,501.
Table 14: Estimated Annualized Cost to the Federal Government
Staff (FTE) |
Average Hours per Collection |
Average Hourly Rate |
Average Cost |
Health Scientist (GS-13): OMB package preparation; review and oversight of assessment design, instrument development, pilot testing, data collection, quality control, data analysis and report preparation |
104 |
43.28 |
$4,501 |
Estimated Total Cost of Information Collection |
|
|
$4,501 |
15. Explanation for Program Changes or Adjustments
There are no program changes or adjustments.
16. Plans for Tabulation and Publication and Project Time Schedule
The quantitative data will be analyzed using frequency distributions, cross-tabulations, chi-square tests, and regression modeling. The qualitative data will be analyzed using thematic analysis.
The results of this assessment will be shared with CDC leadership and CBB staff. The results will be used by CBB for continuous quality improvement of the Learning Community services and products and to improve program processes and operations. CDC staff will receive annually a more detailed summary report of the yearly findings. By March 2023, a report summarizing all cumulative years of data collection will be shared with CDC leadership.
Table 16: Project Time Schedule
Collect, code, and enter data |
Upon OMB approval – 24 to 30 months
Desired need for OMB approval 11/1/21 |
Quality control and analyze data |
12 months after OMB approval |
Prepare report |
24 months after OMB approval |
Disseminate results/reports |
30 months after OMB approval |
17. Reason(s) Display of OMB Expiration Date is Inappropriate
The display of the OMB expiration date is not inappropriate
18. Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification.
REFERENCE LIST
The White House. “National HIV/AIDS Strategy for the United States: Updated to 2020.” Available at https://npin.cdc.gov/publication/national-hivaids-strategy-united-states-updated-2020. Accessed on 6/25/20
Centers for Disease Control and Prevention (CDC). “Capacity Building Assistance for High-Impact HIV Prevention – PS19-1904: Program Backgrounder.” Available at https://www.cdc.gov/hiv/pdf/funding/announcements/ps19-1904/cdc-hiv-1904-CBA-backgrounder.pdf. Accessed at 6/25/20.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | M Phields |
| File Modified | 0000-00-00 |
| File Created | 2021-12-01 |