Site Visit SSA

OMB Control Number 0990-0421, Expires 2/28/2027

Best Practices for COVID-19 Vaccination and Testing

ASPE Generic Clearance for the Collection of Qualitative Research and Assessment

OMB Control Number 0990-0421, Expires 2/28/2027

Supporting Statement – Section A

Submitted: MMMM DD, 2024

Program Official/Project Officer

Deborah Porterfield, MD, MPH

Medical Officer

U.S. Department of Health and Human Services

Office of the Assistant Secretary for Planning and Evaluation

200 Independence Ave SW, Washington, DC 20201

202-730-8794

Deborah.Porterfield@hhs.gov

1. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

The COVID-19 pandemic has exacerbated longstanding health inequities (Simmons et al. 2021). Historically marginalized, systematically under resourced, and medically underserved U.S. communities have disproportionately experienced more COVID-19-related infections, hospitalizations, and deaths, compounded further by limited access to vaccines and other effective mitigation strategies (Khanijahani et al. 2021; Smith et al. 2021; Romano et al. 2021; Webb-Hooper et al. 2020; Islam et al. 2021; Nayak et al. 2020). The evolving nature of the COVID-19 pandemic and the inconsistent approach to testing and vaccination across jurisdictions in the U.S. delayed shared learnings and rapid implementation of effective strategies for all communities (DeSalvo et al. 2021). Contextual factors such as social and cultural beliefs and norms presented additional challenges to COVID-19 testing and vaccine uptake in many communities once tests and vaccines were available (Reitsma et al. 2021; Nguyen et al. 2022).

The Office of the Assistant Secretary for Planning and Evaluation (ASPE) is submitting this generic information collection request (ICR) to the Office of Management and Budget (OMB) to obtain approval to collect data on promising practices to increase (1) COVID-19 testing awareness, access, and uptake; (2) confidence in, access to, and uptake of COVID-19 vaccines in underserved populations; and (3) confidence in, access to, and uptake of routinely recommended vaccines for children and adults during the COVID-19 pandemic. Our contractor, Mathematica, is conducting a national survey (ICR Reference Number 202010-0990-001), sending a survey to organizations across the United States to better understand what organizations are doing to deliver services, who they are serving, and how well these efforts are working. As a complement to that, already approved information collection, our contractor team will also conduct virtual site visit interviews¹ with state health departments, tribal organizations, and local providers such as local health departments, clinics, and pharmacies. This Information Clearance Request (ICR) is focused on the interviews.

The purpose of the interviews is to learn about promising COVID-19 testing and vaccination programs and routine vaccination programs, as well as the contextual factors and implementation strategies that influence their success. Site visit interviews are important because they can provide in-depth detail to supplement the closed-ended survey data.

Building on data collected through the national survey, this site visit interview data collection effort is in continued response to ASPE’s recognition that obtaining broad and diverse perspectives on this area of practice is needed to help state and local health departments promote equity in the delivery and uptake of COVID-19 services.

2. Purpose and Use of the Information Collection

The data collected through site visit interviews will be used by ASPE to fill a gap in previous research which has not identified or examined promising practices for COVID-19 and vaccination and testing and routine vaccination across diverse populations and from diverse perspectives. This research will produce findings on promising practices and contextual factors that support the continued work of various research, program, and practice audiences intending to influence COVID-19 vaccination and testing, routine vaccination efforts, and efforts to support response to future pandemics. The data collection will not be designed or expected to yield results that are generalizable to the entire population.

The contractor will develop and disseminate a memo to practice audiences that provides key considerations for implementing and disseminating promising practices for COVID-19 vaccination and testing services, as well as routine vaccinations. Dissemination strategies will include, among other related materials, issue briefs, technical assistance, manuscripts, conference presentations, and convenings with stakeholders. Dissemination of findings will follow federal guidelines, e.g., the HHS "Guidelines for Ensuring the Quality of Information Disseminated to the Public," and will include specific discussion of the limitation of the qualitative results.

3. Consideration Given to Information Technology

ASPE’s contractor, Mathematica, uses numerous technologies to conduct and manage data collection efforts. Mathematica will use Webex, Cisco’s videoconferencing platform, to conduct all virtual interviews. Collecting data in this manner reduces burden on respondents and improves data quality. For example, the application enables users to (1) audio- or video-record their meeting, which Mathematica will do, for later review, transcription, and validation of the content of interview transcripts; (2) conduct the call exclusively via audio, enabling users with varying bandwidth and technical capacities to successfully engage in interviews; and (3) use chat, upload, and screen-share functions to share links, files, and other data in real time, potentially reducing burden associated with conventional, post-interview email follow-up.

Although we anticipate most, if not all, interviews will be conducted virtually, our contractor will use the Voice Recorder application on their secured and encrypted laptops to audio-record any in-person interviews.

All interview data and related information will be stored on Mathematica’s restricted drive, which complies with the Federal Information Security Management Act of 2002. The drive is secured and encrypted. Only a limited set of project staff can access the folder, and their access is specifically approved by the project director. Access control lists are reviewed at least annually. Staff are removed from folder access when they leave the project.

4. Duplication of Information

Neither ASPE nor our contractor are aware of any previous or ongoing qualitative research that explores promising practices for COVID-19 vaccination and testing and routine vaccination across the diverse populations and provider organization types that this project will engage. ASPE has and will continue to leverage feedback from internal and external experts throughout our design and data collection to ensure that we are not duplicating existing or new efforts.

5. Reducing the Burden on Small Entities

We anticipate that community representative respondents will be from smaller businesses or entities than public health leader or program and partner staff respondents. To minimize burden on community representatives, our contractor, Mathematica, will ask community representatives fewer questions than other respondents. We will also provide a $50 Visa gift card to community representative respondents who complete the interview to compensate them for their time.

6. Consequences of Not Conducting Collection

If ASPE does not conduct this data collection effort, we would be unable to identify promising practices for improving COVID-19 testing among people with disproportionate medical or social risks across a broad range of provider types, hindering ongoing efforts as well as our response to future public health emergencies. The interviews are a one-time data collection effort necessary to collect diverse, contextualized perspectives on this critical area. These data are not available from other sources, so they will contribute to an important field of knowledge.

7. Special Circumstances

There are no special circumstances with this information collection package. This request fully complies with regulation 5 CFR 1320.5 and will be strictly voluntary.

8. Consultations with Persons Outside the Agency

Our contractor developed the interview guides for the study and selected sites in collaboration with a panel of experts. Federal experts on this panel include Theresa Armstead, Samra Ashenafi, Desmond Banks, and Nancy Habarta of the Centers for Disease Control and Prevention; Vanessa Marshall of the National Institutes for Health; Susy Postal of the Indian Health Service; and John Snyder and Shafali Srivastava of the Health Resources and Services Administration. Non-federal experts on the panel include Mara Aspinall of the Health Catalysts Group, The Rockefeller Foundation, and Arizona State University; Alsan J. Bellard, Jr. of Community of Hope; Matthew Bobo of the National Association of County and City Health Officials; Rita Carreόn of UnidosUS; Hannah Fish of the National Community Pharmacists Association; Spero M. Manson of the Centers for American Indian & Alaska Native Health at the Colorado School of Public Health, University of Colorado Anschutz Medical Campus; Emily Messerli of the Association of Immunization Managers; Lily Shen of the Asian & Pacific Islander American Health Forum; and Kamau Stanford of the Black Doctors COVID-19 Consortium at the Dr. Ala Stanford Center for Health Equity.

9. Explanation of Any Payment or Gift to Respondents

We will provide a $50 Visa gift card to community representatives who complete an interview (30 minutes). We will provide a $100 Visa gift card to program and partner staff who complete an interview (60 minutes).

10. Confidentiality

Data collected in this study will be kept private to the extent allowed by law. Organization staff whom ASPE’s contractor staff will interview will be answering in their official roles and will be asked to speak only about their work. Community representatives will be asked only about the specific testing or vaccination services in their communities. We will collect respondents’ names and contact information only to facilitate communications related to this research. Individually identifiable information will not be saved with interview responses, will not be used in any publicly available reports, and will be destroyed following the conclusion of this study. The Institutional Review Board, Health Media Labs, has approved the outreach materials, interview guides, and informed consent language. No data collection will begin until OMB approval has been obtained.

11. Sensitive Nature

No questions will be asked in the interview that are of a personal or sensitive nature. Questions will focus on operation of and experience with COVID-19 vaccination and testing and routine vaccination programs.

12. Burden of Information Collection

The estimate for burden hours for all respondents includes participating in one interview. The length of the interview will vary based on respondent category (60 or 30 minutes). Burden estimates contain an additional 15 minutes to account for coordinating with ASPE contractor staff to schedule the interview. A subset of respondents will incur additional burden related to outreach and recruitment. For example, prior to conducting the site visit interviews, we will invite one primary public health leader from each site to participate in a 30-minute unstructured conversation during which we will explain the study and request recommendations of programs and people to include in data collection for that site.

See the summary of procedures for collecting information in Part B, and the full interview guides attached (Attachment A_Instruments).

Table A.1 shows estimated burden information, using an average salary of $44/hour for each public health leader and program and partner organization staff respondent, based on information on federally qualified health center, community-based organization, pharmacy, and public health employee salaries in September 2023. We used the average United States worker salary of $34/hour for the community representative in September 2023.²

Table A.1. Estimated Annualized Burden Hours to Respondents

Type of respondent	Form	Number of respondents	Number of responses per respondent	Average burden, in minutes per response	Total burden, in hours	Hourly wage rate	Total cost, across respondents
Public health leaders		27
Primary	Conversation and semi-structured interview	9	1	15 scheduling 30 planning conversation 60 interview	15.75	$44	$693
Other	Semi-structured interview	18	1	15 scheduling 60 interview	22.5	$44	$990
Program staff		228
Primary	Conversation and semi-structured interview	57	1	15 scheduling 60 interview	71.25	$44	$3,135
Other	Semi-structured interview	171	1	15 scheduling 60 interview	213.75	$44	$9,405
Community representative	Semi-structured interview	57	1	15 scheduling 30 interview	42.75	$34	$1,453.50

13. Costs to Respondents

There will be no direct costs to the respondents other than their time to participate in the data collection. All participation in this data collection is strictly voluntary and respondents can decline to respond any question they choose not to answer.

14. Costs to Federal Government

The cost of the government task order attributable to the work is $603,996. This cost includes salaried labor for contractor staff and other contractor direct costs associated with planning and execution of the interviews. Federal staff responsibilities include overall management and oversight of the project, provision of content matter expertise in the development of the research strategy and data collection instruments, and overseeing all data analyses and dissemination activities.

Table A.2. Estimated Annualized Cost to the Federal Government

Staff (FTE)	Average Hours per Collection	Average Hourly Rate	Average Cost
Contractor labor	2,527 hours	$180	$454,017
Other contractor direct costs (recruitment, incentives, transcripts, online platform)	N/A	N/A	$62,178
Federal staff costs	N/A	N/A	$63,576
Federal Contract Officer’s Representative (COR) – contract management	N/A	N/A	N/A
Estimated Total Cost of Information Collection			$579,771

15. Reason for Change

This is a new, one-time interview data collection effort. There is no change.

16. Tabulation of Results, Schedule, Analysis Plans

Information shared by respondents in interviews will be audio- or video-recorded (depending on respondent preference on using video during interviews), with study team interviewers taking notes electronically on their computers. Transcripts will be generated from the interviews for later coding and analysis and will be reviewed for completeness and accuracy. Information gathered will yield rich, descriptive information related to successful and novel COVID-19 vaccination and testing programs and routine vaccination programs, as well as the contextual factors and implementation strategies that influence their success. The data collection effort is not designed or expected to yield statistically reliable results. The study team will qualitatively code the transcripts and generate findings that will contribute to answers to the study’s research questions. No personal information will be reported in the dissemination of findings.

Table A.3. Timeline

Completion Date	Major Tasks/Milestones
February 2024	Start outreach
April 2024	Start data collection interviews
June 2024	Complete data collection interviews Start coding and analysis
July 2024	Complete coding and analysis
December 2024	Complete final report

17. Display of OMB Approval Date

We are not requesting an exemption from displaying the OMB approval date.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification. These activities comply with the requirements in 5 CFR 1320.9.

REFERENCES

DeSalvo, K., B. Hughes, M. Bassett, G. Benjamin, M. Fraser, S. Galea, and J.N. Gracia. “Public Health COVID-19 Impact Assessment: Lessons Learned and Compelling Needs.” NAM Perspectives. April 2021. Public Health COVID-19 Impact Assessment: Lessons Learned and Compelling Needs - National Academy of Medicine (nam.edu)

Islam S.J., A. Nayak, Y. Hu, A. Mehta, K. Dieppa, Z. Almuwaqqat, Y. Ko, et al. “Temporal Trends in the Association of Social Vulnerability and Race/Ethnicity with County-Level COVID-19 Incidence and Outcomes in the USA: An Ecological Analysis.” BMJ Open, vol. 11, 2021, p. e048086. http://dx.doi.org/10.1136/bmjopen-2020-048086.

Khanijahani, A., S. Iezadi, K. Gholipour, S. Azami-Aghdash, and D. Naghibi. “A Systematic Review of Racial/Ethnic and Socioeconomic Disparities in COVID-19.” International Journal for Equity in Health, vol. 20, no. 1, November 2021. https://doi.org/10.1186%2Fs12939-021-01582-4.

Nayak, A., S.J. Islam, A. Mehta, Y.A. Ko, S.A. Patel, A. Goyal, S. Sullivan, et al. “Impact of Social Vulnerability on COVID-19 Incidence and Outcomes in the United States.” medRxiv [preprint], April 2020. https://doi.org/10.1101%2F2020.04.10.20060962.

Nguyen, K.H., E. Anneser, A. Toppo, J.D. Allen, J.S. Parott, and L. Corlin. “Disparities in National and State Estimates of COVID-19 Vaccination Receipt and Intent to Vaccinate by Race/Ethnicity, Income, and Age Group Among Adults ≥ 18 Years, United States.” Vaccine, vol. 40, no. 1, January 2022, pp.107–113. https://doi.org/10.1016/j.vaccine.2021.11.040.

Reitsma, M.B., J.D. Goldhaber-Fiebert, and J.A. Salomon. “Quantifying and Benchmarking Disparities in COVID-19 Vaccination Rates by Race and Ethnicity.” JAMA Network Open, vol. 4, no. 10, October 2021, p. e2130343. http://jamanetwork.com/article.aspx?doi=10.1001/jamanetworkopen.2021.30343.

Romano S.D., A.J. Blackstock, E.V. Taylor, S.E. Felix, S. Adjeiet, C. Singleton, J. Fuld, et al. “Trends in Racial and Ethnic Disparities in COVID-19 Hospitalizations, by Region—United States, March–December 2020.” Morbidity and Mortality Weekly Report, vol. 70, April 2021, pp.560–565. https://www.cdc.gov/mmwr/volumes/70/wr/mm7015e2.htm.

Simmons, A., A. Chappel, A.R. Kolbe, L. Bush, and B.D. Sommers. “Health Disparities by Race and Ethnicity During the COVID-19 Pandemic: Current Evidence and Policy Approaches.” March 6, 2021. Available at https://aspe.hhs.gov/sites/default/files/migrated_legacy_files//199516/covid-equity-issue-brief.pdf. Accessed August 29, 2022.

Smith A.R., J. DeVies, E. Caruso, L. Radhakrishnan, M. Sheppard, Z. Stein, R.M. Calanan, et al. “Emergency Department Visits for COVID-19 by Race and Ethnicity—13 States, October–December 2020.” Morbidity and Mortality Weekly Report, vol. 70, no. 15, April 2021, pp.566–569. https://www.cdc.gov/mmwr/volumes/70/wr/mm7015e3.htm?s_cid=mm7015e3_w.

Webb Hooper, M., A.M. Nápoles, and E.J. Pérez-Stable. “COVID-19 and Racial/Ethnic Disparities.” Journal of the American Medical Association, vol. 323, no. 24, May 2020, pp. 2466–2467. http://jamanetwork.com/article.aspx?doi=10.1001/jama.2020.8598.

2. LIST OF ATTACHMENTS

Attachment A: Instruments

A.1: Interview guide: public health leaders

A.2: Interview guide: program and partner staff

A.3: Interview guide: community representatives

Attachment B: Outreach and Recruitment Materials

B.0: ASPE Recruitment Support Letter

B.1: Public health leader: Initial outreach email

B.2: Public health leader: Reminder outreach email

B.3: Lead program respondent: Initial outreach email

B.4: Lead program respondent: Reminder outreach email

B.5: Non-lead program and partner staff: Initial outreach email

B.6: Non-lead program and partner staff: Reminder outreach email

B.7: Community representatives: Initial outreach email

B.8: Community representatives: Reminder outreach email

B.9: Nonresponsive contacts: Initial phone call and voicemail script

B.10: Nonresponsive contacts: Reminder voicemail script

B.11: Calendar hold email

1 On the advice of their subcontracting partner with expertise in research with tribal organizations (KAI), Mathematica will offer all tribal organizations the option of in-person interviews.

2 Current salary information, as of September 2023, for federally qualified health center, community-based organization, pharmacy, and public health employees was obtained from www.ziprecruiter.com, www.salaryexpert.com, www.indeed.com, and salary information for community representatives was obtained from the U.S. Bureau of Labor Statistics at Summary table B. Establishment data, seasonally adjusted : U.S. Bureau of Labor Statistics (bls.gov). We used the average of the hourly rates of public health leaders, program staff, and community representatives to estimate the hourly rate of non-respondents.

Page 11 of 11