Supporting Statement A for
Federal COVID Response - Audience Feedback to Inform Ongoing Messaging and Strategies for "Combat COVID"
OMB# 0925-0769, exp., 12/31/2021
Date:11/10/21
Check off which applies:
New
Revision
Reinstatement with Change
Reinstatement without Change
Extension
Emergency
Existing w/o OMB approval
Federal Government Employee Information:
Name: Ryan Bayha
Address: National Institutes of Health, Office of Science Policy
6705 Rockledge Drive, Suite 750, MSC 7985
Bethesda, MD 20892-7985
Telephone: 301.435.2138
Fax: 301.496.9839
Email: Ryan.Bayha@nih.hhs.gov
Table of contents
A. ABSTRACT
A.1 Circumstances Making the Collection of Information Necessary
A.2. Purpose and Use of the Information COLLECTION
A.3 Use of Information Technology and Burden Reduction
A.4 Efforts to Identify Duplication and Use of Similar Information
A.5 Impact on Small Businesses or Other Small Entities
A.6 Consequences of Collecting the Information Less Frequently
A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency
A.9 Explanation of Any Payment of Gift to Respondents
A.10 Assurance of Confidentiality Provided to Respondents
A.11 Justification for Sensitive Questions
A.12 Estimates of Hour Burden Including Annualized Hourly Costs
A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record keepers
A.14 Annualized Cost to the Federal Government
A.15 Explanation for Program Changes or Adjustments
A.16 Plans for Tabulation and Publication and Project Time Schedule
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions
Attachments (save file names to match what is being referenced: (ex: x.baseline; y.screener)
ATTACHMENTS
Attachment 1: Consumer Recruitment Screener
Attachment 2: HCP Recruitment Screener
Attachment 3: Moderators Guide for Discussions with Consumers
Attachment 4: Moderators Guide for Discussions with HCPs
Attachment 5: Audience Feedback Team Sample Online Bulletin Board Prompt
Attachment 6: Custom Web Survey with Consumer and Provider Paths
A. Justification
The Federal COVID Response (FCR) Team is a cross-agency partnership that includes the U.S. Department of Health and Human Services (HHS), including the National Institutes of Health (NIH) Office of the Director, Centers for Disease Control and Prevention (CDC), the U.S. Food and Drug Administration (FDA), the Biomedical Advanced Research and Development Authority (BARDA), and the U.S. Department of Defense (DOD). The FCR coordinates existing HHS-wide efforts, including the NIH’s Accelerating COVID-19 Therapeutic Interventions and Vaccines (ACTIV) partnership, NIH’s Rapid Acceleration of Diagnostics (RADx) initiative, and work by BARDA.
A.1 Circumstances Making the Collection of Information Necessary
The Federal COVID Response (FCR) Team oversees the Combat COVID initiative (including the combatcovid.hhs.gov website)—a multifaceted effort to provide the general public and healthcare providers with the latest evidence-based information on COVID-19 treatments and the Accelerating COVID-19 Therapeutic Interventions and Vaccines (ACTIV) clinical trials. Together with their contractor, the FCR Team is working to:
Address participation barriers and raise awareness of ACTIV clinical trials, and
Ensure the general public’s and health care provider’s needs are met as it pertains to evidence-based information on these trials.
Because the COVID-19 treatment landscape continues to evolve, new evidence-based information continues to come to the forefront, and audience needs continue to change, it is critical for the FCR Team to collect routine audience feedback from the general public (especially from groups who have not historically been well-represented in clinical trials) and healthcare providers to identify these evolving needs. By understanding target audience needs, the FCR team will be able to properly develop and broadly disseminate relevant COVID-19 treatment and ACTIV clinical trial resources.
In a previous submission (OMB Control No. 0925-0769, ICR Reference No. 202105-0925-001), the FCR team requested emergency clearance to collect timely data on audience feedback in support of efforts to mitigate the pandemic. This request is to extend that data collection for 1 year beyond the 6-month emergency clearance approval period (which ends on December 31, 2021).
The purpose of the information collection is to collect routine feedback from the Combat COVID Initiative’s two target audiences (the general public and healthcare providers) to identify evolving needs and better disseminate relevant information as it relates to COVID-19 treatment and ACTIV clinical trial resources.
Data collected through this effort will be used to inform the development and broad dissemination of Combat COVID resources, including new or enhanced messages, materials, communications strategies, and/or web pages (combatcovid.hhs.gov). This data collection will continue from June 2021 through December 2022; the 1-year approval requested will cover the 12 months following the emergency clearance approval (which ends December 2021). This will allow us to continue gathering feedback over the life of the Combat COVID effort and to develop and test messages for advances in COVID-19 treatment (such as antiviral treatments currently seeking authorization).
The team will employ two strategies to collect this routine audience feedback:
DATA COLLECTION STRATEGY 1: Monthly 60-minute virtual audience feedback teams’ sessions (through focus groups, in-depth interviews, online bulletin boards) for rapid qualitative testing of new Combat COVID messages, concepts, ideas, resources, webpages, and materials.
DATA COLLECTION STRATEGY 2: 15-minute custom web surveys to quantitatively understand target audience needs, awareness of Combat COVID over time, and inform ongoing messages and strategies.
Submitting for Non-Substantial Changes
Because the COVID-19 landscape continues to evolve, we would like to request that we submit to OMB relevant new stimuli (e.g., social media ad concepts, out-of-home ad concepts, digital display ad concepts, webpage mockups) on a rolling basis for non-substantive change reviews. While our team has carefully crafted the questions needed to be asked via DATA COLLECTION STRATEGY 1 and DATA COLLECTION STRATEGY 2 above, the team cannot confirm all stimuli beyond the first data collection endeavor of DATA COLLECTION STRATEGY 1 and 2 at the time of this initial submission given the quick-changing nature of COVID-19 and the messaging that may be needed in the future as it relates to clinical trials for treatments. In terms of submitting non-substantial changes, the team will plan to prepare and deliver new stimuli to OMB no later than 10 business days prior to any data collection effort. Should OMB have feedback to the new stimuli being tested, the team will integrate feedback and will not move forward with the data collection until approved by OMB.
For DATA COLLECTION STRATEGY 2 (the two surveys), there may be a need to slightly adjust question wording depending on the stimuli that we are asking respondents to react to. For example, “Have you seen this webpage before today?” might need to be replaced with “Have you seen this ad before today?” Changes such as this will not impact the length of the survey or estimated burden. We will, however, plan to submit these relevant changes to OMB as non-substantial change requests and will not proceed with the data collection until the non-substantial change requests are reviewed and approved by OMB.
A non-substantive change request for updated stimuli and questions for the custom web survey was submitted on 11/9/21 and approved on 11/23/2021. Attachment 6 (Custom Web Survey with Consumer and Provider Paths) and the Federal COVID Response Consumer and HCP Suppl Materials reflects these changes.
Audience Feedback Teams Sessions: The FCR team will use a market research firm and their participant opt-in database to recruit for four Combat COVID audience feedback teams. These feedback teams will participate in routine qualitative sessions (i.e., online bulletin boards, focus groups, and in-depth interviews). Three audience feedback teams will include representation from the general public and include a mix of age, ethnicity/race, gender, geography, and education (n=60 total participants). The fourth feedback team will include health care providers who represent a mix of years in practice, geography, gender, and specialty type (n=20 participants). By recruiting a cohort of participants at the outset, in lieu of a new cohort of participants for each session, recruitment-related costs and participant recruitment screening burdens are minimized. Participants will be expected to participate in up to 12 qualitative sessions, each no longer that 60 minutes in length.
Custom Web Survey: The team will use a market research/survey panel firm to field two custom online surveys (one for consumers, one for HCPs). Surveys will be approximately 15 minutes in length each, fielded at baseline and at four “checkpoints” based on the creation and dissemination of new Combat COVID resources:
One survey with a nationally representative sample of the general public panel members (n = 2,000).
One survey with healthcare providers panel members who directly treat COVID-19 positive patients (n = 300).
Following the conclusion of each data collection activity, the contractor will submit topline summary presentations to the FCR Team, along with actionable recommendations for the development and/or enhancement of ongoing Combat COVID resources.
A.3 Use of Information Technology and Burden Reduction
Each of those modes will be deployed using online collaboration tools:
DATA COLLECTION STRATEGY 1 will consist of live discussions held via online video- or audio-conferencing (e.g., Zoom) or online bulletin boards for asynchronous discussions. Participants in virtual qualitative data collection efforts have been found to be no less and often more diverse in terms of geography, education level, and racial and ethnic identity than in-person data collection efforts.1,2 Although participants in virtual data collection tended to have higher overall internet use, virtual modes still successfully recruited participants reporting lower rates of internet use.3 Holding sessions remotely across target audiences helps to minimize participant burden overall (i.e., time it would take for a participant to drive to a research facility). For HCPs especially, the use of virtual data collection and asynchronous collection methods reduces the impact of participation on busy professional schedules.4
DATA COLLECTION STRATEGY 2 will recruit from an opt-in web survey research panel maintained by a market research/survey panel firm. This will allow users to participate on their own time using any device with a web browser. Pages are succinct and easy to navigate. Respondents only see questions that apply to them (i.e., a healthcare provider would not see the questions intended for a consumer).
No similar data are gathered or maintained by the agency or available from other sources known to the agency.
All data collection efforts will be conducted with individuals; thus, this will not impact small businesses.
Collecting routine information from the general public and healthcare providers will ensure the federal government stays abreast of changing target audience needs as they relate to COVID-19 treatments and ACTIV clinical trials. This ensures needed resources are developed and broadly disseminated, thus maximizing the return on taxpayer dollars. All data collection efforts however, are voluntary and all participants are free to respond, or not respond, to questions posed at any point during this endeavor. If data are collected less frequently, Combat COVID message development will not be able to respond to rapidly evolving information needs from the target audiences on a topic critical to the public’s health
This survey will be implemented in a manner that fully complies with 5 C.F.R. 1320.5.
The 60-Day Federal Register notice was published on September 7, 2021, (86 FR 50143) No public comments were received.
Federal employee representatives from the FCR Team reviewed the attachments to this package.
A.9 Explanation of Any Payment of Gift to Respondents
Research has shown improved participation in qualitative interviews with adults in the general United States population when given a monetary incentive in comparison to a nonmonetary incentive or no incentive.5
Feedback team participants will receive a token of appreciate of $75 for each feedback team activity in which they participate (individual discussion, group discussion, or online bulletin board post prompt). The proposed incentive amount has been shown to be effective by Federal agencies for focus group activities, offsetting the challenges for these audiences to participate in a focus group. This incentive amount was chosen based on potential expenses occurred for the average participant, such as:
childcare for both participating in the pre-discussion homework (such as reviewing the online bulletin board prompt or setting up the video-conferencing technology) and the focus group or response writing (60 minutes), estimated at $16.20 per hour;6
any loss of income to participants (e.g., if a participant has to take time off work to participate).
This amount is consistent with amounts offered for similar research efforts (virtual qualitative research participation).7,8
Consumer respondents and HCP respondents will receive $3 and $20, respectively, for a complete web survey. The consumer incentive amount is consistent incentives found to increase response rates.9 The difference in incentives for HCPs consistent with studies showing higher response rates from HCP respondents with increased incentives, especially looking for a subset of HCPs who have treated patients with COVID-19.10
A.10 Assurance of Confidentiality Provided to Respondents
No personally identifiable information will be collected. All respondent information will be kept private to the extent permitted by law. The FCR Team (including NIH, BARDA, CDC, FDA, and DoD) will not have access to any personally identifiable participant/respondent data for neither the audience feedback teams, nor the surveys, at any time during or after this data collection endeavor.
A.11 Justification for Sensitive Questions
To identify which messages are clear, relevant, and actionable for different audiences, this data collection will ask questions about experience with COVID-19 prevention and treatment. Audience feedback team respondents and survey respondents are alerted of this prior to each data collection effort and are assured that their responses are entirely voluntary. If a question makes a participant/respondent uncomfortable, they are free to “skip” it. Participants/respondents may also choose to stop participation at any time. For demographic reporting purposes, the team also collects information on age, gender, ethnicity and race, employment status, income range, state of residence, education, and political affiliation. These questions are voluntary, and participants are free to decline a response. A person’s first name will never be connected to their demographic information, including race and ethnicity. The information will only ever be reported in summary form.
A.12.1 Estimates of Hour Burden Including Annualized Hourly Costs
In order to successfully recruit for audience feedback teams, it may be necessary to contact up to 120 members of the general public and 40 HCPs. Screening takes 5 minutes, for a total of 13.33 hours. Each audience feedback team activity (focus group, individual interview, or online bulletin board post) will take 60 minutes. The 80 audience feedback team members (60 general public, 20 HCP) will participate in up to 8 activities over the project time period for a total of 960 hours. (The entire project period runs from June 2021 through December 2022; the requested 1-year extension will cover the period between the expiration of the emergency clearance on 12/31/21 and the close of the Combat COVID project in December 2022.) 2,000 general public survey respondents and 300 HCP respondents will complete each 15-minute survey. Surveys will be fielded at five times over the project period (one benchmark and four follow-up), for a total of 1,725 hours.
Table 12-1 Estimated Annualized Burden Hours
Type of Collection |
No. of Respondents |
No. of Responses per Respondent |
Time per Response (in hours) |
Total Hours |
Consumer Audience Feedback Team Screener (Attachment 1) |
120 |
1 |
5/60 |
10 |
HCP Audience Feedback Team Screener (Attachment 2) |
40 |
1 |
5/60 |
3 |
Consumer Audience Feedback Activity (Attachments 3 & 5) |
60 |
8 |
1 |
480 |
HCP Audience Feedback Activity (Attachments 4 & 5) |
20 |
8 |
1 |
160 |
Benchmark & Follow-Up Web Surveys – Consumer Audience (Attachment 6) |
2,000 |
5 |
15/60 |
2,500 |
Benchmark & Follow-Up Web Survey – HCP Audience (Attachment 6) |
300 |
5 |
15/60 |
375 |
Total |
2,540 |
12,300 |
…. |
3,528 |
A.12-2 Annual Cost to respondent
Table 12-2 Annualized Cost to Respondents
Type of Respondents |
Total Annual Burden Hours |
Hourly Respondent Mean Wage Rate* |
Respondent Cost |
General Public |
2,990 |
$27.07 |
$80,939.30 |
Health Care Providers
|
538 |
$41.30 |
$22,219.40 |
TOTAL |
3,528 |
|
$103,158.70 |
*The General Public and HCP hourly rates were obtained from https://www.bls.gov/oes/current/oes_nat.htm
A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers
No other cost burden is expected.
Staff |
Grade/Step |
Salary* |
% of Effort |
Fringe (if applicable) |
Total Cost to Gov’t |
Federal Oversight |
LTC/GS14 |
$111,330 |
5% |
|
$5,567 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Contractor Cost |
|
|
|
|
$360,600 |
|
|
|
|
|
|
Travel |
|
|
|
|
$0 |
Other Cost |
|
|
|
|
$0 |
|
|
|
|
|
|
Total |
|
|
|
|
$366,167 |
*the Salary in table above is cited from https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/20Tables/html/DCB.aspx
A.15 Explanation for Program Changes or Adjustments
No changes to data collection instruments have been made after the non-substantive change request approved on 11/23/2021.
Results from this data collection will to be published. Data will be use internally for the development and/or enhancement of ongoing Combat COVID resources.
A.16 - 1 Project Time Schedule |
|
Activity |
Time Schedule |
Field Wave 2 of Survey & Analyze Results |
November 2021 – January 2022 |
Audience Feedback Team Recruitment Update |
January 2022 |
Conduct Message Testing with Audience Feedback Team & Analyze Results |
January – March 2022 |
Update (as Needed) & Field Wave 3 of Survey & Analyze Results |
March 2022 – June 2022 |
Updated Message Testing & Analysis (Audience Feedback Teams |
June 2022 – September 2022 |
Final Analysis |
September 2022 – November 2022 |
A.17 Reason(s) Display of OMB Expiration Date is Inappropriate
We are not requesting an exemption to the display of the OMB Expiration date. All data collection instruments will display the OMB number and expiration date.
A.18 Exceptions to Certification for Paperwork Reduction Act Submissions
This survey will comply with the requirements in 5 CFR 1320.9.
1 Rupert D, Poehlman J, Hayes J, Ray S, Moultrie R. Virtual Versus In-Person Focus Groups: Comparison of Costs, Recruitment, and Participant Logistics. J Med Internet Res 2017;19(3):e80. DOI: 10.2196/jmir.6980
2 Namey E, Guest G, O’Regan A, Godwin CL, Taylor J, Martinez A. How Does Mode of Qualitative Data Collection Affect Data and Cost? Findings from a Quasi-experimental Study. Field Methods. 2020;32(1):58-74. doi:10.1177/1525822X19886839.
3 Rupert et al., 2017.
4 Daniels N, Gillen P, Casson K, Wilson I. STEER: Factors to Consider When Designing Online Focus Groups Using Audiovisual Technology in Health Research. International Journal of Qualitative Methods. January 2019. doi:10.1177/1609406919885786
5 Kelly, B., Margolis, M., McCormack, L., LeBaron, P. A., & Chowdhury, D. (2017). What Affects People’s Willingness to Participate in Qualitative Research? An Experimental Comparison of Five Incentives. Field Methods, 29(4), 333–350. https://doi.org/10.1177/1525822X17698958
6 Care.com 2020 Cost of Care Survey, retrieved 4/29/21 from https://www.care.com/c/stories/10278/what-to-charge-for-child-care-jobs-child-care-job-guide/.
7 Rupert D et al., 2017.
8 Kelley et al., 2017.
9 Smith, M.G., Witte, M., Rocha, S. et al. Effectiveness of incentives and follow-up on increasing survey response rates and participation in field studies. BMC Med Res Methodol 19, 230 (2019). https://doi.org/10.1186/s12874-019-0868-8
10 Noel H, Huang AR. The Effect of Varying Incentive Amounts on Physician Survey Response. Evaluation & the Health Professions. 2019;42(1):71-81. doi:10.1177/0163278718809844
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Subject | Supporting Statement A |
| Author | Lopez, Maria (NIH/NICHD) [E] |
| File Modified | 0000-00-00 |
| File Created | 2021-11-24 |