(OMB Control No. 0990-0476)
Justification
Need and Legal Basis
In accordance with 5 CFR 1320.13, the United States Department of Health and Human Services (HHS) plans to submit an emergency processing submission. As a general matter, under sections 1320 (a)(2)(ii) and (2)(iii) of the federal regulations, an emergency processing may be authorized in the occurrence of an unanticipated event and the use of normal clearance procedures is reasonably likely to prevent or disrupt the collection of information. Given the impact of the COVID-19 pandemic on the nation, the Secretary of Health and Human Services (HHS) has also declared a public health emergency effective January 27, 2020, under section 319 of the Public Health Service Act (42 U.S.C. 247d [1]) and renewed it continually since its issuance (see links to the determination here and here).
Information Users
This submission contains five parts: 1. COVID-19 Current Events Tracker; 2. Foundational Focus Groups/Interviews/Dyads; 3. Copy Testing Surveys; 4. Message Matrix Surveys; and 5. Creative Testing Surveys and Experiments. The broad purpose of each effort is as follows:
Current Events Tracker
The primary purpose of the COVID-19 Current Events Tracker (CET) survey is to continuously track key metrics of importance to the Campaign, including vaccine confidence, familiarity with and trust in HHS, and the impact of external events on key attitudes and behaviors. Tracking Americans’ attitudes about, perceptions of, and behavior toward the COVID-19 pandemic will inform the Campaign of key metrics around vaccine confidence and uptake, as well as towards vaccine messengers such as HHS and key public health officials. It will also inform changes in messaging strategies necessary to effectively reach the entire U.S. population or specific subgroups.
The weekly tracking of this information will be critical for the Campaign’s ability to respond to shifting events and attitudes in real-time, helping guide the American public with accurate information about the vaccine rollout as well as on how to take protective actions against COVID-19.
Foundational Focus Groups, Interviews, and/or Dyads
The office of the Assistant Secretary for Public Affairs (ASPA) is collecting information through the COVID-19 Public Education Campaign Foundational Focus Groups, Interviews, and/or Dyads to inform the Campaign about audience risk knowledge, perceptions, current behaviors, and barriers and motivators to healthy behaviors (including COVID-19 vaccination). Ultimately these focus groups, interviews, and/or dyads will provide in-depth insights regarding information needed by Campaign audiences as well as their attitudes and behaviors related to COVID-19 and the COVID-19 vaccines. These will be used to inform the development of Campaign messages and strategy.
Copy Testing Surveys
Prior to placing Campaign advertisements in market, ASPA will conduct copy testing surveys to ensure the final Campaign messages have the intended effect on target attitudes and behaviors. Copy testing surveys will be conducted with sample members who comprise the target audiences; these surveys will assess perceived effectiveness of the advertisements as well as the effect of exposure to an ad on key attitudes and behavioral intentions. The results from these surveys will be used internally by ASPA to inform decisions on Campaign messages and materials; for example, to identify revisions to the materials or determine which advertisement to air.
Message Matrix Surveys
The purpose of the Messaging Matrix surveys is to evaluate, validate, and prioritize Campaign messages for various target audiences. The surveys test Campaign messages, including CDC-approved facts about COVID-19, to help ASPA understand which messages are most effective at influencing COVID-19-related attitudes and behaviors. These surveys will also allow ASPA to identify which messages work across multiple target audiences. Findings from these surveys will be used to inform the development of Campaign messages and strategy. Given the ongoing need for rapid Campaign message testing and development for our target audiences, ASPA will conduct the Messaging Matrix survey every other month. By creating a regular Messaging Matrix cadence, we will be able to pivot quickly to new or emerging issues, and ensure we are able to continuously gather data on messages that work with the Campaign target audiences to help those in the U.S. avoid the worst outcomes of COVID-19.
Creative Testing Surveys and Experiments
The purpose of the Creative Testing Surveys and Experiments is to assess participant reactions to various Campaign materials to inform the selection and development of creative concepts, messages, or material format used for campaign outreach to key audiences. Surveys will be fielded with members of the Campaign’s target audience to assess various features of the Campaign content, such as the perceived effectiveness of messages, the influence of material format (e.g., video vs. static image ad), likelihood of engaging in desired behaviors, and changes in COVID-related knowledge after viewing Campaign materials. Some surveys may use an experimental design (i.e., grouping respondents into control and experimental conditions) while some will expose all respondents to all stimuli, depending on the objectives of the study. Findings from these surveys will be used to inform the development of Campaign messages and strategy (e.g., which ad formats may work best for different types of Campaign information).
Use of Information Technology
Current Events Tracker
The CET will be administered online by a survey panel vendor (Ipsos KnowledgePanel) as part of an omnibus survey that includes questions from other organizations. The CET questions are expected to comprise the majority of the questions in the survey from week to week, but a small number of questions sponsored by other Ipsos’ clients may be included in the full survey. The survey vehicle is designed to be flexible to accommodate many different clients. Often the questions fielded as part of the omnibus are topical in nature and could cover topics such as views of events in the news; however, the additional clients and survey questions included in the omnibus survey for the duration is not known at this time.
Panel members will be invited to the survey through email invitations and reminders and will be asked to log-in to the panel vendor portal to complete the survey. The survey minimizes respondent burden by using clear and easy to read pages with minimal graphics to make it as easy as possible to respond to questions. The survey also incorporates skip logic so respondents automatically skip past any questions that are not applicable to them.
Members of the Ipsos KnowledgePanel update their personal information with the panel vendor on a regular basis, meaning that the vendor collects several additional demographic variables which are available to ASPA. For this effort, ASPA will obtain relevant demographic variables (e.g., age, education, race/ethnicity, gender, household size, household type, household income, census regions, marital status) from the panel vendor to help minimize the number of questions asked in the survey, thus reducing burden.
Foundational Focus Groups, Interviews, and/or Dyads
HHS COVID-19 Public Education Campaign foundational focus groups, interviews, and/or dyads will be conducted remotely using teleconference software (with 4-6 participants per focus group, 1 participant per interview, and 2 participants per dyad). Vendors will recruit participants from their panel databases via invitation emails and a screener questionnaire to participate in focus groups, interviews, or dyads. Vendors will generally screen participants via phone, but an online version of the screener may be used in the event that vendors are having challenges meeting their quotas by using telephone screening only. If an online screener is used, the questions will be programmed to appear exactly the same way as they are asked in the telephone screener.
Copy Testing Surveys, Message Matrix Surveys, and Creative Testing Surveys and Experiments
These surveys will be administered online by a survey panel vendor (Prodege). Panel members have already agreed to be contacted for surveys and have provided Prodege with their demographic information to use for the purposes of sampling for specific surveys. Prodege will initially screen panel members on basic demographics, such as age and ethnicity, based on information within their existing dataset and invite potential participants to the survey through email invitations and reminders. Potential participants will be asked to log-in to the panel vendor web portal to enter the survey, at which point they will answer additional screening questions. These requirements include not working for the U.S. Federal government or media marketing fields, not having participated in a COVID-19 vaccine trial, and not having completed the target behavior (e.g., having received an updated vaccine for COVID-19). If participants pass the screener questions at the beginning of the survey, they will view the informed consent form and then complete the remainder of the survey. Participants who do not pass the screener questions embedded at the beginning of the survey will be screened out and thanked for their time. The survey minimizes respondent burden by using clear and easy to read pages with minimal graphics to make it as easy as possible to respond to questions. The survey also incorporates skip logic so respondents automatically skip past any questions that are not applicable to them.
If multiple advertisements, messages, or concepts are being tested in one survey, respondents may be randomly assigned to one test condition in which they are shown a subset of the stimuli that are being tested in the study, thus minimizing the number of creative assets a respondent views and reacts to. In the Messaging Matrix surveys, participants may be recruited across several target audiences (e.g., parents and updated vaccines). Respondents will only be shown a set of messages that apply to the audience to which they belong.
Duplication of Efforts
The team has not identified any duplicative information collection instruments or processes.
Within the ASPA COVID-19 Public Education Campaign research efforts, the CET assesses broad perceptions and attitudes across the general public to form foundational knowledge to inform the campaign and ASPA. Other third-party survey work focuses on specific topics or outcomes and is not geared to provide the information that is needed to develop an understanding of the audience’s needs and preferences for information that will be disseminated as part of this campaign. The CET was also designed to be able to change rapidly to meet emergent needs of the campaign; for example, to assess the potential influence of external factors that may impact the campaign and cannot be anticipated.
The CET also differs from other surveys being conducted in support of the ASPA COVID-19 Public Education Campaign. Specifically, the CET differs from the Monthly Outcome Survey (MOS), which tracks key campaign variables over time and is not designed to track current events nor foundational aspects of behavior and attitudes towards the pandemic and the COVID-19 vaccine.
While there are other ongoing recurring third-party surveys polling Americans’ behavior and attitudes about the COVID-19 pandemic and vaccine willingness, most of these surveys occur less frequently (typically once or twice a month) or are single administration surveys. Additionally, none of these survey projects have a recurring focus on knowledge of and/or attitudes about the U.S. Department of Health and Human Services – in general or as they pertain to the COVID-19 pandemic and this public education campaign. No other studies collect data that would directly inform the messages and strategy used in the Campaign.
Note: ASPA submitted a prior emergency package (0990-0475). The research efforts covered in that package (longitudinal and cross-sectional campaign evaluations) and those covered herein work in concert to support the HHS COVID-19 Public Education Campaign. There are no duplications between these two submissions.
Small Businesses
All efforts described in this submission (CET, Foundational Focus Groups/Interviews/Dyads, Copy Testing Survey, Message Matrix Surveys, and Creative Testing Surveys and Experiments) will be completed by individuals and not by any organizations, and therefore are not anticipated to impact small businesses.
Less Frequent Collection
Current Events Tracker
This request is voluntary. Sampled members can choose to respond or not respond to the surveys with no penalty for not responding. Data collected through the CET data collection efforts will help ASPA and the Campaign understand changing public perceptions and attitudes towards COVID-19 as well as identify any emergent issues among target groups within the general population. The survey will be conducted on a weekly basis so that the team can track metrics over time such as vaccine confidence, perceptions about organizations providing information about COVID-19 and vaccines, and the use of preventative measures to slow the spread of the virus.
Foundational Focus Groups, Interviews, and/or Dyads
This request is voluntary. Individuals who are invited to and screened for focus groups, interviews, or dyads have the option to not accept the invitation to participate with no penalties attributed to them. Data collected will provide ASPA in-depth information on vaccine confidence, preventative measures taken to slow the spread of infection, and information sources used by people to educate themselves on issues surrounding COVID-19. This information will be used to inform the Campaign in developing and targeting its messaging. These qualitative studies will be conducted as needed as warranted by the current pandemic environment. If the information were collected less frequently, the Campaign may miss out on insights regarding how attitudes change over time, or how the public reacts to key events related to COVID-19 and the COVID-19 vaccine. Given how the COVID-19 pandemic and response have evolved over time, it is important for the Campaign to maintain an ongoing awareness of the public’s in-depth perceptions and motivations as they relate to Campaign-targeted behavior.
Copy Testing Surveys
This request is voluntary. Sampled members can choose to respond or not respond to the surveys with no penalty for not responding. Up to 36 (12 per year) copy testing surveys will be fielded over the course of the Campaign; these will be timed to coincide with the production schedule for each Wave of Campaign messaging. In order to be effective, copy testing must be conducted for each Wave of materials, as each Wave will have a unique message and creative approach. If copy testing were not conducted for any Wave of the Campaign, then we would not be able to proactively determine whether the ads are likely to affect the target behaviors in the desired manner. Success of the Campaign hinges upon being able to test creative assets before placing them in market.
Message Matrix Surveys
This request is voluntary. Sampled members can choose to respond or not respond to the surveys with no penalty for not responding. The Message Matrix Surveys will be fielded every other month, for a total of 6 surveys per year (up to 9 surveys under this package). These surveys must be conducted on a regular basis to keep up with changing guidance and environmental factors surrounding target campaign behaviors, as the situation with respect to COVID-19 is rapidly evolving. Messages and approaches that worked well several months ago may become quickly outdated if vaccination guidance has changed or if the composition of target audiences has shifted (e.g., because a large portion of the target audience engaged in the target behavior). For campaign messaging to be effective, we must have a current understanding of what messages work best with key audiences.
Creative Testing Surveys and Experiments
This request is voluntary. Sampled members can choose to respond or not respond to the surveys with no penalty for not responding. These creative testing surveys and experiments will be fielded on an as-needed basis over the course of the Campaign and will be timed to align with the development and production of campaign messages, concepts, or other assets. Up to 4 creative testing surveys and experiments may be fielded per year (up to 6 under this package). If these surveys were conducted less frequently, then we would not be able to effectively determine which approaches work best for key target audiences. Ultimately, this may lead the campaign to develop campaign materials that are not as effective in reaching the target audiences as they could be. Success of the Campaign hinges upon being able to test these messages and concepts before making final decisions about material development.
Special Circumstances
There are no special circumstances associated with this collection.
Federal Register/Outside Consultation
Before submitting for approval, materials for both the Current Events Tracker, the Foundational Focus Groups/Interviews/Dyads, and Messaging Matrix Surveys (i.e., instrument, discussion guide, screener, and consent form) were reviewed by an expert panel of reviewers including staff from HHS and the Centers for Disease Control and Prevention. The CET was submitted to an external IRB review process and received an “exempt” status. The Foundational Focus Group materials and the Copy Testing Survey were also submitted to an external IRB review process and received approval. The Messaging Matrix Survey was reviewed by an independent panel of researchers and found to be exempt from IRB review. Materials for the Creative Testing Surveys and Experiments will be updated for each round of testing, given the unique objectives of these studies. These materials will be similarly reviewed by an independent panel of researchers prior to each round of fielding to determine whether the study is exempt from IRB oversight; if any additional IRB review is determined to be necessary, we will submit them for an external IRB approval before finalizing study materials.
All supplemental materials were uploaded with the initial submission and remain current.
Payments/Gifts to Respondents
Current Events Tracker
Ipsos KnowledgePanel has their own rewards program that provides panel members with points for completing surveys, which can be redeemed for gift cards or merchandise. These small tokens of appreciation will help maximize response rates across a more diverse respondent pool.
For the CET, respondents will receive the equivalent of $1 in points for completing the survey. Incentives are important for the CET because the survey fielding window is relatively short and the number of required completes is relatively high. Providing incentives will allow the survey vendor to recruit a sufficient number of respondents during the fielding window.
Foundational Focus Groups, Interviews, and/or Dyads
OMB guidance currently sets the incentive for in-person focus groups and interviews at $75 for adults. Due to COVID-19, this data collection must be virtual, which would typically warrant a lower incentive payment. However, HHS requests that participants receive a payment of $75 for participation in one 90-minute virtual focus group or one 60-minute interview.
Providing incentives for recruitment of participants in focus groups, interviews, or dyads is a critical component of planning a successful recruitment effort. Given the critical need to get consistent messaging out to the public as soon as possible, ensuring incentives are high enough will support successful recruitment of all participants, which is imperative especially for those groups disproportionately affected by COVID-19.
Although initially it may appear as a cost savings to have a lower incentive amount, in the long term it takes more resources to recruit respondents to participate with a lower incentive. For instance, with a lower incentive it may require 10 calls before securing a participant while a higher incentive may require only half of those calls. A higher incentive also reduces the number of no-shows, which also means we will have fewer cancellations.
Copy Testing Surveys, Message Matrix Surveys, and Creative Testing Surveys/Experiments
Prodege’s incentive program is Swagbucks, an online reward community in which members can earn and accumulate virtual currency that can be redeemed for gift cards to popular retailers (online and offline) or used for donations toward charities. Respondents will receive the equivalent of $5.00 in Swagbucks for completing these surveys. Providing incentives allows the survey vendor to recruit a sufficient number of respondents within the expedited fielding window.
Confidentiality
Current Events Tracker
ASPA will comply with all Privacy Act, Freedom of Information laws and regulations that apply to this collection. ASPA will not have access to any identifying information for respondents to the CET. Ipsos provides their members with confidentiality assurances when they agree to become members of the survey panels and does not release personally identifying information to data users.
The Ipsos KnowledgePanel member “Bill of Rights” states that responses will be anonymized, and each participant will be assigned a unique ID. Express permission from participants is required for Ipsos to share identifying information with a survey client. KnowledgePanel members are informed that participation in surveys is voluntary and that they do not have to answer any questions they do not wish to answer.
Foundational Focus Groups, Interviews, and/or Dyads
ASPA will comply with all Privacy Act, Freedom of Information laws and regulations that apply to this collection. ASPA will not have access to any identifying information for participants in the foundational focus groups, interviews, and dyads. Participants’ identity will not be linked to their individual responses and all findings will be reported in the aggregate. The data collected, including information collected during screening, video/audio files from the sessions, and transcripts, will be stored on a password-protected computer and/or in locked cabinets that only the project team can access.
Copy Testing Surveys, Message Matrix Surveys, and Creative Testing Surveys and Experiments
ASPA will comply with all Privacy Act, Freedom of Information laws and regulations that apply to this collection. ASPA will not have access to any identifying information for respondents to the creative testing surveys. Prodege provides their members with confidentiality assurances when they agree to become members of the survey panels and does not release personally identifying information to data users. In addition, all data obtained from respondents is stored on secure servers only accessible to the immediate research team. All study related data files use encryption and password protection so that only authorized personnel can access the files.
Sensitive Questions
Current Events Tracker
The survey questions are not sensitive in nature. The general perceptual or attitudinal questions on this survey instrument are not expected to pose any psychological risks to respondents. However, survey questions about COVID-19 may make participants feel uncomfortable or bring up unwanted thoughts or feelings. To minimize these risks, we will emphasize the voluntary and confidential nature of the research and restate the panel vendor’s commitment to confidentiality within the survey.
Foundational Focus Groups, Interviews, and/or Dyads
The foundational focus groups, interviews, and dyads pose no more than minimal risk to subjects. None of the questions/items/data elements will include private or sensitive information. However, questions about COVID-19 may make participants feel uncomfortable or bring up unwanted thoughts or feelings. To minimize these risks, we will emphasize the voluntary and confidential nature of the research in the informed consent form and again at the beginning of each session. Additionally, in the event participants experience any distress from this study, we will provide contact information for the 1) SAMHSA Disaster Distress Helpline, which provides 24/7 crisis counseling and support for individuals experiencing emotional distress caused from natural or human-caused disasters, as well as the 2) Suicide Prevention Lifeline (a 24/7 free and confidential support resource) in the informed consent form.
Copy Testing Surveys, Message Matrix Surveys, and Creative Testing Surveys and Experiments
The survey questions are not sensitive in nature and are not expected to pose any psychological risks to respondents. However, survey questions about COVID-19 may make participants feel uncomfortable or bring up unwanted thoughts or feelings. To minimize these risks, we will emphasize the voluntary and confidential nature of the research and restate the panel vendor’s commitment to confidentiality within the survey. In addition, the informed consent form includes mental health resources in case of unexpected discomfort experienced during the survey.
Burden Estimates (Hours & Wages)
Current Events Tracker
For the CET we estimate that 1,000 complete respondents x 0.12 hours per complete survey submission = approximately 120 burden hours associated with completing this survey each wave. No separate screening of participants will be required because Ipsos stores panel variables that determine the eligibility of each panel member without the need for a screener instrument. Only eligible panel members will be invited to take the survey. Over 138 total waves, the total burden is estimated to be approximately 16,560 total burden hours.
Category of Respondent |
No. of Respondents per Wave |
Participation Time (Hours) |
Burden Hours per Wave |
Survey Completes: Adults 18+ |
1,000 |
0.12 |
120 |
Total, all Waves (138) |
138,000 |
0.12 |
16,560 |
Foundational Focus Groups, Interviews, and/or Dyads
For the foundational focus groups, we estimate screening a maximum of 2,500 potential respondents x .09 hours (5 minutes) = 225 hours associated with screening participants during each round. In addition, each round will include a maximum of 108 respondents x 1.5 hours per focus group = 162 burden hours associated with the discussion for each round of focus groups. (Note that the exact burden hours will vary depending on the type of study conducted; these estimates serve as a maximum number of participants/hours because in-depth interviews or dyads would involve fewer participants). Over the course of the Campaign, this will amount to a maximum of 20 rounds of qualitative research, for a total of 7,740 burden hours.
Category of Respondent |
No. of Respondents per Round |
Participation Time (Hours) |
Burden Hours per Round |
FG Screening: Individuals in the reference audience |
1,250 |
0.09 |
112.5 |
FG Screening: Individuals in priority populations |
1,250 |
0.09 |
112.5 |
FG Participants: Individuals in the reference audience |
54 |
1.5 |
81 |
FG Participants: Individuals in priority populations |
54 |
1.5 |
81 |
Total, per round |
2,500* |
.155** |
387.5 |
Total, all rounds (20) |
50,000* |
.155** |
7,750 |
*Focus group participants are also included in the focus group screening, so are only counted once toward the total number of respondents.
**.1548 is approximately 9.3 minutes; it is the weighted average over the screener and interview for all participants.
Copy Testing Surveys
For the copy testing survey, we estimate screening 15,000 potential respondents x .03 hours (2 minutes) = 450 hours associated with screening survey participants during each wave. Note that this is a maximum estimate that may be necessary to find members of particularly small audiences of interest. In addition, we will obtain 1,500 respondents x .33 hours (20 minutes) per submission = 495 hours associated with completed surveys in each wave of Campaign message testing. Over the course of the Campaign, this will amount to a maximum of 36 Waves, for a total of 34,020 burden hours.
Category of Respondent |
No. of Respondents per Wave |
Participation Time (Hours) |
Burden Hours per Wave |
Survey Screener |
15,000 |
0.03 |
450 |
Survey Completes |
1,500 |
0.33 |
495 |
Total, one Wave |
15,000* |
0.063 |
945 |
Total, all Waves (36) |
540,000* |
0.063 |
34,020 |
*Survey completes are also included in the survey screener, so are only counted once toward the total number of respondents.
**.063 is approximately 3.8 minutes; it is the weighted average over the screener and survey for all participants.
Message Matrices
Each Message Matrix survey will recruit up to 4,000 respondents. We estimate screening 42,000 potential respondents x 0.03 hours (2 minutes) = 1,400 hours associated with screening survey participants. Note that this is a maximum estimate that may be necessary to find members of particularly small audiences of interest. In addition, we will obtain survey responses from up to 4,000 respondents: 4,000 x 0.33 hours (20 minutes) = 1,333 hours associated with survey completion. Over the course of the Campaign, this will amount to a maximum of 9 rounds of data collection, for a total of 24,600 burden hours.
Category of Respondent |
No. of Respondents per Round |
Participation Time (Hours) |
Burden Hours per Round |
Survey Screener |
42,000 |
0.03 |
1,400 |
Survey Completes: Adults 18+ |
4,000 |
0.33 |
1,333 |
Total, per round |
42,000* |
0.065 |
2,733 |
Total, all rounds (9) |
378,000 |
0.065 |
24,600 |
*Survey completes are also included in the survey screener, so are only counted once toward the total number of respondents.
**.065 is approximately 3.9 minutes; it is the weighted average over the screener and survey for all participants.
Creative Testing Surveys and Experiments
Each Creative Testing Survey or Experiment will recruit up to 3,000 respondents. We estimate screening 42,000 potential respondents x 0.03 hours (2 minutes) = 1,400 hours associated with screening survey participants. Note that this is a maximum estimate that may be necessary to find members of particularly small audiences of interest. In addition, we will obtain survey responses from up to 3,000 respondents: 3,000 x 0.33 hours (20 minutes) = 1,000 hours associated with survey completion. Over the course of the Campaign, this will amount to a maximum of 6 rounds of data collection, for a total of 14,400 burden hours.
Category of Respondent |
No. of Respondents per Round |
Participation Time (Hours) |
Burden Hours per Round |
Survey Screener |
42,000 |
0.03 |
1,400 |
Survey Completes: Adults 18+ |
3,000 |
0.33 |
1,000 |
Total, per round |
42,000* |
0.057 |
2,400 |
Total, all rounds (6) |
252,000 |
0.057 |
14,400 |
*Survey completes are also included in the survey screener, so are only counted once toward the total number of respondents.
**.057 is approximately 3.4 minutes; it is the weighted average over the screener and survey for all participants.
Sum of All Studies
TOTAL RESPONDENTS: 1,358,000
TOTAL BURDEN HOURS: 97,330
Capital Costs
There are no direct costs to respondents other than their time to participate in the study.
Cost to Federal Government
Current Events Tracker
The survey will field weekly for three years, for a maximum of 138 waves (note: CET is not collected on holiday weekends), using a panel vendor and will collect an estimated 138,000 responses from participants in total. The total labor price is $804,043.20 with $1,713,449.40 additional for ODCs. Broken down, the price for one wave of the CET is $5,826.40 in labor costs and $12,416.30 in ODCs associated with survey fielding. This price reflects the labor and direct costs to field 138 waves of a 7-minute online survey to 1,000 total participants from the Ipsos KnowledgePanel. Included in these costs are: 1) Development of the annotated questionnaire; 2) Panel vendor services to program and test the survey, field the survey, staff the survey helpline, deliver data tables, and clean and deliver data; 3) Profile variables from the panel vendor; 4) IRB review; 5) Analysis and reporting (key findings included in the Weekly Situation briefing) and 6) Quality control.
Foundational Focus Groups, Interviews, and/or Dyads
In total, ASPA plans to conduct a maximum of twenty rounds of qualitative testing. Broken down, the price to conduct for one round of foundational focus groups is $135,111.84 in labor costs and $56,939.22 in other direct costs, including participant incentives. Interviews and dyads cost significantly less than focus groups; this maximum total cost is estimated with 20 rounds of focus groups, which is $2,702,224.80 and $1,138,784.40 in ODCs.
Included in these costs are: 1) Development of the focus group screener; 2) Development of the focus group discussion guide; 3) Translation of materials into Spanish; 4) IRB review; 5) Participant recruitment; 6) Participant incentives; 7) Execution of the focus groups and moderation; 8) Live interpretation of the two Spanish-speaking groups for those in the virtual back room; 9) Transcription; 10) Analysis and reporting (memo of key findings and recommendations); 11) Quality control.
Copy Testing Surveys
The copy testing survey will field for a maximum of 36 waves. The total labor price for the is $3,293,448.12 with $650,520 additional for ODCs. Broken down, the price for each wave of the copy testing survey is $91,484.67 in labor costs and $18,070 in ODCs associated with survey fielding. This price reflects the labor and direct costs to field 36 waves of a 20-minute online survey. Included in these costs are: 1) Development of the annotated questionnaire; 2) Panel vendor services to program and test the survey, field the survey, staff the survey helpline, deliver data tables, and clean and deliver data; 3) Participant incentives; 4) IRB review; 5) Analysis and reporting (memo of key findings and recommendations) and 6) Quality control.
Message Matrix Surveys
In total, ASPA plans to field a maximum of 9 waves of the Messaging Matrix Survey under this package. The total labor price for these efforts is $823,362.03 with $630,000.00 additional ODCs. Broken down, the price for each wave of the Messaging Matrix Survey is $91,484.67 in labor costs and $70,000 in direct costs associated with survey fielding. This price reflects the labor and direct costs to field 9 waves of a 20-minute online survey. Included in these costs are: 1) Development of the annotated questionnaire; 2) Panel vendor services to program and test the survey, field the survey, staff the survey helpline, deliver data tables, and clean and deliver data; 3) Participant incentives; 4) Analysis and reporting (memo of key findings and recommendations) and 5) Quality control.
Creative Testing Surveys and Experiments
In total, ASPA plans to field a maximum of 6 waves of Creative Testing Surveys and Experiments under this package. The total labor price for these efforts is $548,908.02 with $150,000.00 additional ODCs. Broken down, the price for each survey wave is $91,484.67 in labor costs and $25,000 in direct costs associated with survey fielding. This price reflects the labor and direct costs to field 6 waves of a 20-minute online survey. Included in these costs are: 1) Development of the annotated questionnaire; 2) Panel vendor services to program and test the survey, field the survey, staff the survey helpline, deliver data tables, and clean and deliver data; 3) Participant incentives; 4) Analysis and reporting (memo of key findings and recommendations) and 5) Quality control.
Note: The number of iterations noted for each effort is a maximum and is highly dependent upon the requirements of an ongoing COVID-19 Public Education Campaign. The figure also uses the most expensive qualitative data collection to estimate the total. The campaign may also be terminated prior to the expiration of this package. Therefore, the maximum total cost to the government is $12,454,740, but the actual costs are likely to be much less and as low as $6,000,000.
MAXIMUM TOTAL COST TO THE GOVERNMENT: $12,454,740
Changes to Burden
First revision: The information collection request itself (i.e., CET, qualitative research, copy testing surveys) and accompanying documentation was unchanged from the initial submission except for the addition of two modes to qualitative data collection (). However, the burden calculation initially submitted did not account for the lifespan of the approved package, which is three years (CET accounted for two years, and qualitative and copy testing for one) as at that time, it was impossible to predict the path COVID-19 would take and the need for an ongoing public education campaign. The first emergency revision increased iterations and burden to account for data collection through the life of the package, which is set to expire February 2024.
Second revision: This second emergency revision added two types of market research collection (i.e., message matrix surveys and creating testing surveys/experiments) that were previously covered under waivers relating to the public health emergency the U.S. declared to address COVID-19. In anticipation of the potential end of the public health emergency, these two market research efforts are being moved into the current package. This results in an increase to the requested burden on the package of 39,000 burden hours over the remaining lifetime of the package. All efforts will be discontinued in the event the campaign is no longer necessary.
Publication/Tabulation Dates
Current Events Tracker
The aim of this survey is to understand the public’s information sources, attitudes, and behaviors as they relate to COVID-19 and COVID-19 preventative behaviors to include vaccination. The data will be used internally to inform the development of Campaign messages and to guide strategy. After each survey wave has been completed, results will be presented in weekly data tables and briefing slides that will be used by ASPA and the Campaign team. We do not expect to publish external briefings or reports from this study. We will communicate this to respondents and inform them, should our intentions not to publish external facing materials change. All results will be aggregated and anonymized and explicitly discuss the limitations of this design, including the lack of generalizability beyond the study participants.
Foundational Focus Groups, Interviews, and/or Dyads
The primary purpose of these sessions will be to inform HHS COVID-19 Public Education Campaign strategy about audience risk knowledge, perceptions, current behaviors, and barriers and motivators to healthy behaviors. The findings will be used internally to inform the development of Campaign messages and to guide strategy. Key findings and recommendations will be presented in a memo that will be used by ASPA and the Campaign team to inform decision-making and strategy. We will communicate this to respondents and inform them, should our intentions not to publish external facing materials change. All results will be aggregated and anonymized and explicitly discuss the limitations of this design, including the lack of generalizability beyond the study participants.
Copy Testing Surveys
The primary purpose of the copy testing surveys is to inform final decisions about Campaign advertisements. These findings will be used internally. A briefing with key findings and recommendations will be provided to ASPA. We do not expect to publish external-facing briefings or reports with the findings from these surveys. We will communicate this to respondents and inform them, should our intentions not to publish external facing materials change. All results will be aggregated and anonymized and explicitly discuss the limitations of this design, including the lack of generalizability beyond the study participants.
Message Matrices
The primary purpose of the Message Matrix surveys is to inform decisions about Campaign messages. These findings will be used internally. A briefing with key findings and recommendations will be provided to ASPA. We do not expect to publish external-facing briefings or reports with the findings from these surveys. We will communicate this to respondents and inform them, should our intentions not to publish external facing materials change. All results will be aggregated and anonymized and explicitly discuss the limitations of this design, including the lack of generalizability beyond the study participants.
Creative Testing Surveys and Experiments
The primary purpose of the Creative Testing Surveys and Experiments is to inform decisions about Campaign advertisements. These findings will be used internally. A briefing with key findings and recommendations will be provided to ASPA. We do not expect to publish external-facing briefings or reports with the findings from these surveys. We will communicate this to respondents and inform them, should our intentions not to publish external facing materials change. All results will be aggregated and anonymized and explicitly discuss the limitations of this design, including the lack of generalizability beyond the study participants.
Expiration Date
Current Events Tracker
The OMB approval number and expiration date will be shown in the top right corner of all pages of the survey that include CET questions.
Foundational Focus Groups, Interviews, and/or Dyads
The informed consent form for the foundational focus groups, interviews, and dyads will display the OMB approval number and expiration date.
Copy Testing Surveys
The OMB approval number and expiration date will be shown on all pages of the online programmed survey.
Message Matrices
The OMB approval number and expiration date will be shown on all pages of the online programmed survey.
Creative Testing Experiments
The OMB approval number and expiration date will be shown on all pages of the online programmed survey.
Certification Statement
There are no exceptions to item 19 of OMB Form 83-1.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | CMS |
| File Modified | 0000-00-00 |
| File Created | 2023-07-29 |