Ssb

GenIC Clearance for CDC/ATSDR

Formative Research and Tool Development

Harmful Algal Bloom Shellfish Communication Evaluation

OMB Control No. 0920-1154

1/23/2025

Supporting Statement B

Contact: Amy Jacobi, MPH

Waterborne Disease Prevention Branch

Division of Foodborne, Waterborne, and Environmental Diseases

Centers for Disease Control and Prevention

1600 Clifton Road, NE

Atlanta, Georgia 30333

Phone: (404) 718.3715
Email: puw6@cdc.gov

Table of Contents

1. Respondent Universe and Sampling Methods 3

2. Procedures for the Collection of Information 4

3. Methods to Maximize Response Rates and Deal with No Response 5

4. Tests of Procedures or Methods to be Undertaken 5

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 6

LIST OF ATTACHMENTS

1. Eligibility Screener

2. Recruitment Materials

3. Eligible Participant Screener

4. Privacy Agreement

5. Respondent Consent Form for Focus Groups

6. Standard Invitation for FGs

7. Participant Confirmation Email

8. Focus Group Moderator Guide

9. Messages to be Tested

10. Human Subjects Determination

Supporting Statement B

The collection of data for this project does not involve statistical methods, and the purpose of the collection is not to make statistical generalizations beyond the respondents included in the study. The objectives of the project are to:

• Identify appropriate and effective messages for people who collect shellfish to eat for recreational, subsistence, or cultural reasons to increase awareness on preventing harmful algal blooms (HABs)-associated illness and following safe shellfish collection practices.

• Gather data on the preferred tone, format, and placement of those messages on CDC’s communication channels.

1. Respondent Universe and Sampling Methods

The project team will enlist a national recruitment agency to recruit and manage participant screening. They will recruit from a national proprietary database of individuals. The primary audience for the focus groups is U.S. adults aged 18 and older who collect shellfish to eat for recreational, subsistence, or cultural reasons, and live on the east, west, Alaskan, or Gulf coasts. We have set targets for the research focus group sample by key audiences, based on data about risk for illness associated with harmful algal blooms (HABs) (Hilborn & Beasley, 2015; Lavery, Kieszak, Law et al., 2019). Key audiences will consist of the general public in Alaska who collect shellfish to eat; general public in Washington, Oregon, or California who collect shellfish to eat; general public in the northeastern U.S. and along the east coast who collect shellfish to eat; general public near the Gulf coast who collect shellfish to eat. The project team will aim to recruit a sample of diverse participants that aligns with U.S Census data. The project team will aim to recruit a sample of diverse participants by various demographic and harmful algal bloom risk characteristics segmented as shown in Tables 1 and 2, respectively. We will conduct a series of 4 virtual focus groups lasting 60 minutes each. Each focus group will consist of up to eight participants (n=32).

Table 1. Focus group demographic makeup by key audience, type, and number.

Population	Number of Focus Group Discussions (FGDs)	Number of Participants
General public (adults 18+) in Alaska who collect shellfish to eat	1	8
General public (adults 18+) in Washington, Oregon, and California who collect shellfish to eat	1	8
General public (adults 18+) in the northeastern U.S. and along the east coast who collect shellfish to eat	1	8
General public (adults 18+) near the Gulf Coast who collect shellfish to eat	1	8
Total	4	32

Table 2. Demographic characteristics to be captured.

• Geographic location (coastal region)

• Race and ethnicity

• Income

• Education level

• Occupational status

2. Procedures for the Collection of Information

Recruitment

For the focus group discussions, to identify and recruit participants, we will employ a two-part screening process to assess eligibility for participation. The first screener (Attachment 1) will ensure they are eligible to participate in the focus groups. The second screener (Attachment 3) will allow the recruiters to group the participants into specific focus groups (e.g., individuals who live near the Gulf coast).

Exclusion criteria for participation includes people:

• under 18 years of age;

• who are not comfortable speaking or reading in English;

• who do not collect shellfish to eat for recreational, subsistence, or cultural reasons;

• who do not live along the west coast, east coast, Gulf coast, or Alaskan coast;

• who have participated in a focus group in the last 6 months;

• who work in the following industries: market research, graphic design or website design, advertising or public relations, media (TV/radio/newspapers/magazines), healthcare (e.g., doctor, nurse, pharmacist, dietician), federal government;

• whose occupation is related to HABs; and

• who do not have access to the internet with a computer or mobile device.

These comprehensive screeners were developed by CDC and the contractor. During the recruitment phase, the recruitment firm will provide a respondent report, confirmed attendees, and respondents for review. CDC and contractor staff will review these documents to ensure the recruitment mix is being reached. The recruitment firm will complete confirmation calls and provide a recruitment report after each focus group session is completed. Once participants are chosen, they will receive a standard invitation (Attachment 6). Once they accept the invitation, they will receive the participant confirmation email (Attachment 7).

The contractor and recruitment agency will provide ongoing screening and recruitment updates to CDC and work with the project team to select a diverse sample for the groups during the recruitment phase. Samples of the recruitment materials that may be used by the recruitment agency can be found in Attachment 2 & Attachment 6. These, and similar items, will be used for recruitment.

Implementation

The project team will conduct four virtual focus groups lasting about 60 minutes each. The team has developed a focus group approach designed to gather information about and assess participants’ knowledge, attitudes, beliefs, and behaviors related to HABs and related messages. Specifically, the focus groups will provide responses to targeted questions about interests, behaviors, and opinions related to collecting shellfish, as well as the effectiveness, preferred tone, format, and placement of new and existing messages related to HABs. Experienced moderators will facilitate the focus groups following an approved semi-structured facilitator discussion guide (Attachment 8). The guide will contain multiple items and probes, which start more generally and get more specific, for individuals to respond to throughout the session. Moderators will test select messages and communication materials with the focus groups (Attachment 9). The contractor will conduct the virtual focus groups using a web conferencing platform.

CDC project staff who have signed privacy forms will have access to observe live focus groups. The focus groups will be audio and video recorded and the de-identified transcripts will be provided to CDC upon completion of all focus groups.

To protect the privacy of participants, personally identifiable information (e.g., names) will be redacted from all transcripts and not included in the field notes, final report, or any presentation about the project. Further, to ensure security, the contractor will provide CDC with password-protected files of the transcripts, notes, and analysis files.

Analysis and Reporting

The contractor will use rapid analysis to identify key themes and subthemes captured in the data collected during focus groups. Multiple studies have compared notes-based rapid analysis to in-depth analysis of transcripts and concluded that they are equally effective for structured, primarily deductive analysis (Gale et al., 2019; Nevedal et al., 2021). This research approach has been used to study public health topics including: opioid safety (Midboe et al., 2018), infectious disease transmission (Higham et al., 2022), and COVID-19 response efforts (Keniston et al., 2022).

The contractor will develop a codebook consisting of deductive codes based on constructs (e.g., clarity, resonance) included in the research questions and focus group moderator guide (Attachment 8). These codes will serve as a basis for a summary notes template. The summary notes template will also include space for additional observations in case new themes emerge. After each focus group, each notetaker will synthesize their note into the summary notes template and review their summary notes with the moderator to arrive at a mutually agreed upon summary of the focus group. The final summary notes will be placed in an analysis matrix that will include findings across all focus groups. The contractor will then review the data matrix to identify key themes. The contractor will provide CDC with a final report that summarizes the results of the focus groups and includes illustrative, de-identified quotes.

3. Methods to Maximize Response Rates and Deal with No Response

Tokens of appreciation will be used in the focus groups to increase the likelihood of participation and offer a token of appreciation to participants for their time and input to the study. Based on industry standards and national vendor’s expertise, and a previously cleared project, the team recommends a participant token of appreciation of $75.00 per focus group participant. To optimize and increase the chance of having at least a minimum of eight participants in each focus group, the contractor will over-recruit by 20 percent. This will account for any last-minute cancellations or no-shows and aim to get eight people per focus group.

A similar communication evaluation project that was conducted in the summer of 2023 proposed and was approved for $75 per person for a 60-minute focus group discussion (OMB Co. Number: 0920-1154 Food Safety Communication Evaluation Assessing Food Safety Messages, Knowledge, and Attitudes ). During this project, the team recruited 115 individuals (the goal was to recruit 144). Another communication evaluation project about HABs was conducted in the spring of 2024 and was approved for $75 per person for a 60-minute focus group discussion (OMB Co. Number: 0920-1154, Harmful Algal Blooms Communication Evaluation). For this project, the team successfully recruited 27 individuals (the goal was to recruit 32).

Reviewed literature revealed the payment of incentives can provide significant advantages to the government in terms of direct cost savings and improved data quality. (See references.)

4. Tests of Procedures or Methods to be Undertaken

One technical run-through will be conducted with contractor staff prior to the start of the study.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

No individuals outside of the project team were consulted for statistical aspects of the design. The data being collected are qualitative and descriptive and there will be no statistical aspects of analysis. Target numbers for the focus participants and groups were informed by the project scope of work, public health surveillance data and research, and DFWED priorities. The individuals collecting and/or analyzing data include:

Lead Investigator: Amy Jacobi, MPH, Project Lead, Centers for Disease Control and Prevention (CDC), National Center for Emerging and Zoonotic Infectious Diseases (NCEZID), Division of Foodborne, Waterborne, and Environmental Diseases (DFWED), Waterborne Disease Prevention Branch (WDPB)

Collaborators

Name	Organizational Unit
Tola Aina, Project Director	Banyan Communications (contractor)
Sharanya Thummalapally, Lead Research and Evaluation Specialist	Banyan Communications (contractor)
Bria Berry, Research and Evaluation Specialist	Banyan Communications (contractor)

References

Abreu, D.A., & Winters, F. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. Proceedings of the Survey Research Methods Section of the American Statistical Association.

Bonevski, B., Randell, M., Paul, C., Chapman, K., Twyman, L., Bryant, J., Brozek, I., & Hughes, C. (2014). Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC medical research methodology, 14, 42. https://doi.org/10.1186/1471-2288-14-42

Gale, R. C., Wu, J., Erhardt, T., Bounthavong, M., Reardon, C. M., Damschroder, L. J., & Midboe, A. M. (2019). Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implementation Science, 14(1), 11. https://doi.org/10.1186/s13012-019-0853-y

Higham, R., Pini, S., Quyn, A., Kowal, M., Helliwell, J., Saman, R., Lewthwaite, P., Young, N., & Rousseau, N. (2022). Rapid qualitative analysis in a mixed-methods evaluation of an infection prevention intervention in a UK hospital setting during the COVID-19 pandemic: A discussion of the CLEAN study methodology. Frontiers in Sociology, 7, 958250. https://doi.org/10.3389/fsoc.2022.958250

Hilborn, E. D., & Beasley, V. R. (2015). One health and cyanobacteria in freshwater systems: animal illnesses and deaths are sentinel events for human health risks. Toxins, 7(4), 1374–1395. https://doi.org/10.3390/toxins7041374

Keniston, A., Patel, V., McBeth, L., Bowden, K., Gallant, A., & Burden, M. (2022). The impact of surge adaptations on hospitalist care teams during the COVID-19 pandemic utilizing a rapid qualitative analysis approach. Archives of Public Health = Archives Belges De Sante Publique, 80(1), 57. https://doi.org/10.1186/s13690-022-00804-7

Krueger, R. and Casey, M. (2009) Focus Groups: A Practical Guide for Applied Research. Sage

Publications: Thousand Oaks, CA.

Lavery, A. M., Kieszak, S. M., Law, R., Bronstein, A. C., Funk, A. R., Banerji, S., Brown, K., Sollee, D. R., &

Backer, L. C. (2023). Harmful Algal Bloom Exposures Self-reported to Poison Centers in the United States, May-October 2019. Public health reports (Washington, D.C. : 1974), 138(6), 865–869. https://doi.org/10.1177/00333549221146654

Midboe, A. M., Wu, J., Erhardt, T., Carmichael, J. M., Bounthavong, M., Christopher, M. L. D., & Gale, R. C. (2018). Academic Detailing to Improve Opioid Safety: Implementation Lessons from a Qualitative Evaluation. Pain Medicine (Malden, Mass.), 19(suppl_1), S46–S53. https://doi.org/10.1093/pm/pny085

Nevedal, A. L., Reardon, C. M., Opra Widerquist, M. A., Jackson, G. L., Cutrona, S. L., White, B. S., & Damschroder, L. J. (2021). Rapid versus traditional qualitative analysis using the Consolidated Framework for Implementation Research (CFIR). Implementation Science: IS, 16(1), 67. https://doi.org/10.1186/s13012-021-01111-5

Robinson, K. A., Dennison, C. R., Wayman, D. M., Pronovost, P. J., & Needham, D. M. (2007). Systematic review identifies number of strategies important for retaining study participants. Journal of clinical epidemiology, 60(8), 757–765. https://doi.org/10.1016/j.jclinepi.2006.11.023

Shettle, C., & Mooney, G. (1999). Monetary incentives in U.S. government surveys. Journal of Official

Statistics, 15, 231–250.

Singer, E., N. Gelber, J. Van Hoewyk, and J. Brown (1997). Does $10 Equal $10? The Effect of Framing

on the Impact of Incentives. Paper presented at the American Association for Public Opinion;

Norfolk, VA.

Singer, E., Van Hoewyk, J., and Maher, M.P. (2000). Experiments with Incentives in Telephone Surveys.

Public Opinion Quarterly 64(3):171-188.

U.S. Bureau of Labor Statistics. Economy at a Glance. Retrieved from https://www.bls.gov/eag/eag.us.htm, on December 2, 2021.