SSA-CPR-ThisIsATEST

The Centers for Disease Control and Prevention (CDC) requests approval of a new GenIC titled, “Evaluation on Impact and Effectiveness of Pilot Project: This is a T.E.S.T.” under OMB Control No. 0920-1154 (Expiration Date: 03/31/26).

Recent emergency responses, such as the pandemic and natural disasters, highlighted a significant need for cross-sector collaboration and training for emergency response personnel. Frameworks, like FEMA’s exercise program, exist for federal, state, and local government agencies to exercise and plan together (FEMA 2020). However, training resources and exercises often fail to address all aspects of emergency preparedness and do not realistically address stakeholder engagement and restrictions on resources. As a result, these tools fail to comprehensively train emergency response personnel and accurately identify gaps in agency plans (MacKinnon 2012). Providing an engaging and realistic training environment that fosters cross-agency collaboration and appropriately stresses decision makers is key to building a better prepared emergency response workforce (Cozine 2015, EEF 2021).

CDC has designed a new training and exercise tool titled, “This is a T.E.S.T. – A Tabletop Exercise

Simulation Tool” (TEST). TEST is a collaborative game that bridges the gap between discussion and operational exercises for Community Reception Centers (CRCs) using narrative-based problems to foster teamwork, discuss resource management, and aid in understanding specific roles and responsibilities during a radiation emergency. TEST is a new innovative tool for nuclear/radiological emergency response training that has potential to be expanded to other types of training or emergency response.

The purpose of this project is to evaluate the pilot version of TEST to determine the effectiveness of this tool and how it impacts/improves emergency preparedness and public health capacity. The evaluation will also analyze how TEST improves learning outcomes from exercise participants through measuring retention levels. Findings from this evaluation may inform a new training and exercise framework for CDC and improved emergency response workforce capacity.

A.2. Purpose and Use of Information Collection

The purpose of this information collection is to evaluate CDC’s new training product, “This is a T.E.S.T.” (TEST) (Attachment 1 and 1a), to identify 1) how this product benefits state and local emergency planning partners and 2) opportunities to leverage and improve the product to further enhance state and local preparedness.

The purpose of this tool is to provide participants with pressures and situations that will likely arise in a real nuclear/radiological emergency, highlighting the need to work together to process the population effectively. To determine if the goal of the tool has been met, it is key to perform evaluation of the tool and its implementation. Information collected through survey and focus groups will be used to evaluate tool effectiveness in achieving intended outcomes, identify options for future implementation of TEST that best promote emergency preparedness planning and coordination, and identify areas for improvement of development and implementation of TEST.

CDC’s Program Implementation Branch within the Center for Preparedness and Response and the Emergency Management, Radiation and Chemical Branch within the National Center for Environmental Health will use the information collected in surveys and focus groups to determine the effectiveness and impact of the implementation of TEST, improve the evidence base for using game design to develop innovative learning models in emergency preparedness, enhance training for workforce development based on different learning styles, develop a framework for experiential learning and gamification for CDC/Public Health trainings, and explore improved cohesion with public health response partners. This will support CDC’s public health preparedness partners in training, exercising, and responding to public health emergencies.

CDC will share information and lessons gathered in webinars, presentations, and practice-based journals to support further development of similar innovative trainings.

A.3. Use of Improved Information Technology and Burden Reduction

Participation

Jurisdictions will volunteer to host and use TEST as an exercise. Potential respondents will be identified from those who participate in these exercises. Participation will be voluntary and open to all participants in the exercise.

Survey Review

Draft survey questionnaires were reviewed by evaluation subject matter experts. Feedback provided by the experts included information related to survey content, need for clarification in terminology and wording, survey structure, and response time. The survey questions were designed to collect the minimum information necessary for the purposes of this formative research. Surveys also have branching questions to ensure that respondents only answer the minimum applicable questions based on previous answers.

Survey Administration – Initial Survey (Attachment A)

A QR code for the survey link will be made available during and after the exercise. The initial survey will capture respondents’ initial reactions and takeaways from the exercise.

Survey Administration – Follow-up Surveys (Attachment B)

A short follow-up survey will be emailed to participants one and three months after the exercise, and response will be voluntary. Emailed surveys will be sent by the jurisdictions’ Exercise Coordinators, who will have collected contact information for all participants as part of the standard exercise requirement. Follow-up surveys will capture any preparedness activities attendees and/or agencies and jurisdictions have taken resulting the training.

Focus Groups (Attachment C)

On the initial survey immediately following exercise participation, participants will be asked if they would like to participate in additional focus group interviews. Focus group participants will be selected from those participants that provide voluntary consent.

Focus groups will be conducted virtually via Microsoft Teams approximately 4-6 weeks after the exercise. Focus groups will last for approximately 60 minutes and consist of 8 questions. Feedback from the initial survey may be used to prompt additional, more detailed feedback from participants as well. Data from the focus groups should provide more detailed information on participants' experience with TEST and the actions they/their organization have taken since participating in TEST to improve their preparedness and response efforts. CDC TEST project team will lead the focus groups. Focus group questions (Attachment C ) have been vetted by evaluation subject matter experts. Participants will be allowed to exit the meeting at any time.

A.4. Efforts to Identify Duplication and Use of Similar Information

“This is a T.E.S.T. – A Tabletop Exercise Simulation Tool” (TEST) is a new CDC product that has not been previously evaluated. Therefore, no duplicate or similar information evaluating the product exists. Gamification has been evaluated for training in medicine and other fields, but it has never been evaluated for its impact on improving cross-sector partnerships and collaboration for governmental agency response (Drees 2018, Zagal 2006).

A.5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this data collection.

A.6. Consequences of Collecting the Information Less Frequently

Each TEST exercise participant will have the opportunity to participate in one full survey cycle (initial, 1- and 3-month follow-up and one focus group). If CDC does not collect data from participants after each exercise, then the utility of the tool across diverse jurisdictions will not be fully characterized. TEST exercises will be implemented in different jurisdictions that vary in their background, training, resources, personnel, and preparedness. It’s important to evaluate TEST with each audience to ensure it meets the needs of the stakeholders it aims to serve.

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agencies

The Federal Register notice was published for this collection on August 23, 2019, Vol. 84, No. 164, pp. 44308. No public comments were received.

CDC project staff will solely be responsible for this project.

A.9. Explanation of Any Payment or Gift to Respondents

No incentives will be provided for participation in surveys or focus groups.

A.10. Protection of the Privacy and Confidentiality of Information Provided by Respondents.

The Office of Readiness and Response has determined the Privacy Act does not apply to this collection.

This project will only collect names and emails of participants if they choose to voluntarily provide that information in the initial survey because they would like to be included in focus group interviews. The email will be used to send a Microsoft Teams invite and will not be used by TEST project team for any other purposes. Name and contact information will not be shared outside of the project team nor will it be retained following completion of focus groups. Names and emails will not be associated with comments or documented during the focus group other than to identify who was in attendance.

A.11. Institutional Review Board (IRB) and Justification for Sensitive Questions

This project was reviewed by CPR’s human subjects contact and determined to not meet the definition of research under 45 CFR §46.102(l). IRB review is not required (Attachment 2).

There are no planned sensitive questions.

A.12. Estimates of Annualized Burden Hours and Costs

The annualized response burden is estimated to be 633 hours.

Exhibit A.12.A. Annualized Burden Hours

Category of Respondent	Form Name	No. of Respondents	Frequency of Response	Participation Time (minutes)	Burden (Hours)
Epidemiologist	Initial Feedback Survey including T.E.S.T. Game Play and Instructions	600	1	240	2400
EEpidemiologist	Follow up Survey (1 and 3 months)	600	2	15	250
Epidemiologist	Focus Group Interview	50	1	60	50
Totals					2,750 hours

A.12.B Estimated Annualized Costs

Annualized costs to respondents were calculated using the mean hourly wage rate for epidemiologists from the United States Department of Labor, Bureau of Labor Statistics May 2021 Occupational Employment Statistics.

Exhibit A.12.B. Annualized Cost to Respondents

Category of Respondent	Form Name	Burden in Hours	Hourly Wage Rate	Total Respondent Cost
Epidemiologist	Initial Feedback Survey	534	$41.70	$100,080.00
Epidemiologist	Follow up Survey (1 and 3 months)	200	$41.70	$10,425.00
Epidemiologist	Focus Group Interview	50	$41.70	$2,085.00
Totals	$114,675.00

A.13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

There are no other costs to respondents or record keepers.

A.14. Annualized Costs to the Government

The estimated annual cost to the Federal government is $4,051 based on 100 hours of FTE time. The hourly wage for a GS-13 employee is $40.51.

A.15. Explanation for Program Changes or Adjustments

This is a new generic information collection.

A.16. Plans for Tabulation and Publication and Project Time Schedule

Milestones	Schedule
Site Selection (Rolling)	Immediately following OMB Approval
Site Planning and Exercise Preparations	2 weeks after OMB Approval
Exercise Facilitation and Evaluation	2 months after OMB Approval
Data collection for evaluation	2 months after OMB Approval
Analysis and Reporting	1 year after OMB Approval

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

The display of the OMB expiration date is appropriate.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

REFERENCES

Centers for Disease Control and Prevention (CDC). (2019). Public Health Emergency Preparedness (PHEP) Notice of Funding Opportunity Exercise Requirements.

Cozine, Keith. (2015). Thinking Interestingly: The Use of Game Play to Enhance Learning and Facilitate Critical Thinking Within a Homeland Security Curriculum. British Journal of Educational Studies. 63. 367-385. 10.1080/00071005.2015.1069256.

Drees, S., Geffert, K., & Brynen, R. (2018). Crisis on the game board - a novel approach to teach medical students about disaster medicine. GMS journal for medical education, 35(4), Doc46. https://doi.org/10.3205/zma001192

Education Endowment Foundation (EEF). (2021, July). Collaborative learning approaches. Retrieved September 25, 2022, from https://educationendowmentfoundation.org.uk/education-evidence/teaching-learning-toolkit/collaborative-learning-approaches

Federal Emergency Management Agency (FEMA) (2020). Homeland Security Exercise and Evaluation Program. Retrieved September 25, 2022 from https://www.fema.gov/sites/default/files/2020-04/Homeland-Security-Exercise-and-Evaluation-Program-Doctrine-2020-Revision-2-2-25.pdf

Office of Management and Budget, Statistical Policy Directive No. 2: Standards and Guidelines for Statistical Surveys; Addendum: Standards and Guidelines for Cognitive Interviews. Published in the Federal Register, October 12, 2016, vol. 81, no. 197, pp. 70586.

MacKinnon, Lachlan & Bacon. (2012, April 22–25) Developing Realistic Crisis Management Training. [Paper Presentation]. International Information Systems for Crisis Response and Management (ISCRAM) Conference, Vancouver, Canada.

Zagal, J. P., Rick, J., & Hsi, I. (2006). Collaborative games: Lessons learned from board games. Simulation & Gaming, 37(1), 24–40. https://doi.org/10.1177/1046878105282279

LIST OF ATTACHMENTS

Attachment A – Participant Information Flyer

Attachment B – Immediate Post Exercise Survey Questions

Attachment C – Follow-Up Survey Questions

Attachment D – Focus Group Script

Attachment 1 – T.E.S.T. Game

Attachment 1a – Game instructions

Attachment 2 – Human Subject Research Determination Form

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Supporting Statement for Request for Clearance:
Author	Karen Whitaker
File Modified	0000-00-00
File Created	2025-05-18

SSA-CPR-ThisIsATEST