Form #1 Form #1 Semi-structured interview guide

S

Form Approved
OMB No. 0935-XXXX
Exp. Date XX/XX/20XX

emi-Structured Interview Guide - TeamSTEPPS^® National Implementation Evaluation

Location:	Date and time of interview:
Type of organization: (Type – QIO, hospital, other)	Interview Code: Role:

Testing materials checklist

Interview guide

Interviewer clock

Audio recording equipment (2)

Verify audio recording equipment

Pens and notepads

Informed consent forms (1 copy for participant to sign, plus 1 copy for participant to keep)

Outline (up to 60 minutes total)

Time in minutes for each section*	Topic	Elapsed time at end of section
3	Introduction (confirm consent; welcome; overview)	3
10	SECTION 1: Participant Characteristics	13
15	SECTION 2: Application of TeamSTEPPS Training to the Workplace	28
6	SECTION 3: Patient Safety Culture	34
10	SECTION 4: Barriers and Facilitators to Patient Safety Projects Implemented Post-Training	44
8	SECTION 5: Perceived Outcomes	52
8	SECTION 6: Final Insights	60
TOTAL		60
*Times listed are estimates.

Public reporting burden for this collection of information is estimated to average 60 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-XXXX) AHRQ, 540 Gaither Road, Room # 5036, Rockville, MD 20850.

Site visit goals

• Gather lessons learned from the AHRQ National Implementation of TeamSTEPPS Master Training program.

• Learn about the extent the TeamSTEPPS Master Training program and networking experiences associated with participation have been useful and applicable to the participants’ day-to-day work.

• Capture additional detail on the issues impacting participants’ and organizations’ ability to implement, disseminate, and sustain the TeamSTEPPS concepts, tools, or strategies.

• Describe the impact of TeamSTEPPS at various levels.

• Obtain useful information for AHRQ on how best to implement any future patient safety training activities.

PLEASE NOTE: This is a GUIDE for conducting the interview. It is intended to provide you with questions that may only be relevant based upon the role of the person you are interviewing. Adapt the questions, order, and wording to best suit each respondent according to their role, training experience, and state.

If hospital: Ask about “your organization.”

If QIO: Ask about “your organization” or “organizations you work with” or both depending on the question.

Introduction 3 min

(As interviewees arrive, meet participants and give them informed consent forms (one to sign, one to keep).

1. Welcome

• Thank you for agreeing to participate in an interview today.

• My name is {NAME} and I’ll be talking with you today. I work for the American Institutes for Research (AIR), an independent not-for-profit research organization.

2. Background—explain purpose of the interview

• Our discussion today is part of a project sponsored by the Agency for Healthcare Research and Quality (AHRQ). AHRQ contracted with The American Institutes for Research (AIR) to gather lessons learned from the AHRQ/DoD National Implementation of TeamSTEPPS Master Training program.

• In conjunction with AHRQ, we’ve selected nine locations for site visits. Our selections were based on an effort to identify a diverse group of sites.

• The site visits are intended to help us better understand the extent to which participation in the TeamSTEPPS Master Training program may have affected teamwork, communication, and patient safety practices at the home organizations of individuals and teams participating in the program, and more broadly to their local areas, regions, and states. At each site, we’re interviewing qualified individuals serving in a variety of roles among Master Trainers and non-Master Trainers.

• In addition to the site visits we are conducting a Web-based questionnaire of master training participants to gather a broader perspective on the impact of the TeamSTEPPS Master Training program.

3. Go over ground rules

• Your opinion is very important to us and so to make sure we don’t miss anything, we are audio taping today’s session. Because we’re recording, please try to speak in a voice at least as loud as the one I’m using now so that we can make sure the tape is picking up our voices.

• Everything you tell us will be confidential. To protect your privacy, we won’t connect your name with anything that you say. Also, you may ask us to stop the recording at any time and if you choose you may ask the recording be destroyed at the end of the session.

• The interview will take about 60 minutes.

• We have a lot to talk about today, so there may be times when I need to move the discussion along. Please understand that when I ask that we move to a new topic, I don’t mean to be rude.

• Do you have any questions before we get started? Okay, I’ll start the recorder now.

SECTION 1: Patient Safety Experience 10 min

{Turn on recording equipment.}

1. Individual Experience {TeamSTEPPS Master Trainers}

1. Which of the following best characterizes the organization in which you currently work? [Select one]

   1. Academic health profession program

   2. Public hospital

   3. Federal government health provider

   4. Community Hospital

   5. Children’s Hospital

   6. Critical Access Hospital

   7. Outpatient clinic

   8. Quality Improvement Organization (QIO)

   9. State health department

   10. Hospital association

   11. Patient safety center/commission

   12. Long term care facility, assisted living facility, or home health agency

   13. Regional or state-based healthcare professional association or institution

   14. Other: [Fill in the blank]

2. In what state is your organization located? [Provide drop-down menu of states]

3. What is the name of your employer facility/organization? [Fill in the blank]

4. If your facility is part of a larger healthcare system, what is the name of that larger healthcare system? [Fill in the blank]

5. What is your current job title? [Fill in the blank]

6. In what area did you complete your professional training? [Check all that apply]

   1. Nursing

   2. Medicine

   3. Pharmacology

   4. Public Health

   5. Health Administration

   6. Allied Health

   7. Life Science

   8. Other: [Fill in the blank]

7. How long have you served in your current role (at your current organization)? [Select one]

   1. 0-1 years

   2. 2-5 years

   3. 6-10 years

   4. 11 or more years

8. What was the reason for your participation in TeamSTEPPS Master Training? [Check all that apply]

   1. I wanted to find out more about TeamSTEPPS.

   2. I wanted to find out more about teamwork and team training in healthcare.

   3. My organization/department/unit is implementing TeamSTEPPS to improve quality of care

   4. Our health system has committed to using TeamSTEPPS to improve quality of care

   5. My organization had experienced a sentinel event or potential patient safety near miss.

   6. My organization leaders mandated our participation.

   7. We would like to introduce TeamSTEPPS in our medical or nursing education

   8. We would like to introduce TeamSTEPPS in our in-service program

   9. We are assisting a health provider organization improve patient safety (NOTE: this would be orgs like state hospital associations, state government patient safety orgs and insurers who accompany an hospital team)

   10. We plan to use TeamSTEPPS as part of a research project

   11. Other: [Fill in the blank]

9. Where did you first learn about the TeamSTEPPS Master Training being offered through the National Implementation Program?

   1. TeamSTEPPS.gov website

   2. Another website or web search

   3. At a conference

   4. Webinars

   5. Word of Mouth

   6. AHRQ Publications

   7. A talk given by AIR representative

   8. A talk given by another health care professional

   9. A professional newsletter or publication

   10. Other: [Fill in the blank]

10. Which of the following best represents your implementation of TeamSTEPPS since participating in master training? [Select one]

    1. I/we have implemented TeamSTEPPS concepts, tools, or strategies in my unit/department/facility/organization.

    2. I/we have supported the implementation of TeamSTEPPS concepts, tools, or strategies in organizations that we support as a QIO.

    3. I/we have supported the implementation of TeamSTEPPS concepts, tools, or strategies in organizations that we support as a hospital or health professional association.

    4. I/we have supported the implementation of TeamSTEPPS concepts, tools, or strategies in organizations that we support as teamwork consultants.

    5. I/we have supported the implementation of TeamSTEPPS concepts, tools, or strategies in organizations that we support as insurers.

    6. Other: [Fill in the blank]

11. [FOR Implementers only] Since the master training, which of the following have you accomplished? [Check all that apply]

    1. I/we have created a change team to implement TeamSTEPPS concepts, tools, or strategies in the unit/department/facility/organization.

    2. I/we have defined our teamwork problem or identified an opportunity to improve teamwork using TeamSTEPPS concepts, tools, or strategies in our unit/department/facility/organization.

    3. I/we have conducted a site assessment to define our teamwork problem or opportunity for teamwork improvement.

    4. I/we have collected data using one of AHRQ’s surveys of patient safety culture.

    5. I/we have defined the aim(s) of our TeamSTEPPS intervention.

    6. I/we have designed a formal implementation plan for our teamwork improvement intervention.

    7. I/we have created an action plan for implementation.

    8. I/we have briefed leadership and key personnel about our action plan.

    9. I/we have designed a TeamSTEPPS intervention.

    10. I/we have conducted TeamSTEPPS training for our intervention.

    11. I/we have developed a plan for testing the effectiveness of our TeamSTEPPS intervention.

    12. I/we have measured the impact of our TeamSTEPPS intervention.

    13. I/we have developed a plan for continuous sustained improvement with our teamwork improvement intervention.

    14. I/we have developed a communication plan for our teamwork improvement intervention.

12. [FOR Implementers only] Did you follow the steps in the Kotter model for Culture Change (i.e., Our Iceberg Is Melting)? [Check all that apply]

    1. I/we set the stage and created a sense of urgency in our unit/department/facility/organization. .

    2. I/we pulled together a guiding team for our unit/department/facility/organization.

    3. I/we developed a change vision and strategy for our unit/department/facility/organization.

    4. I/we communicated for understanding and buy-in from others throughout our unit/department/facility/organization..

    5. I/we empowered others to act in our unit/department/facility/organization.

    6. I/we produced short-term wins throughout our unit/department/facility/organization.

    7. I/we did not let up as we implemented change in our unit/department/facility/organization.

    8. I/we created a new culture in our unit/department/facility/organization.

13. [FOR Implementers only] What was the goal of your TeamSTEPPS implementation [Check all that apply]

    1. To reduce the threat of medical error in our unit/department/facility/organization.

    2. To improve patient safety and quality in our unit/department/facility/organization.

    3. To improve team cohesion in our unit/department/facility/organization.

    4. To improve team leadership in our unit/department/facility/organization.

    5. To improve situation monitoring in our unit/department/facility/organization.

    6. To improve mutual support in our unit/department/facility/organization.

    7. To improve communication in our unit/department/facility/organization.

    8. To foster a just culture within our unit/department/facility/organization.

    9. Other: [Fill in the blank]

14. [FOR Facilitators only] Since the master training, which of the following have you accomplished? [Check all that apply]

1. I have trained others outside my organization on any or all of the TeamSTEPPS concepts, tools, information, and techniques.

2. I have served as a TeamSTEPPS coach facilitating the use of concepts, tools, or strategies by others outside my organization by providing guidance, observing their use, and answering questions when possible.

3. I have facilitated the use of TeamSTEPPS concepts, tools, or strategies by others outside my organization by providing guidance, observing their use, and answering questions when possible.

4. I have helped others integrate TeamSTEPPS concepts, tools, or strategies into healthcare professional education or curricula.

5. Other: [Fill in the blank]

15. Since the master training, how have you implemented the TeamSTEPPS concepts, tools, or strategies you learned?

    1. Across our health system or organization

    2. Across our facility but not the entire health system or organization

    3. Within several units/departments but not the entire facility

    4. Within our unit/department only

    5. Other: [Fill in the blank]

16. Since the master training, how have you implemented TeamSTEPPS training for others?

    1. Targeted – Unit-Based implementation focusing on the needs of one unit and tailoring training to these needs (e.g., training SBAR, Briefs, and Huddles to all staff within an Emergency Department).

    2. Targeted – Tool-Based implementation focusing on a specific problem within the organization where the same tool(s) is/are introduced across all units at one time (e.g., training only SBAR to all staff across an organization regardless of variations in specific unit needs).

    3. Transformational – Transformational implementation where a broad application of TeamSTEPPS and all its concepts, tools, and strategies is carried out across an entire organization at once (e.g, training the entire suite of TeamSTEPPS tools and strategies to all staff within an organization at once).

    4. Other: [Fill in the blank]

17. What were your expectations about what you would learn or accomplish through your participation in the TeamSTEPPS Master Training?

    1. I had no expectations of the training program

    2. I expected to learn very little about teamwork

    3. I expected to walk away with a toolkit for improving teamwork

    4. I expected to hear the same things I had heard from other programs

    5. I expected to understand the role of teamwork in health care and patient safety

18. What was your level of experience in developing interventions to improve teamwork and communication prior to participating in TeamSTEPPS Master Training?

    1. I had no experience

    2. I had very limited experience

    3. I had some experience

    4. I had a lot of experience

19. What was your level of experience in developing interventions to improve patient safety or reduce/mitigate the impact of medical errors prior to your participation in the TeamSTEPPS Master Training?

    1. I had no experience

    2. I had very limited experience

    3. I had some experience

    4. I had a lot of experience

PROBE: Prior to your participation in the program, what was your level of experience with:

• Tools used to improve teamwork, communication, and team orientation.

20. {Non- TeamSTEPPS Master Trainers} Prior to attending training, how familiar were you with TeamSTEPPS and the National Implementation Program?

    1. I was not familiar with it.

    2. I was somewhat familiar it.

    3. I was familiar with it.

    4. I was very familiar with it.

2. Current Role

1. Tell me about your current role. What are your primary responsibilities? {Collect information on all responsibilities}

PROBE: Can you tell me a bit about your role in patient safety and quality improvement activities in general, not just this project?

PROBE: How long have you worked at this organization? How long have you worked in the field of patient safety at this organization? Overall?

SECTION 2: Application of TeamSTEPPS Concepts, Tools, or Strategies 40 min

3. Phase I: Assessment

1. How did your organization determine that you needed TeamSTEPPS? For QIOs, how did you determine that the organizations you support needed TeamSTEPPS?

2. Did you conduct a site assessment including measures like the AHRQ Culture Surveys?

3. What other measures or tools did you use to assess the state of teamwork and communication in your organization or organizations you support?

4. Did you identify a change team to plan how TeamSTEPPS concepts, tools, or strategies would be implemented in your organization or the organizations you support? Who comprised this team? What were the goals of this team?

5. Can you share the details of the action plan set forth by this team or the change team at the organizations you support? Did it follow the steps prescribed by TeamSTEPPS? Were there steps that did not seem relevant?

6. Can you share the details of how you set the stage and created a sense of urgency in your organization or those you support?

7. Can you share details of you pulled together a guiding team for your organization or those you support? What change vision and strategy for did the guiding team develop?

8. What kind of support did you have from leadership at your organization for the strategies and initiatives you decided to implement? What about support from your colleagues or other staff?

9. What kind of support did you provide for the strategies and initiatives implemented at your organization? How has your participation facilitated these initiatives?

10. Has your organization undertaken efforts to systematically assess the safety culture in your hospital or hospitals around your state? Tell me about it.

11. How would you describe the safety culture in your organization?

4. Phase II: Planning, Training, and Implementation

1. How did you conduct your action planning? What was the sequence of events? Who was involved in planning? How was leadership buy-in gained?

2. How did you define your TeamSTEPPS-based intervention? Did you focus on a specific unit or facility? Did you focus on a specific tool or tools? Did you plan a transformational intervention where your organizations or those you support engaged in sweeping changes all at once?

3. Since participation in TeamSTEPPS Master Training, how have you implemented any of the concepts, tools, or strategies in the curriculum to improve teamwork and communication within your organization or the organizations you support?

For the next questions, please focus on only those concepts, tools, or strategies that you are most familiar with.

• Tell me about the concept, tool, or strategy chosen.

• Tell me about the implementation process including action planning and the change team involved.

• How did you decide upon an intervention? How did you figure out that something should be done? What data/information did you look at?

• How was the site introduced to this implementation effort/activity?

• How was the implementation effort/activity promoted?

• What was your role on the implementation effort?

• How has the implementation effort/activity changed over time?

4. Did you conduct any formal training when implementing TeamSTEPPS? How was training structured? Who conducted training? Did you use the materials provided in the TeamSTEPPS manual? What modifications or customizations did you make to the instructional content?

5. How often did you communicate about the training? What did communications consist of? How did you communicate for understanding and buy-in from others throughout your organization or those you support?

6. Did you empower others to act in your organization or those you support? What actions did they pursue once empowered?

7. How was the training initiative received by staff? Was there interdisciplinary participation? Did you collect any data on the training?

8. What factors about your organization made it easier to implement concepts, tools, and strategies from TeamSTEPPS?

9. What lessons did you learn or what made it easier during implementation of TeamSTEPPS concepts, tools, or strategies?

• Tell me more about that. What are some of the factors you considered?

• What types of resources were provided to help you implement?

• How, if at all, did your leadership (including organizational, community, and state leadership) react to, work with, or help carry out your implementation efforts?

10. What factors about your organization made it easier to sustain the use of concepts, tools, and strategies from TeamSTEPPS?

11. What lessons did you learn or what made it easier while trying to sustain the use of TeamSTEPPS concepts, tools, or strategies?

• Tell me more about that. What are some of the factors you considered?

• What types of resources were provided to help you sustain?

• How, if at all, did your leadership (including organizational, community, and state leadership) react to, work with, or help carry out your sustainment efforts?

12. What factors about your organization made it harder to implement concepts, tools, and strategies from TeamSTEPPS?

13. What lessons did you learn or what made it harder during implementation of TeamSTEPPS concepts, tools, or strategies?

• How did you overcome these challenges?

• Was there any support you feel would’ve been helpful that you did not get? (PROBE: resources, leadership support)

14. What factors about your organization made it harder to sustain the use of concepts, tools, and strategies from TeamSTEPPS?

15. What lessons did you learn or what made it harder while trying to sustain the use of TeamSTEPPS concepts, tools, or strategies?

• How did you overcome these challenges?

• Was there any support you feel would’ve been helpful that you did not get? (PROBE: resources, leadership support)

16. If you were able to start over with your TeamSTEPPS implementation and sustainment efforts what would you do the same? Differently?

5. Phase III: Sustainment

12. What strategies did you use to sustain TeamSTEPPS implementation? Did you use promotional materials, incentives, continuous information sharing, coaches, etc…?

13. How did you sustain the lessons learned from TeamSTEPPS implementation?

1. Were coaches used? If so, who served in the coaches’ roles?

2. How did you ensure there were opportunities to use the tool(s)?

3. How did ensure new behaviors were valued and reinforced?

4. How did you report progress back to staff?

5. How did you celebrate your initial successes?

6. Did you develop your own sustainment materials like pens, pads, and note cards?

7. Did you host specific events geared toward sustaining the lessons learned during training?

14. How long was the implementation sustained?

15. Were some tools easier than others to sustain? What did you do to sustain?

16. Are you retraining people on some sort of regular basis? What is the cycle for training? Are you still using the same training staff?

17. Did you produce short-term wins throughout your implementation? How were they celebrated or recognized? Did they fuel future success?

18. Did you feel pressure to let up as you implemented change throughout your organization? How did you resist the temptation to let up while implementing change? What motivated you to keep implementing change?

19. Did you create a new culture in your organization or those you support? What signs of this new culture can you point out as evidence?

20. How well did the concepts, tools, or strategies fit the needs of your unit, department, or organization? What would you have liked to have learned or had access to through the TeamSTEPPS Master Training that you did not?

21. How have measured your progress or success? What were your initial successes? What were your initial challenges? What tools did you use to assess your progress or the impact of training?

1. Did you measure changes in attitudes regarding teamwork? Idf no, do you think attitudes have changed?

2. Did you track changes in organizational outcomes such as patient satisfaction, staff satisfaction, event reports, or other measures?

3. Did you observe changes in actual teamwork behavior?

4. When did you conduct your post-training measurements?

5. Did you conduct a formal evaluation of the TeamSTEPPS program you implemented as a teamwork improvement intervention? If so can you describe this study?

6. National Implementation of TeamSTEPPS Program

22. How well did the National Implementation of TeamSTEPPS Master Training program match the needs of your organization?

• What aspects of the training do you feel were most valuable to you in your work?

• Were there particular topics not covered in the training that you feel should have been?

• Did the training properly prepare you to implement TeamSTEPPS concepts, tools, or strategies in your organization?

• What, if anything, do you think your organization (or those you support) is doing better as a direct result of implementing TeamSTEPPS? How can you tell that there have been changes?What made it possible for you to improve? How has your organization benefited from involvement in the TeamSTEPPS National Implementation program?

• How has participation in the program affected your organization’s ability to network with others?

7. {TeamSTEPPS Master Trainers -- Implementers and Facilitators Only} Sharing TeamSTEPPS with External Others

1. Have you trained others outside your organization? Did you share your TeamSTEPPS knowledge or expertise with others in the community?

• What did the training(s) consist of?

• What types of people attended the training?

• Which organization hosted the training?

2. Have you promoted the concepts, tools, or strategies with others within your organization, community, and/or state that you accessed through the TeamSTEPPS Master Training program?

2. How have you made TeamSTEPPS concepts, tools, or strategies accessible to other organizations or individuals from organizations outside your own?

Interviewer reference list of concepts, tools, techniques, resources [Q4, Q5]
General AHRQ Patient Safety Improvement Corps (PSIC) AHRQ Patient Safety Network (AHRQ PSNet) Business Case for Patient Safety Introduction to Patient Safety Medical and Legal Issues Patient Safety	Culture AHRQ Hospital Survey on Patient Safety Culture (HSOPS Survey) AHRQ Nursing Home Survey on Patient Safety Culture AHRQ Medical Office Survey on Patient Safety Culture Just Culture Leading Change?
	Root Cause Analysis RCA Process & Methods Root Causes: Five Rules of Causation (the laminated flip book)
Assessment and Evaluation AHRQ Patient Safety Indicators Cause and Effect Diagramming Evaluation of Patient Safety Programs Healthcare Failure Modes and Effects Analysis (HFMEA) Heuristic (Expert) Evaluation Technique Probabilistic Risk Assessment Safety Assessment Code (SAC) Matrix Usability Testing Technique Assessment Tool (PSAT)
	Patient Safety Design Human Factors Engineering and Patient Safety Designing for Safety Mistake-Proofing: The Design of Healthcare Processes
	Other AHRQ Web M&M High Alert Medications High Reliability Organizations (HROs)

SECTION 3: Final Insights 8 min

6. Final Insights

1. { TeamSTEPPS Master Trainers} If you could do it over again, would you still have participated in the TeamSTEPPS Master Training program?

• Would you change anything about the experience or the program itself? Please explain.

• Should other staff have been included in the master training? What about other organizations?

How would you like AHRQ to support TeamSTEPPS in the future?

What would you like to see added to Master training, removed…

2. What, if any, type of ongoing technical support from AHRQ would be helpful to the successful implementation of teamstepps after training?

3. What advice would you give another organization that wanted to improve their teamwork, communication, or patient safety culture?

4. Before we end, I’d like to give you chance to share any additional thoughts or comments about TeamSTEPPS or the program and the information we talked about today. Is there anything else you would like to add that you didn’t have a chance to say during our discussion today and you think is important for us to know?

Thank you very much for participating in this discussion today. We appreciate your time.