Form 3 Program Staff Focus Group Protocol 2 (Co-definition Phas

OMB No: 0906-XXXX

Expiration Date: XX/XX/20XX

Public Burden Statement: This collection of information will be used to prioritize and select a virtual engagement strategy to test in subsequent phases of the study. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this information collection is 0915/0906-XXXX and it is valid until XX/XX/202X. This information collection is voluntary. Data will be kept private to the extent allowable by the law. Public reporting burden for this collection of information is estimated to average 1.5 hours per response, including the time for reviewing instructions, searching existing data sources, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to HRSA Information Collection Clearance Officer, 5600 Fishers Lane, Room 14N39, Rockville, Maryland, 20857 or paperwork@hrsa.gov. Please see https://www.hrsa.gov/about/508-resources for the HRSA digital accessibility statement.

Program Staff Focus Group Protocol

Phase 1: Co-Definition, Focus Group 2

Thank you for taking the time to talk with us today. My name is [NAME] and this is [co-facilitator/notetaker NAME(s)]. We work as research analysts with The Policy & Research Group (PRG). As a reminder, we are implementing a project designed to identify, develop, study, and disseminate evidence-informed strategies and resources that early childhood home visiting (ECHV) programs may use to strengthen home visiting services, and, ultimately, achieve better outcomes for children and families. The project is being implemented on behalf of the Health Resources and Services Administration (HRSA), and in collaboration with the Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families (ACF).

In our last discussion, we talked about strategies your program has implemented to promote child engagement and active participation, especially in virtual settings. The goal of our discussion today is to identify and define specific strategies your program would like to implement and test over the next few weeks. I will begin by reviewing the information we gathered during our first focus group in Phase 1 of this project: Co-Definition. Then, I will lead you through a brainstorming activity. Lastly, I will ask your thoughts on which strategies you would like to move forward with in this study and the parameters for defining and implementing these strategies.

Your participation in our discussion today is completely voluntary, and you can leave the call at any time. There is no penalty if you choose not to participate in this focus group. I may ask you a question directly, but you don’t have to answer anything you don’t want to. This is just an effort to make sure we hear from everyone today. Please know that the information you share today is confidential and only study staff will have access to the notes, recordings, and transcripts. Study staff may summarize what we discuss today to share with interested parties (e.g., staff in your program, HRSA, or other programs participating in the study), but no identifiable information or individual responses will be shared. We ask you to please respect each other’s privacy by not sharing any personal information or experiences discussed during the focus group.

I’m going to record our discussion today so that we can make sure we accurately capture what is said (that we remember the discussion correctly). By joining the discussion, you are agreeing to participate and to have the discussion recorded. Does anyone have any questions before I start recording? [Answer questions.]

As we are getting things set up to record and get our discussion underway – I’m going to drop a link to a quick survey in the Zoom Chat [or via email, if using phone]. The survey asks you to answer a few questions about yourself, so we have an understanding of who participated in our discussions. Like your participation in this discussion, filling out the survey is voluntary. You do not have to respond to all or any of the questions. Your name is not on the survey and will not be connected to your responses. By filling out the survey you are agreeing to allow the study team to see your responses and use your answers to describe (summarize) who was involved in the focus groups we conducted. Is everyone able to click on the link? [troubleshoot as necessary]. Great. If you are willing to fill out the survey, please take a few minutes to do so now and we’ll get started in [5 minutes, at TIME].

Ok. We are set up and I’m going to begin recording now. [Start to record.] Before we get started, I’d like us to set a few ground rules for the conversation.

[If using Zoom]

Since we are using Zoom today, I’ll keep my video on the whole time, but you can choose to have your video on or off. We want to make sure that everyone has the opportunity to have their thoughts and opinions heard. So, to avoid us talking over each other, I’ll be helping to direct the conversation. If you want to speak, you can let me know in the chat, click the “raise hand” button, or just chime in when there is a pause in conversation. Also, I want to emphasize that there are no right or wrong answers and we are interested in hearing both positive and negative comments and opinions – or whatever you think is important to share. We only ask that you do not use the names of clients or staff outside of this focus group to help maintain everyone’s privacy. Those are all of the guidelines I have - does anyone have additional guidelines for our conversation that you would like us to consider? Does anyone have any questions about Zoom or the guidelines for our discussion? [Confirm that everyone understands Zoom features and can mute/unmute.]

[If using Phone]

We want to make sure that everyone has the opportunity to have their thoughts and opinions heard. So, to avoid us talking over each other, I’ll be helping to direct the conversation. If you want to speak, you can chime in when there is a pause in conversation, otherwise, before we move on to different topics, I will provide an opportunity for people to voice their opinions. Also, I want to emphasize that there are no right or wrong answers and we are interested in hearing both positive and negative comments and opinions – or whatever you think is important to share. We only ask that you do not use the names of clients or staff outside of this focus group to help maintain everyone’s privacy. Those are all of the guidelines I have - does anyone have additional guidelines for our conversation that you would like us to consider? Does anyone have any questions about the guidelines for our discussion? [Confirm that everyone understands how to mute/unmute.]

In all, this should take about an hour to an hour and a half. Does anyone have any questions before we get started? [Answer questions.]

[Facilitator reviews available information gathered during cycle (e.g., initial staff focus group, focus groups with families, administrative data). At least one week before meeting, the facilitator provides a list of the strategies discussed by the site, and also strategies discussed by other sites or discussed in literature that meet the minimum definition set for the project. During the course of the Co-definition phase, additional parameters may be identified and placed on engagement strategies that improve their feasibility to implement and study within the context of this project.]

As a reminder, an engagement strategy may include one or a combination of the follow elements:

• A strategy to initially engage a child in the visit (spark interest and encourage involvement)

• A strategy to maintain child engagement once it is established

• A strategy to reengage or refocus a child who has lost interest/stopped participating.

An engagement strategy may also be an overall approach (holistic) to the visit that is intended to be applied for the duration of the visit in order to continuously engage a child.

[The list created for review will be organized as a table with columns for type (purpose or intent), such that initial engagement, engagement maintenance, refocusing, combination, and wholistic strategies will be grouped separately. The facilitator will provide the list of strategies to all participants via email, prior to the discussion. Additionally, if the discussion occurs via Zoom, the facilitator will use the Whiteboard function or similar feature to share the list so everyone can see it.]

Part 1: Brainstorming Activity

Now let’s transition to the brainstorming activity. We’ll start by reviewing a list of virtual engagement strategies your program has used, as well as strategies used by other home visiting programs.

During our last focus group, your program generated a list of strategies that you would like to consider. [Display list.] When we sent the strategies list to you to review before our discussion today, we included strategies from other sites as well. Are there any strategies from other sites that you would like to consider for your program?

[Before moving on, make sure everyone has time to share and that all identified strategies of interest are noted and the list is final.]

Okay, this is our list of all the potential strategies to study. Next, we will discuss each strategy, with the goal of refining the list and eventually deciding on which strategy we will implement as part of this study. The chosen strategy can have multiple components, but we will want to focus on one strategy so we can clearly implement, study, and refine it throughout the study cycles. Any questions? Let’s begin.

[If Zoom, facilitator shows list in Whiteboard, or similar Zoom feature]

First, take a look at the list of strategies [on the screen]. I’m going to send out a poll through [Zoom Feature] to get an understanding of your familiarity and experiences with each. I’d like to ask each of you to take the next [3-4 minutes], and for each strategy, first indicate if you have used it (by answering yes or no), then indicate how you feel about the strategy based on your experiences with or knowledge of it [mostly positive, neutral, mostly negative].

When you’re deciding what to put next to each strategy, what we are interested in is your experience and opinion. There is no right or wrong answer. You can think about things like how much of a hassle it is to use or how you think it affects the quality of the visit and the experiences of families. If it’s a strategy you haven’t used before, consider what your initial reaction is to the idea.

[Wait until it appears all responses have been received.]

Now, let’s reflect on the activity we just completed.

[If phone]

Please make sure you have a pen/pencil and something to write on. We will walk through the activity together.

First, take a look at the list of strategies. Of these, which are you familiar with or have heard of, either personally or professionally? Please take the next few minutes to write the names of these strategies down on the paper in a list. If there are any new strategies on the list you haven’t used but which look promising to you, write those down too.

Next think about what you know about the experiences you and the families you work with have had with each strategies. For each strategy listed, indicate how you feel about it based on these experiences. Use a smiley face for services or resources you feel mostly positive about, a sad face for ones you feel mostly negative about, and a neutral face you feel neutrally or have mixed feelings about.

When you’re deciding what to put next to each strategy, what we are interested in is your experience and opinion. There is no right or wrong answer. You can think about things like how much of a hassle it is to use or how you think it affects the quality of the visit and the experiences of families. If it’s a strategy you haven’t used before, consider what your initial reaction is to the idea.

Now I’m going to go around the group and ask you to tell me whether you have had any experience with it and what you think about it (mostly positive, neutral, mostly negative).

[Starting with the first strategy on this list, go through the list of the strategies one by one, and tally answers.]

[Zoom or phone]

[Reorganize the list using Zoom feature or table feature available on personal computer. Within each column (for each strategy type), list the strategies in descending order from those receiving most positive to those receiving most negative ratings/assessments. Create separate sections (rows) in the table to delineate strategies that received all positive, mixed, and negative ratings. If conducted over the phone, read the list out loud.]

I’ve reordered the original list to reflect how you have reviewed or assessed the strategies today. Overall, It looks like [list/point out strategies] received more positive reviews; [list/point out strategies] received mixed reviews, and [list/point out strategies] received more negative reviews.

To make the most of our time today, we will limit our discussion to the four strategies with the most positive ratings. We will discuss each of these strategies one by one. I will ask specific questions about each to make sure we all have an understanding of how it works and when it works best, the challenges or barriers it presents, and what the intended outcomes of the strategy are.

[Ask the following questions for each strategy on the list.]

Question: How many people have used [strategy] before?

[If no one has personal experience with the strategy, skip to the additional questions for new strategies indicated by asterisks in the next section.]

Question: Do you think your program has the capacity to implement [strategy]?

• Probe: addition to home visitors’ work/caseload; accessibility to all home visitors (for example, some may be less comfortable doing virtual visits, some families might not be eligible for the study, etc.); other resources needed

Question: Is there anything else we should consider or know about this strategy?

[Ask the questions for new strategies indicated by an asterisk below, as needed. If there are no added strategies, skip this section and move to the next page, (Selecting top choice for “to be considered” list).]

*Question: Can you tell me why you wanted to include [strategy] on your longer list?

• Follow-up: What appealed to you about [strategy]?

• Follow-up: What do you see as the benefits of [strategy]?

*Question: How do you think [strategy] would work at your site?

• Follow-up: What would implementing it look like for home visitors, families, and program staff?

• Follow-up: Do you think [strategy] will be easy to implement? Why or why not?

• Follow-up: What do you think might be some challenges to implementing [strategy]?

*Question: How do you think families/children will feel about [strategy]?

*Question: What is [strategy] intended to do/help with?

*Question: What would [strategy] require to be implemented in this study?

• Probe: materials needed; additional training or support; more time to implement; additional costs

*Question: Do you think your program has the capacity to implement [strategy]?

• Probe: addition to home visitors’ work/caseload; accessibility to all home visitors (for example, some may be less comfortable doing virtual visits, some families might not be eligible for the study, etc.); other resources needed

Question: Is there anything else we should consider or know about [strategy]?

[Selecting top choice for “to be considered” list]

[If focus group members are all on their own computers] Now that we have discussed each strategy, I will ask everyone to write down their top choice for which of these strategies to add to the final “to be considered” list. Please send these to me privately in the group chat. Does anyone have any questions about how to send me a private message?

[If focus group members are all in one room together and not all on individual computers] Please write down your top two choices for strategies to add to the final “to be considered” list. You can pass these to [program administrator], without adding your name to the paper. [Program administrator], can you please read out the strategies everyone listed. We will make notes and generate a new list.

[Generate a new list of final “to be considered” list of strategies and share it with the group in Whiteboard, or similar Zoom feature]

Question: Based on our discussion, this is the list of strategies you believe should be considered for testing [if phone, list out strategies verbally].

• Follow-up: Does anyone have any reactions to this list that they would like to share?

• Follow-up: Is there anything final you would like to add or amend about this list?

[In the eligibility screening calls, study staff will determine who is responsible for making the decision about the strategy or strategies that will be tested. Study staff will also ask this person(s) if they would like for the rest of the staff to remain on the call for the decision-phase of this focus group, or if they would like other staff to leave the call. The next part of the discussion will only include those who were previously identified for this part of the call. This could be only the individual(s) who will make the decision about what strategy/strategies to implement, a smaller subset of the whole group, or the whole group. If it is a subset of the full group, the facilitator will thank everyone else for their time and ask those individuals to stay on the call.]

[If some participants will leave the call.]

This has been a very helpful brainstorming session. We appreciate everyone’s participation and thoughtful responses and ideas. At this time, we are going to go through the information we just discussed, and we will work with [leader’s name and title] to identify and define the parameters for the strategy(ies) your program will be testing in the Installation Phase of this project. I will then email you all an update based on that conversation.

Question: Before we wrap up this brainstorming activity, is there anything you would like to add that we haven’t discussed already?

Thank you again for your time.

[Individual(s) who will sign off on the strategies for implementation and the parameters for the strategies stay on call to continue the conversation.]

Part 2: Strategy Selection

Question: Considering our conversation and what you know about your program, what is your main reaction or takeaway about the strategies your team/staff think are important to consider for testing?

Question: Which of these strategies are you most excited about?

• Follow-up: What is it about the strategy(ies) that interests you?

Ok, now let’s take another look at the “short” list of strategies your team/staff agreed are best to consider and try to identify strategies of interest, discuss what you see as the most promising element(s) of an engagement strategy, and try to come up with an initial understanding of a strategy, or combination of strategies that are similar, that you want to test out in the Installation phase.

Question: Is there a specific strategy on this list that you think you may be interested in testing? If so, please explain. [If they can’t decide on one go to elaboration below.]

• Follow-up: What do you like about this strategy?

• Follow-up: Do you think it will be feasible to implement?

• Follow-up: How well does it work with the program model you use?

• Follow-up: How well is it suited for the children and families you serve?

• Follow-up: How promising do you think it is for initiating/maintaining engagement, refocusing, or as a holistic approach to engagement?

• Follow-up: Are there any challenges that you might expect with these?

• Follow-up: What do you see as the benefits?

Question: Is this the strategy (or strategies) that you would like your program to test?

[If yes, go to Definitions, starting on the next page. If no, continue with the Elaboration section below.]

Elaboration

Okay, so if you’re not fully certain of a specific strategy – let’s work to identify the elements of a strategy you’d like to test.

Question: Why would you say you are having a hard time deciding?

• Probe: number of options, similarity of options, intended outcome, feasibility, scope of strategies

Question: What is most appealing or important for you – what do you want the strategy to accomplish or help with? Please explain.

• Probe: initial engagement, maintaining engagement, refocusing, parental involvement in engaging the child, managing the visit

Question: Are there any strategies or elements on the list that you think could accomplish this goal and meet your needs? If so, please explain.

Question: Based on what you are saying is most appealing, it appears that [list strategies] could be a good fit, perhaps with some adjustments to meet the needs of your program. Would you want to test out one [or two] of these strategies? If so please discuss.

Question: Could any of these strategies be combined to create a more comprehensive or appealing strategy?

• Follow-up: What would that look like?

• Follow-up: What are the benefits of combining these strategies?

• Follow-up: Would this combination of strategies significantly change the ask for home visitors and/or other staff?

• Follow-up: Are there any other considerations anyone can think of with this combination of strategies?

Question: So, based on this conversation it sounds like you are interested in trying [describe strategy as discussed]?

[Once a strategy or combination of similar strategies have been identified as the one the program would like to implement, move on to the Definition section below.]

Definition

[For this set of questions keep in mind, we need to make sure the definition is fully fleshed out – that we understand exactly what the home visitors will be asked to do, when they will be asked to do it, and what considerations they will need to make when deciding how/when to use the strategy.]

Okay, now that we have a better understanding of the engagement strategy/strategies you are interested in testing, let’s work to define what this will look like in practice. You indicated you are interested in testing [type of strategy: initiation, maintenance, refocusing, combination, or holistic approach] that [describe what we know about the strategy, including main components and what home visitors will be asked to do.]

Question: Does this match your understanding, or should we adjust this definition?

• Follow-up: What adjustments should be made?

• Follow-up: What do you see as the main components of the strategy you want to test?

Question: To make sure we have a mutual understanding of the strategy, can you define the strategy in your own words?

• Follow-up: How will you explain this strategy to your home visitors?

Question: How do you see this working in practice?

• Probe: guidance for implementing [strategy]; most important aspects of [strategy]; boundaries to be set for [strategy]; considerations will home visitors be asked to make

Question: Considering how you have defined [strategy] and the parameters you have set for their use, are there any potential challenges for facilitators you see/expect for implementing these?

• Follow-up: What kind of guidance do you think your home visitors will need for dealing with potential challenges?

Question: Are there any specific materials or considerations your home visitors need for implementing [strategy]?

• Follow-up: What is a plan for addressing these needs?

Question: What are the outcomes you expect to see from the implementation of [strategy]?

• Follow-up: What do you think “success” looks like for [strategy]?

Now that we have defined the strategy and have discussed guidance for implementing the strategy in the Installation phase. We will write up the details of how the strategy is initially defined and how it will be implemented. I will send you those details to review, then project staff will work with your staff (those selected to test the strategies) to make sure they understand what they will be doing and how they will be participating in the study.

Part 3: Wrap-up/Final Questions

[This section is the closing of the focus group and is used to make sure we address any outstanding questions and ensure we ask if anyone has anything additional to share before we end the focus group.]

Question: Are there any questions about the next steps in this process?

Question: In closing, is there anything else you’d like to share?

Thank you very much for participating in this discussion! Our next focus group will be [provide timing] and will be Phase 2 (Installation) of the study process where we will provide [findings about X] and discuss your reactions and reflections to these findings and the implementation process. In between now and then we will solidify the plans for this study, ensure your team has what they need to implement the strategy, and begin implementation with cycle one of the study cycles. If you have questions after you leave the call, feel free to contact me.

If you have any questions about the study, please contact Teresa Smith at The Policy & Research Group (teresa@policyandresearch.com or (225) 281-3783).

If you have questions about your rights as a research volunteer, you can call Health Media Lab Institutional Review Board at (202) 549-1982.

The Policy & Research Group 10