Discussion Guide: Focus Groups

ATTACHMENT 5 - Revised

YARH DISCUSSION GUIDE FOR FOCUS GROUPS

October 2014

The Planning Grants to Develop a Model Intervention for Youth/Young Adults with Child Welfare Involvement At-Risk of Homelessness (YARH) process study will use the discussion guide that follows this cover page to guide focus groups conducted during the site visits. The interviewer will begin the interview by reading the introduction and consent statement aloud to the respondents and secure consent to participate before moving on to the discussion guide. It is the same discussion guide as presented in Attachment 4 – Revised: YARH Discussion Guide for Individual Interviews.

The YARH Process Study will include one site visit to each grantee where we interview: (1) planning team leaders, (2) planning team members, and (3) partner organizations that are not represented on the planning team. Interviews will be conducted either one-on-one, or in small groups, depending on staffing structure, roles, and the number of individuals in a role.

Below is a list of questions that could be posed to address the three key research questions presented in the Supporting Statement Part A of the previously approved ICR.

The following scripts will not be read verbatim. As the interviewer, you need to be sufficiently familiar with the script to introduce the study and the focus group process effortlessly. We suggest that you familiarize yourself thoroughly with the text in advance. Your manner should be relaxed and your tone conversational throughout the discussion.

Introductory Script

Thank you very much for agreeing to participate in this conversation. Your participation is very important to the study. I’m ______________________ and I work for Mathematica Policy Research, an independent social policy research company.

We are conducting a study for the Office of Planning, Research, and Evaluation at the Administration of Children and Families (ACF) at the U. S. Department of Health and Human Services. The major goal of the study is to understand how the planning grant supported the work of local communities in building a comprehensive service model to serve youth and young adults at-risk of homelessness. We will analyze survey results, documents submitted as part of the grant—such as semi-annual reports—and notes from conversations like this to understand the activities grantees planned and completed, what grantees see as the major outcomes of the planning process, the target populations for whom the comprehensive service model is designed, the interventions that will be part of the comprehensive service model, and the partnerships and integration that were supported for or rose from the planning grant activities.

We would like to talk with you about your experiences and perceptions of the planning grant process and activities. Our team will use your responses in conjunction with other data sources to identify themes to describe the planning grant experience. Comments will not be attributed to specific individuals or grantees, and no individuals will be quoted by name. Your participation in this discussion is voluntary.

I am going to moderate the discussion. It is really important for everyone to speak up so we can have a lively and informative discussion. It will also be helpful if you speak one at a time, so everyone has a chance to talk. We ask that you respect each other’s point of view.

Your responses will be kept private and used only for research purposes.

There are no right or wrong answers. You are the experts, and we want to learn from you.

We will not share your comments with anyone other than members of the research team. We will not attribute any statements to you, or your organization, in the final report submitted to ACF.

We have many topics to cover during the discussion. At times, I may need to move the conversation along to be sure we cover everything.

I would like to record our discussion with a digital recorder so I can listen to it later when I write up my notes. No one besides our research team will listen to the recording. If you want to say anything that you don’t want recorded, please let me know and I will be glad to pause the recorder. Does anyone have any objections to being part of this interview or to my recording our discussion?

We want to reiterate that being part of this discussion is up to you, and you may choose not to answer a question if you wish. Being part of this discussion will not affect your employment or your involvement with the YARH planning grant.

The discussion will last no more than one and a half hours, and we will not take any formal breaks. Please feel free to get up at any time if you need to do so.

Since there are several of us participating in this discussion, please be sure to speak clearly, and one at a time—this will help ensure that everyone gets a chance to participate, and should minimize the number of times we need to ask people to repeat what they said. If you have a different perspective from the one being presented that is completely fine, and we are absolutely interested in hearing different sides of the issues we will be discussing today. Please allow other group members time to finish speaking before you provide input. We have a number of topics to discuss during our time together, and we want to be sure to hear from everyone today. I will manage the discussion so that we cover everything and hear from everyone in the time we have together today. This may require me to interrupt you to allow another person to speak; I apologize for that in advance but it is necessary to accomplish our goals.

Do you have any questions before we get started?

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this collection is 0970-0445 and it expires 04/30/2016.

After Answering Any Questions

Good, let’s begin.

Once again, thank you for agreeing to meet with us today. Let’s go around the room now and introduce ourselves. Please say your name and the name of the organization you represent.

The following table presents potential questions to be asked organized by program activity. The particular questions will be developed for each grantee and participant to reflect what is known about the planning team work.

Program Activity	Interview Questions
Identifying and engaging the planning team members	Discuss the composition of the planning team. How has this changed from the time you wrote the proposal to now? Why? How were team members identified? What criteria were used to select planning team members? How did you identify youth to serve on the planning team? What were some of the challenges of forming the planning team? Strategies to overcome challenges? Are there any community organizations or individuals that are not involved that you wish would be involved? Why aren’t they involved? What were some of the successes of forming the planning team? What have you learned from the team formation that you wish you had known at the onset?
Establishing structure of planning team	Confirm understanding of the structure of the planning team, from most recent SAR. How was the structure determined? How has the structure changed since the initial proposal, and why? Have there been changes in the leadership structure? Are there sub-groups or committees within the planning team? How were these sub-groups or committees identified? How was membership in these sub-groups determined? To what extent and how is the leadership team involved in the subgroups or committees? How do you envision your partnership structures would change as you begin to implement the new program/services? Would you attempt to bring in new partners?
Establishing formal relationships with partners	Please describe the history or relationship your organization has with your partnering organizations (or vice-versa). Did the leadership convey a clear vision for the team and the planning period? Do you share their vision? For partners (including youth): Explain your experience in participating in planning team work. How have you been included in the planning team work? What were the benefits to your organization of participating in this project? For the lead agency- what were the benefits of including particular agencies as partners? What challenges have you faced in creating formal partnerships with community organizations? What strategies have you used to overcome these challenges? How has your relationship with the partnering organizations changed since the beginning of the grant? Why? Has having a formal commitment changed the nature of your partnerships? Has a formal commitment been an advantage, disadvantage or neither in obtaining/supplying data? Planning for service delivery? How do informal partners contribute to the planning team? How do their contributions differ from formal partners? Would having a formal commitment have made a difference to their contribution? Have any informal partners been incorporated as planning team members since the beginning of the grant? What motivated this change? How, if at all, has the grant changed whom you view as your partners?
Developing informal relationships with partners
Establishing communication system	Describe the communication process employed by the planning team/ employed by the partner agencies. How has the communication process changed since the beginning of the grant? How, if at all, has the grant changed the frequency or means by which you interact with the partners? How has it changed the degree of collaboration? How is this communication process used to foster collaboration between partner agencies and the planning team? What strategies are most effective (e.g. standing frequent meetings?) How receptive is the leadership team to input from others? What communication mechanisms ensure input flows up to leaders? What challenges has your agency had in communicating with other planning team members? With grant leadership (for partners)? With other partnering agencies? What strategies has your planning team used to overcome the challenges? What lessons have you learned related to establishing strong communication processes? The role of leadership in communication? Are there “key ingredients”?
Establishing decision-making process	Does your planning team employ a formalized decision-making process? If so, please describe. (Probe based on most recent SAR responses). How was the decision-making process determined? How has this process changed over time? What has led to this change? How has the grant influenced the ways in which you share decision-making with your partners? Does the decision-making process help or hinder the team’s ability to work together effectively/collaborate? Challenges to following the formalized process? Strategies that enable planning team members and partners to follow the process? What lessons have you learned for establishing and following a collaborative process for making decisions? Are there “key ingredients”? Are there important leadership qualities or actions to foster collaborative decision-making?
Accessing administrative data sources	How did you determine the data sources that would be needed for the data analysis? Did the TA provided, or other TA sought, influence your decisions? In what way? Did you use data from a partnering organization or from an organization represented on the planning team? Did you use different data sources to examine the different target populations? Which data sources or elements did you have access to before the grant and which did you have to acquire? How are you receiving the data? Individual-level, aggregate, de-identified? Do you have access to any integrated data sets or warehouses? If so, please describe the process of creating or getting access to integrated data. Were you able to access all of the data sources that you initially proposed to use in your grant application? Did you access all the data sources you would have liked to use? What contributed to changes in the data that you proposed to access and the data that you were able to access? How did you respond to challenges you encountered in accessing various data sources? What have you learned from your experience accessing new data sources?
MOU/DUA in place	Have any new MOU/DUAs been added since you completed the most recent SAR? With which organizations? Were existing MOU/DUAs altered to reflect any additional data needs as a result of this project? Were there any confidentiality or other concerns about using the shared data for the purposes of this project? How did you overcome these concerns? What strategies did you employ to achieve MOUs with other organizations/agencies? What were some of the greatest challenges you faced developing MOUs? What have you learned from your experience obtaining MOUs? Do you have plans or a vision of maintaining the MOUs or relationships to continue to share data in the future either to continue this work, or to study other populations or programs?
MOU/DUA in progress
Analysis of administrative data	Who took the lead on developing the data analysis plan? How does your data analysis plan differ for each target population? How did your proposed data analysis plan change from your proposal to now? Did the TA provided or other TA sought influence your analysis plan? How? Describe any primary data collection from youth (such as surveys or focus groups) that you will or have engaged in for the data analysis. What analysis have you completed to date? What have you learned about the target populations from the data analysis thus far? Did anything surprise you? What did you learn from the youth surveys or focus groups that you did not or could not have learned through the administrative data? How are you using the findings of your data analysis to inform the development of your intervention? Your evaluation plan? Your partnerships? What challenges did you encounter that delayed or impacted your data analysis plan? What strategies did you employ to overcome these challenges? What have you learned in developing the analysis plan that would be helpful for similar organizations to consider if they were to do something similar? Would you consider using a similar data approach to plan other types of interventions of interventions for other populations?
Integrating individual-level or case/family-level data
Analysis of integrated data
Access to youth surveys or focus groups
Analysis of youth data
Identifying new insights and/or providing support for existing ideas
Documenting starting definition of at-risk	How did you develop your initial definition of at-risk-youth (i.e. looking to previous research, past data analysis, USICH framework, etc.)? How have your data analysis results altered your definition for the at-risk population? What else has changed your definition for the at-risk population? How, if at all, has the offered TA influenced your definitions? How is your definition for at-risk youth different for the three target populations?
Refining definition of at-risk
Developing rationale for changes in definition
Identification of at-risk youth
Assessing current services	Describe the ongoing process used throughout the planning phase to continue to identify the current array and gaps in services. How does your assessment of available services and gaps in services differ across the target populations? How have planning team members and partner agencies been engaged in the identification of available and missing services? How have planning team members and partner agencies reacted to the available and missing services? How has the information on available and missing services informed the development of the intervention? How have youth contributed to the identification of available and missing services in the community? What have you learned from the process of assessing services and gaps that you didn’t know beforehand? What lessons would be useful for other similar communities to keep in mind?
Conducting gap analysis
Documenting partner reaction
Identifying evidence-based or evidence-informed interventions	Describe your process for identifying evidence-based or evidence informed interventions. How has this approach changed throughout the project? What planning team members or partner agencies were responsible for identifying evidence-based or evidence-informed interventions? To what extent are evidence-based or evidence-informed interventions currently available to youth in the three target populations? How were these interventions identified? How have these interventions changed over time? Please describe any plans to alter currently existing evidence based programs in the community as a result of this grant. What led to this decision? Please describe the evidence based programs you plan to implement. How if at all has the planning period influenced these decisions? Have any adaptations or modifications been suggested for evidence based programs that will be implemented in order to better address the unique needs of the target populations? Have you sought input from the developers of the evidence-based interventions in selecting the interventions or adapting them? Why and to what benefit? What will be some challenges to implementing new or modified evidence based programs in the community? What are some strategies you will employ to overcome these challenges? How will you employ planning team members and partner agencies in the modification or implementation of evidence based programs? How if at all will you employ program developers? How have planning team members and partner agencies responded to the proposed services? How have youth responded? Have different evidence based programs been identified for the different target populations? Please describe the referral process for attaching youth to evidence based programs in the community. Does this process differ for the 3 target populations? How does the referral process outlined now differ from the process proposed in the grant? Are you experiencing any challenges in developing the referral procedures? What strategies are you using to overcome these challenges? Please describe the screening and assessment process for determining the best resources and programs for youth. Are you using different screening and assessment tools for the different populations? How have the tools you proposed in the grant application changed over time? Have you experienced any challenges accessing specific screening and assessment tools? How have you approached training program staff on new screening and assessment tools? How has your choice in screening and assessment tools been influenced by your data analysis findings? What have you learned from your experiences identifying, adapting, and selecting evidence-based programs?
Continuing current evidence-based or evidence-informed interventions
Adapting or modifying the evidence-based or evidence-informed services selected by your community.
Referral and Service Priorities
Identifying screening and assessment tools
Documenting theory of change – partnership	Please describe your project’s proposed theory of change at the partnership level. Did your project use any resources in the development of their theories of change (USICH framework, etc.)? How has this theory of change been modified over time? What has motivated modifications to the theory of change? Have there been challenges to adhering to the theory of change? Who was involved in the development of the theory of change? Please describe your project’s proposed theory of change for service delivery How has this theory of change been modified over time? What has motivated modifications to this theory of change? Have there been challenges to adhering to the theory of change? Who was involved in the development of this theory of change? Please describe your project’s proposed logic model. How has the logic model changed over time? Has it changed since the latest SAR? (If so, request a copy). Who was involved in developing the logic model? How were the major outcomes for the planning period identified? Who was involved in this decision? To what extent would the planning team be able to implement the logic model without federal support?
Documenting theory of change – service delivery
Developing a logic model for the service delivery
Describing intended rigorous evaluation design	Who is the lead agency for the evaluation? How have other planning team members and partner agencies been involved in the evaluation planning? What were their main contributions? Was an initial evaluation plan proposed in the grant application? If so, how has this proposed plan changed over time? Please describe the evaluation design. What were some of the tradeoff’s discussed when determining which evaluation design would be used? How did partner agencies and other planning team members respond to the proposed evaluation design? What are some of the anticipated challenges you will face in implementing the evaluation design in the future? What are some strategies you propose to overcome these challenges? To what extent will you be able to complete the evaluation if you do not have funding in Phase II? How will you determine which youth receive intervention services and which youth will make up a comparison group? What services will be provided to the intervention group and what services will be provided to the comparison group? To what extent will there be a feedback loop during the evaluation to make modifications to the enrollment plans or service delivery plans? What outcomes have been identified for the intervention group? What measures are being used to assess impacts? How have the outcomes and measures changed over the life of the project? Will your project be able to measure all proposed impacts without Phase II funding? Are different evaluation plans, services, outcomes, and measures proposed for the 3 target populations? Have you determined the necessary sample size for each analysis? How was this sample size determined? Please describe the recruitment and enrollment process proposed in your grant application, if any. How has the recruitment and enrollment process been modified throughout the life of the project? What are the anticipated challenges to recruitment and enrollment for the evaluation? What are some strategies you will employ to overcome these challenges? Please describe the consent process, if you used one, outlined in your grant proposal. How has this process been modified or developed over the course of the project? What have led to these changes? What challenges do your foresee with the consent process? How will you overcome these challenges?
Describing intervention services to be evaluated
Describing the comparison services–to what will the new services be compared?
Describing the youth/young adults who will be the target of the intervention
Assessing partner support for evaluation design
Selecting outcomes for the evaluation
Determining target sample size
Developing recruitment and enrollment processes
Developing consent process	Describe the consent process your evaluation may use to gain informed consent and assent to participate in the evaluation study. This may involve consent from biological parents or guardians for youth under 18, assent for youth under18, and consent from young adults over 18.