Combating Autism Act Initiative Evalutation - Training Programs - LEND

Leadership Education in Neurodevelopment Disabilities (LEND) interview protocol

LEND Interview Protocol

Thank you for taking the time to speak with me today. My name is [name], and I’m a researcher at Insight Policy Research. The purpose of this interview is to obtain some descriptive information about your LEND program’s activities and outcomes for the evaluation of MCHB’s Autism CARES (or Collaboration, Accountability, Research, Education, and Support) Act Initiative. I will be asking you questions about your activities and accomplishments related to the goals and objectives of Autism CARES. I’ll focus on a few key topic areas, including training, awareness building, research, and improving systems of care.

Throughout our discussion, I will also be asking you about activities related to supporting youth with autism spectrum disorder (ASD) during transition to adulthood because this is a new reporting requirement for MCHB.

Touch on bulleted topics as needed

• Review of additional data sources. We have already reviewed some quantitative and qualitative data on your program from other sources, including [name relevant documents]. The information you share today can supplement the other sources we have reviewed. You may be able to provide additional context or share new information.

[Note to interviewer: To the extent possible, try to focus the conversation on new information or supplemental details that are not adequately covered in the progress reports.]

• Evaluation results. The results from this evaluation will be presented in two separate documents. The first is a brief (< 10-page) fact sheet summarizing the major activities of LEND grantees. The second is a larger, more detailed report that will focus on all grant programs that received funding under Autism CARES.

• Evaluation period. This evaluation builds on the previous evaluation, which covered Autism CARES activities and accomplishments through roughly summer 2014. For each set of questions, I’d like you to report on activities or outcomes that have occurred since approximately fall 2014.

• ASD focus. I know your program works in many areas outside of ASD and other developmental disabilities (DDs). Since this evaluation focuses on Autism CARES funding, please focus on activities pertaining to ASD and DDs unless otherwise indicated. Similarly, if possible, please try to focus on activities supported by LEND funding.

• New grantees. I realize your program is new and some of the activities you hope to implement may be in the planning phase. It’s completely fine to discuss your early steps in these areas, or to tell me that you do not have anything to report in certain areas at this time.

[Note to interviewer: Throughout the discussion, you may need to adjust your language to reflect the current stage of new grantees’ work; for example, asking about plans and initial steps rather than completed accomplishments. Some questions are specifically intended for either new or continuing grantees].

Do I have your permission to record our conversation? The purpose of the recording is to be certain we accurately capture the information you share today. The recording will not be shared with anyone outside the Insight evaluation team.

[Confirm permission once recording starts.]

Training

For the first set of questions, I’d like to talk about the training your program provides.

[Interviewer may summarize or ask about the program’s specific training activities as described in progress reports, etc.]

1. [New grantees] Why did your institution decide to apply for a LEND grant? Was any similar training in place prior to the LEND? What are the main goals of your program?
   [Continuing grantees] For the current evaluation, we are primarily interested in activities that have occurred since fall 2014. What are the biggest changes to your program since that time? Have you made any substantial changes to your training program?

2. [If not already covered] Has the number of long-, medium-, or short-term trainees changed since 2014? Who are the trainees and what disciplines do they come from?

3. How would you describe the diversity of your faculty and trainees (racial and ethnic diversity, geographic diversity, disciplinary diversity)? How many family trainees and faculty do you have? How many self-advocates or people with disabilities?

[Note to interviewer: Some grantees may have people with disabilities in their programs who do not disclose their status, and many programs do not want to make them disclose.]

4. [New grantees] How have you incorporated content on ASD in your didactic curriculum?

[Probe] Valid and reliable screening and diagnostic tools? Interventions? Transition to adulthood?)

1. Do you have training opportunities that focus specifically on screening tools, diagnostic instruments, or interventions, particularly for ASD?

1. [Probe] Instruments covered? Type of training? Audience?

1. What clinical opportunities do you offer to trainees? What are the settings for clinical training (hospital-based, schools, other community settings)?

2. Are there other hands-on or observational training opportunities: other community-based opportunities, outside of strictly clinical settings; for example, observing services being provided to children with ASD or participating in a social skills group?

5. [Continuing grantees] How, if at all, has the curriculum changed (since fall 2014)?

1. Has the didactic curriculum been modified to include any new content, especially content specific to ASD?

[Probe] Valid and reliable screening and diagnostic tools? Interventions? Transition to adulthood?

2. Have you incorporated any new training opportunities that focus specifically on screening tools, diagnostic instruments, or interventions, particularly for ASD?

[Probe] Instruments covered? Type of training? Audience?

3. Have the clinical opportunities for trainees changed? Any new clinical opportunities in hospital-based settings, schools, or other community settings?

4. Have there been any other new hands-on or observational training opportunities: other community-based opportunities outside of strictly clinical settings; for example, observing services being provided to children with ASD or participating in a social skills group?

6. Do you collaborate with other LENDs or other organizations around training?

[Examples] Regional LEND collaborations, interstate LEND training, UCEDDs. What advantages do these collaborations provide?

7. What are some of the ways your program prepares trainees to become leaders in their field?

[Continuing grantees] Has this component of the program changed at all since 2014?

8. To what extent has your program engaged family members and self-advocates in the training process as faculty members, trainees, and in other capacities? What sort of training do you provide to family trainees and self-advocate trainees?

9. In general, where would you say your training efforts have had the greatest impact or been the most successful? What factors do you think facilitated their success?

1. What activities have been particularly effective in reaching the goal of early identification and intervention for ASD?

Increasing Awareness

Now I’d like to learn about ways your program may be helping to increase awareness and disseminate information about ASD/DDs to a broad range of audiences. This is in addition to the formal training you provide to trainees within your program. In the past, grantees have disseminated Learn the Signs. Act Early. materials to families, provided continuing education courses to various kinds of professionals, or disseminated findings from their research. You may have different examples.

10. Can you tell me about your program’s efforts to increase ASD/DD awareness?

[For each effort described, probe]

1. What types of audiences/professional fields were targeted (e.g., practicing pediatricians aiming to get the latest information on screening, rural health care providers, social workers, educators)?

2. Approximately how many people did you reach?

3. What topics/content areas did these efforts focus on (e.g., early signs and symptoms, evidence based screening tools, DSM 5 changes, transition issues and services)?

4. What was the primary communication strategy used (e.g., dissemination of print materials, Webinars, PSAs, conferences)?

11. [If not already covered] How, if at all, have your program’s awareness activities targeted or focused on—

1. Accommodating people with ASD/DDs in public spaces (e.g., Cincinnati’s SOAR program that trains airport employees to work with people with ASD/DDs)

2. Transition issues

12. What roles do trainees play in efforts to raise awareness about ASD/DDs?

13. In general, where do you think your awareness-building efforts have had the most impact or been the most successful?

[Probe] Demographics where message has been successful, particular events with good turnout and satisfaction, types of communication strategies that have worked well?

1. Have you seen any changes in practices or attitudes as a result of your awareness efforts? How are these changes measured?

Contributing to Research

Next I’d like to talk about your program’s research activities, particularly any activities related to ASD.

14. To what extent are your LEND faculty members or trainees currently involved in planning or conducting any research studies related to ASD/DDs?

15. Can you tell me about some of the most significant studies related to ASD/DDs that have been conducted by LEND faculty or trainees since fall 2014?

[For each study described, probe]

1. What is being investigated, and who is doing the research?

2. What is the anticipated impact of the results/how might the results be used (e.g., will they provide evidence about an intervention’s effectiveness)?

3. How is the research being funded? Does Autism CARES funding support these efforts?

16. [If not already discussed] How are trainees involved in research? In what capacity? What sort of training do they receive in conducting research and the research process?

17. With whom do you collaborate on research projects?

[Probe] Other departments in the university? Local hospitals? Schools? Other MCHB grantees such as CARES-funded research programs (Research networks, R40s)?

18. In general, where have your research activities had the most impact or been the most successful?

Reducing Barriers to Care, Improving Systems of Care

Next I’d like to talk about activities related to reducing barriers and improving systems of care. By systems of care, I mean the full continuum of care for individuals with ASD/DDs including screening, diagnosis, intervention, treatment, and transition, etc. We recognize there may be some overlap between this objective and the others, so feel free to call back to earlier examples if you like.

19. What are the most significant barriers to care in your [area/region/State/communities you serve]? What opportunities exist to address these barriers?

a. Who is best positioned to do the work? What organizations or individuals can best address these barriers?

b. What resources are needed?

c. Which of these barriers, if any, has your program specifically been working to address since receiving the Autism CARES grant/since the fall of 2014? What strategies are you using to address them?

20. What sort of work has your program done to improve the system of care for children with ASD/DDs?

[Potential follow-up questions]

1. Has your program provided technical assistance to other organizations, such as medical practices?

2. Has your program collaborated with other organizations, including State agencies, schools, family groups, Title V agencies, and community-based organizations to improve the system of care for children and families in your area?

3. Has your program worked with State officials to improve systems of care in your State or region?

4. Has your program worked with medical providers to improve the services they offer to children with ASD/DDs and their families?

5. Has your program developed resources/organizations/collaborations to assist children with ASD/DDs and their families through the transition to adulthood?

21. What changes have you seen, if any, in children’s access to care since fall 2014?

1. How are these changes tracked?

2. Is there any evidence of improved access to early screening and diagnostic services? Is your LEND providing additional diagnostic evaluations?

22. To what extent has your program reached children from underserved populations with respect to screening, diagnostic, intervention, or transition issues?

1. Have you developed or carried out any specific plans to reduce barriers to care among underserved children?

[Probe] Any preliminary results you may have observed from these activities?

2. Has your program developed/used innovative models of care, such as telehealth or tele-education to expand the reach to underserved populations?

23. [If needed] What (other) efforts has your program undertaken to build systems of care that improve capacity or are coordinated, comprehensive, and community-based?

1. What about efforts to enhance transition-related systems of care?

24. In general, where do you think your efforts to reduce barriers and improve systems of care have had the most impact or been the most successful? What efforts have been made, or what efforts do you think are needed, to ensure any advances achieved are sustainable?

National Resource Centers

The national resource centers are also being evaluated as part of this study. I have a few brief questions about your interactions with the Interdisciplinary Technical Assistance Center (ITAC), which is run by the Association of University Centers on Disabilities (AUCD). Please feel free to share any details you think are relevant. If you don’t have any information to share, that’s OK. Just let me know.

25. How have you interacted with ITAC (AUCD) since fall 2014? Have you attended any conferences they hosted (e.g., annual meeting, LEND directors meeting)? Do you receive emails from them? Have you participated in a Webinar or held other types of meetings with their staff?

26. What kinds of support, if any, has ITAC (AUCD) provided to your program under Autism CARES? How did this help you meet your objectives?

1. Did they provide technical assistance in any form?

2. Did they facilitate any collaboration with other grantees or organizations?

3. Did they disseminate or produce information that contributed to your success? For example, have you viewed any webinars or used anything from the training toolbox? Do you receive any newsletters from AUCD?

4. Did they help you address any specific challenges?

27. Can you think of any other examples of things ITAC (AUCD) is doing especially well? What about ways they could improve or better support your program?

Overarching Questions, Wrap-Up

Now I have a few overarching questions about your activities and some final questions about the broad impact of your program.

28. [If not already covered, check for adequate coverage of the “buzzwords”]: I want to make sure we cover a few key concepts important to MCHB. What can you tell me about your program’s main activities related to—

[Probe on any relevant items]

[Probe] What activities have been particularly successful? What factors made them successful?

1. Transitions

2. Collaboration with other MCHB grantees

3. Cultural and linguistic competency

4. Family-centered care

5. Interdisciplinary care and collaboration

6. Medical homes

7. Underserved populations (rural and minority)

29. Thinking about all the activities we’ve discussed today, where would you say you’ve faced the biggest challenges? What barriers have impeded your progress?

1. [Probe if needed] What have been your main challenges related to training? Awareness? Research? Systems of care? Transition-related services?

30. How, if at all, has your program evaluated the activities we’ve discussed today?

1. How have you evaluated your activities related to training? Awareness? Research? Systems of care?

2. Do you have any initial results to share from these evaluations?

31. What would you say are your main goals related to Autism CARES?

1. Are there any areas where you fell short of achieving your goals? If so, what challenges prevented you from meeting the goals? What would it take for you to meet these goals in the future?

32. To the best of your knowledge, how are your former trainees continuing to serve the ASD/DD population?

1. Are you aware of any former trainees serving in leadership roles in the community or professional practice?

2. Can you think of any other examples of the kinds of things trainees might do when they leave the program?

33. [For long-term grantees] For the most part, this evaluation is focusing on activities beginning with the fall of 2014. However, we’d also like to think about the long term. The passage of the initial Combating Autism Act in 2006 brought an increased focus on autism and related disabilities. Will you tell me a little about the long term effects of this legislation, from your perspective?

34. Is there anything else you would like to share about your program at this time?

35. Is there anything else you would like to add that I did not ask about?

1. Public Burden Statement:  An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number.  The OMB control number for this project is 0915-XXXX.  Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to HRSA Reports Clearance Officer, 5600 Fishers Lane, Room 10C-03I, Rockville, Maryland, 20857.

That concludes my questions for you. Thank you very much for speaking with me.