Attachment D. CCBHC Leadership Interview Protocol

TELEPHONE INTERVIEW QUESTIONS – CCBHC LEADERSHIP INTERVIEWS

In year three of the evaluation, we will conduct telephone interviews with CCBHC leaders at up to 15 clinics. We plan to conduct joint interviews with the CEO, medical director, and any other key leadership staff recommended by the CCBHC at each clinic. These interviews will gather information on clinics’ implementation experiences with the model. We will tailor protocols to reflect information about the demonstration in that state and what we know about the CCBHC from the survey fielded in the second evaluation year. The transcripts will be professionally transcribed and coded. We will probe on barriers and facilitators to implementing various aspects of the CCBHC model throughout the interview as appropriate. We have identified by name and color-coded questions or parts of questions below that are specific to new states (blue text) and original states (red text).

1. Introduction and Warm-up

   1. Interview introduction

      1. Obtain verbal consent to record

   2. We would like to learn about your current role(s)/position(s).

      1. For how long have you held this position?

      2. What are your key responsibilities related to the CCBHC demonstration?

      3. [Kentucky and Michigan] If you were at this agency prior to CCBHC certification, has your role changed with CCBHC certification? Please describe if so.

   3. How has implementation of the CCBHC model changed the way your organization provides care?

4. What are some differences in the way your CCBHC is administered versus other clinics in your region?

5. What have been some of the key successes your clinic has had implementing the CCBHC model? What about key challenges?

6. Have any of the demonstration requirements been particularly challenging to meet? Why?

7. Does your CCBHC have a process in place to monitor its ongoing compliance with the CCBHC certification criteria? Please describe if so.

8. Does your state monitor your CCBHC’s ongoing compliance with the CCBHC criteria? How if so?

2. Access to Care

   1. [Original states] Has your clinic generally been able to maintain the activities to increase access to care that were added as a result of participating in the demonstration?

   2. What new activities has your clinic engaged in to increase access to care as a result of the demonstration?

      1. Have these changes led to increased access as expected? Why or why not?

   3. What are some of your main outreach strategies to attract new CCBHC clients? (For example, media, word of mouth?) What about to keep existing clients engaged in care? Are any of these strategies more recently implemented as a result of the demonstration?

   4. Which populations, if any, are you specifically targeting in your outreach? Why?

      1. [For each population mentioned] How is your clinic targeting these populations?

         1. How do these activities differ from outreach strategies for different populations, if at all?

   5. Have you used any technology to extend your clinic’s reach?

   6. What services and supports does your clinic offer for children, adolescents, and their families? Please describe these activities.

      1. What kinds of outreach does your clinic conduct to reach children, adolescents, and their families?

      2. About what percentage of your client population do children and adolescents comprise? Has this changed as a result of the demonstration?

      3. How do you adapt services and care to this population?

      4. How are families of children involved in their treatment planning and care?

      5. How do you assess whether your clinic is meeting the needs of children, adolescents, and families?

   7. How does your CCBHC incorporate clients into treatment planning?

   8. In what other locations, if any, are CCBHC services offered? How did you select these other locations? Did you offer care in these other locations prior to the demonstration?

   9. What challenges have you encountered in increasing access to care under the demonstration?

      1. How is your clinic working to overcome these challenges?

   10. How do your CCBHC’s activities to increase access compare to non-CCBHC behavioral health clinics in your area?

   11. Which of your clinics’ activities to improve access have clients responded most favorably to? Which have clients not taken advantage of?

   12. [Kentucky and Michigan] What activities did your clinic implement in response to the findings from the needs assessment that was conducted when the state applied for the demonstration?

       1. Have there been activities that you planned to implement but were unable to?

          1. Why were you unable to implement these activities?

   13. Has the clinic conducted any formal needs assessments since the demonstration began? If yes:

       1. How frequently has your CCBHC conducted needs assessments?

       2. What data source did your clinic use for the needs assessment?

       3. How have you addressed findings from your needs assessment related to access, if at all?

       4. Has your CCBHC added any services or evidence-based practices as a result of these needs assessments?

   14. Could you please reflect on how the CCBHC model affects clients’ use of behavioral health services? What about their use of physical health care?

       1. Have utilization patterns changed? Are clients receiving any new services? Are they receiving services in a different way? Are they visiting the clinic or receiving care more or less frequently than before the demonstration?

       2. Have referral patterns changed?

       3. Do CCBHC clients continue to receive services elsewhere in the community from other behavioral health providers?

          1. [If yes]: What types of services? Why?

   15. The CCBHC criteria specify the timeframes that clinical assessments and first clinical appointments must occur within. Has your clinic experienced any challenges in meeting these timeframes?

       1. Have you heard any feedback from clients regarding the timeliness of these initial contacts?

       2. How do these timeframes compare to what your clinic was able to offer before the demonstration?

       3. What changes did your clinic need to make to meet this component of the criteria?

   16. How has COVID-19 affected access to required demonstration services at your clinic?

       1. What did your clinic do to maintain access during the pandemic?

       2. Have these activities continued?

3. Scope of Services

   1. [Kentucky and Michigan] What services and evidence-based practices did your clinic add to meet certification requirements?

      1. Were any services particularly challenging to add initially? Why?

         1. What strategies helped you to overcome these challenges? Did the state provide any support or guidance?

   2. How do you feel about the service requirements that the state has established?

      1. Are they applicable to your clinic’s client populations?

      2. Do the requirements include any services that your clients are not using?

      3. What small or big changes to the scope of services required by your state would you find helpful?

      4. How is the provision of evidence-based practices going?

   3. Has your clinic been able to maintain the required services over time?

      1. What has helped you maintain the services?

      2. What challenges have you encountered in providing particular types of services?

         1. How have you tried to overcome these challenges?

      3. What challenges have you encountered in providing services to particular target populations?

         1. How have you tried to overcome these challenges?

      4. How has the COVID-19 pandemic affected delivery of the required services?

         1. Did the state provide any support or guidance during the pandemic?

         2. [If not discussed yet] Could you please talk about the clinic’s use of telehealth?

• To what extent were you using it before the pandemic?

• What technological investments did you need to make either before or during the pandemic to provide telehealth, if any?

• Were all services provided via telehealth or only some during the pandemic? Why?

4. Could you reflect on how your formal and informal relationships with external partners are going?

   1. Did your clinic establish relationships with DCOs? Why or why not?

   2. How do you decide whether to establish a formal vs. informal relationship with a partner?

   3. Have you needed to add or discontinue any partnerships over the course of the demonstration (or for DCOs, change the services they provide)? Why?

5. How is your CCBHC integrating the crisis services it provides with existing or new crisis service programs in your community?

   1. How does your CCBHC support or interact with the 988 crisis line?

6. Could you briefly describe the types of care management your staff provide as a result of the demonstration?

   1. How, if at all, have you changed your care management activities [original states: over time]?

      1. What were the reasons for these changes?

7. Could you briefly describe the types of coordination activities your CCBHC engages in with external organizations?

   1. How is your referral process to other healthcare providers going? What about to non-healthcare providers or supports?

   2. How, if at all, has your clinic changed its coordination activities [original states: over time]?

      1. What were the reasons for these changes?

[Probe on collaborating and sharing information with the following example entities]:

• Hospitals

• EDs

• Child/adolescent service providers

• Schools

• External behavioral health providers

8. How, if at all, have your EHR/HIT systems helped your clinic to deliver higher-quality care coordination? What have been some of the challenges in using your EHR/HIT systems for this work?

   1. Does your clinic exchange information with other providers electronically? Please describe if so.

   2. Is your clinic part of a health information exchange in your state? Has this been helpful if so?

   3. Are you alerted if your client is seen by another provider and how?

   4. To what extent are non-technological workflows being used?

   5. What other kinds of technology is the clinic using to increase access to or coordination of care?

   6. What information and/or services are available to CCBHC clients through a secure electronic system/patient portal? (e.g., health information, clinical visit summaries, treatment plan, two-way communication with the practice, emails to notify clients about needs)? Does the CCBHC have an interactive website or patient portal to support CCBHC client access?

   7. To what extent did you have these technologies and workflows in place prior to the demonstration? Which were added because of the demonstration?

   8. How has your CCBHC financed its EHR/HIT systems or changes to its EHR/HIT systems?

9. How about internal coordination activities? How does your clinic staff collaborate to monitor and address clients’ behavioral health conditions and their treatment?

   1. How do providers share information with one another?

   2. Who is responsible for coordinating clients’ care?

10. [Kentucky and Michigan] What type of primary care screening and monitoring activities has your CCBHC implemented?

    1. How did your clinic change its practices and workflows to integrate primary care screening and monitoring?

11. How, if at all, has your CCBHC changed its primary care screening and monitoring activities [original states: over time]?

    1. What were the reasons for these changes?

12. How do CCBHC clients access primary care? What arrangements, if any, help to facilitate clients’ access to primary care?

    1. Do clients have access to on-site primary care? What about access through partner organizations? How is this going?

13. How does the scope of services provided by your clinic compare to non-CCHBCs in your area?

4. Quality of Care

   1. Reflecting on the CCBHC model overall, which aspects of it do you think most improve the quality of behavioral health care (compared to standard practice)? Why?

      1. Which CCBHC requirements do not significantly improve the quality of care? Why? (For example, because they are too burdensome, not well-received, etc.)

   2. How has the COVID-19 pandemic affected the quality of care your clinic provides, particularly related to the required demonstration services?

   3. How do you monitor the performance of your CCBHC?

   4. Could you reflect on your experience reporting the CCBHC quality measures?

      1. What are some of the challenges you have encountered?

      2. How have the state or other groups supported you, if at all?

   5. How has your clinic used performance results on the quality measures to improve care [original states: recently]?

   6. Does your state share performance on the demonstration quality measures with you?

      1. What specifically does your state share? For example, does it share your clinic’s performance only or does it share performance of other CCBHCs in your state?

      2. How frequently does the state share quality measure performance with your clinic?

      3. Have you made any changes to the way your clinic implements the model as a result of the information provided by the state?

   7. Does your CCBHC’s EHR generate the quality measures required for the demonstration?

      1. [If yes] Are the quality measures generated by the EHR easily and quickly accessible to your CCBHC?

         1. Which staff at your CCBHC can access them? For example, are they only available to your quality management staff or are they available to all staff?

         2. Could you describe where they are located or embedded within your EHR?

      2. [If no] Why not?

8. Has your clinic been eligible to receive quality bonus payments for meeting certain quality measure thresholds under the demonstration?

   1. [If yes] Have you found that the quality bonus payment system offers an incentive to provide improved quality care? Why or why not?

      1. What specifically is it about the quality bonus payment system that incentivizes higher quality? For example, is it the amount of the payment? The measures the state is using? The ability to benchmark your clinic to others in the state?

9. Thinking about EHR/HIT more broadly now--how, if at all, has your EHR/HIT system helped your clinic to deliver higher-quality care?

   1. Are there any other recent challenges you would like to share or lessons learned?

5. Impacts on Costs

   1. How did the PPS system support your clinic’s efforts to improve access, quality of care, and scope of services, if at all? Are the impacts from net increases in reimbursement, reductions in administrative burden, or coverage of previously uncovered services?

      1. How does your experience being paid under a PPS compare to your experience being paid before the demonstration or at other organizations you have worked for?

      2. Has the PPS facilitated any specific innovations? For example, innovative ways of service provision, use of technology?

   2. Have you encountered any difficulties reporting costs via the cost reports?

   3. To what extent do the payment rates cover the costs of your Medicaid clients?

      1. Have the requirements to operate as a CCBHC increased the costs of covering other clients, such as the uninsured?

         1. How do you cover these increased costs? What additional funding sources do you rely on? What strategies have you employed? For example, do staff help uninsured clients sign up for health insurance?

4. Has your clinic instituted any billing process improvements over the last few years?

   1. Has the state provided any billing process-related technical assistance to your CCBHC? Please describe if so.

5. [PPS-2 states]: Are there any patient populations that are difficult to cover under the PPS? Does the PPS not fully cover the costs for some populations?

6. How did the COVID-19 pandemic affect costs?

   1. Did it affect operation costs?

   2. Did it affect costs of specific types of required CCBHC services?

   3. Did it affect staffing costs?

6. Crosscutting

   1. What changes did your clinic make to your governance structure [KY/MI: as a result of certification/original states: since the end of the second year of the demonstration]?

      1. To what extent do you include clients and families in oversight and performance review?

   2. How does your clinic solicit feedback from clients and families, both formally and informally? For example, does the CCBHC collect data related to satisfaction beyond the experience of care surveys?

      1. What information is collected?

      2. How does your clinic use the information collected?

   3. Has your clinic been successful in retaining staff needed to implement the CCBHC model during the demonstration?

      1. [If yes]: What has helped your clinic to retain staff?

      2. [If no]: What challenges has your clinic faced in retaining the required staffing? For example, have certain staff types been more difficult to retain than others?

         1. What caused these challenges?

         2. How has your clinic and the state addressed these challenges?

   4. What other state or local policy initiatives, if any, are affecting your CCBHC currently? How so? Do they align well with the goals and implementation of the CCBHC model or do they present any challenges?

   5. How has the state supported your implementation of the model?

      1. Do you feel like you have received enough support from the state for implementing the CCBHC model? Why or why not?

      2. What additional trainings or support would be helpful?

   6. Do you participate in any learning collaboratives or peer learning/TA opportunities?

   7. What are you hearing from the state in terms of ways it plans to sustain the model (or not)?

      1. Are you planning to sustain the model for this clinic if the demonstration ends? If so, how? Would you continue to provide all of the services you are providing under the demonstration and retain all types of staff? Why?

7. Interview Closing

   1. What have we missed? What else do we need to know that we haven’t asked you?

      1. What are you most proud of with your CCBHC work?

      2. Are there any creative innovations you’d like to briefly highlight?