Form Grantee-Level Inte Grantee-Level Inte Grantee-Level Interview

March 2017

PROGRAM EVALUATION FOR
PREVENTION: STRATEGIC
PREVENTION FRAMEWORK FOR
PRESCRIPTION DRUGS
State/Tribal Grantee Interview Protocol
This instrument is not OMB approved and is provided for informational purposes
only while the instrument is under OMB review. The final items are subject to
change as part of the OMB approval process.

Contents
Section

Page

Overview and Purpose

1

Preparation for Interviews

1

Interviews

1

Postinterview Activities

1

Interview Guide for State/Tribal Grantees

3

1.

Introduction ............................................................................................... 3

2.

Interview Questions..................................................................................... 4
2.1

Infrastructure and Capacity ................................................................ 4

2.2

Prescription Drug Monitoring Program Data .......................................... 4

2.3

Collaboration .................................................................................... 7

2.4

Leveraging Funds and Resources ......................................................... 8

2.5

Interventions .................................................................................... 9

2.6

Monitoring, Performance Measurement, and Evaluation........................ 10

2.7

Behavioral Health Disparities and CLAS Standards ............................... 11

2.8

Concluding Questions ...................................................................... 12

Closing

March 7, 2017

12

ii

OVERVIEW AND PURPOSE
The PEP-C team will conduct initial telephone interviews with each SPF-Rx grantee Project
Director or designated staff in fall 2017. Follow-up interviews will be conducted using a
similar protocol in Years 3 and 5 of the project. The primary purpose of the interviews is to
obtain the perspective of the implementing Project Directors or their staff on several
important topics, including infrastructure and capacity, collaboration, leveraging funding and
resources, selection and implementation of interventions, monitoring and evaluation, and
health disparities. This information is critical for the SPF-Rx cross-site evaluation reports
and understanding of how SPF-Rx is being implemented around the country. These
semistructured interviews will last approximately 90 minutes and will proceed using the
included guide.

PREPARATION FOR INTERVIEWS
Before each interview:


The interviewer will arrange and confirm the interview time and telephone number
with each grantee Project Director or designated staff using an email template
approved by SAMHSA and will obtain written (email) consent to record the interview.



The interviewer will review grantee-specific information, including quarterly reports,
proposal information, annual implementation instrument, and other documents as
applicable.

INTERVIEWS
Data collection will consist of individual telephone interviews with the Project Director or
designated staff for the state-level (or tribal-level) SPF-Rx Program.
The interviewer will document the interviews through notes taken on the interview forms in
this document. The files containing the completed interview forms will be placed in the
relevant grantee’s folder under the Data Collection folder on SharePoint (see [LINK TBD]).
With participant consent, the interviews may be audio-recorded to facilitate reporting after
the interview.

POSTINTERVIEW ACTIVITIES
Within a week after each interview, the interviewer will email the Project Director or
participating staff member to thank him or her for taking the time to be interviewed. The
interviewer will follow up by telephone within 2 weeks for any outstanding information or
document requests (e.g., contact information, plans, reports, local analyses).

March 7, 2017

1

Program Evaluation for Prevention: Strategic Prevention Framework for Prescription Drugs

The interviewer will complete the interview process by:

2



Filing the completed interview forms (see remaining sections of this document);



Providing a draft copy of the interview notes to the interviewee for review;



Obtaining any feedback from the interviewee and making any additions to the notes;
and



Filing any written materials obtained from the grantee.

March 7, 2017

Grantee Interview Protocol

INTERVIEW GUIDE FOR
STATE/TRIBAL GRANTEES
1.

Introduction

Hello, my name is ______________ and I work for the Program Evaluation for Prevention
Contract (PEP-C). PEP-C is part of the team conducting an evaluation of the Strategic
Prevention Framework for Prescription Drugs (SPF-Rx), which is sponsored by SAMHSA’s
Center for Substance Abuse Prevention (CSAP). The overall goal of the cross-site evaluation
is to document and assess the effectiveness of SPF-Rx in preventing prescription drug
misuse. We also want to gain insights to improve SPF-Rx and future efforts to reduce
prescription drug misuse.
This interview is one of a series of interviews that will be conducted at the beginning of the
second year of your SPF-Rx grant and during the third and fifth years of the grant. You are
one of 25 grantees who will participate in this study. We are contacting grantees to get your
unique perspectives on several important topics related to the implementation of SPF-Rx.
The interview will help the PEP-C team understand your approach to planning and
implementing SPF-Rx and the progress of your SPF-Rx efforts in preventing prescription
drug misuse and abuse in your (state/tribe/jurisdiction). Through these interviews, we want
to understand what kinds of challenges and barriers grantees are experiencing. We also
want to hear about success stories and lessons that could help other grantees. If you or
your subrecipients have developed any innovative approaches, please share them.
Information gathered during these interviews will be summarized in the SPF-Rx cross-site
evaluation reports. Selected information from this interview may also be used for brief
summaries of each grantee’s program as a resource for SAMHSA project officers. As part of
the SPF-Rx grant, SAMHSA requires grantees’ participation in the cross-site evaluation,
including this interview. Therefore, data are not confidential or anonymous. We will provide
you with a copy of the interview notes to review for accuracy.
All grantees will be asked the same questions. As you answer the questions, try to focus on
the role and contributions of your SPF-Rx strategies rather than on activities your state (or
tribe) was doing before the grant. What is the value added of SPF-Rx?
This interview will take about 90 minutes of your time. In addition, we may contact you
after this interview if necessary to clarify responses or to obtain outstanding information.
In preparation for this interview, you responded positively to an email requesting your
permission for audio recording. We are audio-recording this interview as a backup to our
written/typed notes. This recording is only for the use of our evaluation team and will be
destroyed after interview notes are finalized. Is it OK if I begin recording now?

March 7, 2017

3

Program Evaluation for Prevention: Strategic Prevention Framework for Prescription Drugs

Thank you for agreeing to participate in this interview. Do you have any questions for me
before we begin?

2.

Interview Questions

2.1

Infrastructure and Capacity

The PEP-C team and CSAP want to understand the many ways in which the SPF-Rx program
may affect prevention capacity and infrastructure at the grantee and community levels. You
can think of infrastructure as the foundation that supports a system, community, or society
in achieving desired outcomes. For SPF-Rx grantees, key infrastructure components include
groups such as an advisory body, epidemiological outcomes workgroup (EOW), and
evidence-based practice workgroup (EBPW). Infrastructure can also include collaborative
relationships, workforce development policies, and monitoring and evaluation systems. The
following questions are intended to help us understand processes affecting your capacity
and infrastructure for preventing prescription drug misuse.

Organizational infrastructure
1. [In the first year of the grant/Now that you are about mid-way through the grant/Now
that you’re near the end of the grant], how has the SPF-Rx initiative contributed to your
state’s (or tribe’s) planning and decision-making process for prevention of prescription
drug misuse? This process might involve your Advisory Council, EOW, and EBPW.
2.

How has the operation of your Advisory Council shifted to address SPF-Rx priorities?

3. How has the EOW contributed to your SPF-Rx efforts?
4. How has the EBPW contributed to your SPF-Rx efforts?

2.2

Prescription Drug Monitoring Program Data

Now I’d like to ask you a few questions related to your use of data from the prescription
drug monitoring program, also called PDMP.
5.

[BASELINE ONLY] Have you been able to receive any PDMP data or PDMP reports yet to
use for planning or implementing your SPF-Rx effort at the state (or tribal) level?
Yes

No

[IF YES] How do you receive the data?
Do you receive summary statistics from the PDMP office?
Yes

No

Do you receive a deidentified data file that you can analyze?
Yes

No

[IF NO] Please describe the challenges or barriers to accessing any PDMP data for SPFRx prevention work. [SKIP TO QUESTION 7]

4

March 7, 2017

Grantee Interview Protocol

6. [BASELINE ONLY] Were you able to use PDMP data to help make decisions about
selection and funding of SPF-Rx community subrecipients?
Yes

No

[IF YES] Please describe how you used PDMP data for decision-making about selecting
and funding community subrecipients.
[IF NO] Please describe the challenges or barriers to using PDMP data for subrecipient
selection.
[EVERY INTERVIEW]
7.

[BASELINE] Since the beginning of your SPF-Rx grant… [OR, FOR FOLLOW-UP
INTERVIEWS] Since our last interview in (YEAR), what has changed in your capacity,
understanding, or ability to use PDMP data for prevention?
[PROBES]
a. Have there been any significant changes in the quality of the PDMP data?
[IF YES] Please summarize the changes.
b. Have there been any changes in your access or capacity to use PDMP for your
prevention efforts?
For example, this could relate to using PDMP data in new ways to monitor
prescribing trends, or making more use of unsolicited reports.
[IF YES]

8.



In what ways has your capacity improved at the grantee level?



How were you able to accomplish these improvements in capacity for using
PDMP data?

How has your use of PDMP data as part of the SPF-Rx effort affected your prevention
efforts at the state (or tribal) level?

March 7, 2017

5

Program Evaluation for Prevention: Strategic Prevention Framework for Prescription Drugs

9.

How [are you planning to/have you been] facilitating your community subrecipients’ use
of PDMP data reports for prevention?
[PROBES]
a. Are you encountering any major challenges or barriers to using PDMP data reports
at the subrecipient community level?
Yes

No

[IF YES] What are the major challenges or barriers?
[BASELINE PROBE]
Are privacy concerns a key barrier to using PDMP data reports at the
community level?
Yes

No

b. How are you able to address the main challenges or barriers to using PDMP data
reports at the community level?
[PROBE – SECOND AND FINAL INTERVIEWS]
c. Since the start of SPF-Rx, how has capacity improved to use PDMP data at the
community level?
10. [SECOND AND FINAL INTERVIEWS] Do you have any lessons to share for other grantees
or communities related to building capacity to use PDMP data for prevention?
11. As part of SPF-Rx, have you been able to do any workforce development for your or
your subrecipients’ prevention staff related to PDMP use?
Yes

No

[IF YES] Please tell me about some of your workforce development efforts at the
grantee level and for community subrecipients.
[IF NO] What are the main challenges or barriers?

6

March 7, 2017

Grantee Interview Protocol

2.3

Collaboration

Now I would like to find out more about the role of your collaborations with other
organizations in implementing SPF-Rx.
12. At the grantee level, what progress are you making in your goals in working with the
medical and pharmaceutical communities in terms of policy or practice?
[PROBES]
Please talk about your SPF-Rx…
a. …progress in having health care providers use PDMP data when prescribing opioids.
b. …progress in safe prescribing (adhering to CDC guidelines for prescribing opioids
for chronic pain).
13. At the grantee level, what have been factors for success in collaborating with the
medical and pharmaceutical communities to achieve your SPF-Rx goals?
[PROBE]
Do you have any other promising approaches or innovations to share that might be
helpful to other grantees?
[IF R. ASKS WHAT WE MEAN BY INNOVATIONS, WE MEAN PROMISING NEW,
ENHANCED OR ADAPTIVE APPROACHES IN YOUR STATE]
14. At the grantee level, what have been challenges or barriers to successful
collaboration with the medical and pharmaceutical communities to achieve your
goals?
15. Now, turning to your subrecipient communities, please describe some of the
successes and challenges or barriers your communities have experienced in
collaborating with the medical and pharmaceutical communities to address
prescription drug misuse.
[PROBES]
a. For subrecipients, what have been major factors for successful collaboration with
the medical or pharmaceutical communities?
[PROBE]
Have you observed any promising approaches that might be helpful for other
communities?
b. For subrecipients, what have been major challenges or barriers to successful
collaboration with the medical or pharmaceutical communities to achieve their
goals?

March 7, 2017

7

Program Evaluation for Prevention: Strategic Prevention Framework for Prescription Drugs

16.

Last, I would like to ask you about your collaboration with other state agencies and
partner organizations to achieve SPF-Rx goals.
a. First, what are some of the ways that you have been successful in advancing
collaboration?
[PROBE] What has been helpful to your progress?
b. What are some of the challenges or barriers you have been experiencing?

17.

Finally, what highlights or themes can you identify regarding any prominent
successes or challenges your subrecipients are encountering in their collaborative
partnerships?
a. What are some of the key factors for success or examples of successes that you
have observed so far?
b. What are biggest challenges that you hear about?

2.4

Leveraging Funds and Resources

Leveraging resources describes the process of combining SPF-Rx-funded resources with
other resources, including non-SAMHSA funds, to enhance prevention strategy delivery (i.e.,
to do more together than with SPF-Rx resources alone).
Neither of these definitions includes using SPF-Rx funds to “free up” previously programmed
funds for other uses, as this is not permitted under the grant.
18. [BASELINE AND FINAL] What strategies did your jurisdiction employ for leveraging
prevention funds and resources (e.g., staff) in support of your SPF-Rx efforts?
IF NEEDED, EXAMPLES YOU CAN SHARE WITH THE GRANTEE INCLUDE…


Building off other prevention funding streams like CDC Prevention for States



Conducting joint trainings with other agencies on PDMP use and access



Jointly implementing (or coordinating) interventions with other agencies



Engaging prevention providers or coordinators from other funding sources



Collaborating with other agencies to develop reports using PDMP data

19. [BASELINE AND FINAL] What strategies did your jurisdiction employ for redirecting or
realigning prevention funds and resources in support of your SPF-Rx efforts? Redirecting
or realigning resources describes the process of shifting resources (fiscal and nonfiscal)
from one focal area to a focal area that complements SPF-Rx efforts.
IF NEEDED, EXAMPLES YOU CAN SHARE WITH THE GRANTEE INCLUDE…


8

Redirecting training to high-priority areas for prescription drug (including opioid)
misuse or diversion

March 7, 2017

Grantee Interview Protocol



Realigning focus from solely individual services to also include population-focused
practices for prescription drug misuse or diversion

20. [SECOND AND FINAL INTERVIEWS] How has your jurisdiction encouraged the use of
local funds to supplement resources provided by SPF-Rx?
IF NEEDED, EXAMPLES YOU CAN SHARE WITH THE GRANTEE INCLUDE…

2.5



Providing training and technical assistance on obtaining local funds



Encouraging subrecipients to apply for locally available grant funding

Interventions

Now I’d like to ask you about your selection and implementation of interventions.
[BASELINE AND SECOND INTERVIEWS ONLY]
21. How are you selecting your SPF-Rx interventions?
[PROBES]
a. What was helpful to you in identifying promising or evidence-based interventions for
this grant?
b. What challenges have you faced in finding promising or evidence-based interventions
to meet the goals of the SPF-Rx grant to prevent prescription drug misuse?
[IF RESPONDENT ASKS: Criteria for a promising or evidence-based intervention
might be publication in a scientific journal, inclusion in government lists of evidencebased interventions, review by a panel of informed expertise and key community
leaders, and so on.]
22.

To what extent have you or your agency been involved with your community
subrecipients in identifying promising or evidence-based interventions, policies, or
practices for SPF-Rx?
[PROBE]
Please describe any resources or technical assistance available to assist subrecipients
in the selection, implementation, and adaptation of promising or evidence-based
approaches for the focal areas of SPF-Rx.

[EVERY INTERVIEW]
23.

How is SPF-Rx contributing to your progress in planning or implementing any legal
or policy changes to reduce prescription drug and opioid misuse? For example, legal or
policy changes could relate to PDMP use, opioid prescribing practices, or health care
provider training.
What is the value added of SPF-Rx beyond what the state/tribe was already doing?

March 7, 2017

9

Program Evaluation for Prevention: Strategic Prevention Framework for Prescription Drugs

[PROBES]
a. What factors have been most helpful to you in making progress in planning or
implementing policy or legal changes?
b. [SECOND AND FINAL INTERVIEWS] Do you have any other lessons to share that
might be helpful to other grantees in terms of policy or legal change?
24.

What is the unique contribution of SPF-Rx to your state’s (or tribe’s) social marketing
campaigns to prevent prescription drug and opioid misuse?
[PROBE – BASELINE AND SECOND INTERVIEWS]
How does your SPF-Rx social marketing campaign fit within your state’s overall social
marketing efforts to prevention prescription drug and opioid misuse?

2.6

Monitoring, Performance Measurement, and Evaluation [SKIP FOR
BASELINE INTERVIEW]

Now I’d like to turn briefly to the subject of performance measurement.
25.

Have you been using PDMP data to monitor the progress of your SPF-Rx efforts?
Yes

No [IF NO, SKIP a. and b.]

[PROBE FOR ALL] What challenges or barriers have you faced in trying to use PDMP
data for monitoring and performance measurement?
[IF YES]
a. How have you been using the PDMP data for monitoring and performance
measurement?
b. Were you able to combine PDMP data with other data, such as opioid overdose
morbidity or mortality data?
Yes

No

[IF YES, PROBE]:
•

Please discuss what capabilities you developed in this area. For example,
did you conduct geographic analysis at the aggregate level or merge
individual-level data for analysis?

•

How did you use these combined data?

[PROBE FOR SECOND AND FINAL INTERVIEWS]
c. As part of your SPF-Rx effort, has your (state/tribe) developed any promising
innovative practices using PDMP data for monitoring that might be helpful to other
grantees?

10

March 7, 2017

Grantee Interview Protocol

2.7
26.

Behavioral Health Disparities and CLAS Standards
[BASELINE ONLY] Which behavioral health disparities are you addressing in your
jurisdiction through the SPF-Rx?
[PROBES]


Gender?



Race/ethnicity?



Lesbian, gay, bisexual, transgender, questioning (LGBTQ)?



Rural/urban?



Military families?



Other?

27.

[BASELINE AND FINAL] Please describe your [plan/processes] for addressing
behavioral health disparities for SPF-Rx for your state/tribe/jurisdiction.

28.

[BASELINE ONLY] What has been your biggest challenge in developing your Disparity
Impact Statement (DIS)?
[PROBES]
a. What strategies have you used to overcome this challenge?
b. How has this challenge affected your implementation of behavioral health
disparities-related efforts at the subrecipient level?

29.

[BASELINE AND FINAL] What have been the major challenges in implementing your
DIS statement?
[PROBES]
a. What challenges have you experienced related to data to document and assess
behavioral health disparities?
b. What challenges have you experienced related to implementing activities to
address behavioral health disparities related to access to prevention services?
(This refers to effort that are being made to increase access to prevention
services.)
c. What challenges have you experienced related to implementing activities to
address behavioral health disparities related to use and reach of prevention
programs?

30.

[EVERY] How [will you ensure/are you ensuring] that strategies reach populations
affected by behavioral health disparities?

31.

[BASELINE AND FINAL] How are you [planning to support/supporting] subrecipients in
addressing behavioral health disparities?

March 7, 2017

11

Program Evaluation for Prevention: Strategic Prevention Framework for Prescription Drugs

32.

2.8

[BASELINE AND FINAL] How are you [planning to support/supporting] subrecipients in
adapting the CLAS standards? (By CLAS standards, I mean the National Standards for
Culturally and Linguistically Appropriate Services in Health and Health Care.)

Concluding Questions

Now, I have a couple of last questions to wrap up the interview.
33.

In your view, what is the most important thing that SPF-Rx is helping your state (or
tribe) to do above and beyond your other funding resources? What is the unique
contribution of SPF-Rx?
[PROBE – FINAL INTERVIEW ONLY]
What aspects of your SPF-Rx effort and infrastructure do you think will be most
sustainable after SPF-Rx funding ends?
[IF NEEDED, EXAMPLES YOU CAN SHARE WITH GRANTEE INCLUDE…]
• Regular access to PDMP data for prevention
• Workforce development (e.g., prevention staff, medical community)
• Changes in laws or policies
• Ongoing collaborations (e.g., other state agencies, local communities, medical
community)
[FINAL INTERVIEW ONLY]

34.

As you are nearing the end of your SPF-Rx grant, what are the most critical gaps that
remain in your state’s (or tribe’s) efforts to address prevention of prescription drug,
including opioid, misuse? What would be your next priorities?

CLOSING
Thank you for talking with me today. We appreciate your sharing your experiences.
If you have any questions or concerns about this interview, please contact the PEP-C Help
Desk through the Management Reporting Tool, by telephone message (at XXX-XXX-XXXX),
or by email (to email address). You can request assistance at any time and someone will
respond to you on the next business day.
Do you have any final questions or concerns today?
Thanks, again, for your time.

12

March 7, 2017