Form Director Survey Director Survey Director Survey

OMB No. 0930-xxxx

Expiration Date: xx/xx/xxxx

Assertive Adolescent &

Family Treatment (AAFT) Program

AAFT Implementation Survey

Principal Investigators

& Project Directors

Public reporting burden for this collection of information is estimated to average 45 minutes per response per year, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 7-1044, Rockville, Maryland, 20857.

Thank you for agreeing to participate in the

Program Evaluation for Assertive Adolescent & Family Treatment (AAFT) Program!

This survey is being conducted by Advocates for Human Potential, Inc. (AHP). We are a research and consulting firm based in Sudbury, MA and Albany, NY. We’re conducting this study as part of our contract to assist the Center for Substance Abuse Treatment (CSAT) in the national evaluation of the implementation of AAFT, funded by the Substance Abuse and Mental Health Services Administration (SAMHSA).

At the end of each project year, we are asking program administrators, clinical, and research staff across the entire AAFT3 grantee cohort to provide information about their professional background as well as opinions/thoughts on a variety of topics, including substance abuse treatment. We are gathering data from many sources and believe it is important to collect information from many perspectives as well. As you complete this survey, PLEASE RESPOND TO THE QUESTIONS FROM YOUR OWN PERSPECTIVE—choosing an answer that best describes your experience or opinion.

Your responses to the survey will be kept in a private record. Your responses will come directly to AHP staff. Any reports generated as part of this evaluation will contain only aggregate responses (for example, “50% of the program staff had been working for this program for less than two years”).

If you have any questions, concerns or comments about the questionnaire or the study, please feel free to contact us by phone or email:

Dr. Terri Tobin Denise Lang

Evaluation Director Research Associate

1-800-795-5486 x418 401-323-9678

ttobin@ahpnet.com dlang@ahpnet.com

Thank you again for your participation!

Section A

Respondent ID: Please provide your First, Middle, & Last initials, and the Month & Day of your birthday. |               | (For example: ALA0415|)

What is your primary role in the project? (select one)

Principal Investigator

Clinician/Counselor

Other, please specify      

Project Director

Researcher/Evaluator

Clinical Supervisor

Data Manager

What is the highest degree that you have obtained?

No high school diploma or equivalent

Bachelor’s degree

High School Diploma or Equivalent

Master’s degree

Some college, but no degree

Doctoral degree or equivalent

Associate’s degree

Other, please specify      

What is your professional background? (check all that apply)

Addictions Counseling

Social Work/Human Services

Nurse Practitioner

Other Counseling

Physician Assistant

Administration

Education

Medicine: Primary Care

None, unemployed

Vocational Rehabilitation

Medicine: Psychiatry

None, student

Criminal Justice

Medicine: Other

Other, please describe      

Psychology

Nurse

Which licenses, credentials, or certificates do you currently hold? (check all that apply)

ACDP/ACPS

LMHC

MD/DO

None

LCDP/LCDCS

LSW/LCSW/LICSW

LMFT

Other, please describe      

How many years of experience do you have in the substance use treatment field?      

Do you have experience providing substance use treatment services to:

Adolescents (ages 12-17)

YES, # years      

NO

Transition Age Youth (TAY; ages 18-24)

YES, # years      

NO

Families

YES, # years      

NO

Please list on what date you began working at your current agency (MONTH/YEAR):      

Please list on what date you began working on this AAFT project (MONTH/YEAR):      

Please tell us a bit about how this AAFT program is structured at your site:

Who, at your agency, is collecting the following data (check all that apply):

GAIN-I (Baseline interview)

Clinical staff

Research/evaluation staff

GAIN M-90 (Follow-up interviews)

Clinical staff

Research/evaluation staff

Treatment Satisfaction Index (TxSI)

Clinical staff

Research/evaluation staff

Who is serving as the data manager for your site?

Internal staff (agency/program staff/clinicians)

External evaluator (research/evaluation staff)

Please indicate which of the following evidence-based treatment models are currently being used throughout your agency’s programs. (check all that apply)

Adolescent-Community Reinforcement Approach /Assertive Continuing Care (A-CRA/ACC)—only check if have used prior to this grant

Motivational Enhancement Therapy (MET)

Motivational Interviewing (MI)

Brief Strategic Family Therapy (BSFT)

Multidimensional Family Therapy (MDFT)

Cognitive Behavioral Therapy (CBT)

Multi-systemic Therapy (MST) for juvenile offenders

Dialectical Behavior Therapy (DBT)

Relapse Prevention Therapy (RPT)

Family Behavior Therapy (FBT)

Seeking Safety

Family Support Network (FSN)

The Seven Challenges

MET/CBT5 (or MET/CBT12)

Other, please specify:      

Grantee experience

This grant may not be your first experience working with A-CRA/ACC, Chestnut Health Systems or research/evaluation. The following questions ask about your experiences with A-CRA/ACC, Chestnut Health Systems, data collection and research/evaluation projects.

Prior to this project, did you participate in a CSAT-funded adolescent project?

Yes

No (If NO, skip to next question)

IF YES, please indicate each type(s).

AAFT

EAT

YORP

TCE/HIV

OJJDP

SCY

Other      

DK

Please indicate what your level of A-CRA/ACC/GAIN certification was prior to working on this AAFT project. [check all that apply]:

I was a certified GAIN local trainer.

I was a certified GAIN administrator.

I was an A-CRA-certified clinical supervisor.

I was an A-CRA-certified clinician.

I was an ACC-certified clinical supervisor.

I was an ACC-certified clinician.

None of the above

Section B: Please indicate…

1. How great a change is it/has it been for your organization to adopt/establish (e.g., change agency policies, program procedures, paperwork) each of the listed components of the AAFT project?

2. How well do each of these components work within your organization (e.g., how well do they “fit” with your organization)?

	How great a change is it/has it been for your organization to adopt/establish each component listed below?				How well do each of these components work within your organization (e.g., how well do they “fit” with your organization)?
	Great change	Moderate change	Slight change	No change	Fits extremely well	Fits moderately	Fits a little	Not a fit
A-CRA approach
ACC approach
GAIN assessment at Intake
Treatment Satisfaction Index (TxSI)
EBTx website
Using ABS web-based system
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
Follow-up interviews (3, 6, 12 months)
Data management activities (e.g., monthly data submission to CHS, responding to GAIN Edits)
GAIN certification
GAIN Clinical Interpretation Certification (GCIC)
ACRA/ACC certification
Monitoring & compliance reports (e.g., follow-up rates, enrollment rates, DSRs, 13+week reports)

Section C: Based on your experience, please rate the reactions to each of the components of this project. If you feel that the question does not apply to you or you do not know the answer, please mark “N/A” or “DK” (respectively).

Adolescent/TAY reactions to…	Always positive	Mostly positive	Mostly negative	Always negative	N/A	DK
A-CRA sessions
ACC sessions
Use of digital recorders for DSRs
GAIN interviews (Baseline/Intake)
Follow-up interviews (3, 6, 12 months)
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
Family reactions to…	Always positive	Mostly positive	Mostly negative	Always negative	N/A	DK
A-CRA sessions
ACC sessions
Use of digital recorders for DSRs
GAIN interviews (Baseline/Intake)
Follow-up interviews (3, 6, 12 months)
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
Clinician reactions to…	Always positive	Mostly positive	Mostly negative	Always negative	N/A	DK
A-CRA sessions
ACC sessions
Use of EBTx website
Use of digital recorders for DSRs
GAIN interviews (Baseline/Intake)
Follow-up interviews (3, 6, 12 months)
Using ABS web-based system
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
The use of monitoring & compliance reports (e.g., follow-up rates, enrollment rates, DSRs, 13+week reports)

Project leadership (e.g., project director, supervisor) reactions to…	Always positive	Mostly positive	Mostly negative	Always negative	N/A	DK
A-CRA sessions
ACC sessions
Use of EBTx website
Use of digital recorders for DSRs
GAIN interviews (Baseline/Intake)
Follow-up interviews (3, 6, 12 months)
Using ABS web-based system
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
The use of monitoring & compliance reports (e.g., follow-up rates, enrollment rates, DSRs, 13+week reports)
Evaluator/Data Manager reactions to…	Always positive	Mostly positive	Mostly negative	Always negative	N/A	DK
Use of EBTx website
Use of digital recorders for DSRs
GAIN interviews (Baseline/Intake)
Follow-up interviews (3, 6, 12 month s)
Using ABS web-based system
Tracking for follow-up interviews (e.g., contacting clients after discharge from program)
The use of monitoring & compliance reports (e.g., follow-up rates, enrollment rates, DSRs, 13+week reports)

Section D: The following questions ask about your experiences with the support provided for the AAFT grant from Chestnut Health Systems, CSAT and other outside sources. For each item, please indicate how helpful each support has been. Lastly, please add any comments or recommendations for improvement for each.

Have you had involvement with A-CRA/ACC, EBTx, and/or DSRs?			YES		NO		IF “NO”—SKIP questions below, continue on next page
How helpful has it been in helping you implement the AAFT Program?	Extremely	Moderately		Slightly		Not at all	Not enough participation to rate this item	Any comments or recommendations for improvement?
A-CRA/ACC Training								     
A-CRA/ACC certification								     
A-CRA/ACC coaching calls								     
A-CRA/ACC Training manual (e.g., refer back to procedures, certification process)								     
A-CRA/ACC Materials (e.g., Happiness Scale, Functional Analysis worksheet)								     
EBTx website & DSRs								     
EBTx Support team staff								     
Program reports (e.g., 13+weeks report)								     
A-CRA/ACC team staff								     

Have you had involvement with the GAIN, GAIN data, and/or ABS?			YES		NO		IF “NO”—SKIP questions below, continue on the next table
How helpful has it been in helping you implement the AAFT Program?	Extremely	Moderately		Slightly		Not at all	Not enough participation to rate this item	Any comments or recommendations for improvement?
GAIN Training								     
GAIN certification								     
GAIN coaching calls								     
GAIN Training manual (e.g., intent of questions, certification process, etc.)								     
GAIN Clinical Interpretation Training								     
GAIN Clinical Interpretation Certification Individual-level reports (e.g., GRRS, PFR)								     
ABS web-based system								     
Reports (e.g. Site Profiles Report)								     
GAIN data (e.g., analytical files)								     
GAIN Support team staff								     

How helpful has it been in helping you implement the AAFT Program?	Extremely	Moderately	Slightly	Not at all	Not enough participation to rate this item	Any comments or recommendations for improvement?
Implementation calls/site visits to monitor/address progress at grantee sites						     
Individualized coaching calls to address areas for improvement (e.g., low recruitment/follow-up rates, DSR uploads, increase TxSI completion, etc.)						     
List Serve Information						     
Technical assistance available through NIATx						     
Special topic calls (e.g., Cultural Responsiveness)						     
Monthly calls with CSAT project officer/Chestnut						     
Interactions with other grantees at meetings/ calls						     
Initial/Annual Grantee Meetings						     

Section E: To meet the needs of your target population, you may have felt the need to modify or adapt the A-CRA/ACC treatment model. Please tell us about any changes your program has made to the A-CRA/ACC model for this project.

Has your program made any modification/adaptations to the A-CRA/ACC treatment model?
YES (If YES, proceed to next question below)			NO (If NO, skip to next page)
Has well have these modifications worked for your program?
Extremely well	Quite well	Not too well		Not well at all

Section F: Based on your experience, please indicate if the issues listed below have been barriers to implementation at your site. Then, describe the strategies you have used to overcome the MOST CHALLENGING BARRIER at your site.

POSSIBLE BARRIERS	Was this a barrier to implementation at your site?
Program/Organizational Issues	YES	NO
Internal communication (e.g., program staff)
External communication (e.g., CSAT, Chestnut Health Systems)
Staff attitudes (e.g., morale, enthusiasm, resistance)
Leadership/management attitudes
Recruiting clinical staff
Recruiting supervisory staff
Turnover; significant loss of staff
Budget issues
Service Delivery	YES	NO
Enrolling clients
Client engagement
Family engagement
Client retention
Grant-related Activities	YES	NO
A-CRA/ACC training
GAIN training
Collecting GPRA data
Collecting GAIN data
Collecting follow-up data
Using ABS
Using the SAIS system for GPRA
Recording treatment sessions (DSRs) & using EBTx website
Other Grant requirements (e.g., certification, coaching calls, data management)
Research/Evaluation Activities	YES	NO
Working with the local evaluator
Use of data, monitoring, compliance reports (e.g., Site Profile Data, follow-up/enrollment rates, 13+week reports)
Any other barriers not included above (specify):     
Please describe the strategies you have used to overcome the MOST CHALLENGING BARRIER at your site.      

Section G: Staff turnover is common in this field. Please tell us about any staff turnover and effects it may have had or is having on your program.

During the last project year, did the project hire new…			If YES, how would you rate the overall effect of this change on the grant program?
Program Manager(s)	YES	NO	Positive	Neutral	Negative	Don’t know
Clinical Supervisor(s)	YES	NO	Positive	Neutral	Negative	Don’t know
Clinician(s)/direct care staff	YES	NO	Positive	Neutral	Negative	Don’t know
Evaluator(s)	YES	NO	Positive	Neutral	Negative	Don’t know
Data manager(s)	YES	NO	Positive	Neutral	Negative	Don’t know

Section H: To better understand barriers adolescents and families may have experienced when starting treatment prior to the AAFT grant, please tell us about the barriers adolescents and family may have encountered prior to starting the AAFT grant and after implementation of the AAFT grant.

	Barrier for adolescents/families before the grant?				Barrier for adolescents/families now?
	An extreme barrier	Very much a barrier	A slight barrier	Not a barrier at all	An extreme barrier	Very much a barrier	A slight barrier	Not a barrier at all
Transportation
Child care
Need for mental health treatment
Neighborhood safety (e.g., home visits, gang turf)
Insurance/cost of treatment
Family issues (e.g., lack of family involvement)
Language/cultural issues
Treatment resistance (e.g., low readiness for change, not understanding substance abuse disorders/treatment, stigma, shame)
Other (specify):      

Section O: Please read each statement carefully and check the answer that you feel best characterizes your agency’s experience with research and evaluation activities.

	Very True	Somewhat True	Slightly True	Not True
Policies and procedures are in place to ensure integrity of data collected.
Staff capacity is adequate to meet the demands of data collection efforts.
Our program regularly conducts surveys with consumers to identify program strengths and weaknesses.
Program staff use data to evaluate program services and consider opportunities for improvement.
Our agency relies on data to set goals and measure success.
Data collected is analyzed and used to evaluate/enhance program implementation.
Management shares data with staff about program effectiveness in meeting the needs of clients.
Data are provided to/discussed with staff.

Section P. Sustainability: Please tell us about any plans your program may have to sustain this program after CSAT funding has ended.
Are you currently engaged in any activities aimed toward sustainability of your program?	YES (if YES, proceed to next question)	NO (if NO, skip to FINAL THOUGHTS below)
If YES, please describe the activities.
What parts/components are likely to continue and why? FINAL THOUGHTS Given your experience implementing this project, what do you think you would have done differently if you could turn back the clock to the time when you first got involved with this grant?       Please tell us about your biggest success(es) with this project. This is your opportunity to brag about the work you are doing to help adolescents, transition-age youth, and their families. To what would you attribute the successes (e.g., strong staff, additional funding, morale…)?

Thank you for your time and contribution!

Please e-mail to ttobin@ahpnet.com or FAX to (978) 443-4722, Attn: Terri Tobin

4