0216_omb Ss-b_4.9.13

S

UPPORTING STATEMENT FOR THE ADDICTION TECHNOLOGY TRANSFER CENTERS (ATTC) NETWORK PROGRAM MONITORING

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

The Center for Substance Abuse Treatment has funded the Addiction Technology Transfer Center Network to provide training and technical assistance and to meet with an estimated 35,000 individuals per year across 10 Regional Centers, 4 National Focus Area Centers, and the Network Coordinating Office. A wide variety of event participants are anticipated, ranging from addictions counselors to professionals in other fields who work with individuals with substance use disorders. Participants self-refer to all events provided by the ATTC Network. Since this is the case, the respondent universe is made up of participants of events of the ATTC Network.

Consistent with the approach used in the GPRA Customer Satisfaction Surveys for the CSAT Knowledge Application Programs, sampling methods will be used for the follow-up portion of this project. It was determined that the use of sampling in the follow-up phase of the performance monitoring would be most efficient because of the wide variety of training topics and formats offered by the ATTC Network. Sampling allows the ATTC Network to collect information on the entire variety of events, and to allow for comparisons by topic of event and by format of event. The Event Description Form collects information about the format and topic of events, allowing for comparisons between topics and formats. A random sample of 25% of participants who have consented for follow-up will be contacted 30 days after the event by e-mail or traditional mail to complete follow-up forms.

In 2000, a follow-up response rate pilot showed that the methodology that consistently reached 80% was phone follow-up; however, the cost was prohibitive for adoption across the ATTC Network. The Center using that methodology compared doing 80% response rate with a 20% random sample, and found that there was no difference between the results from the two groups. ATTC Directors and CSAT agreed on a 25% sampling plan for follow-up, with the aim of 80% response rate of that sample. Participants are asked at each event if they are willing to participate in a 30-day follow-up survey, and if so, how they would prefer to be contacted (e-mail or traditional mail). The sample is drawn randomly from the pool of consenting participants. ATTCs use a standard method for follow-up, with changes based on the contact method (e-mail or traditional mail). Sample cover letters for follow-up are attached (Attachment 6).

2. Information Collection Procedures

As explained previously, CSAT intends to use seven (7) instruments for program monitoring of ATTC events as well as for ongoing quality improvement. Five of the seven instruments have been previously approved and are being used to monitor the ATTC Network’s performance (OMB No. 0930-0216). CSAT is requesting approval to continue to use these instruments as is. The remaining two instruments are currently approved by the OMB through a separate approval for CSAT Government Performance and Results Act (GPRA) Customer Satisfaction instruments (OMB No. 0930-0197). These instruments (the Meeting Follow-up Form and the Technical Assistance Follow-up Form) are not being revised, and since they are covered under a separate OMB approval than is the focus of this application, CSAT is not seeking any action related to those two forms at this time. CSAT is, therefore, requesting approval to continue to use five data collection instruments in total to monitor the ATTC Network at this time. The five data collection instruments for which CSAT is seeking an extension in this application are described below, summarized in Table 4, and are provided in Attachments 1 through 3. Attachments 4-1 and 4-2 provide the Follow-up forms for Meetings and Technical Assistance which are currently being used by CSAT Best Practices Programs. Although these forms are not a part of the OMB application, CSAT has included them in the Attachments to provide clarity and context.

• Event Description Form (EDF). The EDF collects descriptive information about each of the events of the ATTC Network. This form is completed by ATTC staff.

• Training Post Event Form. This form is distributed to training participants at the end of the training activity, and collected from them before they leave. For training events which take place over an extended period of time, this form is completed after the final session of training. The form asks participants to report demographic information, education, profession, field of study, status of certification or licensure, workplace role, employment setting, satisfaction with the quality of the training and training materials, and to assess their level of skills in the topic area. There are no revisions to the form currently in use. CSAT is requesting approval to continue to use the form as is.

• Training Follow-up Form. The follow-up form is sent to a random sample of training participants 30 days after the end of the training event. The form asks participants to report demographic information, satisfaction with the quality of the training and training materials, and to assess their level of skills in the topic area. No revisions are being made to the form currently in use. CSAT is requesting approval to continue to use the form as is.

• Meeting Post Event Form. This form is distributed to meeting participants at the end of the meeting, and collected from them before they leave. This form asks participants to report demographic information, education, profession, field of study, status of certification or licensure, workplace role, employment setting, and satisfaction with the quality of the event and event materials, and to assess their level of skills in the topic area. No revisions are being made to the form currently in use. CSAT is requesting approval to continue to use the form as is.

• Technical Assistance Post Event form. This form is distributed to technical assistance participants at the end of the TA event. This form asks participants to report demographic information, education, profession, field of study, status of certification or licensure, workplace role, employment setting, and satisfaction with the quality of the event and event materials, and to assess their level of skills in the topic area. No revisions are being made to the form currently in use. CSAT is requesting approval to continue to use the form as is.

Event participants will be asked to complete the Post Event Form for the type of event they attended (meeting, technical assistance or training) at the end of the event. Prior to completing the Post Event form, participants will receive the Cover Page for Instruments (Attachment 7) and will be told about the purpose of the data collection (See Model Statement, Attachment 8). Participants will also be asked if they are willing to participate in the 30-day follow-up and if so, to provide contact information for their preferred method of contact (e-mail or traditional mail; see Attachment 9). This is often done in conjunction with the completion of a consent form (See Model Consent Form in Attachment 10).

Follow-up forms will be sent 30 days after the end of the event to a random sample of 25% of the participants willing to be contacted (See Attachment 6 for sample cover letters).

Table 4: Data Collection Methods

Form	Data Collection Method	Timeline
ATTC Faculty/Staff
Event Description Form	Paper and pencil instrument	Beginning of each training event
Event Participants
Post Event Training Form	Paper and pencil instrument (10% online)	Completion of each training event
Post Event Meeting Form	Paper and pencil instrument (10% online)	Completion of each training event
Post Event Technical Assistance Form	Paper and pencil instrument (10% online)	Completion of each training event
Follow-up Training Form	Mail-out instrument (Estimated 75% online via online survey tools)	30 days after completion of training event

Methods to Maximize Response Rate

The ATTC Network anticipates an 80% overall response rate for this performance monitoring project. The ATTC expects receiving at least 90% of the Post-event forms, as these forms are completed at the time and location of the event. The ATTC anticipates a 100% response rate on the Event Description Form, as this is completed by ATTC staff and/or faculty, and is a requirement of their position. The ATTC Network anticipates receiving 80% of the follow-up forms. The 25% sampling strategy described above is in place to allow for repeated attempts to contact participants for follow-up.

Methods to maximize response rates include:

• Brief and easy to complete instruments;

• One of the instruments is completed at the time and location of the training event;

• One instrument is completed by ATTC training staff;

• Standard language in the grant announcement requiring all ATTC sites to participate in the program monitoring;

• Semi-annual reports summarizing findings to ATTC directors and CSAT;

• Annual summary reports of findings for broader dissemination;

• All sites will emphasize the importance of completing these instruments in order to continue the funding that provided the opportunity for the event for no cost or at low cost;

• E

  
  

  ase and convenience of electronic reporting; and

• The ATTC Network has a history of a strong positive relationship with event participants who are receptive to the request for follow-up participation with a traditional single mailing methodology.

4. Tests of Procedures

For approximately the past sixteen years, the ATTC Network has successfully collected data for performance monitoring. Since no revisions are being made to the forms since the last OMB approval, no difficulty is anticipated in maintaining and continuing the past successes.

5. Statistical Consultants

Government Project Officer

Suzan Swanton

Substance Abuse and Mental Health Services Administration

Center for Substance Abuse Treatment

1 Choke Cherry Road, Room 5-1118

Rockville, MD 20857

(240) 276-1775(p)

(240) 276-2970 (f)

Suzan.swanton@samhsa.hhs.gov

ATTC Network Coordinating Office Coordinator

Laurie Krom

Director of Education and Communications

Addiction Technology Transfer Center Network Coordinating Office

School of Nursing and Health Studies

University of Missouri-Kansas City

2464 Charlotte

Kansas City, MO 64108

816.235.6985 (p)

www.ATTCnetwork.org

kroml@umkc.edu

ATTACHMENTS

Attachment 1: Event Description Form

Attachment 2: Post-Event Forms

2-1 Training Post Event Form

2-2 Meeting Post Event Form

2-3 Technical Assistance Post Event Form

Attachment 3: Training Follow-up Form

Attachment 4: Government Performance and Results Act Customer Satisfaction Surveys for the Center for Substance Abuse Treatment Knowledge Application Programs (OMB No. 0930-0197)

4-1 CSAT Customer Satisfaction Survey ─ Meeting Follow-up Form

4-2 CSAT Customer Satisfaction Survey ─ Technical Assistance Follow-up Form

Attachment 5: ATTC Network Centers

Attachment 6: Sample Cover Letters

Attachment 7: Cover Page for Instruments

Attachment 8: Model Statement

Attachment 9: Contact Information Form

Attachment 10: Statement of Informed Consent for Follow-up Interview

5