S
A. JUSTIFICATION
1. Circumstances of Information Collection
The Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT) is seeking Office of Management and Budget (OMB) approval for the revision of data collection activities for monitoring program performance of the Addiction Technology Transfer Center (ATTC) Network. SAMHSA’s CSAT funds this program under legislative authority of Section 509, Priority Substance Abuse Treatment Needs of Regional and National Significance, of the Public Health Service Act, as amended.
CSAT intends to use seven (7) instruments for program monitoring of ATTC events as well as for ongoing quality improvement. Three of the seven instruments are currently approved by the OMB (OMB No. 0930-0216) for use through February 28, 2010. These are:
The Event Description Form,
The Training Post Event Form, and
The Training Follow-up Form
CSAT is requesting approval at this time to revise the Event Description Form and the Training Post Event Form. The Training Follow-up Form will not be revised. CSAT is requesting to continue to use the Training Follow-up Form as is.
The remaining four instruments are currently approved by the OMB through a separate approval for CSAT Government Performance and Results Act (GPRA) Customer Satisfaction instruments (OMB No. 0930-0197). These are:
The Meeting Post Event Form,
The Meeting Follow-up Form,
The Technical Assistance Post Event Form, and
The Technical Assistance Follow-up Form
The Meeting Follow-up Form and the Technical Assistance Follow-up Form are not being revised. CSAT is not seeking any action related to those two forms at this time since they are covered under a separate OMB approval than is the focus of this application. These two forms are included in the description of the forms CSAT intends to use in order to provide clarity and context. The Meeting Post Event Form and the Technical Assistance Post Event Form are, however, being revised. Therefore, CSAT is seeking approval in this application to utilize the revised versions of these forms in order to monitor the performance of the ATTC Network. Table 1 summarizes the forms currently in use, the status of the request related to each form for this application, and the forms CSAT is proposing to use after February 28, 2010. In addition, each of the seven instruments intended for use after February 28, 2010 is included as an attachment to this support statement (see Attachments 1 through 4).
Forms Currently in Use |
2010 Approval Request Details |
Proposed 2010 – 2013 Forms |
Event Description Form - (ATTC Form Approved under OMB No. 0930-0216) |
Requesting to revise this form. |
(1) EVENT DESCRIPTION FORM |
Training Pre-Event Form – (ATTC Form Approved under OMB No. 0930-0216) |
This form is being eliminated. |
|
Training Post-Event Form – (ATTC Form Approved under OMB No. 0930-0216) |
Requesting to revise this form. |
(2) TRAINING POST-EVENT FORM |
Training Follow-up Form – (ATTC Form Approved under OMB No. 0930-0216) |
Requesting approval to continue to use this form as is.
|
(3) TRAINING FOLLOW-UP FORM |
Meeting Pre-Event Form – (ATTC Form Approved under OMB No. 0930-0216) |
This form is being eliminated. |
|
Meeting Post-Event Form – GPRA Customer Satisfaction Surveys for the CSAT Knowledge Application Programs (CSAT Form Approved under OMB No. 0930-0197) |
Requesting to revise this form. |
(4) MEETING POST-EVENT FORM |
Meeting Follow-up Form – GPRA Customer Satisfaction Surveys for the CSAT Knowledge Application Programs
|
No change – Approval is not being requested in this submission since this form continues to be approved under OMB No. 0930-0197 |
(5) MEETING FOLLOW-UP FORM (covered under OMB No. 0930-0197) |
Technical Assistance Pre-Event Form – ATTC Form Approved under OMB No. 0930-0216 |
This form is being eliminated. |
|
Technical Assistance Post-Event Form - GPRA Customer Satisfaction Surveys for the CSAT Knowledge Application Programs CSAT Form Approved under OMB No. 0930-0197 |
Requesting to revise this form. |
(6) TECHNICAL ASSISTANCE POST-EVENT FORM |
Technical Assistance Follow-up Form - GPRA Customer Satisfaction Surveys for the CSAT Knowledge Application Programs
|
No change – Approval is not being requested in this submission since this form continues to be approved under OMB No. 0930-0197 |
(7) TECHNICAL ASSISTANCE FOLLOW-UP FORM (covered under OMB No. 0930-0197) |
Table 1: Forms Currently in Use Compared to Forms Proposed |
||
The ATTC Network, a nationwide, multidisciplinary resource that draws upon the knowledge, experience and latest research of recognized experts in the field of addictions, is a unique CSAT initiative formed in 1993 in response to a shortage of well-trained addiction professionals in the public sector. The ATTC Network works to enhance the knowledge, skills and aptitudes of the addiction treatment and recovery services workforce by disseminating current health services research from the National Institute on Drug Abuse, National Institute on Alcohol Abuse and Alcoholism, National Institute of Mental Health, Agency for Health Care Policy and Research, National Institute of Justice, and other sources, as well as other SAMHSA programs. To accomplish this, the ATTC Network (1) develops and updates state-of-the-art, research based curricula and professional development training, (2) coordinates and facilitates meetings between Single State Authorities, Provider Associations and other key stakeholders, and (3) provides ongoing technical assistance to individuals and organizations at the local, regional and national levels.
Currently, CSAT funds a network of fourteen Regional Centers and a National Office, which serve all 50 states, the District of Columbia, Puerto Rico, the U.S. Virgin Islands, and the Pacific Islands (Attachment 6). Of the fifteen sites, twelve are located at academic institutions, two are located within nonprofit institutes, and one is a project of a state certification board. Although the individual sites vary in the number of states served and areas of emphasis, each is charged, as is the Network as a whole, with seven key objectives:
Building and maintaining collaborations and facilitating systems change to improve the provision of substance use disorders treatment and recovery services among key organizations and groups;
Using innovative technologies to create partnerships with, and disseminate current behavioral health research from various federal sources;
Developing and updating state-of-the-art research-based curricula;
Enhancing the clinical and cultural competencies of professionals in a variety of disciplines to address the treatment/recovery needs of individuals with substance use disorders;
Upgrading standards of professional practice for addictions workers in a variety of settings;
Serving as a technical resource on state-of-the-art treatment and recovery services for community-based and faith-based organizations, consumers and recovery organizations, and other stakeholders; and
P
The ATTC Network is a major resource in ensuring that treatment and recovery services professionals have the latest information on recommended approaches to addictions treatment and recovery management techniques. The Network provides ongoing dissemination of research-based addictions knowledge in a number of ways, including training events, technical assistance, meetings, print resources such as brochures and newsletters, and electronic media such as Web sites.
Training events take the form of workshops, conferences, continuing education courses, university courses, and community college courses. While the majority of training events are done in the traditional classroom format, there are a growing number of distance education opportunities, including online and video courses. Training participants come from diverse populations, ranging from addictions counselors to correctional officers and health professionals. Technical assistance events are jointly planned consultations generally involving a series of contacts between an ATTC and an outside organization/institution during which the ATTC provides expertise and gives direction toward resolving a problem or improving conditions. Meeting events are ATTC sponsored or co-sponsored events in which a group of people representing one or more agencies other than the ATTC work, cooperatively on a project, problem, and/or a policy.
2. Purpose and Use of Information
SAMHSA’s legislative mandate is to increase access to high quality prevention and treatment services and to improve outcomes. Its mission is to improve the quality and availability of treatment and prevention services for substance abuse and mental illness. To support the Agency’s mission, SAMHSA’s overarching goals are:
1) Accountability—Establish systems to ensure program performance measurement and accountability
2) Capacity—Build, maintain, and enhance mental health and substance abuse infrastructure and capacity
3) Effectiveness—Enable all communities and providers to deliver effective services
SAMHSA strives to coordinate the development of these goals with other ongoing performance measurement development activities. Below are the measures delineated in the Government Performance and Results Act (GPRA) Performance Plan for the Best Practices programs, including the ATTC Network:
Measure 1: Increase the number of individuals trained per year - This is a key measure tracking CSAT’s mission of promoting effective treatment through the adoption of evidence-based practices. Tracking the number of individuals trained is critical in documenting the delivery of service and dissemination of relevant information to the field.
Measure 2: Increase percentage of drug treatment professionals trained by the program who (a) would rate the quality of the events as good, very good, or excellent; (b) shared any of the information from the events with others; (c) report implementing improvements in treatment methods on the basis of information and training provided by the program.
Measure 3: Increase the percentage of grantees in appropriate cost bands.
Outcome data reflect the Agency’s desire for consistency in data collected within the Agency. SAMHSA is implementing specific performance domains called the National Outcome Measures (NOMS) to assess the accountability and performance of its discretionary and formula grant programs. These domains represent SAMHSA’s focus on the factors that contribute to the success of substance abuse treatment, one of which is the use of evidence-based practices. The ATTC Network directly assists CSAT promote effectiveness through evidence-based practice programs, which help communities and providers to identify, adapt, implement, and evaluate evidence-based practices. Programs include activities to bridge the gap between knowledge and practice by promoting the adoption of evidence-based practices, and by ensuring that services availability meets targeted needs. These programs also are used to disseminate information about systems and practices shown to be most effective.
Information collected from these instruments will continue to assist CSAT in documenting the numbers and types of participants in ATTC events, describing the extent to which participants report improvement in their clinical competency, and which method is most effective in disseminating knowledge to various audiences. This type of information is crucial to support CSAT in complying with GPRA reporting requirements and will continue to inform future development of knowledge dissemination activities. Moreover, this information will be collected and examined on a regular basis by the ATTC Network itself for ongoing quality improvement purposes. Questions that the ATTC Network considers while examining the data include:
W
Do ATTC training events lead to an increase in knowledge and/or skills in participants?
Are certain training formats more effective than others in transferring knowledge and skills to participants?
How is training effectiveness affected by participant type, training format and/or training topic?
Event Definitions
The definitions for the three types of events from which data will continue to be collected have not changed. They are as follows:
Training Event --A training event is defined as an ATTC sponsored or co- sponsored event lasting at least three (3) hours that focuses on the enhancement of knowledge and/or skills. Higher education classes must be included in this definition with each course considered as one training event.
Technical Assistance --Technical assistance is defined as a jointly planned consultation generally involving a series of contacts between the ATTC and an outside organization/institution. During the consultation, the ATTC provides expertise and gives direction toward resolving a problem or improving conditions. This may be a time-limited consultation or an ongoing series of consultations. The ATTC reports technical assistance at the end of the series of contacts or yearly if contacts are ongoing.
Meeting --A meeting is defined as an ATTC sponsored or co-sponsored event in which a group of people representing one or more agencies other than the ATTC work cooperatively on a project, a problem, and/or a policy. These groups may be established and ongoing, or may exist only to accomplish a single purpose. Included in this definition would be consortia meetings and workgroup meetings. The ATTC reports activities as "meetings" only when they are NOT appropriate to report under any other category.
Description of Data Collection and Purposes
Data collected on the forms continue to be entered into an online system maintained by a CSAT contractor. Data entered into this online system are immediately live and accessible to CSAT Project Officers for administration purposes. Table 2 summarizes the data collection instruments. As described above, CSAT intends to use seven forms to monitor the work of the ATTC Network. CSAT is requesting approval for five of these seven forms in this application. Out of the five forms for which CSAT is requesting approval in this application, four of them will be revised from previously approved versions (See Table 1).
Event description data will continue to be reported by ATTC faculty/staff on all three types of events using a revised version of the Event Description Form (EDF). The EDF collects event information. This instrument asks approximately 10 questions of ATTC faculty/staff relating to the event focus and format, as well as publications to be used during the event. It allows the ATTC Network and CSAT to track the number and types of events held (See Attachment 1). The EDF includes questions related to which SAMHSA Matrix priority areas and cross-cutting principles are covered by the event. SAMHSA’s Matrix has been revised since this form was last approved. The EDF will, therefore, be revised to match the updated Matrix. In addition, the EDF asks which of SAMHSA’s Technical Assistance Publications (TAPs) and Treatment Improvement Protocols (TIPs) were used during the event. New TIPs and TAPs have been published since the form was last approved. Those new TIPs and TAPs will be added to the form.
Post-event data will be collected on participants of all three types of events according to CSAT requirements.
Post Event Form for Training: The Training Post Event Form, which is administered immediately following the event, currently asks approximately 25 questions of each individual that participated in the training (Attachment 2-1). The instrument asks the participants to report demographic information, satisfaction with the quality of the training and training materials, and to assess their level of skills in the topic area. The instrument currently in use is nearly identical to the CSAT Customer Satisfaction instrument for training (OMB No. 0930-0197) except that it includes two (2) additional questions. The Training Post Event form is being revised. The five current demographic questions on the form will be modified to reflect a more current understanding of the field, and five additional demographic questions will be included. The form will ask participants to report demographic information, education, profession, field of study, status of certification or licensure, workplace role, and employment setting.
Post-event Forms for Meetings and Technical Assistance: The Meeting and Technical Assistance Post Event forms ask approximately 25 questions of each individual that participated in the event (Attachment 2-1 and 2-3). Currently, CSAT uses the Government Performance and Results Act Customer Satisfaction Surveys for the Center for Substance Abuse Treatment Knowledge Application Programs (OMB No. 0930–0197) as the Meeting and Technical Assistance Post Event forms. These forms ask the participants to report satisfaction with the quality of the event and event materials, and to assess their level of skills in the topic area. They are administered to participants of technical assistance and meeting events immediately following the event. The Meeting and Technical Assistance Post Event forms are being revised. The five current demographic questions on the forms will be modified to reflect a more current understanding of the field, and five additional demographic questions will be included. The forms will ask participants to report demographic information, education, profession, field of study, status of certification or licensure, workplace role, and employment setting.
Follow-up data will be collected on participants of all three types of events according to CSAT requirements.
Follow-up Form for Training: The Training Follow-up form, which is administered 30-days after the event, also asks about 25 questions (Attachment 3). The approved instrument asks the participants to report demographic information, satisfaction with the quality of the training and training materials, and to assess their level of skills in the topic area. This instrument is nearly identical to the CSAT Customer Satisfaction instrument for training (OMB No. 0930-0197) except that it includes four (4) additional questions. The Training Follow-up form is not being revised at this time. CSAT is requesting approval to continue to use this form as is.
Follow-up Forms for Meetings and Technical Assistance: The Meeting and Technical Assistance Follow-up forms, which are administered 30-days after the event, ask about 20 questions of 25% of consenting participants (Attachments 4-1 and 4-2). The approved form asks the participants to report satisfaction with the quality of the event materials, to assess their level of skills in the topic area, and to report whether or not they have shared information from the event at their place of work. This form is already approved by OMB and will not be revised (OMB # 0930-0197).
While the instruments administered immediately at the end of each event are given to all participants, the instruments administered 30 days after each event are sent to a random sample of 25% of those participants who consented to follow-up. This sampling rule applies to all three types of events.
T
Form |
Timeline |
Type of Information |
ATTC Faculty/Staff |
||
Event Description Form (EDF) (Form 1 in Table 1; See Attachment 1) |
Prior to each event
|
Format and content of event
|
Participants |
||
Post-event Form for Training (Form 2 in Table 1; See Attachment 2-1) |
Completion of each training event (training participants only). |
CSAT Customer Survey (OMB No. 0930-0197) instrument for training, PLUS 2 questions on knowledge and skills in topic area of training event. Revisions to form currently in use will include modification the five current demographic questions on the form and addition of five new demographic questions to reflect a more current understanding of the field. The form will ask participants to report demographic information, education, profession, field of study, status of certification or licensure, workplace role, and employment setting. |
Follow-up Form for Training (Form 3 in Table 1; See Attachment 3)
|
30 days after completion of training event (random sample of 25% of consenting training participants only).
|
CSAT Customer Survey (OMB No. 0930-0197) instrument for training follow-up, PLUS 4 questions on knowledge, skills and effectiveness in topic area of training event, and barriers to implementation. No revisions to form currently in use. Requesting approval to continue to use the form as is. |
Post Event Forms for Meetings (Form 4 in Table 1) and Technical Assistance (Form 6 in Table 1) (See Attachments 2-2 and 2-3) |
Completion of each event (event participants only). |
CSAT Customer Survey (OMB No. 0930-0197) instrument for meetings and technical assistance revised to include modification the five current demographic questions on the forms and addition of five new demographic questions to reflect a more current understanding of the field. The forms will ask participants to report demographic information, education, profession, field of study, status of certification or licensure, workplace role, and employment setting. |
Follow-up Forms for Meetings (Form 5 in Table 1) and Technical Assistance (Form 7 in Table 1) (See Attachments 4-1 and 4-2) |
30 days after completion of each technical assistance event or meeting (random sample of 25% of consenting training participants only). |
Government Performance and Results Act Customer Satisfaction Surveys for the Center for Substance Abuse Treatment Knowledge Application Programs (Attachments 4-1 and 4-2)
Approval is not being requested in this submission.
CSAT Form Approved under OMB No. 0930-0197 |
3. Use of Information Technology
Sixty percent of the ATTC performance monitoring instruments are administered in person to participants at ATTC events, who complete the forms by paper and pencil. Ten percent of the training courses are online, and thus, those instruments are administered online. Data collection for the Post Event instruments is incorporated into the class procedure. The remaining thirty percent is made up of those 30- day follow-up instruments that are distributed to consenting participants via electronic mail using an online survey tool.
All data collected will be managed in electronic databases. The ATTC Regional Centers are responsible for data collection and entry for their events. Data collected on all the instruments are entered into the online database maintained by CSAT's contractor. Once data are entered into the system, they are available to CSAT for review. These data can also be downloaded by the Regional Centers for their use. At least once a year, the ATTC National Office requests all ATTC data from that year from the CSAT contractor. The ATTC National Office keeps a record of the data collected, and merges files for data examination as requested.
4. Efforts to Identify Duplication
The data to be collected are unique and are not otherwise available.
5. Involvement of Small Entities
Participation in the ATTC Network program monitoring will not be a significant burden on small businesses or small entities or on their workforces.
6. Consequences If Information Collected Less Frequently
Comparisons of data are crucial for CSAT and the ATTC Network to be able to monitor the effectiveness of the training events, and to make necessary adjustments accordingly. The Post Event Questions are needed to give a baseline measure of participants' knowledge, skills and effectiveness in the topic area of the event attended. The Follow-up Questions provide a comparison point to measure a change in knowledge and skills. They also demonstrate whether the information learned in the event has been used in the workplace.
All of the information collected from participants is critical for assessing the effectiveness of the events of the ATTC Network. Without this information, CSAT will be unable to:
Determine the effectiveness of ATTC events in increasing the knowledge and skills of participants;
Determine the effectiveness of various training formats to transfer knowledge and skills to participants.
7. Consistency with the Guidelines in 5 CFR 1320.5ld}(2}
This information collection fully complies with the guidelines in 5 CFR 1320.5(d)(2).
8. Consultation Outside the Agency
The notice required by 5 CFR 1320.8(d) was published in the Federal Register on October 6, 2009 (74 FR 51283). No comments were received in response to this notice.
No further outside consultation has been initiated.
9. Payment to Respondents
Some ATTC sites may provide minimal payment for the follow-up instruments. This varies across the Network due to regional and local differences. For those ATTC sites that do provide payment, survey research literature suggests that monetary incentives have a strong positive effect on response rates and no known adverse effect on reliability.
10. Assurance of Confidentiality
Each of the instruments asks a number of questions that are used to create a unique identification code. This code does not change over time, but also does not include personal identifiers. This code is used to match responses from post-training to follow-up without personal identifiers. See the Personal Code section at the top of each of the attached instruments in Attachments 1 through 4.
11. Questions of a Sensitive Nature
No forms collect information that is sensitive to individuals.
12. Estimates of Annualized Hour Burden
The total annualized burden to an estimated 42,750 respondents for the ATTC Network program monitoring is estimated to be 6,662.50 hours. Burden estimates are based on previous use of related data collection instruments by the ATTC Network. The annualized hourly costs to respondents are estimated to be $118,592.50. Hourly wage information is based on estimated median hourly wages of $17.80 an hour for substance abuse and behavioral disorder counselors as reported in the Occupational Employment Statistics available from the Bureau of Labor Statistics, U.S. Department of Labor (http://www.bls.gov/oes/current/oes211011.htm). There are no direct costs to respondents for participation aside from their time. Burden estimates are detailed in Table 3. The Event Description Form is filled out by ATTC faculty or training staff.
Table 3: Annualized Burden Estimates
Type of Respondent |
Number of Respondents |
Responses per Respondent |
Hours per Response |
Total Annual Burden Hours |
Hourly Wage Cost |
Total Hour Cost |
Faculty/staff Event Description Form |
250 |
1 |
.25 |
62.50 |
|
|
Meeting and Technical Assistance Participants |
|
|
|
|
|
|
Post-Event Form |
5,000 |
1 |
.12 |
600 |
$17.80 |
$10,680.00 |
Follow-up Form |
Covered under CSAT Government Performance and Results Act (GPRA) Customer Satisfaction form (OMB # 0930-0197) |
|||||
Training Participants |
|
|
|
|
|
|
Post-Event Form |
30,000 |
1 |
.16 |
4,800 |
$17.80 |
$85,440.00 |
Follow-up Form |
7,500 |
1 |
.16 |
1,200 |
$17.80 |
$21,360.00 |
TOTAL |
42,750 |
|
|
6,662.50 |
|
$118,592.50 |
|
|
|
|
|
|
|
13. Estimates of Annualized Cost Burden to Respondents
There are neither capital or startup costs nor are there any operation and maintenance costs.
14. Estimates of Annualized Cost to the Government
The annual estimated cost to the government for the ATTC Network program is $7.61 million. This includes an estimated annual amount of $7.6 million for cooperative agreements for three years. Approximately $78,234 per year represents SAMHSA costs to manage/administrate the network for 90% of one employee (GS-13).
1
Currently, there are 9,200 burden hours in the OMB inventory. The Program is requesting 6,662.50 hours, a decrease of 2,537.50 hours. This decrease is due to the elimination of the Pre Event forms for all event types (See Table 1).
16. Time Schedule. Publication and Analysis Plans
Data collection will occur as individuals participate in ATTC-sponsored events across the 14 regional sites and the National Office. Because this assessment is used to monitor and improve upon the quality of ATTC services, ongoing examination is critical. Fortunately, CSAT’s electronic database in which the data will continue to be entered allows reports to be run on the data in a quick and timely manner. ATTC sites will, therefore, periodically run such reports to examine their data. Furthermore, each ATTC site must, according to CSAT funding requirements, prepare two semi-annual reports each fiscal year. In these reports, each ATTC site is required to include a summary report of its performance monitoring data describing whether the site is meeting its annual event and participant targets as well as maintaining a response rate to the follow-up forms of at least 80%. The semi-annual reports are completed regionally by ATTC staff and Directors using a Network template, and are sent to CSAT electronically. In addition, at least once a year the ATTC National Office requests aggregated, Network-wide data on the following categories from CSAT’s electronic database contractor:
Total events;
Total events in each of the three event categories;
Total participants;
Total participants in each of the three event categories;
Percentages of participants of various races and ethnicities combined for all three event types;
Percentages of participants of each gender combined for all three event types;
Percentages of participants who are overall satisfied with ATTC events taken as a whole;
Percentages of participants who are overall satisfied with each of the three types of events;
Percentages of participants who shared information from all ATTC events combined with others;
Percentages of participants who shared information from each of the three types of events with others;
Percentages of participants who shared materials from all ATTC events combined with others;
Percentages of participants who shared materials from each of the three types of events with others;
Percentages of participants who applied what they learned from all ATTC events combined to their work;
Percentages of participants who applied what they learned from each of the three types of events to their work.
The National Office then presents the data in a brief, easy-to-read format for dissemination to key stakeholders.
17. Display of Expiration Date
The expiration date for OMB approval will be displayed on all data collection instruments for which approval is being sought.
18. Exceptions to Certification Statement
There are no exceptions to the certification statement.
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
1. Respondent Universe and Sampling Methods
The Center for Substance Abuse Treatment has funded the Addiction Technology Transfer Center Network to provide training and technical assistance and to meet with an estimated 35,000 individuals per year across 14 Regional Centers and the National Office. A wide variety of event participants are anticipated, ranging from addictions counselors to professionals in other fields who work with individuals with substance use disorders. Participants self-refer to all events provided by the ATTC Network. Since this is the case, the respondent universe is made up of participants of events of the ATTC Network.
Consistent with the approach used in the GPRA Customer Satisfaction Surveys for the CSAT Knowledge Application Programs, sampling methods will be used for the follow-up portion of this project. It was determined that the use of sampling in the follow-up phase of the performance monitoring would be most efficient because of the wide variety of training topics and formats offered by the ATTC Network. Sampling allows the ATTC Network to collect information on the entire variety of events, and to allow for comparisons by topic of event and by format of event. The Event Description Form collects information about the format and topic of events, allowing for comparisons between topics and formats. A random sample of 25% of participants who have consented for follow-up will be contacted 30 days after the event by e-mail or traditional mail to complete follow-up forms.
In 2000, a follow-up response rate pilot showed that the methodology that consistently reached 80% was phone follow-up; however, the cost was prohibitive for adoption across the ATTC Network. The Center using that methodology compared doing 80% response rate with a 20% random sample, and found that there was no difference between the results from the two groups. ATTC Directors and CSAT agreed on a 25% sampling plan for follow-up, with the aim of 80% response rate of that sample. Participants are asked at each event if they are willing to participate in a 30-day follow-up survey, and if so, how they would prefer to be contacted (e-mail or traditional mail). The sample is drawn randomly from the pool of consenting participants. ATTC Regional Centers are using a standard method for follow-up, with changes based on the contact method (e-mail or traditional mail). Sample cover letters for follow-up are attached (Attachment 7).
2. Information Collection Procedures
As explained previously, CSAT intends to use seven (7) instruments for program monitoring of ATTC events as well as for ongoing quality improvement. Three of the seven instruments have been previously approved and are being used to monitor the ATTC Network’s performance (OMB No. 0930-0216). CSAT is requesting approval at this time to revise two of these instruments (the Event Description Form and the Training Post Event Form) and to continue to use one of these instruments as is (the Training Follow-up Form). The remaining four instruments are currently approved by the OMB through a separate approval for CSAT Government Performance and Results Act (GPRA) Customer Satisfaction instruments (OMB No. 0930-0197). Two of these instruments (the Meeting Follow-up Form and the Technical Assistance Follow-up Form) are not being revised, and since they are covered under a separate OMB approval than is the focus of this application, CSAT is not seeking any action related to those two forms at this time. The two forms (the Meeting Post Event Form and the Technical Assistance Post Event Form) are being revised. CSAT is seeking approval in this application to utilize the revised versions of these forms in order to monitor the performance of the ATTC Network. CSAT is, therefore, requesting approval for five data collection instruments in total to monitor the ATTC Network at this time. The five data collection instruments for which CSAT is asking approval in this application are described below, summarized in Table 4, and are provided in Attachments 1 through 3. Attachment 5 clarifies the revisions being made to the Event Description Form and the Post Event Forms for Trainings, Meetings and Technical Assistance. Attachment 3 provides the Follow-up form for Training. Attachments 4-1 and 4-2 provide the Follow-up forms for Meetings and Technical Assistance which are currently being used by CSAT Best Practices Programs. Although these forms are not a part of the OMB application, CSAT has included them in the Attachments to provide clarity and context.
Event Description Form (EDF). The EDF collects descriptive information about each of the events of the ATTC Network. This form is completed by ATTC staff.
Training Post Event Form. This form is the same as the GPRA Customer Satisfaction Survey for CSAT KA Programs for Training (OMB No. 0930-0197), but with two additional questions on the participants' perceptions of their skills and knowledge in the topic area directly after the training event. This form is distributed to training participants at the end of the training activity, and collected from them before they leave. For training events which take place over an extended period of time, this form is completed after the final session of training. Revisions to the form currently in use will include modification the five current demographic questions on the form and addition of five new demographic questions to reflect a more current understanding of the field. The form will ask participants to report demographic information, education, profession, field of study, status of certification or licensure, workplace role, and employment setting.
Training Follow-up Form. This is the same as the one used for the GPRA Customer Satisfaction Survey for CSAT KA Programs for Training Follow-up, but with four additional questions on the domains covered in the POST, and on actual barriers to implementation. The follow-up is sent to a random sample of training participants 30 days after the end of the training event. No revisions are being made to the form currently in use.
Meeting Post Event Form. This is the same as the one used for the GPRA Customer Satisfaction Survey for CSAT KA Programs for Meetings immediately following the end of the meetings (OMB No. 0930-0197), but CSAT is revising the instrument by modifying the five current demographic questions and adding five new demographic questions to reflect a more current understanding of the field. The form will ask participants to report demographic information, education, profession, field of study, status of certification or licensure, workplace role, and employment setting.
Technical Assistance Post Event form. This is the same as the one used for the GPRA Customer Satisfaction Survey for CSAT KA Programs for Technical Assistance immediately following the end of the technical assistance (OMB No. 0930-0197), but CSAT is revising the instrument by modifying the five current demographic questions and adding five new demographic questions to reflect a more current understanding of the field. The form will ask participants to report demographic information, education, profession, field of study, status of certification or licensure, workplace role, and employment setting.
Event participants will be asked to complete the Post Event Form for the type of event they attended (meeting, technical assistance or training) at the end of the event. Prior to completing the Post Event form, participants will receive the Cover Page for Instruments (Attachment 8) and will be told about the purpose of the data collection (See Model Statement, Attachment 9). Participants will also be asked if they are willing to participate in the 30-day follow-up and if so, to provide contact information for their preferred method of contact (e-mail or traditional mail; see Attachment 10). This is often done in conjunction with the completion of a consent form (See Model Consent Form in Attachment 11).
Follow-up forms will be sent 30 days after the end of the event to a random sample of 25% of the participants willing to be contacted (See Attachment 7 for sample cover letters).
Table 4: Data Collection Methods
Form |
Data Collection Method |
Timeline |
ATTC Faculty/Staff |
||
Event Description Form |
Paper and pencil instrument |
Beginning of each training event |
Training Participants |
||
Post Event Training Form |
Paper and pencil instrument (10% online) |
Completion of each training event |
Post Event Meeting Form |
Paper and pencil instrument (10% online) |
Completion of each training event |
Post Event Technical Assistance Form |
Paper and pencil instrument (10% online) |
Completion of each training event |
Follow-up Training Form |
Mail-out instrument (Estimated 75% online via online survey tools) |
30 days after completion of training event |
Methods to Maximize Response Rate
The ATTC Network anticipates an 80% overall response rate for this performance monitoring project. The ATTC expects receiving at least 90% of the Post-event forms, as these forms are completed at the time and location of the event. The ATTC anticipates a 100% response rate on the Event Description Form, as this is completed by ATTC staff and/or faculty, and is a requirement of their position. The ATTC Network anticipates receiving 80% of the follow-up forms. The 25% sampling strategy described above is in place to allow for repeated attempts to contact participants for follow-up.
Methods to maximize response rates include:
Brief and easy to complete instruments;
One of the instruments is completed at the time and location of the training event;
One instrument is completed by ATTC training staff;
Standard language in the grant announcement requiring all ATTC regional sites to participate in the program monitoring;
Semi-annual reports summarizing findings to ATTC directors and CSAT;
Annual summary reports of findings for broader dissemination;
All sites will emphasize the importance of completing these instruments in order to continue the funding that provided the opportunity for the event for no cost or at low cost;
E
The ATTC Network has a history of a strong positive relationship with event participants who are receptive to the request for follow-up participation with a traditional single mailing methodology.
4. Tests of Procedures
For approximately the past thirteen years, the ATTC Network has successfully collected data for performance monitoring. Since minimal revisions are being made to the forms since the last OMB approval, no difficulty is anticipated in maintaining and continuing the past successes.
5. Statistical Consultants
Government Project Officer
Donna Doolin
Substance Abuse and Mental Health Services Administration
Center for Substance Abuse Treatment
1 Choke Cherry Road, Room 5-1126
Rockville, MD 20857
(240) 276-2965 (p)
(240) 276-2970 (f)
Government Branch Chief
Catherine Nugent
Substance Abuse and Mental Health Services Administration
Center for Substance Abuse Treatment
1 Choke Cherry Road, Room 5-1095
Rockville, MD 20857
(240) 276-1577 (p)
(240) 276-2970 (f)
ATTC National Office Coordinator
Laurie Krom
Director
Addiction Technology Transfer Center National Office
University of Missouri-Kansas City
5100 Rockhill Road
Kansas City, MO 64110
816.235.6985 (p)
816.235.6580 (f)
ATTACHMENTS
Attachment 1: Event Description Form
Attachment 2: Post-Event Forms
2-1 Training Post Event Form
2-2 Meeting Post Event Form
2-3 Technical Assistance Post Event Form
Attachment 3: Training Follow-up Form
Attachment 4: Government Performance and Results Act Customer Satisfaction Surveys for the Center for Substance Abuse Treatment Knowledge Application Programs (OMB No. 0930-0197)
4-1 CSAT Customer Satisfaction Survey ─ Meeting Follow-up Form
4-2 CSAT Customer Satisfaction Survey ─ Technical Assistance Follow-up Form
Attachment 5: Summary of Revisions
Attachment 6: ATTC Network Regional Centers
Attachment 7: Sample Cover Letters
Attachment 8: Cover Page for Instruments
Attachment 9: Model Statement
Attachment 10: Contact Information Form
Attachment 11: Statement of Informed Consent for Follow-up Interview
| File Type | application/msword |
| File Title | SUPPORTING STATEMENT |
| Author | proth |
| Last Modified By | DHHS |
| File Modified | 2010-04-06 |
| File Created | 2010-04-06 |