Cmhs Mhcpe Ss-a 2.21.14

PARTICIPANT FEEDBACK FORMS FOR THE

MENTAL HEALTH CARE PROVIDER EDUCATION

IN HIV/AIDS (MHCPE) PROGRAM

SUPPORTING STATEMENT

A. JUSTIFICATION

1. Circumstances of Information Collection 

The Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Mental Health Services (CMHS) is requesting a revision from the Office of Management and budget (OMB) for the approval of the use of standardized forms to collect systematic feedback from trainees participating in the Mental Health Care Provider Education in HIV/AIDS (MHCPE) Program. CMHS supports education for mental health providers through its HIV/AIDS education programs. The feedback forms and program assessment design for this program are used by education site staff in the current CMHS MHCPE Program and are approved under OMB No. 0930-0195, which expires March 31, 2014. CMHS is authorized to collect the data under 42 USC 290aa (Section 501(d) (4)) of the Public Health Service Act.

The overall goal of the education program is to help create a cadre of traditional and non-traditional mental health service providers who possess and utilize state-of-the-art information on the psychological and neuropsychological sequelae of HIV/AIDS, and to enhance the nation’s ability to have an impact on the HIV/AIDS epidemic. CMHS has used the participant feedback forms and over-all assessment design for over 10 years in its MHCPE Program. CMHS has used the multi-site assessment data to verify the integrity and efficacy of these organizations’ efforts to educate mental health workers, and thereby enhance the quality of services available to HIV-affected individuals. This information allows CMHS to continue to assess its success in creating a cadre of mental health service providers for HIV/AIDS-affected populations.

The 2012 CDC HIV Surveillance Supplemental Report, Estimated HIV Incidence in the United States, 2007-2010 includes national HIV incidence (new infection) estimates for the Unites States. Based on the 2010 data, CDC reported an estimated 47,500 new HIV infections occurred, and also confirmed that African Americans, and Hispanics/Latinos, and gay and bisexual men of all races/ethnicities were most heavily affected by HIV.¹ It is estimated that in the United States more than 1.1 million people are currently infected with HIV.² In addition, people of color living with HIV/AIDS continue to become critically ill and/or die at distressing rates despite widespread availability of highly effective HIV/AIDS medical treatments in the U.S.³

There is a continued growth in the need for mental health treatment for HIV affected individuals. Untreated and undiagnosed neuropsychiatric complications related to HIV and AIDS often lead to more serious problems, such as non-adherence with the treatment regimen, impaired quality of life, and increased morbidity and mortality. Individuals affected by HIV/AIDS confront critical life altering decisions in view of changing options for medical treatment particularly protease inhibitors and Highly Active Anti-Retroviral Therapy. Given the effects of HIV itself, coupled with effects of medication used to treat it, continuing education and relatively frequent updates for mental health services providers about developments in the treatment and psychological aspects of HIV care are crucial. The mental health practitioner’s role has become increasingly significant as the psychosocial and cultural issues surrounding the treatment of HIV/AIDS continue to grow in complexity. Mental health practitioners more than ever need to acquire training specific to the mental health needs of HIV-affected individuals across a wide variety of populations.

The MHCPE Program currently provides funding to three mental health professional associations: the American Psychological Association (APA), the American Psychiatric Institute for Research and Education (APIRE), and the National Association of Social Workers (NASW), and potentially for additional education site grantees, thus, the estimates of burden/cost are based on 10 sites. These trainers help to train and educate mental health professionals in their respective disciplines; taken together, the cadre of mental health professionals trained by these associations comprise a significant proportion of mental health providers that serve the HIV/AIDS affected population in our nation.

The theoretical and practical foundation for this round of funding comes from over 20 years of prior CMHS experience through its HIV/AIDS education programs. The CMHS MHCPE Program was designed to develop model approaches to educate mental health care providers in the neuropsychiatric, ethical and psychosocial aspects of HIV/AIDS. For over 10 years the MHCPE Program has funded education for mental health providers, and has conducted a multi-site assessment of the program. Over this period the MHCPE Program conducted more than 2,278 training sessions, and collected feedback regarding, for example, satisfaction with training and knowledge gained through training from over 36,300 participants. This represents an over-all response rate over 80% across the three training organizations. CMHS is able to effectively assess its MHCPE Program through this process over-all. Table 1 summarizes the two year response.

Table 1: Two Year Summary

	APIRE	APA	NASW	Total
Total Attendees	2,305	2,253	1,881	6,439
Total Returning Forms	1,350	2,030	1,834	5,214
Response Rate	58.6%	90.1%	97.5%	81.0%

CMHS funds the MHCPE Program to continue to enhance the nation’s impact on the HIV/AIDS epidemic. For each of their 5 years of funding, each professional education site is expected to train 1,000 mental health professionals. They reach primary target audiences of psychologists, psychiatrists and social workers, all of whom play significant roles in treatment for individuals affected by HIV and AIDS. Each site utilizes their own site-specific curricula and the CMHS curricula to educate mental health providers on the neuropsychiatric, ethical, psychosocial and treatment aspects of HIV/AIDS. CMHS is seeking approval from OMB to continue conducting a systematic multi-site assessment of the education provided by the funded education sites. The multi-site effort logically builds on and extends their activities. This multi-site assessment involves collecting information on the organization and delivery of the training sessions, as well as assessing the effectiveness of trainings. The multi-site feedback instruments collect descriptive information on each HIV/AIDS education training session using a Session Report Form to be completed by education site staff. Information on the effectiveness of the training as measured by participant satisfaction and increases in participant knowledge, skills, and abilities will be collected by feedback forms completed by participants. Participants attending sessions complete a single feedback form at the end of the training session. As of 2012, education sites collect these feedback forms in hard copy for in-person training and electronically for trainings conducted online (webinar, video, etc.). The education sites’ evaluators or their designees continue to be responsible for administering the instruments at training sessions and for determining whether participant feedback will be collected on paper or electronically. On a monthly basis, the education sites will submit the hard-copy data, for processing and preliminary analysis, to the CMHS subcontractor (electronic participant data is submitted directly to the CMHS subcontractor). Table 2 summarizes the proposed multi-site data collection strategy.

Table 2: Summary of Overall Data Collection Strategy

Curriculum	Feedback Form
	Participant Feedback Form	Participant Feedback Form (Neuropsychiatric Version)	Participant Feedback Form (Adherence Version)	Participant Feedback Form (Ethics Version)
General Education	X
Neuropsychiatric		X
Adherence			X
Ethics				X

2. Purpose and Use of Information

The information collected through the CMHS multi-site assessment effort benefits CMHS, the training sites, and the HIV/AIDS affected populations. The assessment data helps CMHS to continually improve and ensure high quality education programs that meet the needs of mental health providers serving those individuals most affected by the HIV/AIDS disease. This information also facilitates planning for future programs. For example, feedback from participants trained under prior years has helped CMHS to identify the need for additional education in specialized mental health issues.

The multi-site assessment activities are designed to help CMHS to fully describe the training sessions and participants served through the programs. CMHS uses the data collected under these programs to monitor the number of mental health providers attending trainings, participants’ demographic characteristics, and the effectiveness of training sessions. The data collected allows CMHS to understand the following organizational-level issues:

• The characteristics of participants attending CMHS-funded sessions, including demographic characteristics, primary work settings and extent of prior experience working with HIV-affected individuals;

• Topics covered at CMHS-funded trainings; and

• Educational methods employed to deliver the curriculum, including educational strategies used, material distributed, and involvement of HIV-positive individuals in the training.

This information is important to CMHS for ensuring that the education sites are serving the intended populations of traditional and non-traditional mental health service providers, delivering training sessions that cover the breadth of topics specified in their contracts (general, neuropsychiatric, ethics, adherence and other curricula), and documenting the methods employed in delivering the various training sessions. Ultimately, this feedback helps both CMHS and the individual sites to continuously monitor and improve the education curricula, including their design, implementation and methodology.

The multi-site program assessment also provides a quality improvement mechanism to help individual sites to monitor the effectiveness of the tools they use to deliver trainings, the organization of individual training sessions, and the training environment. The program assessment also allows CMHS to address individual-level issues:

• The extent to which trainees are satisfied with the trainings they receive;

• The extent to which trainees indicate that the training enhanced their ability, willingness and comfort in working with HIV-infected/affected individuals;

• The most effective types of trainings;

• Whether or not particular types of educational strategies and training delivery methods result in higher satisfaction levels than others;

• The characteristics of the education sites and sessions that are most effective in increasing trainees’ perceptions of enhanced work performance; and

• The characteristics of trainees who report greater satisfaction.

This project benefits CMHS, the education sites, and the HIV/AIDS service population by:

• Enabling CMHS to monitor the quality of its education programs;

• Enabling CMHS to assess the repertoire of skills and abilities of traditional and non-traditional mental health service providers;

• Allowing CMHS to provide feedback and design technical assistance for funded education sites in order to improve efficiency and training effectiveness;

• Helping CMHS to ensure that the education programs are disseminating current information to HIV/AIDS mental health service providers, thereby enhancing service provision to service populations; and

• Guiding CMHS in identifying model approaches to educating HIV/AIDS mental health service providers that can be widely disseminated.

Without this multi-site assessment, CMHS cannot empirically determine whether the funding of HIV/AIDS education is reaching the intended traditional and non-traditional mental health provider audiences. In addition, failure to conduct the program assessment would result in the diminished capacity of CMHS to provide targeted technical assistance to the education sites in order to improve the quality of education and training delivered. Without the assessment data, CMHS would lack the feedback needed to support continuous quality improvement and to ensure the needs of mental health providers and the HIV-affected populations they serve are being met, particularly for minority populations. Failure to collect this information and ensure the efficacy of educational trainings for mental health providers would potentially result in diminished capabilities of service providers and lower quality of services for HIV/AIDS-affected populations.

Changes

To ensure the efficiency of the MHCPE participant data collection process, participant feedback form data collected over the past two years of the contract were reviewed to identify outdated and/or rarely-used response options. The purpose of updating and streamlining a data collection tool is to enhance its effectiveness as a data collection source, to simplify completing the tool, and to reduce the burden associated with collection data.

Based upon this review of the data, CMHS identified some outdated and rarely-used response options for all participant response forms and the session reporting form, and removed these items from the individual data collection tools. No new questions or response options have been added to any of the data collection forms.

Table 3 shows the response options removed from the previous iterations of the MHCPE participant feedback forms and session reporting form.

Table 3: Changes to Participant Feedback Forms

Type of Feedback Form	Question #	Change(s)	Reason for Change
All Participant Feedback Forms (General Education, Neuropsychiatric, Adherence, Ethics)	Q7	Removal of response option “other”	Rarely /never used response option(s)
	Q8, Q9A	Removal of response option “Dentist/Dental Assistant”	Rarely /never used response option(s)
Session Reporting Form	Q6	Removal of the following response options: State/Local Department of Public Welfare HMO/Managed Care Organization Migrant Health Center Other MHCPE Program State/Local Department of Corrections	Rarely /never used response option(s)
	Q11	Removal of response option “Audio tapes”	Outdated response option

3. Use of Information Technology

Procedurally, each of the education sites mails or the participant electronically submits completed participant feedback forms to the CMHS evaluation subcontractor for data capture/ entry/analysis, depending upon whether the training was conducted in-person or online (via internet).

In 2012, the CMHS evaluation subcontractor piloted use of a web-based version of the General Education participant feedback form using Fluid Survey technology. Access to electronic versions of the form decreased the coordination required for education site staff regarding collection of feedback forms and for trainees to access and submit the required data. Upon reviewing training organization and participant feedback, and the efficiency of web-based data entry, electronic versions of all participant feedback forms (General Education, Neuropsychiatric, Adherence, Ethics versions) were made available to MHCPE trainers and training attendees in 2013. It is anticipated that the option to submit data electronically will increase response and form completion rates for trainings conducted online and reduce the burden on both education site staff and trainees.

The proposed multi-site data collection process increases the efficiency and practical utility of the assessment of these programs. The CMHS multi-site procedures and participant feedback forms were developed and tested, and have been used to evaluate the MHCPE Program for over 10 years efficiently and effectively. The participant feedback forms and the procedures for collection and transmission of data files have been used and improved based on program feedback, continually increasing the efficiency and minimizing the burden on both training participants and education site staff.

4. Efforts to Identify Duplication

The data to be collected are unique to the CMHS HIV/AIDS education programs, are collected only for the CMHS programs, and are not available elsewhere. No other multi-site assessment activities are planned for the education sites. The data collected through the multi-site effort will be non-duplicative, minimize burden on respondents, and be of use to both CMHS and the education sites.

In its assessment design, CMHS has developed procedures to minimize burden on trainees who attend multiple MHCPE training sessions. Participants are asked to complete feedback forms to provide demographic information and feedback specific to each of the training sessions they attend. In the event that participants attend more than one MHCPE-supported training session, they are requested to complete the training-specific questions for each session, but are asked to complete the demographic information only once. The demographic information can then be mapped back to each training session for which the individual provides feedback information.

5. Involvement of Small Entities

This project will have no significant impact on small entities.

6. Consequences If Information Collected Less Frequently

The data is collected one time only from respondents attending CMHS-funded training sessions. Each trainee completes a participant feedback form only once near the end of a training session.

Failing to collect the information from all participants attending CMHS-funded educational training sessions would result in a missed opportunity by CMHS to fully describe the participants served under these education programs, and to conduct a comprehensive assessment of the effect of the education programs. The information provides a quality improvement mechanism for CMHS to continually monitor and refine its education programs to ensure they meet the needs of mental health providers. Without this information:

• CMHS would not be able to determine the extent to which it has helped to build a cadre of mental health providers, especially minority mental health providers;

• CMHS would not be able to monitor the quality of its education program and determine how it can be improved to ensure continued success at meeting the needs of mental health providers and the mental health needs of individuals with HIV and AIDS;

• CMHS would not be able to fully describe the range of mental health service providers being trained, and the representation of minority mental health service providers;

• CMHS would not be able to ascertain if participants are more knowledgeable about HIV/AIDS as a result of attending the education session; and

• CMHS would not be able to identify additional mental health service provider needs, including the potentially unique needs of minority mental health service providers.

7. Consistency With the Guidelines in 5 CFR 1320.5(d)(2)

This information collection fully complies with 5 CFR 1320.5(d) (2).

8. Consultation Outside the Agency

The notice required in 5 CFR 1320.8(d) was published in the Federal Register on December 26, 2013 (78 FR 78374). No comments were received.

The multi-site design and participant feedback form design were based on initial consultation with experts in the field, and pilot testing. During the early stages of feedback form design, CMHS benefited from consultation with experts in the field of HIV training and education, design for collecting feedback, and feedback form development. Consultation with experts outside the agency was meant to minimize the burden on individual respondents and education site staff, to ensure the integrity of the form development, and to verify the appropriateness of the design for the program assessment. CMHS solicited input from consultants with expertise in HIV/AIDS, including clinical psychologists and psychiatrists, nurses, social workers, evaluation experts, HIV trainers, and directors of HIV/AIDS provider education programs. Input on the initial program assessment design and participant feedback forms was also solicited from four professional mental health provider associations that conducted HIV/AIDS education: the American Psychiatric Association, the American Psychological Association, the National Association of Social Workers, and the American Nurses Foundation. Additionally, as detailed in Section B4, a limited field test of the assessment design and instruments was conducted when the forms were initially designed for the MHCPE Program. The purpose of soliciting input from HIV/AIDS education site staff and participants was to gather feedback regarding the feasibility of the proposed multi-site program assessment and feedback forms. This initial feedback was used to modify the overall design and feedback forms to ensure consistency with ongoing training activities. Feedback was additionally solicited from the American Psychiatric Association, the American Psychological Association, and the National Association of Social Workers relative to changes to the participant feedback forms and session reporting form outlined in Section A2.

The assessment design and participant feedback forms have been used by MHCPE education sites for over 10 years. Current users of the forms have requested no revisions.

The assessment design and participant feedback forms were developed based on input from experts listed in Table 4.

Table 4: List of Experts Consulted

Experts Consulted Prior to the MHCPE II Program

John Anderson, Ph.D. American Psychological Association Office on AIDS (202) 336 – 6051	James Halloran, M.S.N., R.N., A.P.N. American Nurses Foundation (202) 651 – 7295
Charles Clark, M.D., MPH Florida Mental Health Institute (303) 442 – 6536	Carol Svoboda, M.S.W. American Psychiatric Association AIDS Program Office (703) 907-8668
Michael Dunham HI-Tech International, Inc. (703) 998 – 0287	Evelyn P. Tomaszewski, A.C.S.W National Association of Social Workers HIV/AIDS Spectrum Project (202) 408 – 8600, ext. 390
Michael Knox, Ph.D. Director, University of South Florida Center for HIV Education and Research Florida Mental Health Institute (813) 974 – 1925

Experts Consulted from the MHCPE II/III Program

John Anderson, Ph.D. American Psychological Association Office of AIDS (202) 336-6051	Cervando Martinez, Jr., M.D. University of Texas Health Science Center at San Antonio Department of Psychiatry (210) 567-4768
Francine Cournos, M.D. Columbia University (212) 543-5412	J. Stephen McDaniel, M.D. Emory University (404) 616-6310
Sally Dodds, Ph.D., LCSW University of Miami Department of Psychiatry & Behavioral Sciences (305) 355-9191	Ali Naqvi, Ph.D. Wayne State University AIDS Research and Education Program (313) 962-2000
Thomas Donohoe, M.B.A. UCLA Center for Health Promotion and Disease Prevention (310) 825-4750	Lisa Razzano, Ph.D. University of Chicago Mental Health Services Research Program (312) 422-8180, ext. 20
Abraham Feingold, Psy.D. (MHCPE II Steering Committee Chairperson) Boston, Massachusetts (617) 859-3953	Carol Svoboda, MSW American Psychiatric Association / Office on AIDS (202) 682-6104
Evelyn Tomaszewski, ACSW National Association of Social Workers (202) 336-8390	Diane Pennessi American Psychiatric Association/Office of HIV Psychiatry (703) 907-8668
David Devito, MPA American Psychological Association/HOPE Program (202) 216-7603	Jeremy Goldbach, Ph.D., LMSW National Association of Social Workers (832) 244-5437
David Martin, Ph.D., ABPP American Psychological Association/ Office on AIDS (202) 336-6051

9. Payment to Respondents

Respondents will not receive any payments.

10. Assurance of Confidentiality

CMHS has designed the multi-site feedback data collection strategy so that no identifying information such as names or complete social security numbers will be requested of trainees. All feedback forms only request the respondent’s month and day of birth. This information is not specific enough to be considered a unique identifier, but will nevertheless enable CMHS to estimate the extent to which trainees attend multiple training sessions at specific sites. To further ensure the privacy of individual responses, all data will be reported at the aggregate level so that individual responses cannot be identified; no data will be reported at the individual participant level.

11. Questions of a Sensitive Nature

No sensitive information will be requested in the multi-site participant feedback forms.

12. Estimates of Annualized Hour Burden

The total annualized burden for respondents for the Mental Health Care Provider in HIV/AIDS Education Program is estimated to be 1,843 hours.

The total burden to each of 10 potential respondent sites is estimated to be 184 hours. The total annualized hourly costs to Program participants across ten sites are estimated to be $4,713. The Center for Mental Health Services supports up to 10 HIV/AIDS education sites and each education site is required to provide training to at least 1,000 individuals per year. The estimates of annual hourly burden are therefore based on the assumption of 10 sites each serving 1,000 participants per year. The burden estimates also assume that education sites will provide on average 5 training sessions per month or 60 per year.

All trainees attending the CMHS-funded training programs are asked to fill out a hard copy or electronic evaluation form at the end of the training session that is expected to take a maximum of 10 minutes to complete.

There is considerable diversity in the types of participants attending the training sessions and in their wage rates. Occupations range from physicians and nurses to outreach workers and clergy. For the purposes of calculating the total annualized cost, a wage rate of $25.00 per hour was used since the Program intends to serve both traditional and non-traditional service providers. The burden estimates and resultant annualized costs are summarized below in Table 5.

The MHCPE Program is a continuation effort. This program consists of three associations and potentially seven grant supported education programs. All ten education sites are required to train a minimum of 1,000 mental health professionals per year using general, ethics, neuropsychiatric, neuropsychiatric for non-psychiatrists, and adherence curricula (all curricula are based on culturally competent mental health service provision). All sites have prior experience in providing HIV/AIDS related mental health training to traditional and non-traditional mental health providers. Each education site conducts about 60 trainings per year. Each site conducts the following types of training sessions: about 25 using the general curriculum, 12 using the neuropsychiatric curriculum for non-psychiatrists, 10 using the ethics curriculum, 8 using the neuropsychiatric curriculum, and 5 using the adherence curricula. The appropriate participant feedback form will be administered to trainees after each session.

Table 5: Annual Burden Estimate

Mental Health Care Provider Education in HIV/AIDS Program (10 sites)
Form	Number of Respondents	Responses Per Respondent	Total Responses	Hours per Response	Total Hour Burden	Hourly Wage Cost	Total Hour Cost ($)
All Sessions One form per session completed by program staff/trainer
Session Report Form	600	1	600	0.08	48	$25.00	$1,200
Participant Feedback Form (General Education)	5,000	1	5,000	0.167	835	$25.00	$20,875
Neuropsychiatric Participant Feedback Form	4,000	1	4,000	0.167	668	$25.00	$16,700
Adherence Participant Feedback Form	1,000	1	1,000	0.167	167	$25.00	$4,175
Ethics Participant Feedback Form	2,000	1	2,000	0.167	125	$25.00	$3,125
Total	12,600		12,600		1,843		$46,075

13. Estimates of Annualized Cost Burden to Respondents

No capital or start-up costs are involved nor is there any cost to respondents or record keepers resulting from the collection of information.

14. Estimates of Annualized Cost to the Government

The average annual estimated cost to the Federal Government for the multi-site program assessment is $395,000 for the 5-year MHCPE Program. This includes the costs associated with collecting feedback data, multi-site assessment and information dissemination. CMHS will fund ten education sites. For the purposes of calculating the annualized cost to the government, it is estimated that each education site will devote approximately 10% of their average annual award to multi-site assessment activities. Per site of the 10 sites, annual multi-site assessment-related costs are expected to be $18,500 for a total of $185,000, for conducting assessments with 1,000 participants each year/site. It is estimated that approximately $200,000 will be spent annually for overseeing the multi-site program assessment, processing and analyzing data, and preparing reports for their respective education sites. An additional $10,000 per year in Government monitoring costs, including travel, is anticipated. The total per year cost estimated for this program is estimated to be $395,000.

15. Changes in Burden

There is no burden change.

16. Time Schedule, Publication and Analysis Plans

The education sites in the MHCPE Program are funded for a period up to 5 years with annual awards being made subject to the continued availability of funds and progress achieved. The current program began its first funding cycle on approximately September 30, 2009 and a new program is scheduled to be released in 2014. A request for approval of use of the participant feedback forms is being re-submitted to OMB, now, in the middle of the last year of the 2009 funding cycle to ensure ongoing feedback as part of the cross-site evaluation.

Data collection will continue after CMHS has received OMB clearance for use of the proposed assessment design and participant feedback forms for the current forms set to expire on March 31, 2014. Education sites will receive a PDF version of the newly approved OMB forms for their use.

Education sites will mail completed forms to the CMHS subcontractor for data capture/entry for in-person trainings. Online training participants will submit completed electronic feedback forms directly through the web-based data entry system to the CMHS contractor.

The mental health professional association contractors are required to submit quarterly progress reports to CMHS. Additional specialized reports may be required.

Table 6 shows the major activities of the professional association education sites, and the anticipated dates of completion for the current project, which is to be completed in September 2014. CMHS expects to award a new grant program at that time.

Table 6: Projected Schedule of Activities and Timelines

Major Activity	Date
1. Education sites submit feedback forms to CMHS subcontractor	Monthly
2. CMHS subcontractor sends quarterly reports to their respective education sites	Quarterly
3. Education sites send quarterly reports to CMHS	Quarterly
4. All sites submit annual report to CMHS	October (yearly)
5. Final report from CMHS subcontractor	December 2014

On a monthly basis, the education sites submit hard copy multi-site participant feedback forms to the CMHS subcontractor for processing; electronic forms are provided automatically on submission by the participant. Upon receipt of the hard-copy feedback forms, the forms are briefly reviewed to ensure that information to be manually entered (e.g., session number and date, training and education site number) has been recorded. Forms then are keyed, and electronic datafiles are produced and electronically mailed to the CMHS evaluation contractor. Electronic feedback forms are submitted directly to the CMHS evaluation contractor through the web-based data entry system. Table 7 contains a data analysis plan that shows the major study questions, instrument items, and types of analysis used to answer the questions at the end of the program. Descriptive (e.g., frequencies, measures of central tendency), bivariate (e.g., chi square, paired t-tests, ANOVA), and multivariate (e.g., regressions) analyses will be conducted as appropriate. The CMHS subcontractor produces quarterly and annual reports on the aggregated data, across sites, for CMHS use in program monitoring. These reports are shared with the education sites for their reference.

Table 7: Data Analysis Plan

Program Assessment Question	Items on Instrument	Types of Analyses
Organization and Delivery of the Training
Characteristics of participants attending trainings.	Number of participants in session; demographic data; primary work settings; number of years provided services.	Descriptive statistics: Frequencies and Measures of Central Tendency
Topics covered by individual sites and across the Program.	Topics covered during training (e.g., epidemiology of HIV/AIDS, substance abuse issues, adherence to treatment).	Descriptive statistics: Frequencies and Measures of Central Tendency
Training methods used at education sites.	Type of curriculum used (general, ethics, neuropsychiatric); workshop length; training delivery method.	Descriptive statistics: Frequencies and Measures of Central Tendency

Impact of Training

Were the trainees satisfied with the trainings?	Questions on the organization of the training session and the usefulness of information/skills training.	Inferential statistics: Paired t-tests, ANOVA
Did trainees indicate that attendance enhanced their ability, willingness and comfort in working with HIV-infected/affected individuals?	Willingness to treat and/or care for HIV-positive/affected individuals; comfort working with HIV-positive/affected individuals; capability in treating and/or caring for HIV-positive/affected individuals.	Inferential statistics: Paired t-tests, ANOVA
Did trainees return to sites for additional training or updates?	Received any additional HIV/AIDS-related education since attending training session.	Descriptive statistics: Frequencies and Measures of Central Tendency
Were some types of trainings more effective than others?	Types of curriculum used; satisfaction with training; knowledge gained from training.	Chi Square Test of Significance; Content analysis of open-ended comments
Do particular types of educational strategies and training delivery methods result in higher satisfaction levels than others?	Types of Strategies/methods employed; type of curriculum used.	Regression Analysis; Content analysis of open-ended comments from trainees
What are the characteristics of education sites and sessions that are most effective in increasing trainees’ perceptions of enhanced work performance?	Type of curriculum used; involvement of HIV+ individuals in training; strategies/methods employed; materials distributed.	Regression Analysis; Content analysis of open-ended comments from trainees
What are the characteristics of trainees who report greater satisfaction?	Demographic data; type of curriculum used.	Regression Analysis

The CMHS Government Project Officer may also request special focused analyses. Among the statistical techniques that may be employed in producing special reports or publications are descriptive statistics, regression or logistic regression depending on the dependent variable, analysis of variance, t-tests and outlier analyses. These reports and publications also may also be presented at periodic meetings as well as regional and national conferences.

17. Display of Expiration Date

The expiration date will be displayed.

18. Exceptions to Certification Statement

There are no exceptions to the certification statement.

1 Centers for Disease Control and Prevention. Estimated HIV incidence in the United States, 2007–2010. HIV Surveillance Supplemental Report 2012;17(No. 4). http://www.cdc.gov/hiv/topics/surveillance/resources/reports/#supplemental. Published December 2012. Accessed November 2013.

2 Centers for Disease Control and Prevention. Monitoring selected national HIV prevention and care objectives by using HIV surveillance data—United States and 6 U.S. dependent areas—2010. HIV Surveillance Supplemental Report 2012;17(No. 3, part A). http://www.cdc.gov/hiv/topics/surveillance/resources/reports/. Published June 2012. Accessed November 2013.

3 Kaiser Family Foundation (2013). The HIV/AIDS Epidemic in the United States. Accessed November 2013 from http://kff.org/hivaids/fact-sheet/the-hivaids-epidemic-in-the-united-states/.

15