Evaluation Methodology

Evaluation Plan

Federal Strategic Action Plan on Services for Victims of Human Trafficking:

Enhancing the Health Care System’s Response to Human Trafficking

This project aims to strengthen the health systems’ response to human trafficking in four key ways:

1. Increasing knowledge about human trafficking among health care providers;

2. Building the capacity of health care providers to deliver culturally appropriate and trauma-informed care to victims of human trafficking;

3. Increasing the identification of victims of human trafficking; and

4. Increasing services to survivors of human trafficking.

To achieve these aims four sites throughout the country will be chosen to pilot the SOAR to Health and Wellness Training. The learning objectives for the training match the project aims, these objectives are for healthcare providers to

1. Describe the scope, severity, and diversity of human trafficking in the United States;

2. Recognize the common indicators and high risk factors for human trafficking;

3. Use trauma-informed techniques when interviewing a potential victim of human trafficking; and

4. Identify and engage local, state and national service referral options for trafficking victims.

The target population is 200+ healthcare providers in 4 sites (Site selection under way).

The evaluation is an impact evaluation, measuring immediate outcomes (i.e., from pre-intervention to post-intervention, with the intervention being 2-3 hours depending on the format to be used at each site), as well as intermediate outcomes at 3-month post intervention.

The evaluation methodology will include the following key components:

1. A standard pre-test administered 2-3 weeks prior to the training program via an email survey. Since participants will be required to register ahead of time, we will have their names and email addresses prior to the training event and will contact them with information about the training and a link to complete the pre-test. Up to three reminders will be sent to ensure that all participants complete the pre-test before the training.

2. A post-test with retrospective pre-test at the end of the training program. Standard pre- and post-test administrations in short (e.g., 1-day) trainings often fail to accurately measure increases in knowledge because respondents’ pre-test scores are naturally inflated; that is, participants often “don’t know what they don’t know” and their pre-test scores are often unrealistically high. During the actual training they often become aware of their gaps in knowledge and their post-test responses tend to be more tempered. As a result, the pre- and post-test instrument fails to accurately measure gains. To address this issue, some research and evaluation professionals have begun to use post-tests with retrospective pre-tests. That is, no pre-test is used at the start of the training event, and at the end of the training participants receive an instrument that asks them, for each item, to respond according to two separate time frames: before the training, and now, after the training. Since we are also using a standard pre-test emailed prior to the training, we will have the ability to compare the three time points.

3. A 3-month follow-up survey with ALL participants, via email. A modified pre-/post-test instrument will be emailed to all training participants 3 months after the training to assess whether gains in knowledge and skills or shifts in attitudes have been maintained and generalized, and whether the participants have changed their practice behavior as a result of the training. All participants will be told prior to and during the workshop that we will follow-up with them after three months, and we will obtain significant locator information so we can ensure contact at the 3-month mark. This locator information will include not only work-related data (address, telephone, email, etc.), but also their secondary or private email addresses, cell phones, and names of colleagues who will “always know how to contact them”.

4. A brief (20-minute maximum) 3-month phone interview with a subsample of participants. A second random subsample of participants will be selected and these individuals will be invited to participate in a follow-up phone call to get more specific qualitative data about changes in practice behaviors as a result of the training. The discussion will center on changes in their practice behaviors related to human trafficking since the training, number of victims they have identified and what the disposition of these cases was, and a sharing of lessons learned and challenges encountered. The subsample will be 20% of the participants; based on the target of 200 provider participants this represents 40 individuals, which is around the sample size considered sufficient to achieve saturation in qualitative research.

The evaluation instruments will assess the following domains (the actual items will be developed in tandem with the training program, but we list some examples of themes below):

1. Increases in knowledge about human trafficking:

1. Definition of human trafficking and types of human trafficking

2. Laws that stipulate how to handle suspected cases of human trafficking.

3. Vulnerabilities and at risk populations in urban and rural areas.

4. Types of medical conditions victims of various types of human trafficking are likely to present with.

5. Signs or indicators of possible human trafficking.

6. Agencies to collaborate with in delivering services to victims of human trafficking

(e.g., legal services; social services; federal, state, and local agencies; not-for-profit organizations; advocacy groups; etc.).

7. Elements of culturally competent service provision (e.g., knowledge of the cultural values, beliefs, and practices of various subgroups that could potentially impact how a victim of human trafficking might present)

2. Increases in skills to work with victims of human trafficking:

1. Assessment skills (e.g., what questions to ask and how to ask them).

2. Referral and navigation skills (e.g., what legal agencies, social services, etc. to tap into, how to make the referrals, and how to ensure that the victim follows through with the referrals).

3. Cultural competency skills (e.g., how a provider can use the knowledge he/she might have about the cultural values, practices, or beliefs of a client who is a victim of human trafficking to more effectively provide services to that client).

4. Trauma-focused/trauma-informed skills (e.g., how a provider can use knowledge of trauma and its aftermath to more effectively serve clients who are victims of human trafficking).

3. Measuring changes in the attitudes of healthcare providers about human trafficking and human trafficking victims:

1. Challenging erroneous or biased beliefs about human trafficking (e.g., confusing labor trafficking with day labor activities by unauthorized citizens, or sex trafficking with prostitution).

2. Challenging biases or stereotypes about who the victims of human trafficking are (e.g., “it only happens to folks who are illegal” or “only women are trafficked”).

4. Measuring intention to change practice behaviors regarding human trafficking and human trafficking victims.

1. How often provider thought about human trafficking (e.g., “this person might be a victim of human trafficking”) before training, to how likely they are to proactively think about it after the training.

2. How likely provider is to actually act on his/her suspicion that the client might be a victim of human trafficking, rather than assuming a stance of “not my problem, let somebody else deal with it”.

3. How likely the provider is to return to their practice setting and continue to inform him/herself and colleagues about human trafficking (e.g., updated a referral list, sharing what they have learned with colleagues through an in-service event, inviting other local legal or social service provides to come and present on the topic at their practice site, etc.).

5. Measuring actual changes in practice behaviors regarding human trafficking and human trafficking victims.

1. At the 3-month follow-up, how many in-service events on the topic have been held in provider’s practice setting?

2. How many new agencies have been added to the referral list (increases in referral networks)?

3. How proactively has the provider been in looking for signs and symptoms of human trafficking?

4. Increasing the identification of victims of human trafficking: How many victims of human trafficking has the provider identified in the 3-months since the training? How many had he/she identified in the 3 months before the training?

5. How many calls has the provider made to a human trafficking hotline in the 3-months since the training? How many had he/she made in the 3 months before the training?

6. Customer Satisfaction and Feasibility of Scale-Up Efforts. Additionally, since this is a pilot project, scale-up efforts will depend on how well the intervention is received by providers. Healthcare providers can be a challenging group to train due to their time constraints and their unstated reluctance to be trained by individuals who are not medical providers themselves. As such, we will measure some of the following domains as well:

1. Overall satisfaction with the training (e.g., length, content, audiovisual aids, facilitators, etc.).

2. Suggested changes to the content, gaps in the curriculum, areas they wish we had covered in more depth.

3. Likelihood to recommend the training to a colleague.

4