ADVHOCaT_SS_B - edited 1-24-18_clean

Accomplishments of the Domestic Violence Hotline, Online Connections and Text (ADVHOCaT) Study

OMB Information Collection Request

OMB No.: 0970-0468

Supporting Statement

Part B

March 2017

Submitted By:

Office of Planning, Research and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

Mary Switzer Building

330 C Street SW, 4^th Floor

Washington, DC 20201

Project Officers:

Samantha Illangasekare, OPRE/ACF

Tia Zeno, OPRE/ACF

B1. Respondent Universe and Sampling Methods

The Office of Planning, Research, and Evaluation (OPRE) and the Family and Youth Services Bureau (FYSB) within the Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services seeks approval to collect phone or web-based survey data from 2,200 individuals who contact the National Domestic Violence Hotline (The Hotline) and the loveisrespect helpline. This information collection is part of the Accomplishments of the Domestic Violence Hotline, Online Connections, and Text (ADVHOCaT) study. The purpose of the ADVHOCaT study is to describe the activities and outcomes of The Hotline and loveisrespect.

The possible universe of respondents to the phone and web-based exit and follow up surveys includes all adults 18 years of age and older who interact with an advocate at the National Domestic Violence Hotline (The Hotline) or loveisrespect by telephone or by online chat. Callers and chatters who interact with an advocate (i.e., “contactors”) include victims/survivors of domestic or dating violence, their friends and family, service providers, and others including batterers/abusers. Contactors who are transferred directly to a service agency immediately after interacting with the advocate will not be eligible to participate, so as not to interrupt their receipt of services.

Power Analysis

In order to answer the relevant ADVHOCaT research questions (see Supporting Statement A and Attachment A: ADVHOCaT Research Questions and Data Sources), we will examine the distribution of the responses to the categorical survey questions, qualitatively code the open text box responses, and conduct sub group analyses to determine if differences exist in the categorical responses by the mode of contact (phone or chat), demographic characteristics, and type of assistance received. The initial frequency analyses will determine the number of respondents and the percentage of total respondents for each question that prompt a categorical response category. The follow up survey results will be compared with the exit survey results to assess for changes in satisfaction and behavior over time (approximately two weeks).

In order to detect small to medium differences between subpopulations at follow up, we estimate requiring a total sample of 2,200 participants (1,100 participants for phone and 1,100 for chat). A previous evaluation of Safeline, a suicide crisis hotline, achieved a follow up rate of 45.2% and 35% for a follow up survey after 4 weeks and 6 weeks, respectively. ¹ Assuming the more conservative rate of follow up (35%), we project as few as 385 callers and 385 chatters will complete the follow up survey. This sample size would still allow the detection of small differences in outcomes by modes of contact (phone and chat). However, we are estimating burden based on the total sample size should 100% of the callers and chatters complete both the exit and follow up surveys. The evaluation will also seek to determine if there are differences between outcomes for adult and young adult contactors. This sample size will allow for bivariate analyses to detect medium to small differences between short-term outcomes for adults and young adults, which would require a sample of at least 176 per category (see Table B below).²

Table B: Sample size needed to detect small and medium differences between two groups   (based on two-tailed hypothesis, 80% power, group1 centered at 0.50, and significance level of 0.05 or less)
Effect size	Equals a difference of	Minimum number per category
Effect size of 0.2 (small)	10%	384
Effect size of 0.3	15%	176
Effect size of 0.4 (medium)	20%	91

B2. Procedures for Collection of Information

After IRB and OMB approval, all contactors over the age of 18 who interact with advocates at The Hotline or loveisrespect via phone or chat will be asked if they would like to complete an exit survey and a follow-up survey approximately two weeks later (see Attachment B: ADVHOCaT Recruitment Script). As noted above, contactors who are transferred directly to a service agency immediately after interacting with the advocate will not be eligible to complete the surveys.

Phone contactors who agree to complete the exit and follow up survey will be transferred to a Hotline research staff person. This Hotline research staff person will be trained using the CITI research ethics training program. This trained research staff person will review informed consent information with the participant and ask for verbal consent (see Attachment C: ADVHOCaT Informed Consent). The research staff person will then read the exit survey questions (see Instrument 1: ADVHOCaT Exit Survey). All responses will be inserted manually by the research staff into the Advocate Caller Application database (ACA), which is the database that advocates use to record information about each contactor.

Chat contactors who agree to complete the exit and follow up survey will be provided a link to the web-based survey. The first page of the survey will contain the informed consent information and ask for consent (see Attachment C: ADVHOCaT Informed Consent). If the participant agrees to participate, the exit survey questions will appear on subsequent web pages (see Instrument 1: ADVHOCaT Exit Survey). All information from the web-based chat exit survey will be directly inputted into the exit survey databases.

After completing the exit survey research questions, the research staff person or web-based survey will ask the participant to choose one of the following options for completing the follow up survey:

1. The Participant contacts The Hotline directly in two weeks. If the participant chooses this option, s/he would be asked if s/he would like to receive a reminder via text message. The reminder would contain a safe message (such as "have a nice day") that only the contactor would understand as a reminder to call a toll-free number or access the the URL (https://goo.gl/TmYGWu) to complete the follow-up survey.

   1. If the participant would like a text or email reminder, the research staff person will ask for the participant’s phone number. The research staff person will send a safe reminder³ message by phone in approximately 12 days to give notice to the participant that a follow-up survey should be completed in two days.

   2. Whether the participant decides to receive a text reminder or not, the research staff person will provide information to the participant on how to complete the follow up survey by phone and online and will ask the participant to write this information down, as well as the date and time to call or go online, in a safe place. The participant can decide at any time to complete the survey by phone or online⁴.

2. The participant receives a call from The Hotline in two weeks. If the participant chooses this option, the research staff person will ask the participant for the best day and time to receive a call. This day and time should be within the next 13-15 days. The research staff person will ask the participant to write down the selected day and time. The research staff person will then ask for the participant’s number to safely call in two weeks’ time. The research staff person will instruct the contactor to only accept the call at that day/time if and only if it is safe to do so. The research staff person will ask the participant if s/he would like the person who calls them to say initially that s/he is calling from somewhere other than the hotline (e.g. a flower shop, a restaurant, or a drugstore) in case someone else answers the phone.

The research staff person or web-based survey will ask the participant to select a code word, such as a favorite fruit or place. When contacted for the follow-up survey, the participant will be asked to provide this code word. The participant can also use this code word as a ‘safe’ word at any time to alert the research staff person that the participant no longer feels safe to complete the survey. Alternately, the research staff will also let the participant know that s/he can hang at anytime if s/he does not feel safe.

The participant will be asked to provide any unique 4-digit number. The research staff will then include this number in the ACA database and provide this number to the participant and suggest that s/he write it down and store it in a safe place. In addition, the research staff person will then ask the participant to choose a pseudonym that he/she will be referred to during the follow-up interview.

The research staff person or web-based survey will inform all participants that they can opt out of any contact with The Hotline/ loveisrespect by texting STOP to a provided phone number to a dedicated text line for this project. The research staff person will let the participant know that if the participant choose to text STOP, the phone number they use must be the same as they provide to the research staff. If a participant texts STOP to the text line (512-203-9775), the research staff person will manually enter this information into the ACA databause.

The research staff person or web-based survey will provide guidance about how to remember these pieces of information (i.e., places where s/he can save the information if remembering it might be difficult), including the date for the follow-up interview. The participant will be asked to think about ways of remembering the date of the follow-up interview and calling or accessing the follow-up survey. For instance, the research staff person might say "The 10th of February will be the Friday before Valentine’s Day. Where will you be?"

The research staff person or web-based survey will record the date and time that the participant chose to complete the follow up survey, and her/his phone number, and date for the text reminder (as applicable) into the ACA database

When the research staff person begins a conversation by phone with a participant to complete the follow up survey for detailed language), he/she will first determine whether the participant is able to talk at that time, and then ask for the participant’s pseudonym, code word, and 4 digit case number. If the participant correctly provides two out of the three pieces of information, the research staff person will proceed with asking the follow-up survey questions (see Attachment F: ADVHOCaT Follow Up Survey Script). The research staff person will use a script to conduct the follow-up survey via phone or online. A participant can complete the follow-up survey in one of three ways: (1) the participant calls a dedicated phone line at The Hotline/ loveisrespect; (2) The Hotline/ loveisrespect calls the participant at a number provided by the participants; or (3) the participant accesses an online survey via a dedicated password protected URL link, https://goo.gl/TmYGWu.

We estimate that the exit survey will take approximately 8 and half minutes to complete and the follow up survey will take approximately 6 minutes to complete.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Expected Response Rates

The Hotline and loveisrespect have conducted surveys of their contactors in the past on a range of topics related to domestic violence and dating abuse for their own organizations’ purposes. The number of responses to these surveys varied based on the specific topic. Most recently, The Hotline and loveisrespect conducted phone and chat exit surveys for their own operational purposes that included questions similar to those for this ICR. Approximately 5.62% of all eligible callers and 17% of all eligible chatters responded to these recent exit surveys. Based on the power analysis described in Section B1, we plan to offer the exit survey for up to 19,563 callers and 6471 chatters in order to achieve a sample size of 1100 by phone and 1100 by chat. As noted above, studies of other hotlines have had rates of follow-up as low as 35%. Assuming this follow-up response rate, as few as 385 callers and 385 chatters could complete the follow up survey. While this sample size would still allow the detection of small differences in outcomes by modes of contact (phone and chat), we are estimating burden based on the total sample size should 100% of the callers and chatters complete both surveys.

Dealing with Nonresponse

Only those individuals who are interested in participating in the surveys will be included in this data collection. If contactors do not want to participate, they do not have to agree to do so. Additionally, if they do not consent after hearing or reading the consent information, they do not have to participate in the surveys. Survey participation will have no bearing on the services that contactors receive.

If participants who choose to receive a call from a research staff person to complete the follow-up survey are ultimately unavailable at their predetermined time, no further attempt will be made to contact the participant, in order to protect the participant’s safety.

If in the original follow-up contact, the research staff was able to complete only half or less of the follow-up survey, the research staff will ask if the participant wants to continue to complete the survey on the phone at a later time. In addition, the research staff person will remind the participant they can complete the follow-up survey online via the dedicated URL link, https://goo.gl/TmYGWu, and can complete it at any time.

Once data collection is complete, the data will be analyzed to determine missing data and determine if there are any noticeable patterns (i.e., all males skip a question). We will monitor the distribution results to assess for missing data and patterns on a monthly basis. Any published information will discuss possible nonresponse biases.

Based on response rates to past surveys conducted by the Hotline and loveisrespect, the study team expects to receive 1100 responses for each modality (i.e., 1,100 for chat and 1,100 for phone) by offering the exit survey to up to 19,563 callers and 6471 chatters. If fewer individuals choose to partake in the surveys, then the study team will work with The Hotline and loveisrespect to examine reasons for nonresponse and to strategize ways to decrease nonresponse.

Maximizing Response Rates

In order to maximize response rates, staffers will be trained to effectively communicate with contactors in a compassionate manner, taking time to listen to and answer contactor questions. Staffers will ensure contactors are aware of the precautions for maintaining the safety and confidentiality of the contactors information.

We recognize that allowing participants the option of contacting The Hotline to complete the follow up survey (instead of asking all participants to provide contact information to be contacted directly by The Hotline) might reduce response rates to the follow up survey. However, providing this option is important to those contactors who do not want to be contacted directly by The Hotline in order to maintain their safety. In order to maximize response rates, we will provide an option to receive a reminder via text, as suggested by members of our expert panel. So as to protect the safety of participants, the reminder message will simply say "Have a nice day” and will be sent from an unidentifiable number.

B4. Tests of Procedures or Methods to be Undertaken

The study staff will use mock scenarios to internally test the protocols for data collection via the phone and the online survey. This internal testing will include a test of the survey links (URLs) in order to determine that they are operational and that the data collected through the survey is accurately captured in the Advocate Caller Application database.

B5. Individual Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The study design plan and data collection protocols were developed by study staff at the George Washington University Milken Institute of Public Health who have experience in the area of domestic violence research and statistical analyses of large data sets. These include Dr. Karen McDonnell (Principal Investigator), Dr. Bart Bingenheimer (Lead for Quantitative Methods), Dr. Holly Mead (Lead for Qualitative Methods), Dr. Susan Wood (Policy Expert) and Nitasha Nagaraj (Research Scientist).

Input was also provided by the ADVHOCaT project officers at OPRE (Samantha Illangasekare, Aleta Meyer and Tia Zeno) and FYSB program staff (Angela Yannelli and Rebecca Odor). Further input was provided by the ADVHOCaT expert panel members: Wendi Cross, Tami Jerue, TK Logan, Terri Poore, Cris Sullivan, and Carmen Williams.

1 Gould MS et al. (2007). An evaluation of Crisis Hotline Outcomes Part 2: Suicide Callers. Suicide and Life-Threatening Behavior (37) 3: 338-352.

2 Based on the distribution of current contactors of The Hotline/loveisrespect, a total sample size of at least 770 will enable us to detect differences between outcomes for young adults and adults. Among a sample of 770 respondents, 60% (n=462) are projected to be adults and approximately 40% (n=308) are expected to be young adults based on previous follow-up rates as seen in the work of Gould MS¹. Given our expected follow up rate of 35%, we could therefore expect that a minimum of 176 adults and 176 young adults will complete the follow up survey.

3 So as to protect the safety of participants, the reminder message will simply say "Have a nice day” and will be sent from an unidentifiable number.

4 Participants will be instructed to complete the follow-up survey in 2 weeks, but will be able to access the survey by phone or online at any time. The time at which the follow-up survey is completed will be controlled for in analysis.

7