Memo to OMB on CCVS

U.S. Department of Justice

Office of Justice Programs

Bureau of Justice Statistics

Washington, D.C. 20531

MEMORANDUM

To: Office of the Chief Statistician of the United States

Office of Management and Budget

Through: Kevin M. Scott, Ph.D.

Acting Director

Bureau of Justice Statistics

Shelley S. Hyland, Ph.D.

Senior Statistical Advisor

Bureau of Justice Statistics

Heather Brotsos

Deputy Director, Statistical Programs Division

Bureau of Justice Statistics

Rachel E. Morgan, Ph.D.

Unit Chief, Victimization Statistics

Bureau of Justice Statistics

From: Alexandra Thompson

Statistician, Victimization Statistics Unit

Bureau of Justice Statistics

Date: April 2, 2025

Subject: Request to cognitively test questions for a campus climate victimization survey (CCVS), under the BJS OMB generic clearance agreement (OMB Number 1121-0339)

1. Introduction

The Bureau of Justice Statistics (BJS), through a cooperative agreement with RTI International (Award: 15PBJS-23-GK-02626-CAMP), is developing a campus climate victimization survey (CCVS) questionnaire that will enable all types of postsecondary institutions to collect data to understand the magnitude and nature of the types of interpersonal violence (IV) students at postsecondary institutions experience. IV includes domestic and dating violence, sexual harassment, stalking, and sexual assault. The Violence Against Women Act Reauthorization Act of 2022 (VAWA 2022) directs the Department of Education (ED) to “develop, design, and make available through a secure and accessible online portal” a standardized CCVS that all postsecondary institutions receiving federal assistance will use to collect the data and information they need to address IV¹. BJS is requesting clearance for tasks related to cognitive interviewing and testing of the proposed CCVS instrument, under the BJS OMB generic clearance agreement (OMB Number 1121-0339). BJS is leading the design and development of this survey instrument that can be used by ED to administer per the VAWA 2022 requirements. BJS has coordinated with ED throughout the instrument development process to ensure final deliverables align with priorities for future implementation. BJS is well positioned to lead this instrument development work given we have conducted significant research and development work on this topic.

The CCVS is designed for use in all postsecondary institutions in the United States and focuses on students’ general experiences of campus climate and safety, experiences of interpersonal violence and domestic violence, and knowledge of reporting and institutional policies. These data will inform efforts to improve the content, timing, and delivery of prevention and responses designed to reduce IV among all students and to better address their service-related needs, thereby improving public safety and public health.

This memo first provides information on previous CCVS research efforts, details the proposed campus climate victimization modules, and the item review and selection processes. Next is a description of the proposed testing procedure, followed by a description of language, burden hours, reporting, protection of human subjects, informed consent, and data confidentiality and security.

This memo requests clearance for cognitive interviewing and testing only. BJS is seeking approval to administer two rounds of cognitive testing with 30-60 respondents (total across two rounds) focused on testing the proposed CCVS. Further deployment of the proposed CCVS instrument through a pilot test will be requested through a separate clearance.

2. History of Campus Climate Surveys (CCSs)

BJS and RTI collaboratively developed and fielded a larger CCS at nine postsecondary institutions in 2016 (Campus Climate Survey Validation Study), building on work initially conducted by the National Institute of Justice (NIJ). The IV experiences of students at 4-year institutions, as measured through existing CCSs, are fairly well-documented, as are the factors that are believed to facilitate IV, such as binge drinking, fraternity and sports culture, social norms, and university policies^2,3. However, much less is known about the experiences of students attending other types of postsecondary institutions. This project seeks to design and test a survey instrument that is both responsive to the VAWA 2022 statutory requirements, extant literature, and existing validated CCSs, and can be deployed by ED to collect relevant information from students at all types of postsecondary institutions.

3. Proposed CCVS Modules, Item Review, and Selection

Table 1 presents the modules and constructs BJS and RTI are recommending be included in the CCVS instrument. Some modules are intended for all respondents to receive, designated as core modules, and some are modular or randomized to a subset of respondents. Core modules are intended to be administered to all respondents and randomized modules are those that will be randomized across a respondent pool (i.e., only a portion of respondents from each institution receive a module). The purpose of having core and randomized modules is to minimize burden on respondents, while maximizing sample sizes and statistical power for events with relatively low prevalence.

Table 1: Proposed CCVS Modules and Constructs

Modules	Construct(s)
Core Modules—All Respondents Receive
Demographics	Education, employment, and living/housing; Personal characteristics (e.g., age, race/ethnicity, disability status, sex, sexual orientation)
Domestic Violence/ Dating Violence	Physical abuse; Emotional/psychological abuse, Coercive/financial control; Prevalence, incidence, tactics, and mode (e.g., cyber, in-person)
Sexual Assault	Sexual assault, rape, and sexual battery; Prevalence, incidence, tactics, and mode
Sexual Harassment	Prevalence, incidence, tactics, and mode (e.g., cyber, in-person)
Stalking	Prevalence, incidence, tactics, and mode (e.g., cyber, in-person)
Incident Characteristics	Severity of incident, including injury; sought medical care; location, month/year, situation; Relationship to perpetrator(s) (v-o relationship and student status of perpetrator); alcohol- or drug-facilitated
Outcomes/Impact/ Consequences of Victimization	Changes in education, employment, residence; Academic satisfaction; Academic disengagement; Time and cost (e.g., injury, time away from school/work, health-related costs, relationship changes); Mental health (e.g., depression, anxiety, posttraumatic stress disorder, self-esteem)
Victimization Reporting	Unofficial reporting or help-seeking (friends and family); Official sources (e.g., campus/local police and service providers; administrators, faculty, staff); why victim chose to report/not report; whether an investigation was conducted if reported
Randomized Modules—Respondents Randomly Assigned to Receive a Subset of 3 Modules
Campus Climate and Attitudes	Institutional response; Knowledge of campus resources; Sense of safety; and Perception of sexual misconduct as part of campus life; Attitudes about violence
Bystander Intervention	Bystander behavior and experiences; Bystander attitudes
Prevention and Training	Awareness, participation, and satisfaction; Topics covered in training (consent, mandatory reporting, confidentiality)

RTI, BJS, and ED reviewed many existing and validated CCSs to ensure the project team is responsive to the needs and offerings of the field. In addition, RTI, BJS, and ED ensured the proposed CCVS instrument is responsive to the statutory requirements of VAWA 2022. This review informed the final draft CCVS instrument, which is called the Student Perspectives and Experience Survey (SPES; Attachment A). Some of the surveys reviewed and relied on most heavily include:

• Administrator-Researcher Campus Climate Collaborative (Georgia State University)

• Association of American Universities (Westat)

• Campus Climate Survey Validation Study (BJS and RTI)

• Campus Climate and Sexual Violence Experiences of Students Attending Historically Black Colleges and Universities (NIJ)

• Cultivating Learning and Safe Environments (The University of Texas System)

• iSPEAK Campus Climate Survey (Rutgers University)

After assessing all of the content in these surveys, RTI, BJS, and ED undertook an iterative process through which all modules, items, response options, instructions, etc. were reviewed and rated. The results of this process yielded a full draft of the SPES that was formatted for visual appeal and survey programming ease. The project team then requested guidance and input from an Expert Panel. After receiving written feedback and guidance through an interactive meeting, the team was able to finalize the draft SPES instrument for cognitive testing.

Cognitive testing of the SPES will help determine whether respondents understand the questions, response options, and terminology used, and whether any misunderstanding has the potential to impact measurement and data quality. Cognitively testing the SPES will provide an empirical basis for determining whether questions are understood as intended and qualitative insights into how the SPES can be improved. Incidents of IV discussed during cognitive testing would have occurred sometime in the prior 12 months to survey administration, which is the same time frame that will be used in the pilot test. The proposed testing efforts include a particular focus on the following issues.

1. False positive responses – are respondents able to accurately identify the types of victimization incidents that are covered by the questions; are respondents answering IV questions affirmatively based on experiences that are within the definitions and scope of the survey; are respondents able to determine if/when incident(s) took place within the reference period(s)?

2. Understanding of terminology – do respondents understand the terms being used and what the questions are asking; are there other terms and phrasing that better or more simply convey the intended meaning of the questions?

3. Wording preference – do respondents prefer certain language approaches over others (e.g., hooking up with vs. dating)?

3. Testing Procedures

4.1 Cognitive Interviewing Overview

Cognitive interviews are an important tool for evaluating respondents’ understanding and ability to accurately answer survey questions. Cognitive interviews involve an interviewer administering the survey questions to a potential respondent and asking that respondent about how they interpreted the question, how difficult it was to answer, and their process for formulating their response. Cognitive interviews are generally conducted prior to fielding BJS survey instruments that are new or have been altered considerably.

Through cognitive interviews, RTI will delve into the thought processes of an array of postsecondary students, assess whether the SPES questions are comprehensible and widely understood, and learn how they can be improved iteratively, maximizing validity and minimizing measurement error across respondents and groups. BJS is proposing to conduct two rounds of cognitive testing with 30-60 respondents (total across two rounds) age 18 or older via a videoconferencing platform, such as Zoom. RTI will recruit students from a wide variety of postsecondary settings who have experienced various types of IV. RTI will ensure representation of students from different communities and whose experiences are underrepresented in CCVS research. The survey will be revised and improved iteratively between rounds (see Attachment B for the Initial Draft Cognitive Interview Protocol).

The cognitive testing is expected to take place from April 2025 through June 2025. Cognitive interviewing provides the opportunity to ask victims, as well as some non-victims, about their perceptions and understanding of the questions. For non-victims, hypothetical scenarios will be used to understand how and why respondents answer the IV and campus climate questions based on varied elements of potential victimization scenarios.

4.2 Recruitment and Screening

RTI plans to recruit respondents through relationships with colleagues and staff at all types of institutions around the country; and if necessary, via sites like Facebook and Instagram. These platforms have been used to successfully recruit respondents for other IV cognitive testing studies, in an effort to ensure students and victims of all types are included and heard (see Attachment C for Recruitment Language). RTI anticipates up to 500 students will fill out the screener because of recruitment efforts, including high participation from social media outreach.

RTI will employ an online brief screening tool to aid in recruitment efforts (see Attachment D for the Online Screener). They will be asked a few demographic questions (e.g., age, current attendance at different types of postsecondary institutions, and experiences with interpersonal violence), some general questions about IV experiences, and if they are interested in participating in a cognitive interview. Those who are interested will also be asked to confirm that they have access to:

1. A private and safe area of their home (or another setting) where they can complete the interview out of earshot of other people and without interruption.

2. A device with both audio and video capabilities for completing the interview, including a laptop, desktop, tablet, or smartphone.

3. Wi-Fi, internet, or cellular service with enough available data to participate in a 60-minute video interview.

Respondents who are not at least 18 years of age at the time of the study will not be eligible to participate. Screener participants will not be compensated. Once interested persons who meet the eligibility criteria are identified, the recruiter will reach out to eligible respondents via email to:

• provide an overview of the study,

• explain that respondents who complete the cognitive interview will be offered a $40 electronic Amazon.com gift card to compensate for the costs associated with data and internet usage,

• share an electronic copy of the informed consent form, and

• provide a calendar for the potential respondent to enter availability for the cognitive interview, if he or she agrees to participate.

The goal will be to recruit about 15-30 respondents who identify as interpersonal violence victims and about 15-30 non-victim respondents. In round 1, about 30 participants will be recruited from the pool of respondents who filled out the screener. In round 2, up to 30 more participants will be recruited from those who answered the screener. If the recruitment effort results in a larger number of victims who express interest in participating, they will be prioritized over the non-victims. RTI will attempt to recruit victims and non-victims with as much variation in geographic location, demographic characteristics, postsecondary institution type, and incident characteristics as possible.

Once the respondent has provided dates of availability, the recruiter will send a calendar invitation, including a link to access the videoconferencing platform, to both the potential respondent and the interviewer (see Attachment E for Email and Phone Scripts). The recruiter will again attach a copy of the informed consent form to the calendar invitation. A day before the scheduled interview, the recruiter will send the potential respondent a reminder email or text with the date and time of the scheduled interview. A final follow-up reminder will be sent on the morning of the interview.

4.3 Consent/Assent Procedures

At the start of the interview, the interviewer will introduce themself to the participant, confirm the participant’s name, and confirm that the participant is on video and can hear the interviewer well. The interviewer will then ask the participant to confirm that he/she is in a private area of their home or other private setting (out of earshot distance of other people). The interviewer will ask the participant to let her know if at any point during the interview, the respondent is interrupted or if they no longer feel they are in a private setting.

The interviewer will then read through the entire informed consent form (see Attachment F for Cognitive Interviewing Informed Consent), providing an opportunity for the respondent to ask any questions. The interviewer will document the respondent’s decision to participate, including the respondent’s willingness to have the interview recorded, and the interviewer will sign and date the consent form as a witness. The form for the respondent includes a list of national numbers to contact for resources on and assistance with interpersonal violence.

4.4 Cognitive Testing Procedure

The respondent will not fill in the questionnaire prior to the cognitive interview. Questions will be displayed on the screen and trained RTI cognitive interviewers will lead respondents through a cognitive interviewing guide (Attachment B). RTI plans to administer the entire instrument in one hour, but will be prepared to rotate modules, if needed, to guarantee that all content gets tested thoroughly by multiple respondents.

Each cognitive interview participant will be compensated $40 upon completion of the interview. The respondent has the option of receiving an electronic Amazon.com gift card via email or text to compensate for the costs associated with data and internet usage.

3. Burden Hours for Testing

The maximum burden associated with the proposed cognitive testing is presented in Table 2. In total, respondent burden is estimated to be 82 hours. This burden comprises three components: the initial screener survey to recruit respondents, scheduling interviews with respondents, and conducting the cognitive interview. The current request for approval is for two rounds of cognitive testing of the SPES instrument with up to 30 respondents in the first round and 30 in the second round. The second round of cognitive testing will use an instrument updated based on feedback from the first round of interviews.

Table 2. Maximum Burden Associated with Planned CCVS Cognitive Testing Activities

	Maximum # of Respondents	Average Administration Time (minutes)	Burden (hours)
Screener survey	500	2	16.7
Recruitment Scheduling	80*	4	5.3
Cognitive interviews –Round 1	30	60	30
Cognitive interviews – Round 2	30	60	30
Total	500	~	82

* Assumes some natural recruitment and scheduling attrition so that a maximum of 60 interviews can be conducted.

3. Language

The online screening and cognitive interviews will be conducted in English only.

3. Timeline

Milestone	Start Date	End Date
Obtain OMB generic clearance	04/07/25	04/21/25
Recruitment and testing period	4/21/25	06/02/25
Analyze data and implement CCVS instrument revisions	06/03/25	06/16/25
Draft final cognitive interviewing report	06/16/25	06/30/25

3. Data Confidentiality and Data Security

BJS is authorized to conduct this data collection under Title 34 United States Code, Section

10132. BJS may use the information it collects only for statistical and research purposes and

must gather it in a manner that precludes its use for law enforcement or any purpose relating to a

private person or public agency other than statistical or research purposes [Title 34 U.S.C.

Section 10134]. By law, BJS is required to protect the confidentiality of information identifiable

to a private person and safeguard it against unauthorized disclosure or misuse. [Title 34 United

States Code Sections 10134 and 10231]

The BJS Data Protection Guidelines provide more detailed information on how BJS and its data

collection agents will use and protect data collected under BJS’s authority. All project staff are required to sign a pledge of confidentiality and privacy certificate which confirms the maintaining of data and following the procedures outlined above. Furthermore, all cognitive interviews will be conducted by RTI project staff, who must complete official security training.

All data related to the cognitive interviews will be stored on a secure drive, including the recordings of interviews, on password-protected RTI-issued laptops with restricted access to those directly involved in the effort. The files will be destroyed upon completion of the project.

3. Cost

The cost of incentives for the cognitive interviews will be a maximum of $2,400 ($40 x 60 respondents). The estimated annual cost to the Federal government is $34,124 for RTI’s portion of the work and an estimated $12,058 for the GS-13 project manager’s work (10% of salary), resulting in a total estimated cost of $48,582 [$34,124 + $12,058 + $2,400].

3. Reporting

Upon completion of the cognitive interviewing, RTI will provide BJS with a report describing the findings and recommendations regarding which version of the questions appeared to function best in terms of clarity and accuracy of information collected and any suggested changes to the instrument based on the cognitive interviewing. The report will provide detailed information on the testing methodology, respondent characteristics, data quality measures, such as response rates, break offs, and skipped questions, and findings related to the key questions of interest (described in the background section of the memo).

3. Protection of Human Subjects

There is a slight risk of emotional distress for respondents given the sensitive and personal nature of the topic; however, appropriate safeguards are in place. RTI’s Institutional Review Board (IRB), which has Federal-wide assurance, has reviewed the planned testing activities, and designated these activities as ‘not human research.’ (see Attachment G).

Contact information

Questions regarding any aspect of this project can be directed to:

Statistical Programs Division

Bureau of Justice Statistics

U.S. Department of Justice

999 N. Capitol Street NE

Washington, DC 20531

Phone: (202) 307-0765

Email: askbjs@usdoj.gov

Attachments

1. Draft of Full CCVS/SPES Instrument

2. Draft CCVS/SPES Cognitive Interview Protocol

3. Recruitment Language/Materials

4. Draft of Online Screener Survey Instrument

5. Cognitive Interviewing Email and Phone Scripts

6. Cognitive Interviewing Informed Consent

7. RTI IRB Approval

1 Violence Against Women Act Reauthorization Act of 2022, §1161l–6. Online survey tool for campus safety. https://uscode.house.gov/view.xhtml?path=/prelim@title20/chapter28/subchapter9/partL&edition=prelim

2 Swan, L. E. T., Mennicke, A., Magnuson, A., & MacConnie, L. (2020). Social Risk Factors for Interpersonal Violence Victimization among College Students: Findings from a Mixed-Gender Sample. Journal of Aggression, Maltreatment & Trauma, 30(5), 605–624. https://doi.org/10.1080/10926771.2020.1832170

3 Cantor, D., Thomas, G., Fisher, B., Lee, H., et al. (2020). Report on the AAU Campus Climate Survey on Sexual Assault and Misconduct. https://www.aau.edu/sites/default/files/AAU-Files/Key-Issues/Campus-Safety/Revised%20Aggregate%20report%20%20and%20appendices%201-7_(01-16-2020_FINAL).pdf

7