Cognitive Interview Report

June 4, 2020 SURVEY NOTE OPA Report No. 2020-###

Workplace Gender Relations Cognitive Interview Report

Introduction

As a part of the survey development process for the 2020 Workplace Gender Relations survey, OPA’s H&R division conducted cognitive interviews. Prior to beginning the interviews, members of the Workplace Gender Relations survey team met to identify items from previous survey instruments of specific interest for examination and to develop a protocol containing specific survey items and probes to guide discussion. At time of writing, data from four, sixty-minute teleconference interviews were available for analyses. Those interviewed were conducted with active duty military members, some officers and enlisted ranked, and from the Army and Marine Corps branches of the military.

The interviews were conducted by a trained moderator. Two members of the survey team were also present to take notes and manage logistics. Participants were able to see survey items via a screen shared and controlled by the moderator. Recordings were transcribed and content coded for themes by two coders independently who then compared results to come to consensus.

This initial outline of findings is organized by original topic area and contains eleven sections:

• Topic 1: Instructions and Telescoping Questions

• Topic 2: Bystander Intervention

• Topic 3: Chain of Command, Supervisors, and Leadership

• Topic 4: In Your Unit

• Topic 5: Same Person

• Topic 6: Current, Prior, and Previous

• Topic 7: Before and After Joining the Military

• Topic 8: Military Work and Workplace

• Topic 9: Women in Your Unit

• Topic 10: Outreach

• Topic 11: Miscellaneous

Each section has a brief introduction providing background on why the questions were included in the interview process, an image similar to what participants would have been shown¹, a bulleted summary of findings, and a more detailed explanation of those findings with relevant supporting quotes. Specifically, to Topic 11, there are subsections for each unique area.

Findings

Topic 1: Instructions and Telescoping Questions

The goal of this set of instructions and questions was to examine how survey respondents interpret BEFORE and AFTER statements. The focus was on the temporal aspects and if the question was clearly asking about events that occurred in a certain period, particularly that the research team and the participant were discussing the same period of time.

Figure 1.
Instruction and Telescoping Question

• Service members generally found the time frame in question easy to understand.

• Providing very specific dates can introduce some confusion without necessarily providing better information.

• Using the between language had mixed receptions.

Service members that were interviewed seemed to agree that the time frames in question were clear and easy to understand. However, it was noted that providing incredibly specific dates, to the exact day (Monday, June 1, 2019) could potentially cause some confusion while not offering much in the way of meaningful data.

“I'm not sure, though, what Thursday gets you, or even in particular, the first. I don't think I'd be able to remember specifically when in November, unless it's some kind of visceral, or maybe related to break or birthday ball, because that's normally a typical time period after the first, or Thanksgiving.” – Officer, Marine Corps

Further, when participants were asked to compare the current language used and the newer between two dates language (see figure 2.) there were mixed responses. Some participants were able to more specifically identify the time frame while others pointed out that it just introduced more noise and raised the question of if the dates were inclusive of the day they listed or if they should consider literally in between.

“If it happened on 1 November, would that count? Are you really saying from 2 November '18 to 31 October, or is it inclusive November to November? It just seems to add another level of precision that makes it more confusing for the reader.”
– Officer, Marine Corps

Figure 2.
Between Two Dates Response Options

Topic 2: Bystander Intervention

The goal of this topic was to check that the question was capturing all the ways in which people can think of to intervene in a bystander situation. The second focus was to better understand what “addressing something” means to participants. Particularly, with a focus on outcomes, does addressing the situation mean that the problem stopped?

Figure 3.
Bystander Intervention Question

• Addressing the situation means actively doing or saying something while the event is occurring.

• Non-verbal (for example, walking out of a room or giving a disapproving look) interventions are not represented.

The participants all stated that addressing the situation meant doing something active and in the moment to intervene in a situation they were witnessing. One specifically stated that the goal was to stop the behavior in question while others described saying or doing something to show their disapproval.

“For me, it would be directive, directly intervening in stopping it, or getting in the middle of the situation. More active voice than passive voice.” – Officer, Marine Corps

Participants mentioned a few ways they believed a person could intervene. One suggestion was the use of non-verbal communication to show dissent. For example, a person could give a stern look or exit the room in protest if something offensive had occurred.

“I showed dissatisfaction. For me, my body language and my facial expression can show concerned discontent. I think those are other ways that people can react to a situation without necessarily using verbal communication.” – Officer, Army

Topic 3: Chain of Command, Supervisors, and Leadership

The goal of this topic was to better understand what terms military members recognize when the survey refers to their leadership. The survey uses four different terms to represent someone in authority, so the focus was on assess whether or not each term was understood to represent similar entities. Four questions were asked each using a different term (figure 4).

Figure 4.
Leadership Terms Questions

See Appendix A for full survey questions with response items.

• Civilian Leadership is not addressed but including "military" signals to people who they should be talking about.

• Chain of command is much broader and includes more people.

• Leadership refers to anyone in a leadership role, not strictly chain of command leaders.

• E3s can sometimes be supervisors under certain circumstances.

One finding echoed by all the participants was a lack of options to speak about civilian leadership. Important to note, this may have been the result of all of the participants at the time of writing coming from higher ranking positions that were likely working at installations and/or for organizations that had large numbers of DoD civilian employees. Positively, the inclusion of the word “military” in front of supervisor and workplace helped to signal to the participants that we were interested in knowing about their uniform military leadership even if they technically were directly managed by a GS rating civilian on a day to day basis.

“I could get confused if your immediate supervisor and a civilian, if I had a civilian as my immediate supervisor, those could get confusing.” – Officer, Army

Participants also stated that when it came to the specific terms that chain of command was a broad category that could include many people upstream from their position. Leadership also proved to have some nuance with one participant mentioning that all military members strive to be leaders, which suggested that leadership may be a term that does not always invoke ideas regarding formal chain of command leaders.

“The use of my chain of command or supervisor, so military supervisor is more direct, immediate military supervisor. Chain of command supervisor is a little bit more generic to me and honestly could apply to a civilian or my whole chain of command. It'd be a lot broader of a question when you say chain of command/supervisor.” – Officer, Army

“I would have trouble, if I received great support from my immediate supervisor, the GS15, but two rungs up, the commanding general is kind of an [Expletive] about it. It's like, all right. How do I now judge the entire chain? I have very different feelings about different rungs on the ladder for that. That's why it's very broad from that perspective.” – Officer, Marine Corps

Finally, one point mentioned by a Marine was that occasionally an E3 can be responsible for other E3 and below if there are not any E4s suitable for the position. Currently, OPA surveys only account for supervisors at the E4 level and up.

“The only other thing I can think of is maybe on the enlisted side, maybe there's an E3 block where your really junior personnel ... It just depends on how you consider military leadership, where there are ... We say that corporal is the lowest leader by doctrine in the Marine Corps, but a lot of times there's not enough, so there's a lance corporal in charge of other lance corporals and PFCs and privates.” – Officer, Marine Corps

Topic 4: In Your Unit

The goal of this topic was to test the new response options for categories of individuals that could have perpetrated an action that were reworked in the last reserve survey administration². The distinction between “in your unit” versus “not in your unit” was of specific interest.

Figure 5.
Alleged Offender Categories

• Unit can mean different levels depending on how highly ranked a person is (a higher rank person might think of their unit as all of the companies in their division, whereas a lower rank person might think of only their platoon).

• Civilians as immediate supervisor are not captured in the responses.

• Military members are people "in uniform."

One of the most important points regarding the term “unit” was the perception that different people may consider different levels of the military structure based on their own rank and the scope of their duties. It was explained that a person at the rank of O5 may view their unit as the whole battalion they oversee while an E2 might think of their unit at the platoon or squad level. This represents a difference in scale, in the case of the O5 they have hundreds to thousands of military members in mind whereas the E2 may only be thinking of between ten and fifty people.

“I would think of that being someone outside of my battalion or squadron, so the O5 level being a unit. But certainly for, if you ask me that question now with junior ranked guys, that could be outside of their platoon, outside of their company. I think it's very subjective to what your rank structure, where you are in the pecking order, of how you identify what your unit is.” – Officer, Marine Corps

Topic 5: Same Person

The goal of this topic was to assess accuracy of participant understanding of this question. More specifically, the goal was to clarify whether the response options are interpreted as referring to repeat offenders (as intended) or the number of offenders.

Figure 6.
Same Person Question

• Participants generally seem to understand that this is talking about the same perpetrator committing multiple offenses.

Participants were able to easily identify that this meant the same person committing multiple acts. There did not seem to be any confusion about the number of different offenders.

“I'm definitely reading it as, it's not involving the same people in terms of, hey, the same person's being picked on. It's definitely more towards the perpetrator or perpetrators with the S on the end.” – Officer, Marine Corps

Topic 6: Current, Prior, and Previous

The goal of this topic was to clarify the language that is used when asking about which duty station and command a survey respondent was referring to. It is another temporal issue, and the distinction between prior, previous, and current could be adding to the confusion. Additionally, there was a focus on if the terms duty station and command are meaningful, or if they cause confusion, especially for deployed active duty members.

Figure 7.
Current or Prior Question

• Temporary duty creates an issue where a service member is not sure which command or duty station they would talk about.

• Deployment would not make it difficult to decide on current or previous duty stations/commands.

• Separating duty station and command produces more accurate data because one can change while the other does not.

As expected, the participants generally believed, often unprompted, that temporary duty (TDY) would make this question difficult to answer. TDY represents a circumstance where the military member may still be technically assigned to a certain command but working for another. Interestingly, one participant talked about deployment and expressed that they did not think deployment would complicate the matter.

“With this one, I think it's fairly clear, except for the fact of what happens if it happens on [TDY]? I could be at my current duty station assigned to my current command, but I could've been farmed out to someone else, and that would never reflect on any of my records.” – Officer, Marine Corps

“If I was deployed and let's say this event happened in Afghanistan and I was assigned to the current unit. I would interpret my current duty station, yes. I wouldn't talk about prior duty station because if I was under the same command, and even though I was deployed but I came back to the same command, I would interpret it as that. Assigned to previous command, no it would all be together if I was deployed and I came back.”
– Officer, Army

The separation of duty station and command was thought to be helpful in order to collect accurate data since one could change while the other stayed the same. Finally, there were no concerns raised about the specific time-oriented (for example, prior versus previous) words used. The question as worded was perceived as relatively clear without the considerations for TDY.

“I've been in a unit before to where within that brigade, I was in a duty station for five years. Within that one brigade, I was in three different battalions. So if I was trying to recur an event within the last 12 months, and I was in another battalion, then yeah, it'd have been a different command…[I would have] been at that same duty station but under a different command.”
– Enlisted, Army

Topic 7: Before and After Joining the Military

The goal of this topic was to investigate another temporal issue. Specifically, the focus was on identifying when a military member considers themselves as part of the military; are they considering things that happen at Military Entry Points (MEPs) or at Basic Training to be part of their military time?

Figure 8.
Before or After Joining the Military Question

• When you "ship", or slightly before when you sign enlistment paperwork, seems to be the start point for being in the military.

• Things that happen during the recruiting process, even if you have already signed up and have not "shipped" yet, are before.

• Does Academy Service count as having "joined the military"?

Most of the participants agreed that they considered the time when a recruit “ships”, or the point where a newly enlisted military member relocates to an installation to begin their basic training, as the starting point for being in the military.

“My shipping day is in three days, and officially you're in the Army, I guess, even though you hadn't completed anything.” – Enlisted, Army

Interestingly, things that occur during the recruitment process, but before you officially sign the paperwork were considered before joining the military despite the possibility that the experience could involve military members.

“…before I joined the military would be the recruiting process. If something happened to me as I was trying to enlist or join, I would include that misconduct within that.”
– Officer, Marine Corps

Finally, one participant wondered if they should consider time spent at a U.S. Military Academy, like the Naval Academy at Annapolis or the Military Academy at West Point. This finding is particularly relevant due to OPAs extensive work with the Academy populations for the Service Academy Gender Relations Survey (SAGR).

Topic 8: Military Work and Workplace

The goal of this topic was to understand how service members interpret terms that are common in the civilian world to describe office spaces. Workplace, work, and coworkers might not accurately speak to a military career. Additionally, how service members think of their unit in these terms was a point of interest.

Figure 9.
Military Work and Workplace Question

• Military workplace is thought of as the primary installation they are assigned too, while military work can occur at a variety of locations.

• Military work can be confined to being at a specific location in uniform.

• Military work is your job and the place you do it.

• Grabbing, pushing, or insulting someone represents at least two different behaviors.

Military work and military workplace are terms that seemed to be simultaneous synonymous and different. Some participants expressed that military work is what your job is and where you do it or being in uniform at a specific location. Those same and other participants also acknowledged that while their military workplace could be thought of as their primary installation, not all their military work occurred in that workplace. For example, a person could be posted at a base in the greater DMV area but still be expected to attend meetings at the Pentagon which they would consider military work.

“I understood military work, but when he said workplace, I thought only installation, or where I work all the time, like my office desk. Sometimes I have meetings in other locations that I consider military work.” – Officer, Army

In response to the phrase “see someone from work grabbing, pushing or insulting someone” used in the response options to this question one participant pointed out difficulty. They expressed that insults are something that can happen in a work environment but grabbing and pushing was a much more bold and serious action. This response option presents a situation in which two behaviors that are likely different are being presented, one potentially less serious and verbal and the other potentially more serious and physical.

“I think somebody's getting insulted if somebody uses passive aggressive methods to say, ‘Yeah your slides suck. What kind of officer are you?’ I've seen that. Is that really insulting? To the point where it's grabbing or pushing, it almost seems separate from grabbing and pushing. It's like grabbing, pushing, or insulting someone. It's kind of like, ‘Oh that's kind of extreme.’ Are they talking about extreme insulting?” – Officer, Army

Topic 9: Women in Your Unit

This topic focused on the question about the proportion of women in the participant’s unit. The goal was to see if this question was accurately assessing what participants consider common since previous surveys have attempted to set a threshold of varying percentages (most recently less than 10% was considered uncommon).

Figure 10.
Women in Your Unit Question

• People view the word uncommon as more binary then proportionally.

• Civilian coworkers are not considered when the phrase "military coworker" is used.

• Military unit is dependent on rank and location.

Some participants tended to view the term uncommon in binary terms, for example either there are women or there are no women, as opposed to a proportional threshold, for example, 1 out 10 people in my unit are women. Further, many thought of this question from a full force perspective and not strictly just their units. Women are uncommon overall in the Marine Corps, so it may be viewed more uncommon to have even one woman in a unit for some jobs.

“I have mixed feelings about the less than 10% of your military coworkers, because I have worked in almost every unit I've been to, there are women there, and they've always been there. So it's not uncommon, but they are certainly always less than 10%, just because the Marine Corps is, I think, 8% right now.” – Officer, Marine Corps

This topic also created discussion around two topics that were previously discussed. First, using the word military signals to participants that we are not asking about potential civilian coworkers, similarly to when that language is used for immediate military supervisors. Second, once again a survey participant might view their unit at a different level depending on their rank; this was also supported by higher ranking participants immediately thinking in terms at the total branch level as previously mentioned.

“Because if you say, if I work with civilians, and you say your military coworkers, then I'm only thinking about those who are in the military, not the civilians.”
– Enlisted, Army

Topic 10: Outreach

The goal of this topic was to attempt to improve outreach efforts when trying to get Service members to take surveys. The majority of communication attempts occur by email. The focus was to assess what would encourage the target population to read those communications. There was also interest in identifying other common avenues that currently unknown that could be used to communicate in the future.

Figure 11.
Outreach Question

• The title is important and should not include "survey" or potential respondents will ignore it.

• The messaging should talk about the importance of taking the survey.

• Leaders up the chain of command need to stress the importance to Service members directly under them.

The two major points that all or most of the participants raised was that the title must catch their attention and the importance of taking the survey need to be stressed. Some participants admitted that if they see the word “survey” in the title line of an email, they tend to push it off until later without much other thought. One participant talked about how leaders throughout the chain of command need to talk about potential surveys so that their subordinates know that it is something that they should be making time to participate in.

“A catchy subject, when I see the word "survey" I flag it and I don't follow up for months.”
– Officer, Army

“Definitely the title, in terms of subject line, and really the first three lines describing what the survey, why it's actually important, are the big ones. Then who is sending it is the other one. If the survey administrator is sending it and it's just the staff sergeant in charge of doing it, and the CO doesn't send one with the same thing, very quickly, if not before the staff sergeant, about why it's important, then I know it's not important. I'm less likely to take the survey.”
– Officer, Marine Corps

Topic 11: Miscellaneous

The goal of this topic was to address all the remaining items of interest. The items include whether the offender was in the same MOS as the victim in the One Situation questions, what resolving an issue looks like in the reporting process, if the bank of intimate partner categories makes sense and includes options for all kinds of relationships, who military members consider “Military Authorities”, specific aspects of retaliation, the clarity of the dating app question, and the combination of race and ethnicity when asking about the participants supervisor. Many of these had specific follow ups and, generally, focused on what certain word and phrasing choices mean to the participants.

Same MOS

The goal of this question was to assess how participants are interpreting the Military Occupational Specialty (MOS) question and if the bank of acronyms and terms that speak to the language commonly used in each branch provide additional clarity or detract from understandability.

Figure 12.
Same MOS Question

• There can be multiple, similar jobs in a single unit that have differing MOS designations.

• Including the abbreviations provides more clarity.

One finding was that MOS can be very granular when it comes to specific designations. There may be several people in a single unit that do similar things as part of a team that have slightly different MOS codes.

“…when you meet occupational specialty, do you mean the specific specialty or just the overall occ field? Was it a motor transport operator 3531 that bullied me or just someone in the motor team field that's at the motor pool? Some of them are so thin sliced that it may not tell you anything.” – Officer, Marine Corps

However, providing the specific language did help one participant to differentiate between Service members doing specific jobs and Service members that work within a specific field. For example, they believed that you would include doctors and nurses in the medical field, but they would not be considered in the same MOS.

“If I was medical I would think you're talking about somebody in my medical field or specialty, like nurses or emergency room something. I would think you're talking about that without the MOS being there.” – Officer, Army

What Resolved Means

This question sought to assess how military members interpret the word resolved when it comes to the reporting process.

Figure 13.
Complaint Resolution Question

• To some, resolved means the complaint process is final or completed.

• Most agree that our response options for the time frame for how long a complaint takes should include times longer than a month.

• Shifting the response ranges from hours to days adds some confusion.

Most participants considered resolution to be when the reporting process was officially concluded, some saying regardless of whether the outcome was favorable or not.

“To me resolved means it's done, complete, whatever. So resolved means, in my interpretation, it's done, not still in the works.” – Enlisted, Army

Interestingly, many agreed that the time frame options the survey offers for a formal complaint to go through the complete process are not realistic and that the process tends to take significantly longer than any of the options listed. One participant pointed out that the options switch from hours to days in the same response set which made it difficult to understand particularly if the resolution occurred right at the shift.

“…I think you're limiting yourself in terms of, I know some of these complaints that have gone on for six months, just because it takes at least two weeks to do a preliminary investigation, typically, for some of these hazing incidents, as compared to a more serious one. Hey, I need answers now. I think you probably want to add more sensitivity to 31 to 60, or 31 to 90, and then 91 plus.” – Officer, Marine Corps

“…why did you change scales, because there seems to be a gap between 49 hours and 72 hours that's not covered, whereas if you just said one to two days, three to seven days, eight to 30, and 30 plus, that gives me a continuous understanding…”
– Officer, Marine Corps

Intimate Partner Categories

This question was asked to understand how clear the intimate partner categories were and to see if there was a category missing that could explain large number of “none of the above” responses.

Figure 14.
Intimate Partner Categories Question

• Generally, the response options seem clear to participants.

• "Friends with benefits" or more casual sexual partners do not have an adequate category.

The participants found this question easy to answer and there was no confusion even when prompted to focus on some of the more complicatedly worded categories.

One point of interest was raised by a participant regarding missing categories. They do not necessarily consider someone they had a casual sexual encounter with to fit into any of the categories because they interpret a friend or acquaintance as a platonic relationship and a significant other as an exclusive relationship.

“One of the options that I'm not seeing is, what if it is a ... for lack of a better word, just friends with benefits. Would that fall under current or former significant other, or would that fall under friend or acquaintance, where I don't think it would fall under either, to be honest. I read friend or acquaintance more as platonic, and then current or former significant other is more exclusive. Where is that dating overall, non-exclusive but still sexually involved?”
– Officer, Marine Corps

Military Authorities

The reason for this question was to assess what military members consider military authorities that they can report to.

Figure 15.
Military Authorities Question

• People think that military authorities are a wide range of people that have authority and are in uniform, not necessarily just the Military police (MP).

The participants mentioned the MP often first, but many did go further to talk about other individuals that are uniformed military members and have the authority to initiate the reporting process. Some specific examples would be chaplains, SARCs, and members of their military chain of command.

“That could be my chain of command. That could also be my medical provider who is a uniformed service member. That could be my chaplain, is the way I would read that. If that's something you want sensitivity to, those need to be options, since they all can result in a different report.” – Officer, Marine Corps

Retaliation

The primary aim of including this question was to assess the language used to describe collateral misconduct, or getting into official trouble for reporting an event that occurred while engaged in some other act that was against the rules (for example, reporting hazing that occurred while underaged drinking was also occurring).

Figure 16.
Collateral Misconduct Response Option

For full question and response set see Appendix A

• Collateral misconduct is understood correctly.

All participants understood what collateral misconduct represented when prompted. Of note, this question was particularly challenging for participants to engage with. This is likely due to the fact that it is several tiers of skip logic down on the actual survey³ and establishing a mock situation where they could orient to this question proved difficult.

Dating Apps

This question was asked in order to assess the clarity of the dating app question and the understanding and necessity of the inclusion of the term “location services.”

Figure 17.
Dating App Question

• The question is straight forward and understandable.

• Everyone understands what is meant by location services.

All participants found this question to be understandable with few noting that some of the specific apps mentioned were unknown to them. Location services was a term that was easily understood, and one participant talked about the nature of these apps basically requiring real time location to function.

“…has your locations turned on your phone, so they can know where you're located, like say here or wherever you're at. Wherever you're at, it's always the town next over or something or the next city over, whatever. They can be like, hey, this person is five miles away from you, or this person is in the next town over, 15 minutes away. It tells you how close or far they are.”
– Enlisted, Army

Race/Ethnicity

The inclusion of these questions was primarily to assess whether participants run into difficulty between the terms race and ethnicity when merged into a single item.

Figure 18.
Gender and Race/Ethnicity Question

• The question seems clear up front, but people seem to get confused when they think harder on the specifics.

• Participants tend to focus on one’s physical characteristics, such as skin color, when answering this question.

• It would be difficult to answer the gender question if you had a transgender supervisor.

• Service members who have a civilian as their immediate supervisor would not know how to answer.

All participants were able to easily answer the question at first read without much additional thought. However, when probed specifically on the combination of race and ethnicity the participants had a more difficult time. One opinion is that people are not frequently considering the ethnicity of people that they do not know well and instead rely on what they can tell visually. Others disagreed and said that the combination would change their answers. For example, where they may mark yes to race and no to ethnicity, they would now have to mark no for both.

“No, because if I was white, if they had a different ethnicity like if I was Irish and they weren't or if I was Jewish and they weren't I would be able to put no.” – Officer, Army

Finally, one participant spoke to the gender question. They felt that they would not be sure how they would answer the gender question if their supervisor were transgender.

“Really reading into it would be seeing gender would be if you are in the rare situation where there is a transgender military supervisor, would you debate? I think it's really subjective on you, you as the respondent being, do you go by the birth gender or identification.”
– Officer, Marine Corps

Conclusion

The results of the cognitive interviews reveal the following key insights:

Participant’s interpretation of their unit is subjective and dependent on their rank and military duties. Participants in higher ranking positions tend to consider their unit at a higher structure level which is made up of hundreds or thousands of military members while more junior ranking members may interpret their unit at the squad or platoon level which is made up of ten to fifty members. This difference could impact the ambiguity of the data that is being reported.

Using the terms “immediate military supervisor”, “military workplace”, and “military coworkers” helps those participants who may have a civilian supervisor or civilian coworkers to clarify who they should be thinking about while answering the questions. A few participants noted that if they have a civilian as their immediate supervisor and the question asked about their military supervisor, they will think about their uniformed supervisor rather than their civilian supervisor who may manage them day to day. Participants also emphasized the importance of keeping the language consistent throughout the whole survey.

Generally, participants had a clear sense on how they interpreted military coworkers and military work. When asked about their military coworkers or military work, participants explained that to mean people in uniform or work done while on military duty in uniform.

Participants identified a few missing response options that currently are not being captured in several questions’ response set. When asked to consider other ways to respond to a particular question, participants identified additional response options that are not captured in the current response set. Those include using non-verbal communication as a way to respond to an unwanted situation in which someone encountered, having a non-exclusive sexual relationship with someone as a type of association one may have with a preparator of an unwanted event, and having an E3 as an immediate military supervisor.

In general, participants indicated that the survey questions were direct and easy to understand. When asked about their overall feelings of the survey questions, most participants felt the questions were clear and made sense on their first read. One participant noted that the questions were designed well enough for him to answer quickly after reading the questions once.

These findings are based on opinions expressed by respondents from higher ranking positions. All participants who have been interviewed at the time of writing come from higher level paygrades. To gain a better understanding of Service members’ thoughts overall it will be important to expand our interviews to include some lower level paygrades as well as interviews from all service branches.

Appendix A: Moderator Guide

1 The exception to this is Figure 4, participants would have seen the full questions one by one, see Appendix A.

2 This refers to the 2019 Workplace Gender Relations Survey of Reserve Component Members (WGRR1901)

3 Specifically, in order to see this question on a survey a participant would have 1) had to have experienced an event, 2) formally file a report or complaint about the event, and 3) experience retaliation from some superior as a result of making a formal complaint.

1