Genderswap logo Genderswap

Permalink to original version of “New study does not show that gamers are sexist” New study does not show that gamers are sexist

…but that does not prevent Caitlin Dewey of Washington Post from reporting it as such. On July 21, his article reprinted in the Sydney Morning Herald stated that;

“Some female players, … who were less-skilled at the game, and performing worse relative their peers (sic) — made frequent, nasty comments to the male gamers. In other words, sexist ladies are literally losers.”

The study in question is a joint effort between Universities of Miami and New South Wales, titled “Insights into Sexism: Female Status and Performance Moderates Male-Directed Hostile and Amicable Behaviour,” and published in the online journal PLOS ONE. Its full text is available on their website and will take the inquisitive reader an hour or so to thread through. The last paragraph of its results section reads:

“We found that the presence of sexist statements was not determined by differences in maximum skill achieved…, the number of deaths…. or the number of kills… relative to the experimental player.”

In other words the study does not show that bad players are sexist, or that “hostile sexism” is a wide-spread phenomenon in online gaming. Yet Dewey and the author continue to talk about sexism throughout The Washington Post article. If one takes the time to read the study, they would see that it appears to reveal a correlation between the player’s performance and skill level and the types of comments they received from other players, where the male avatar seemed to be subjected to greater criticism (which is distinct from sexism by the study’s own terms) than the female. It is as if, the researches might have been looking for a specific outcome – blanket condemnation of the male avatar. There is an earlier study (2013) by Kuznekoff , using the same dataset, whose abstract reads, “…on average, the male voice received three times as many negative comments as the female voice or no voice. In addition, the male voice received more queries and more messages from other gamers than the female voice or no voice.” The obvious question one might be asking here is, “Why did the male avatar receive more messages and is the number of negative messages simply proportional to the overall number received in comparison to the female avatar?” We have no evidence of the researchers’ bias other than the text of the 2015 study. Its abstract begins with the words:

“Gender inequality and sexist behaviour is prevalent in almost all workplaces and rampant in online environments.”

In the introduction, they continue to state that:

“…video games remain a bastion of sexual stereotypes and inequality for several reasons. First, women are often graphically depicted as aggressive and hyperfeminine, while men are portrayed in an overly sexualized manner and are more often depicted as squires in distress.”

The language here sounds eerily similar to the discourse surrounding the recent #gamergate controversy.

The study data seems to show that lower-ranking players are more negative to the male avatar, while the higher-ranking one are more positive. Dewey reports that the researcher interprets this as evidence of low-status female players responding to an intrusion of men into a social hierarchy where they have more to lose than the incoming men or the higher-status players. Next, he quotes her to the effect that this is a reflection of a real life dynamic where, “sexism is a kind of Neanderthal defence mechanism for low-status, non-dominant women trying to maintain a shaky grip on their particular cave’s supply of men.” They go on to say that “the increase in hostility towards a man by lower-status females may be an attempt to disregard a male’s performance and suppress his disturbance on the hierarchy to retain their social rank.” Aside from the obvious masculist bias, Kuznekoff’s cavalier attitude to what Neanderthal society may have been like is a curious one for a scientist.

This study is unusually open for scrutiny, having published a part of its data set. On the surface, it offers scientific evidence that men are treated negatively in the almost exclusively female environment of online gaming (no males spoke during the trial indicating that either none were present or that none were willing to speak). The author’s fanciful projection of her results onto real life can be set aside with the simple observation that people play games as a means of escape from the real world and to state that sexual competition take place as soon as a male appears in the midst of mostly female players is a big, big stretch. Nevertheless, the study is worth further attention.

The methodology is as follows; a number of simple, inoffensive phrases were recorded with a female and male voice and three new Xbox 360 accounts were set up to play Halo 3, resulting in entry-level players. As the games progressed, the player would broadcast the pre-recorded phrases in a female or male voice exclusively. The games were recorded on video and their dialogue, outcomes and player rankings were transcribed and analysed. The authors published the list of phrases used, some examples of the comments the avatars received and the data table containing the outcomes of the games and the associated number of negative and positive comments. This table contains a special column “Sexism” with a value of either yes or no to indicate which games were deemed to contain sexist commentary. There is also a “Game” column, containing a code that allows the entries to be sorted in chronological order. The experimental player’s skill can be seen to rise from zero to 26.

The statistical analysis they presented in the study can be reproduced using this data, however the context of the commentary cannot be determined without seeing the videos. Without the video, we cannot verify if the negativity might have stemmed from inappropriate use of pre-recorded phrases, whose arsenal was limited (3-4 for each situation). Broadcasting, “That was a good game everyone” after a thrashing may not have produced a positive response. Also, could the human players tell if the avatar was using pre-recorded messages? If they could, did that affect their responses? How this might be a problem as far as the reality of the gaming experience encountered is concerned, can also be illustrated by looking at some of the instances of “hostile sexism,” where the male avatar might receive 2 negative comments of which one or both were deemed as “sexist” but also receive 7 positive and 1 neutral from the same player in the same game (Player ID 178, Game F061709A).

Also absent from the table is the control group of games, where no voice was used – the players could not tell if the avatar was female or male. Had it been included, it would have provided a “background radiation” reading of the types of comments a player might expect with the gender cue removed. Furthermore, the number of comments made by the avatars is not recorded. This could have been significant. As the researchers note themselves, the male avatar received a lot more comments than the female. Was this a factor of how talkative the male avatar was? Finally, the Sexism criteria could arguably have been extended onto the female avatar for comments such as “You suck dick” (example as per the study’s addendum), however this does not appear to have been attempted. Finally there is very limited discussion of the small sample size which may have distorted the results.

It is up to the researcher to ask such questions of themselves and their data, in lieu of which we shall do so here at AVfM.

82 Male and 81 female games were recorded with 82 individual teammates speaking to the male avatar and 65, speaking to the female avatar (roughly 5:4 proportion). The results can be summed up as follows:


Even though the win/loss ratio is about 5:3 for both female and male, we can see that, the male avatar received more than twice the negative comments than the female. But, they also received more comments of any type. Additionally, they played with higher-ranking players in both Team Slayer and overall game skill then the female. Looking at the deaths and kills – these too are much higher than in the female avatar games, indicating a different and more dynamic game experience. So, what the researchers assert can, arguably be evidenced by looking at the negative comments alone. But the other statistics suggest otherwise or even the opposite. These were team efforts and what we might be looking at is the result of team effort rather than individual performances.

Running a pivot table on type of comment vs game outcome reveals an interesting dynamic:


Setting aside that the players seem to be more willing to talk to the male avatar, the proportion of negative comments is slightly higher for men – 100:56 vs 100:53; i.e. when one divides the negative comments in games lost by the same in games won, a higher number is produced indicating that men receive proportionally more criticism when the game was lost as compared to women. However, players are much more supportive or neutral to the male avatar when the game is lost, to wit, 100:55 for men vs 100:34 for women, for positive comments. This statistic contradicts the researchers’ original assertion that men are expected to encounter predominantly negative treatment in an all-female gaming environment. Could it be that the study’s results are not a factor of sexism in gaming, but of player or team performance, that seems to be, gasp, positively modified by gender? Of the 11 instances of sexism in the comments, 9 occurred in games that were won, which as discussed earlier always occur together with positive and neutral comments. The researcher points out that the sample size containing “hostile sexism” is not sufficient to be useful in statistical analysis. Is it possible that this low “sexism” score made the researchers look at other ways to interpret the data?

When the male avatar’s rising skill level through the course of the 82 games they play is plotted against the negative comments they receive, the following picture emerges:


A similar picture is observed for the female avatar:


In fact, nothing in the 2015 report suggests that the avatar’s rising gaming skill was a consideration, despite there being a control group with no voice that could have controlled for non-gendered cues. Judging by the language used and sources cited, the researchers’ sole focus was gender and not performance.

Confirmation of bias is all that one will find if that is all they look for. It is no small wonder that this study attracts the attention of a masculist ideologue such as Caitlin Dewey or any number of others.

In conclusion, it would be fair to address the argument that is sure to arise here: “The study does show that lower-ranking players are more negative to a male avatar that to a female one.” To which the answer is – is it does show that, but does not show it is because the avatar is male. There are plenty of other variables in the study, some indicating that the male avatar is treated preferentially by the players. In fact, the only link between ill treatment and “hostile sexism” is deemed to be statistically insignificant and is discarded by the researchers themselves.