Sections

Homophobia, sexism, racism reduced to 2% of League of Legends matches

Riot Games social systems designer says abusive behaviors have been deeply curtailed through collaboration with players and academics

Riot Games has been researching its League of Legends player community for years, trying to understand and eliminate toxic behavior for the benefit of everyone. In a recent post on Re/code, Riot lead game designer of social systems Jeffrey Lin shared some of those insights.

"Our team found that if you classified online citizens from negative to positive, the vast majority of negative behavior (which ranges from trash talk to non-extreme but still generally offensive language) did not originate from the persistently negative online citizens; in fact, 87 percent of online toxicity came from the neutral and positive citizens just having a bad day here or there," Lin said.

To help address the negative behavior, Riot implemented a "Tribunal" system where players could report each other for negative acts, and other players would review each case and vote on whether the behavior in question was acceptable or not.

"These cases were public, so players could see and discuss the behaviors, and the results were inspiring," Lin said. "The vast majority of online citizens were against hate speech of all kinds; in fact, in North America, homophobic slurs were the most rejected phrases in the English language. It turns out that people just need a voice, a way to enact change."

After having collected 100 million such reports, Riot began collaborating with academics and scientists to create a machine-learning system to detect questionable behaviors and offer a customized response to them based on how players voted in comparable Tribunal cases, whether it be a punishment for toxicity or an incentive to behave better.

"As a result of these governance systems changing online cultural norms, incidences of homophobia, sexism and racism in League of Legends have fallen to a combined 2 percent of all games," Lin said. "Verbal abuse has dropped by more than 40 percent, and 91.6 percent of negative players change their act and never commit another offense after just one reported penalty. These results have inspired us, because we realize that this isn't an impossible problem after all."

He added, "Is it our responsibility to make online society a better place? Of course it is, for all of us. It is our society. As we collaborate with those outside of games, we are realizing that the concepts we're using in games can apply in any online context. We are at a pivotal point in the timeline of online platforms and societies, and it is time to make a difference."

Related stories

Riot Games introduces revenue sharing in EU LCS overhaul

Developer rethinks Challenger Series and looks to reward teams that "positively contribute to the success of the LCS"

By Haydn Taylor

Riot co-founders Brandon Beck and Marc Merril to lead development on new title

Senior leadership team to handle company operations while duo work on studio's next project

By James Batchelor

Latest comments (22)

Darren Adams Managing Director, ChaosTrend2 years ago
2% too much?
1Sign inorRegisterto rate and reply
Paul Jace Merchandiser 2 years ago
Very true Darren.
1Sign inorRegisterto rate and reply
James Berg Games User Researcher, EA Canada2 years ago
But a lot less than other games :)

As a researcher, I've followed Lyte's work for years - the incredible stuff that Riot has done to improve their community using research is fantastic. And, most impressively to me, they've *shared* that with everyone.
3Sign inorRegisterto rate and reply
Show all comments (22)
Bill Young Head of Strategic Partnerships & Sponsorships, esports, Twitch2 years ago
2% too much? Absolutely...but give Riot credit for addressing this issue, and for their progress. Well done, Riot.
2Sign inorRegisterto rate and reply
Robert Mac-Donald Game Designer, Lethe Games2 years ago
I don't think a company should put themselves in a place of being the morality police of a game's culture, trying to steer the community one way or the other. There is an ignore button for that, so each individual person can ignore something if they can't handle it or it annoys them. Frankly this kind of thing scares me a bit, and is disrespectful to each person's individuality because in many instances one person's hate speech is actually another's person "good speech". People have different opinions of what is good and bad, and often these opinions will trigger insults and hate from others from any side, and there is nothing wrong with letting people deal with these as they wish.

And if we are merely trying to stop bad words from being said, foul language filters are cheap and on most games already, no need to create a "machine-learning system to detect questionable behaviors" which is honestly sounds like the Inquisition by robots
0Sign inorRegisterto rate and reply
Benjamin Crause Supervisor Central Support, Nintendo of Europe2 years ago
I applaud Riot for the effort and I'd like to hear more about how they did it and how they encourage the community to communicate but in a more peaceful way.
one person's hate speech is actually another's person "good speech"
Not when it comes to racism, homophobia, general hate speech, threats of violence, etc. There shall be no place for that!
Especially not in a game where we want people from all over the world to play together in competition but peaceful.
1Sign inorRegisterto rate and reply
Robert Mac-Donald Game Designer, Lethe Games2 years ago
Not when it comes to racism, homophobia, general hate speech, threats of violence, etc. There shall be no place for that!
Things are rarely simple though Benjamin. Who defines the limits of what is hate speech and what isn't? Of what is fair and what isn't? It is very subjective, and often criticism of someone's behavior can be seen as hate speech when it isn't. I would not like to be banned from a game because someone interpreted what I said as hateful.

Let me give you a real life example. I have a female friend who was a vegan. When people on her WoW guild found out about it, they started picking on her, calling her names, saying she was stupid amongst other things. Do you consider that hate speech? Do game developers need to control the amount of hate vegans are experiencing on online games? No? Why not? Because you deem one thing hate speech, but not another?

And what if in the near future saying an argument favourable to meat consumption becomes hate speech? Would you like to have your speech monitored and impaired, or censored with chilling effects as you may be banned by the questionable behaviors detector?

People act like hate speech is something objective, but it is mostly subjective, and always tied to what a majority perceive as bad. Edit: which can be understood if you want to sell a product to the majority and you want to cater to their feelings and what they see as a good product. Just don't mix selling a product to an audience with the political aspects of hate speech and free speech.

Edited 1 times. Last edit by Robert Mac-Donald on 10th July 2015 10:14pm

0Sign inorRegisterto rate and reply
I have a female friend who was a vegan. When people on her WoW guild found out about it, they started picking on her, calling her names, saying she was stupid amongst other things. Do you consider that hate speech? Do game developers need to control the amount of hate vegans are experiencing on online games? No? Why not? Because you deem one thing hate speech, but not another?
What you're describing isn't really considered 'hate speech' as vegans aren't exactly a historically oppressed minority, but it is targeted harassment which is also considered negative behaviour under LoL's Tribunal system. And yes, we need to control the amount of hate anyone experiences in our games. Players being assholes to other players and going unchecked can only be a detrimental thing to everyone's experience.

The interesting thing about the Tribunal system, from what I read about it, is that it's all based on player interactions. Players report players for what they consider harassment and hate speech, and so the community itself polices unwanted behaviours. Very neat, and an absolutely laudable system. Give your players the tools to deal with the problem, and it seems that they do a fairly neat job of it. I am pleased to see these results, and I hope more developers consider employing similar systems in their own games. I certainly will next time I'm working on a large-scale online game :3
2Sign inorRegisterto rate and reply
Robert Mac-Donald Game Designer, Lethe Games2 years ago
Players being assholes to other players and going unchecked can only be a detrimental thing to everyone's experience.
Not to everyone's experience, but only to those that don't like those experiences. Those that consider being playful and humorous what you may consider hateful will not have a better time online due to your censorship of them.

There are ignore buttons and word filters. Why can't that be enough?
0Sign inorRegisterto rate and reply
Ignore buttons and word filters don't change bad behaviour(and in no world outside of a troll's head is harassment, homophobia, sexism or racism 'playful and humorous'), they just try to mask it and they place the onus on people being harassed or insulted to change their environment rather than making the person actually causing the problem to shape up or get out.

Riot clearly believes, and has shown, that negative behaviours can be changed. Maybe you think that's a bad thing for whatever reason, but I believe that teaching people - by calling out negative behaviours and rewarding good ones - to be polite to each other on the Internet is a very valuable thing indeed.
3Sign inorRegisterto rate and reply
Robert Mac-Donald Game Designer, Lethe Games2 years ago
and in no world outside of a troll's head is harassment, homophobia, sexism or racism 'playful and humorous'
Some people have a darker sense of humor than others.
You should be more accepting of diversity, not everyone is like you.
0Sign inorRegisterto rate and reply
Just curious: Purely hypothetically, if I punched you in the face and broke your nose because I figured it would be fun, would you find yourself perfectly tolerant of my dark sense of humour? Or would you possibly be a little bit upset?

Being harassed and threatened by strangers online, spat at with racial or homophobic or sexist slurs, is not fun for the people being targeted. It hurts. Online harassment campaigns(aka cyberbullying) have driven far too many people to commit suicide. It's not about your oh-so-edgy 'dark sense of humour', it's about the safety and happiness of other human beings.

If being an arse to people is your idea of fun, you need to take a good long look in the mirror and ask yourself why that is.
4Sign inorRegisterto rate and reply
Robert Mac-Donald Game Designer, Lethe Games2 years ago
Physical abuse and opinions/speech are not in the same category Jessica.

And consider the following. Imagine that you join an MMO where the majority of people have an anti "social justice warrior" mentality, where they don't think taking extra measures to control what people say is a good thing. Then you go on in there, and you start saying that some people who are saying some things need to be "dealt with". Now that the tables are turned, how would you feel about the game having tools that would censor your, preventing you from expressing those opinions because the majority of the community deems it as questionable?

Would you still be in favor of these systems when you are the target of them?

The day gamesindustry's staff considers the opinions you are having here as hate speech because they undermine free speech, and your comments start getting deleted, then you'll understand what we were trying to point out all along: letting people subjectively decide what is ok behavior and what isn't is a scary thing. Ignore buttons and word filters should be enough for most cases.
0Sign inorRegisterto rate and reply
Physical abuse and opinions/speech are not in the same category Jessica.
No? Harassment hurts people. Harassment kills people. Being tormented by online harassment and everyday hatred can be just as harmful as physical abuse, though the marks it leaves may be invisible. I used the example of a punch to the face for a reason; frankly I'd rather get hit in the face than have to read some of the shit that comes flying my way when I dare speak out against harassment and abuse and sexism.
Imagine that you join an MMO where the majority of people have an anti "social justice warrior" mentality
I'd go play somewhere else. That kind of toxic environment would not be fun or frankly healthy to exist in.

I've discussed feminism and social justice in many online games though, and been screamed at by overgrown children who can't handle the concepts of empathy and caring about other human beings plenty. And every time, I've recieved supportive messages from people who feel just like me but don't want to have to deal with the reactionary shitstorm when they speak up. So I don't think the majority in any game are pro-harassment or pro-sexism or pro-racism or pro-homophobia - just look at the numbers in this very article, where Riot have discovered that the vast, vast majority of their millions of players rightly abhor such anti-social and selfish behaviour. The toxic few are loud and vociferous, but they are outnumbered by the quiet majority of people who believe in treating each other with respect and common decency.
Ignore buttons and word filters should be enough for most cases.
They are not. Even websites like this have seen fit to remove grossly offensive posts and ban users for spewing hatred and vitriol, although those have been mercifully few. Your vaunted concept of 'free speech' simply does not apply to websites or games run by private companies, nor frankly in most countries in the world which have laws against hate speech, bullying and harassment. When we create private spaces for communication, we become responsible for curating them. There is no value in words used to harass and spread hatred, and no amount of 'but I think it's funny!' will save them.

Edited 2 times. Last edit by Jessica Hyland on 11th July 2015 8:09pm

4Sign inorRegisterto rate and reply
Honestly though, how is a word filter not 'censorship'? I mean, if people can't see your ~totally hilarious~ racial slurs because The Man decided those naughty words might upset somebody, isn't your paramount right to free speech being impinged upon?

No. What you want is the right to say whatever you want, however you want, with no consequences. It's not censorship when you get banned for hounding women with ~super funny wow why are these stupid girls not laughing~ jokes about rape and tired-ass get-back-in-the-kitchen lines: it's consequences. Consequences for your behaviour are part of being an adult. Grow up.
5Sign inorRegisterto rate and reply
Robert Mac-Donald Game Designer, Lethe Games2 years ago
Being tormented by online harassment and everyday hatred can be just as harmful as physical abuse
Yes it can, and I suggest people that are affected by that to seek therapy. You should not be getting this mad at people online if all they are doing is talking about you. Here are the list of things I have to deal with online:

1) My country of origin. Is enough for people to not want to be friends with me and I've been pushed away by people online after they find it out. I remember the first time I've come in contact with that, it was in the 90's. My friend was going to let me play an online game on his computer, and when I started typing he said "don't let them know our country or they won't play with us anymore"
2) Like my wow friend, I'm also a vegan. Once people find that out it's over, you won't hear the end of it even if you never speak about it, others will. I've experienced the same working on game companies IRL. As soon as people notice you are not putting meat on your plate every day, teasing begins daily.
3) There are more things but I'd rather keep them private.

My position however remains that either I ignore people I don't like, or I am free to discuss these things with people on online games as I see fit, which I prefer. No point in hiding from debate in my opinion. Nothing would be gained from putting a barrier between me and other people being able to have discussions because some topics are deem offensive by a majority. I wouldn't want to censor you on anything either, as you've said:
I'd go play somewhere else. That kind of toxic environment would not be fun or frankly healthy to exist in.
You must understand that you too may be ruining other people's fun by supporting having extreme measures and policies being used in games.
I've discussed feminism and social justice in many online games though, and been screamed at by overgrown children who can't handle the concepts of empathy and caring about other human beings plenty.
You've discussed, and the other side has discussed, and both got screamed at overgrown children. As long as all sides are getting to discuss it, then you have a civilized MMO.
No. What you want is the right to say whatever you want, however you want, with no consequences. It's not censorship when you get banned for hounding women with ~super funny wow why are these stupid girls not laughing~ jokes about rape and tired-ass get-back-in-the-kitchen lines: it's consequences. Consequences for your behaviour are part of being an adult. Grow up.
You have no way of knowing what I do and what I don't, and the fact you seem to be weighting the whole pro censorship on these issues that affect you show how you are lacking a greater view of things. These monitoring and censorship measures will not be used only for the most obvious, direct racist/prejudice jokes, they will be used for things that someone may perceive as borderline. Or maybe someone expresses them opinion without being hateful, but the opinion itself is viewed as hateful and then you lose everything you've done on a game for years. The point you keep failing to understand is that ultimately it will always be a subjective system. Someone gets to decide what is bad and what isn't.

And what about jokes on white men that are being done all over the internet? Should people be banned/penalized on games for saying those? Would you want that?
0Sign inorRegisterto rate and reply
You should not be getting this mad at people online if all they are doing is talking about you.
You do not get to dictate to anyone how they 'should' respond to harassment and slurs. You do not get to refer people to therapy because some asshole upset them online. You have had to put up with some people's bad behaviour and you brush it off, which is well and good for you, but for some of us the harassment and the background radiation of prejudices we have to deal with every day is not so easily brushed off. It would be real nice if we didn't have to brush it off - it would be great if it didn't happen at all. Why you are so insistent on defending people's 'right' to harass, to threaten, to throw sexist and homophobic and racist language around however they like, regardless of how that affects others - and it does, even if you don't think so - is utterly beyond me.

You need to stop throwing around the word 'censorship' like paranoid confetti when we are talking about playing videogames. Your attachment to your level 30 LoL summoner and their totally epic win:loss ratio does not justify behaving however you like to other players under the excuse of 'free speech'. Game developers giveth toys and we taketh away when you break the very clearly-stated rules - and yes, it's all terribly subjective, because that is how human beings operate. Word filters can be evaded and are blunt instruments at best. Ignores can be worked around. Sticking to the letter of the law while breaking the hell out of its spirit is not how codes of conduct should work, and initiatives like the Tribunal are solutions to the inflexibility and crudeness of automated systems.

You have no god-given right to say whatever you like to other people online without consequences(and my goodness, the harshest of penalties - being banned from a videogame!). You have no god-given right to play an online game and break the rules without consequences. If you don't like those consequences, you can go play elsewhere.

Riot and many other developers are determined to cultivate civil player communities, and long may that determination bear them fruit. I'm cheering them on. You can go cry about it.
4Sign inorRegisterto rate and reply
Robert Mac-Donald Game Designer, Lethe Games2 years ago
for some of us the harassment and the background radiation of prejudices we have to deal with every day is not so easily brushed off
Therapy.
Why you are so insistent on defending people's 'right' to harass, to threaten, to throw sexist and homophobic and racist language around however they like, regardless of how that affects others - and it does, even if you don't think so - is utterly beyond me.
I'll explain to you with a very practical example then.
Coincidentally, to draw on the example of my vegan friend again, she appeared on the University's newspaper once because she got the kitchen to adjust their menu to fit vegetarians and vegans better. I told her that she was being a typical girl, because she only does revolution in the kitchen.

I am genuinely scared that when I try to play everquest next, next year, I may be penalized or even banned for saying jokes online. Which is just what that is, a joke.

I don't play mobas by the way. So again you are way off on my actual behavior.
0Sign inorRegisterto rate and reply
Morville O'Driscoll Blogger & Critic 2 years ago
Therapy.
Just... Pardon? Your answer to strangers being nasty, abusive, harassing dicks online is not for them to be nice, but for me to pay someone else to talk through the issues with me? If people stopped being "funny" dicks online then not only would I be happier, but that therapist you're suggesting I see? They would have time to help someone who truly needs it.
I told her that she was being a typical girl, because she only does revolution in the kitchen.
I am genuinely scared that when I try to play everquest next, next year, I may be penalized or even banned for saying jokes online. Which is just what that is, a joke.
Jokes amongst friends are different to jokes amongst strangers. If you don't believe me, tell that joke to a waitress serving you dinner in a restaurant, and gauge her reaction. I dare say you'd have to leave a pretty generous tip to get her to forgive you. In any event, whilst your friend may have thought it was funny, that joke is based on stereotyping, and there's many who don't find humour in being cast as a stereotype. We are more than the sum of people's assumptions, and it can be incredibly hurtful (in a variety of ways) to have someone "joke" like that - something which you obviously cannot see or empathise with.

I get the feeling, though, that this conversation is as pointless as trying to convince a troll online that their actions are insensitive.

Edited 1 times. Last edit by Morville O'Driscoll on 12th July 2015 9:40am

5Sign inorRegisterto rate and reply
Chris Payne Managing Director & Founder, Quantum Soup Studios2 years ago
I am genuinely scared that when I try to play everquest next, next year, I may be penalized or even banned for saying jokes online. Which is just what that is, a joke.
If that is the best argument against Riot's community policing strategy, then bring it on. This style of community moderation could easily be built into Wordpress and other off-the-shelf blog/forum backends. I look forward to the day when all online interactions are moderated democratically.

But one of the features of democracy is that an activist minority can sometimes outvote an indifferent majority (eg. the Sad Puppies in the Nebulas). I'd be interested to know if there's any way of protecting against that. Still, the core principle of unpleasant commenters being called on their unpleasant comments clearly works well. Good job, Riot.
1Sign inorRegisterto rate and reply
Craig Page El Presidente, Awesome Enterprises2 years ago
That's great! Maybe they can do some contract work for YouTube to clean up that community. The report/tribunal system could work there too.
0Sign inorRegisterto rate and reply
Robert Mac-Donald Game Designer, Lethe Games2 years ago
Jokes amongst friends are different to jokes amongst strangers. If you don't believe me, tell that joke to a waitress serving you dinner in a restaurant, and gauge her reaction.
Well Morville the internet is not real life is it? it is a place where you get millions of anonymous people saying things, where you can never know if they are being serious or not, and unlike real life, you have buttons that can make you ignore and stop listening to people you don't want to. It takes a different stance to be on the internet. I remember times when I got mad, outraged at the internet when I was much, much younger, and then I learned to have a different stance. And in any rate I am not a bully - it is not my interest to go on a moral crusade on the internet preventing people from saying what I don't like, especially things that are intended to be merely jokes.

And you know, I don't really feel amongst strangers on the internet, for some reason, even when I am amongst them. Maybe it is because of me not equating it to real life, I don't know. I think it was John Carmack that said, when confronted with arguments that violent video games cause violence, that he mentioned Quake con had the most friendly people he'd ever seen. And you know, the things people would say to each other on quake were certainly not nice, yet people didn't really get mad at it, it was just friendly banter.
In any event, whilst your friend may have thought it was funny, that joke is based on stereotyping, and there's many who don't find humour in being cast as a stereotype.
And there are many that like. You don't get to tell what is acceptable and what isn't, so yes Chris Payne:
If that is the best argument against Riot's community policing strategy
I look forward to the day when all online interactions are moderated democratically.
That is why I'm concerned and speaking about things like this. The day the majority gets to decide what can be said on online games is the day they stop being fun for whoever is the 49%. And the day these kinds of censorship leave games and take hold on the whole internet we go back to the middle ages. Might as well put on your Inquisition helmet Chris and go after the heathens and pagans.
0Sign inorRegisterto rate and reply

Sign in to contribute

Need an account? Register now.