Riot: Don't ban your players, reform them

Player behavior in multiplayer online games has been a problem since the beginning of the genre. Efforts to change that behavior have been generally ineffectual, and the toolbox used by publishers has mostly been limited to warnings and the banhammer. Riot Games has been working on changing that using scientific methods, and the results are part of the reason League of Legends has become so popular. GamesIndustry International spoke with Riot's Jeffrey Lin, lead designer of social systems, and producer T. Carl Kwoh about what Riot has been able to accomplish.

Lin, who previously worked at Valve and has a PhD in neuroscience, has approached the player behavior issue with scientific methodology. Riot's enormous player base provides a statistically significant amount of data to work with as the developer strives to improve the player experience. However, the issue of players behaving badly is not something that a company can fix by itself, Riot believes. "Player behavior is really a problem that needs to be tackled by the whole community," said Kwoh. "It's not going to be game developers or academics or players; it has to be everyone together."

Riot sees providing players with the power and letting them take control of their own community as an important step, but it's an evolving process. "Even since the world championships, we've completely changed the way that we do things," Lin said. The player-moderated Tribunal for judging behavior infractions is one part of this. "One of the key things that we noticed was that focusing on Tribunal was only a very small piece of the player pie, because 98 percent of our players are actually very good," said Lin. "What we're seeing is they all have bad days. That's where your toxicity comes from."

"When you have a bad day at work, when you go home you have a higher frustration level before you even play a game," Lin explained. "You also have lower tolerance levels for what happens in the game. As soon as something happens in the game, you snap. That person might not do it in a hundred games, but we just have so many players that those mental instances keep happening throughout many games."

"Now we're really pushing the boundaries of science in general, and also the games industry"

Jeffrey Lin

Riot's been working to improve the behavior before it becomes a problem. "Our last couple of months have been focused on how do we run experiments that prevent these players from going toxic," Lin said. "It's not the people that are toxic, it's all about the context. The insight we had is take a look at the same player playing in normal mode and playing in ranked mode; it's exactly the same person, but the behavior is very different. The tolerance level for behavior is different, the reaction to things is completely different. As game designers we have to figure out how we can create a feature that will also influence behavior outside the game."

"We want to reform toxic influences," added Kwoh. "We made that word choice very specifically because we want to try to reform people back to more sportsmanlike behaviors, rather than necessarily remove them. In a free online game, banning only goes so far." Lin provides an example of how to guide people to better behavior. "Instead of saying 'Hey, you're a jerk, stop doing that,' that is less effective than saying 'Hey, everyone else is good; what are you doing?' Just the angle slightly has much more power," Lin said.

The nature of modern society is affecting the online culture, Lin believes. "If you look at online culture, it's basically a different society that's developing over time," he said. "More and more we have kids and people growing up only online these days; they're not doing as much team sports as they used to. They're not learning about teamwork and cooperation. They're learning it all online, where there are no rule sets at all."

"A lot of these players wrote letters in to us," Lin continued. "One player wrote, 'I just didn't realize how offensive the f-word was.' He wrote in just to apologize; he didn't realize how bad it was. He's been using it in all these other games. There was another player, we showed him his logs, and this player said 'I'm really disgusted by my own behavior. Can you guys give me a little guidance?' Now we're seeing that people online in general really aren't bad; they lack self-awareness in many cases. In many other cases they've learned that there are no consequences for these behaviors."

Lin feels like it's Riot's duty to properly help its players, to build in systems that help them behave better. Is Riot making progress towards that goal? "We don't feel it's an uphill battle," Lin said. He provides an example of how the Player Behavior unit has affected things. "We wanted to experiment: Do the people go to the Tribunal for currency, or do they go to contribute to society? We removed the currency reward for thirty days, and then we introduced justice reviews. That shows you how you contributed to the site and the community; how many toxic games did you help prevent? When we removed the currency we lost about 10 percent of our players in the Tribunal. When we introduced justice reviews, we got double the number of players. People actually go to make a difference."

One of the concerns players raise is whether the play environment might get too bland. "A lot of players complain, 'Riot Games, are you against profanity? This is a mature game, so why can't people swear?,'" said Lin. "If you look at the data, that's not what players are punishing. What they're saying is that profanity is OK, but you can't direct it at somebody else. As soon as they target somebody else, that becomes offensive; that becomes toxic. Now it's become really clear: As soon as you target somebody, they're like 'We're punishing this.'"

Kwoh pointed out that players are policing other players carefully, not just arbitrarily. "A really nice thing to see is that level of nuance," Kwoh noted. "People on the forums like to say 'Well, do people even read the cases?' Our players are reading these cases; we have confidence that people are regularly making these very nuanced determinations."

Pro sports has lots of trash talking, but there are well-defined lines that you can't cross without consequences. Riot sees the problem similarly; some level of aggression helps make sporting matches more interesting, but there's a certain level at which it is not desirable. "In general we work with the eSports team to help them develop their sportsmanlike code of conduct, which is very common in many competitive organizations," Kwoh said. "If they're saying something to another player and they're both bantering back and forth, and both players accept it, that's not a subject for the Tribunal. What we're seeing is this very clear line that it's OK if all parties are OK with it, but as soon as someone isn't it should stop."

"Even though we have used bans, as a developer that is the worst action. As much as possible, we want to see if we can work things out"

Jeffrey Lin

The punishments meted out to professional players have affected regular players. "For a long time our players had the perception that pro players were immune, that no matter what they did we were not going to do anything because they were professionals," Lin said. "In the past year we've had our incidents with the pro players, and the eSports team has handed out some discipline. A lot of players have apparently turned around and said 'Oh wow, Riot really did something; all players are treated to the same standard.' Pro players are meeting an even higher standard. The players do appreciate it."

Are there differences in player behavior around the world? "What we're seeing is that being a gamer is like being a culture in itself," observed Lin. "In many ways we are more similar as gamers than we are different as real cultures. A lot of scientists and developers often come to us and say, 'What about country B? Aren't they far more toxic than country C?' Well, no, they're not. Gamers will be gamers."

Now that Riot has shown some of these techniques work on a massive scale in League of Legends, the company is planning to try some new things. "We're going to create experiments that are interesting to developers to improve the game experience, but also extremely interesting to scientists," said Lin. "Now we're really pushing the boundaries of science in general, and also the games industry."

"At GDC we talked about our priming experiment. Priming is a memory effect where exposure to a stimulus can affect later behavior," explained Lin. "If I show you a brief flash of red and then you do a test, you'll do worse on the test by up to 10, 20 percent. Scientist have been studying priming for years and years, and to this day they still don't understand why our brain makes these color associations."

Riot has been using priming to make significant improvements in the game. "We actually combined priming with an in-game feature," Lin noted. "We did an experiment with 217 unique conditions, where every single game we had a different in-game tip, and we also did stuff like changing the colors or varying the locations. What we found is that one sentence can have massive effects on behavior; it can reduce verbal abuse by 10 percent."

Riot's research into player behavior continues. "We're trying to figure out stuff like the Tribunal; one of the weaknesses is speed," Lin said. "You have a behavior, and if you can't punish the player until a week later using the community to help analyze and engage on that behavior, is it possible for us to somehow interact earlier with that player? We're going to do the research to find out. How do we get all of our effects to be long term and persistent character changes and not simply short-term? Even though we have used bans, as a developer that is the worst action. As much as possible, we want to see if we can work things out."