Sections

Steam removes "off-topic review bombs" from overall game review scores

Reviews will still be visible, scores viewable for users who opt out of new system

Responding to ongoing concerns about "review bombs" negatively affecting games' review scores for issues unrelated to the games themselves, Valve has announced a change to how it calculates those scores across Steam.

"It's clear to us that players value reviews highly, and want us to ensure they're accurate and trustworthy," reads an official blog post on the subject. "Developers understand that they're valuable to players, but want to feel like they're being treated fairly. We've also spent a bunch of time building analysis tools to help us better understand what's happening in the reviews across all titles on Steam. With that feedback and data in hand, we think we're ready to make another change."

Going forward, Valve has implemented a system that will identify what it calls "off-topic review bombs" and notify a team to investigate them. If the team determines the game is being targeted by an off-topic review bomb, it will notify the developer and remove all review scores submitted during the time the review bomb was occurring from the game's average review score.

The reviews will still be visible as written, and users who opt out of the function in their settings will see the score with the review bomb incorporated.

Valve defines a review bomb as when "players post a large number of reviews in a short period of time, aimed at lowering the Review Score of a game," and an off-topic one as "where the focus of those reviews is on a topic that we consider unrelated to the likelihood that future purchasers will be happy if they buy the game, and hence not something that should be added to the Review Score." The company has clarified this includes review bombs focused on issues of DRM or End User License Agreement changes.

Related stories

Steam update changes In-Home Streaming to Remote Play

Games from your Steam client can now be streamed to PCs and devices across different networks

By Rebekah Valentine

Valve making its own version of popular Dota 2 mod

After talks for a collaboration are abandoned, Valve will make a competing version of Dota Auto Chess "with Drodo's blessing"

By Brendan Sinclair

Latest comments (12)

Dariusz G. Jagielski Game Developer 3 months ago
Except DRM issues and EULAs that for example allow to collect all your data with basically impunity is very related "to the likelihood that future purchasers will be happy if they buy the game".
0Sign inorRegisterto rate and reply
Luzarius Rollo 3D Modeler 2 months ago
This could rob consumers of the ability to give vital feedback over what they feel are valid concerns. What one person considers a valid concern can be misinterpreted as off topic by a steam employee. A rule like this can be abused to silence people with valid criticism.

When a game gets reviewed bomb that sends a red flag for me to research the company and find out why they got review bombed. It's important review bombs be allowed because it informs consumers of the companies wrong doing. I can then research why it got review bombed and draw my own conclusion. Hiding that from me is wrong.

If a product gets review bombed, there's almost always a valid reason for it. Empower the consumers and let them draw their own conclusions. If you don't empower the consumer, you'll face backlash.

Edited 1 times. Last edit by a moderator on 18th March 2019 2:05pm

2Sign inorRegisterto rate and reply
Antoine Baker Artist/Designer/QA Tester 2 months ago
@Luzarius Rollo:
Product reviews are not for moral grandstanding. Reviews are a way to give objective feedback and also reflect on the quality of the product and your overall experience with the product. The companies use the collective data of the feedback given to change design, improve features make additions or subtractions or adjust price.

For example, lets say you made models for a game. It looks and plays gorgeously. And in an interview you said that you like sexy women in your games. No harm in that. Let's also add that the game is published by your own game studio. Now lets say the game had the best looking graphics and models out of every other release. The music score is perfect. The story had great pacing with a lot of plot/character development and twists. It's practically a candidate for GOTY. Now let's say I saw the interview and took what you said out of context. Would it be fair for me to write a review saying "Duh graphikz and gaime sux cuz da moddler Iz uh PuRvArT"! How is that a fair critique to you and your craft? How does that help someone who may have general interest in the game? How does that help you improve upon the genre or sequel? Now lets say I didn't play the game at all and just review bombed the game due to my misinterpretation of your opinion.

Do you deserve to be review bombed? I just sank your hopes of future funding for games and I didn't even play the game. Now lets say people who might want to go buy the game look at the review and look into your company. Now let's say they look into your company and see that "dreaded" interview. Do you feel they should boycott your game? Now let's bring this hypothetical home. Let's say no one plays your masterpiece of a game. It doesn't get nominated for any awards or considerations and you were forced to apologize.

And what if I told you that you are now forced to lay off your employees and shut your studio down all because the "empowered" consumer grandstanded you because of difference of opinion. Do you deserve every bit of the review bombings and ill intent that came your way?

I implore you to really place yourself in that scenario and really think about what you just said. Because you basically just said if you were on the receiving end of massive review bombs, the customer is completely vindicated of wrong-doing and that your product, by default, is faulty because of a misinterpretation/difference of opinion and not because of any concerns with the way the game was crafted.
6Sign inorRegisterto rate and reply
Show all comments (12)
Morville O'Driscoll Blogger & Critic 2 months ago
When Captain Marvel was review bombed, it helped me research why and I discovered Brie Larson was discriminating against movie critics on the basis of race
Except... She wasn't? The only way you could come to that interpretation is if you uncritically believed conservative media. Now, this seems off-topic for this article, but it's really not:

Review-bombing a lot of the time has to do with a) a majority of "fans" being told something they don't like, and b) bias/privilege from those "fans". The Washington Examiner (conservative/right-wing paper) took Brie Larson's words and framed them as her discriminating on the basis of race. What she actually argued for was elevating minority voices within film criticism, and, for example, Variety reported this accurately. But the film was review-bombed, you got side-tracked, and then you got misled/misinformed by people expressing their bias/sexism/hate. Review bombing is politicised, and it is used quite often as a cudgel against companies who promote diversity.

As it is for Captain Marvel, so it was for, say, the Total War films. "Female generals are historically inaccurate!" the review-bombers say. Except, 1) the Total War films are not meant to be 100% accurate, and 2) the claims of historically inaccurate indicate a bias - many of those review-bombing neglected to educate themselves (or were wilfully ignorant) about women generals during the Rome time period.
If a product gets review bombed, there's almost always a valid reason for it.
Captain Marvel was hate for Brie Larson, and for women superheroes. Total War was for women generals. Chucel was for redesigning to avoid blackface associations. Skyrim was because of paid mods. Firewatch because Campo Santo having problems with PDP. Devotion because of a Winnie the Pooh/Xi JinPing reference. Dota 2 because of something Marc Laidlaw wrote.

There have been a few occasions - mostly revolving around DRM - where review-bombing has done some form of good. But for the most part review-bombing is antagonistic to the developers, the publishers, and society, being as it is mostly a form of temper tantrum against diversity. Valve's efforts here will, if nothing else, make it so that a woman who wants to play Rome 2 won't have sexist dicks shoving their sexist views in her face on the store page as she buys the game.

Edited 2 times. Last edit by Morville O'Driscoll on 18th March 2019 9:47am

7Sign inorRegisterto rate and reply
Robin Clarke Producer, AppyNation Ltd2 months ago
If we're acknowledging that abuse of the user review system is a problem, why does the option to opt out of this anti-abuse mechanism exist?

Someone at Valve has suggested a genuine improvement to their ecosystem, and someone else at Valve has argued that pushing the toxic element of the community away will hurt their bottom line. As long as they keep listening to that second voice, they'll increasingly see users and developers seeking alternatives to Steam.
2Sign inorRegisterto rate and reply
Luzarius Rollo 3D Modeler 2 months ago
@Morville O'Driscoll:
>What she actually argued for was elevating minority voices within film criticism

I'm a minority and I don't need special privileges. I don't need to be elevated as if I'm less than anyone. A lot of minorities despise this way of thinking. Progressive ideology is in conflict with Martin Luther King's basic principle of don't judge by skin color, but by content of character. Apply MLK's ideal to Brie's argument and what conclusion can we draw? That the race of the movie critic does not matter, it's the content of their review that matters. This is what separates Progressives from Traditional liberals, which has caused many traditional liberals to be forced onto the side of Conservatives.

>Review-bombing a lot of the time has to do with a) a majority of "fans" being told something they don't like, and b) bias/privilege from those "fans".

As a minority I believe these fans don't have bias or privilege. I also prefer they have a means of responding with negative criticism when they're told something they don't like. I'm on the side of the consumer, I'm on the side of the people.

>But the film was review-bombed, you got side-tracked, and then you got misled/misinformed by people expressing their bias/sexism/hate.

This kind of rhetoric drives a wedge between progressives and traditional liberals. You think everyone who disagrees with you is biased, sexist and filled with hate, but in most cases they're not. Wild accusations like this cause friendly fire against people who might have originally been on your side politically, be careful.

>Captain Marvel was hate for Brie Larson, and for women superheroes.

Wrong. Traditional liberals like me love women superheroes and women leads. We have loved their presence in lead roles since the 60's. A lot of conservatives feel the same way. There's been tons of empowered women in lead roles since the 60's, yet progressives won't acknowledge this fact supported by clear objective evidence.

>Captain Marvel was hate for Brie Larson, and for women superheroes. Total War was for women generals. Chucel was for redesigning to avoid blackface associations. Skyrim was because of paid mods. Firewatch because Campo Santo having problems with PDP. Devotion because of a Winnie the Pooh/Xi JinPing reference. Dota 2 because of something Marc Laidlaw wrote.

I consider these all valid criticisms that should be eligible to be in a consumer feedback review. These are the kinds of things the professional game reviewers won't say. Consumers tell it like it is. I NEED that valuable feedback. I do not read professional game reviews because they hide things that are important to me. I trust the consumer more than game journalists. The consumer holds nothing back and can tell me EVERYTHING wrong with a company.

>There have been a few occasions - mostly revolving around DRM - where review-bombing has done some form of good. But for the most part review-bombing is antagonistic to the developers, the publishers, and society, being as it is mostly a form of temper tantrum against diversity.

People are not against diversity as long as no one was initially judged by skin color. The problem now is that progressives openly make decisions based upon race & gender. This has caused a rift between progressives & traditional liberals, because you're acting against MLK's most basic ideal.

Also, who are you to decide who is right & wrong with their review? Ditch that mentality and instead empower ALL consumers to leave the feedback they feel they need to express & share, let them say whatever they want. If you do that, the people will love you, if you do not, you'll make them your enemy.

I implore you to change how you think to a way that empowers people with more freedom to express their concerns & ideas, not less. Don't impose too many restrictions or you'll face needless backlash that was never there to begin with.

Edited 2 times. Last edit by Luzarius Rollo on 18th March 2019 10:19pm

1Sign inorRegisterto rate and reply
Benjamin Solheim Sole Shareholder, ESPN2 months ago
If the people who created the game are reviewing it, then those are artificially boosting the reviews.

I am glad that steam is admitting the review system is currently worthless, as removing negative posts because people stopped buying mostly negative games, means that games that have more ad and publishing dollars to make sure they succeed makes it more likely to get a trend downward in quality. Too many artists and developers do not take pride in what they make, because beating the mile stones results in more projects and getting off the project they are unhappy working on.

That said there are some companies that pay to post bad reviews of games, so if they switched to the number of hours played reviews first and kept the ratio of negative to positive reviews a better view of the games would likely show up. If a review bomb campaign has to spend a hundred hours in game to write a bad review and people can see that player 1 played five hours and wrote a best game ever post, while player 2 wrote that they story fails after the first ten hours due to steams no refunds after two hours, so as long as the first two hours are good, then those ten hour viewpoints might be people that have not gotten to where the writer quit, or was shifted to another game pipeline or the assets like in age of conan were not done ten hours into the game.
0Sign inorRegisterto rate and reply
Morville O'Driscoll Blogger & Critic 2 months ago
@Luzarius Rollo:

I could respond to your other points and talk about how just because you're a minority, does not mean all minorities feel like you do, and how that was actually the point of what Brie said ("I want to hear what a woman of color, a biracial woman has to say about the film. I want to hear what teenagers think about the film"); I could talk about how MLK "looked forward to a day" when colour of skin was not an issue, but how currently being colour-blind perpetuates bias; about how you're making this between liberals and progressives, when it's really mostly between liberals of every shade and conservatives who scream cuck and SJW; and how I think you misstate my position as "everyone who disagrees with you is biased, sexist and filled with hate". I don't. I do believe that anyone who supports the Conservative Party or UKIP in the UK, and the Republican Party in the US perpetuates hate and furthers fascist tendencies. I could argue all these points, but instead, I'll just go to your last point:
I implore you to change how you think to a way that empowers people with more freedom to express their concerns & ideas, not less.
and then ask you to look at this screenshot of the review-bombing of Rome 2 on Steam - https://cdn.mos.cms.futurecdn.net/UctJcUmvjhFeSQRrJWcUdh.jpg . ThenI'll ask you two questions:

Why do you feel the need to empower hate mobs? And why do you feel the need to let hate-filled content - unnecessary to determine if the game is good - remain in place because of "freedom"?

Edited 4 times. Last edit by Morville O'Driscoll on 19th March 2019 12:34am

2Sign inorRegisterto rate and reply
Luzarius Rollo 3D Modeler 2 months ago
@Morville O'Driscoll
>https://cdn.mos.cms.futurecdn.net/UctJcUmvjhFeSQRrJWcUdh.jpg

I agree with most of those reviews in the screenshot. Those are the exact reviews I look for before making a purchasing decision for a game. It's important our views (like the ones in the screenshot) are reflected for the sake of viewpoint diversity, or else you'll be missing out on what 50% of your audience thinks of your game.

>Why do you feel the need to empower hate mobs?

There is no hate in that screenshot, only valid concerns & criticisms. You need to educate yourself as to why labeling everything 'hate' is wrong. You're trying to silence valid criticism: https://pbs.twimg.com/media/Ckrkh2EVEAEmGs1.jpg - Check out this diagram. Jonathan Haidt is a respected academic from New York.

Most people do not like historical revisionism and dislike radical, marxist, anarchist third wave feminism. I bet you every one of those people in that screenshot have no problem with traditional feminism & some forms of modern liberal feminism. They also value historical accuracy and there's nothing wrong with that. If you listen to both sides, you can easily find a solution that caters to both audiences.

Take Battlefield 5 for example, there's tons & tons of women who fought in World War 2 that EA/Dice could've drawn from to accurately represent women in that war, but instead they relied on historical revisionism which betrays real actual women who fought in those battles.

I'm enjoying this conversation by the way. I love talking to people outside my echo chamber. Keeping consumer reviews as unrestricted as possible increases viewpoint diversity so you can truly understand what all your customers are thinking.

Professional Reviews have to hold back and their review goes through a political & ideological filter, but consumer reviews don't have that problem so we can get the whole truth. The only people who are afraid of consumer reviews are the ones with an agenda they're trying to push.
1Sign inorRegisterto rate and reply
Daniel Trezub QA Analyst, Ludia2 months ago
The reviews will still be visible as written, and users who opt out of the function in their settings will see the score with the review bomb incorporated.
It's beautiful and touching how Valve cares about not hurting the feelings of players who get hurt by every single thing (and, by logic, are the ones who actually write the bomb reviews).

Wanna complain about the feminist treatment the developer gave to that single one, totally not important character, buried in an obscure sidequest, somewhere completely out of the way from the main storyline? Go to Twitter or Youtube or whatever channel the cool entitled kids use nowadays.
0Sign inorRegisterto rate and reply
Daniel Trezub QA Analyst, Ludia2 months ago
@Luzarius Rollo:
Professional Reviews have to hold back and their review goes through a political & ideological filter, but consumer reviews don't have that problem so we can get the whole truth. The only people who are afraid of consumer reviews are the ones with an agenda they're trying to push.
By looking at the screenshot from the reviews for TotalWar, I can clearly see who's going through a political/ideological filter and who is trying to push an agenda. All the reviewers there are pushing the same agenda, going through the same political and ideological filter. And even if you think you are not, you are, too.

Everybody and their moms have their own views of everything, this is how we're wired. Some views are opposite to others, and this is ok. The problem is when one's view goes against SOMEONE, and not against someone's own view. Can you see the difference?

What we're talking here is about those views which go against other people. Saying that a game is not a place for a woman character is saying women are not welcome in videogames. And that's NOT ok, no matter how you phrase it.
2Sign inorRegisterto rate and reply
Luzarius Rollo 3D Modeler 2 months ago
@Daniel Trezub:
>By looking at the screenshot from the reviews for TotalWar, I can clearly see who's going through a political/ideological filter and who is trying to push an agenda.

Historical accuracy is not a political/ideological agenda, it's just stating truth. Do you want people requesting the truth in the form of historical accuracy to have their reviews not count?

If a game clearly advertises it is not going for historical accuracy, then it's totally fine to perform historical revisionism. However, when you call a game ROME: Total War, you're giving the impression the game is about ROME & WAR, and people will expect historical accuracy. Now if you advertise the game as ROME: TOTAL WAR RE-IMAGINED, then you can add things that never happened because you're 100% transparent.

>And even if you think you are not, you are, too.

My goal is to allow as much freedom as possible, so every consumer has a voice.

>Some views are opposite to others, and this is ok.

I agree.

>The problem is when one's view goes against SOMEONE, and not against someone's own view. Can you see the difference?

If you're saying reviews should not attack people on a personal level, then I totally agree with you. We should attack the persons idea, not on a personal level. If the person has a radical political view that has negatively impacted the game, it's okay to criticize the ideal, but if you attack the person on a personal level, that is unacceptable.

>What we're talking here is about those views which go against other people. Saying that a game is not a place for a woman character is saying women are not welcome in video games. And that's NOT ok, no matter how you phrase it.

If you're game is known for historical accuracy and you add something to your game that is not historically accurate, you will be criticized. It's not because your ideological opponents hate women. Look at Warhammer 2 for example, there's tons of women generals in that game and in a fantasy world you can make up whatever you want, no one had any problem with women generals in Warhammer 2. The same people you accuse of 'hate' actually loved women generals in Warhammer 2. This is why it's hard to take people like you seriously because your arguments are easily debunked.

Take the time to understand your opponents instead of immediately resorting to claim they 'hate' and you'll find you'll make more friends & allies. Progressives have already declared war on traditional liberals AND conservatives, good luck with that fight, you're gonna need it. The only progressive argument I understand is their anti-war stance, I love how progressives are anti-war. That's very cool.

*Cheers* to everyone who responded. We're making progress by having this dialogue. Take care & *HUGS*
1Sign inorRegisterto rate and reply

Sign in to contribute

Need an account? Register now.