Players less concerned by bugs than bad game design, study finds

Study of over 10 million Steam user reviews unearths valuable data for developers

Players complain more about game design than bugs, a data study conducted by Queen's University in Canada has revealed.

An Empirical Study of Game Reviews on the Steam Platform, which appeared in this month's Empirical Software Engineering journal, concluded that only eight per cent of reviews featured any mention of bugs.

Considering reviews which contained negative feedback, only 17 per cent mentioned bugs, while 57 per cent noted game design issues.

Additionally, 42 per cent of instances that mention bugs in a game were from positive reviews, "suggesting that having bugs in a game does not necessarily lead to negative reviews".

The report suggested that it is important for developers to read reviews, noting that 42 per cent of all Steam user reviews contained valuable feedback (i.e. qualified rather than unsupported statements).

However, the report also found that 71 per cent contained statements of emotion without providing a specific reason and were therefore "not helpful".

Positive reviews provide plenty of valuable information however, with 29 per cent also discussing cons of the games, and seven per cent reporting bugs.

"Moreover, positive reviews contain a higher portion of pros of the games, and a slightly higher portion of suggestions, than negative reviews," the report reads.

"Knowing what players appreciate about a game is important for developers, as they can ensure that these pros remain or are further improved in future updates."

Overall, negative reviews were found to be more useful to developers. Not only did they contain fewer "not helpful" statements, but also a higher portion qualified negative feedback and more evidence of bugs.

The data comes from a March 2016 snapshot of 6,224 games on Steam and 10,954,956 reviews (9,393,546 positive and 1,561,410 negative).

Related stories

Telltale: "Multiple potential partners" offered to save The Walking Dead's final season

Clementine's story may yet be finished, but Netflix is looking for other options to make its Stranger Things game

By Matthew Handrahan

Fans delay football game with esports protest

Young Boys-FC Basel supporters united in opposition of competitive gaming, interrupt match by throwing tennis balls, controllers onto the field

By Brendan Sinclair

Latest comments (1)

Chris Payne Managing Director & Founder, Quantum Soup Studios4 months ago
This is interesting because the data is inherently self-reported. So it really tells us what Steam reviewers THINK it's important to call out in reviews. I wonder how much criticism is actually just personal preference justified as design flaws? I look forward to digging into their findings in more detail when I have time...
0Sign inorRegisterto rate and reply

Sign in to contribute

Need an account? Register now.