Players less concerned by bugs than bad game design, study finds

Study of over 10 million Steam user reviews unearths valuable data for developers

Players complain more about game design than bugs, a data study conducted by Queen's University in Canada has revealed.

An Empirical Study of Game Reviews on the Steam Platform, which appeared in this month's Empirical Software Engineering journal, concluded that only eight per cent of reviews featured any mention of bugs.

Considering reviews which contained negative feedback, only 17 per cent mentioned bugs, while 57 per cent noted game design issues.

Additionally, 42 per cent of instances that mention bugs in a game were from positive reviews, "suggesting that having bugs in a game does not necessarily lead to negative reviews".

The report suggested that it is important for developers to read reviews, noting that 42 per cent of all Steam user reviews contained valuable feedback (i.e. qualified rather than unsupported statements).

However, the report also found that 71 per cent contained statements of emotion without providing a specific reason and were therefore "not helpful".

Positive reviews provide plenty of valuable information however, with 29 per cent also discussing cons of the games, and seven per cent reporting bugs.

"Moreover, positive reviews contain a higher portion of pros of the games, and a slightly higher portion of suggestions, than negative reviews," the report reads.

"Knowing what players appreciate about a game is important for developers, as they can ensure that these pros remain or are further improved in future updates."

Overall, negative reviews were found to be more useful to developers. Not only did they contain fewer "not helpful" statements, but also a higher portion qualified negative feedback and more evidence of bugs.

The data comes from a March 2016 snapshot of 6,224 games on Steam and 10,954,956 reviews (9,393,546 positive and 1,561,410 negative).

Related stories

Former Heroes of the Storm team files suit against ex-owner for unpaid wages

Five ex-Naventic members filed a complaint in August, attempted to serve court documents 14 times before reaching owner James Ross Elliot II

By Rebekah Valentine

Media Create: New Super Mario Bros. U Deluxe sells through over half of first-week shipment

Tales of Vesperia: Definitive Edition sees strong launch in Japan on both PS4 and Nintendo Switch

By Rebekah Valentine

Latest comments (1)

Chris Payne Managing Director & Founder, Quantum Soup Studios8 months ago
This is interesting because the data is inherently self-reported. So it really tells us what Steam reviewers THINK it's important to call out in reviews. I wonder how much criticism is actually just personal preference justified as design flaws? I look forward to digging into their findings in more detail when I have time...
0Sign inorRegisterto rate and reply

Sign in to contribute

Need an account? Register now.