Sections

Players less concerned by bugs than bad game design, study finds

Study of over 10 million Steam user reviews unearths valuable data for developers

Players complain more about game design than bugs, a data study conducted by Queen's University in Canada has revealed.

An Empirical Study of Game Reviews on the Steam Platform, which appeared in this month's Empirical Software Engineering journal, concluded that only eight per cent of reviews featured any mention of bugs.

Considering reviews which contained negative feedback, only 17 per cent mentioned bugs, while 57 per cent noted game design issues.

Additionally, 42 per cent of instances that mention bugs in a game were from positive reviews, "suggesting that having bugs in a game does not necessarily lead to negative reviews".

The report suggested that it is important for developers to read reviews, noting that 42 per cent of all Steam user reviews contained valuable feedback (i.e. qualified rather than unsupported statements).

However, the report also found that 71 per cent contained statements of emotion without providing a specific reason and were therefore "not helpful".

Positive reviews provide plenty of valuable information however, with 29 per cent also discussing cons of the games, and seven per cent reporting bugs.

"Moreover, positive reviews contain a higher portion of pros of the games, and a slightly higher portion of suggestions, than negative reviews," the report reads.

"Knowing what players appreciate about a game is important for developers, as they can ensure that these pros remain or are further improved in future updates."

Overall, negative reviews were found to be more useful to developers. Not only did they contain fewer "not helpful" statements, but also a higher portion qualified negative feedback and more evidence of bugs.

The data comes from a March 2016 snapshot of 6,224 games on Steam and 10,954,956 reviews (9,393,546 positive and 1,561,410 negative).

Related stories

Adi Shankar's next video game animated series is Devil May Cry

Capcom franchise will join Castlevania as part of Shankar's proposed "bootleg multiverse"

By Rebekah Valentine

Microsoft reportedly planning release of digital-only Xbox One

Disc-less console would be considerably cheaper, come alongside disc-to-digital program

By Rebekah Valentine

Latest comments (1)

Chris Payne Managing Director & Founder, Quantum Soup Studios6 months ago
This is interesting because the data is inherently self-reported. So it really tells us what Steam reviewers THINK it's important to call out in reviews. I wonder how much criticism is actually just personal preference justified as design flaws? I look forward to digging into their findings in more detail when I have time...
0Sign inorRegisterto rate and reply

Sign in to contribute

Need an account? Register now.