Having played video games on Steam since his childhood, Nalin Shani, a PhD student at The Kellogg School of Management realized that the platform offered an ideal setting to study online reviews.
So Shani collaborated with Achal Bassamboo, a Kellogg professor of operations, and Maria Ibanez, an associate professor of operations at Kellogg, to learn more.
Not only does Steam have a massive store of publicly available, unsolicited customer reviews, but it also automatically tracks and reveals the amount of time a user spent playing a game at the time of the review (their experience level), the user’s final recommendation (a thumbs up versus thumbs down), and the number of people who flagged the review as helpful.
“What makes this data very unique is that the credibility is measurable in terms of the experience the reviewer has with the product,” Bassamboo explains. “And the fact that the reviews gave a very clear, black-or-white answer.”
In all, the researchers analyzed 26.8 million reviews for 27,170 games written by 10.6 million unique users from October 2010 through October 2019. And they found that there was a strong link between how much time a user spent playing a video game at the time of their review and how helpful other people found the review to be.
Infographic: Michael Meier
“Reviewer experience [with a product] is a very good signal,” Shani says. “But it is not a uniform signal. It depends on the verdict of the review.”
Among positive reviews, users who had either the shortest playing time or experience with the game or the longest playing time provided the most-helpful reviews. But for negative reviews, users with a moderate amount of playing time—that is, with a moderate amount of gameplay experience—wrote the most-helpful reviews.
Read more about the research in “Leveling Up the Helpfulness of Online Reviews” at Kellogg Insight.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.