The trouble with game of the year awards
Are we electing the best games or the most popular ones?
By Blake Snow
(Note from Shoe: Blake’s a cool dude and an established freelancer who asked me if he could run this story on Sore Thumbs, and I said yes but that might’ve been because he bought me a grilled cheese sandwich. But do read on, because Blake provides an interesting inside look at how game of the year awards get done.)
Since becoming a full-time critic three years ago, I’ve been involved in selecting several game of the year awards, the summa cum laude of video game honors.
As a newbie, I was surprised to learn that voters are rarely, if ever, required to play all nominees before casting final ballots. Rather, a group of judges throws their favorites into a hat and lobbies against one another, whether or not they are qualified to do so. The process is very political. In most cases, the loudest voice always wins.
“I think the people organizing things would like it if everyone has played any game they nominate to conclusion, but no one’s ever said anything to me about it,” says Paul Semel, an independent reviewer of eight years.
And therein lies the rub, underscoring how unorganized, unscientific, and naive some game of the year awards have become.
Are they really awards?
“It’s important to remember that game of the year awards are being given by individual outlets, not the industry as a whole,” says Kyle Orland, the unofficial media watchdog of video games. “It’s not the Oscars or the Emmys or even the Golden Globes. It’s just a subjective ranking of important games by an editorial staff.”
Since there is no governing body for video game awards, at least not yet, calling them “awards,” is a bit of a stretch. “The moniker of ‘Game of the Year’ needs to change,” says Patrick Seybold, director of communications for team PlayStation. “Without a governing body, there’s no validity to ‘awards,’ although they give marketing teams good fodder for putting logos on boxes. Something like ‘2008 best overall,’ ‘2008 best shooter,’ etc. is more fitting for individual media.”
Obviously, both “best” and “award” are subjective. But the latter generally implies official widespread recognition for something, not just a single organization’s favorite pick. In that spirit, “best of” might be more applicable.
There are two notable award ceremonies, however, that attempt to mimic industry-wide honors — The Game Critics Awards and the Interactive Achievement Awards. But neither are recognized as a sanctioned entity, and each have their share of problems. The Game Critics Awards allows seeing a game as a substitute for actually playing when it comes to voting, and the Academy of Interactive Arts & Sciences requires developers to pay a fee if they want to be considered for an award.
It’s difficult to predict if or when an official “Academy” of video games will ever exist. Until then, the fragmented, shiny, and official-looking “game of the year” medals will have to do.
How do you define “best”?
The interpretive definition of “best” is another problem when it comes to electing a game of the year. What makes one game better than another in any given year? What is “best”? Is it the most popular, the most fun, the most innovative? Or is it the game you keep coming back to?
“For me, the best games are those that show me something I haven’t seen before, either complete innovation (Spore) or perfection of a form (Fallout 3),” says Troy Goodfellow, a freelance critic. “Sequels are, in a way, handicapped here.”
Semel disagrees. “I typically go with what game I had the most fun playing, not the game that pushes the art form or anything like that,” he says. “While Spore and LittleBigPlanet highlighted concepts, I didn’t like them as much as Gears of War 2, so the sequel got my vote.”
GamePro’s Sid Shuman says it can be both, contingent on your audience. “It all depends on what kind of media outlet you are,” the senior editor says. “If you’re Game Developer magazine, maybe it makes more sense to focus on innovation and production values. Ask any gamer on the street, though, and you’ll find one benchmark: fun.”
Sony’s Seybold, whose company is regularly on the receiving end of judgment, says it’s all of the above. “My opinion is that game of the year should be determined by a collection of criteria: innovation, engagement, capitalizing on hardware (even if cross platform, needs to take advantage of all hardware it is on), staying power, and a factor of emulation (i.e. will other game developers want to build on the idea).”
However one defines “the best” game of the year, most agree it should always be the most deserving. Unfortunately, that doesn’t always happen.
No balls required
Given the number of games and the amount of time it takes to complete them, it’s virtually impossible to expect game of the year voters to play everything, especially when there’s writing to be done on the handful of games already played.
Under the current system, four of the five outlets I spoke to rely more on “anything goes” round-table discussions when nominating and electing game(s) of the year as opposed to a set methodology with voter eligibility requirements. The system works more than it fails, but how can a critic responsibly elect the best game of the year (however defined) if they haven’t at least sampled all nominees?
As a result, popularity rules. “The short list of game of the year awards usually ends up being the most popular games that got good reviews and a lot of exposure when they came out,” says Orland, who regularly attempts to counter inflated review scores with his admirable and comment-inducing column dubbed Nega-reviews. “This is because judges are more likely to have played the high-profile games and prefer to follow the crowd.”
As much fun as participating in game of the year awards is, it takes guts to exclude oneself from the process by deeming yourself unqualified.
“Deciding who is eligible to vote is a big problem because much of the gaming press fancies itself as being an expert at every game,” says Goodfellow. “It takes a lot of confidence and humility to say ‘I didn’t play LittleBigPlanet enough to judge it.’”
Many critics, myself included, feel they will be ostracized by their peers if they can’t say they’ve played every acclaimed game, and thus play along. I once lied to an editor about playing Gears of War (it’s kind of a big game) before I had because I desired his professional acceptance. It was a juvenile thing to do, something I regret.
Kyle Orland explains the behavior. “Our personal tastes are really driven largely by the society around us and the desire to have our tastes validated by the larger group,” he says. “It’s risky as a critic to stand out from the crowd by picking an unknown game over the big hit that sold millions and was showered with praise — there could be a backlash among readers who are expecting their favorites to get on the list. It’s just easier and safer to pick the usual suspects.”
A work in progress
Despite what conspiracy theorists may say, game of the year awards mean well — they exist to help players navigate the voluminous waters of available games. They’re not the result of payola (at least not on widespread accounts), and any flaws are the result of innocence as opposed to being calculated.
“Game journalism is a new field that has plenty of maturing to do,” says Shuman, who has been writing for GamePro for more than a decade. “The field hasn’t matured into a full art form yet, so nebulous concepts such as ‘fun’ tend to dominate.”
Dan “Shoe” Hsu, purveyor of Sore Thumbs and former editor-in-chief for Electronic Gaming Monthly says the same. “Within our group at EGM, we had inconsistencies in voting procedures from year to year because we were always evolving the process,” he says, eight months after leaving the magazine for greener pastures. “We’d see the wrong games float to the top, have to make adjustments, and then carry those lessons into next year’s awards.”
Shortcoming aside, there are two simple solutions to fine-tune and improve the current process: voter play stripes and scientific polling.
“There should be some minimum leveling to qualify as a voter,” Seybold says. “Not everyone will complete every game, but there should be a way of proving that a critic is qualified to vote.”
That could be as easy as requiring voters to familiarize themselves with all nominees before casting final ballots, says Orland. “I think some basic experience with all the major releases is expected and not that hard to do — say a requirement of at least 1 hour of play time with each nominated game.”
Few would argue that a slow-starting game is worthy to be named game of the year.
If that isn’t an option, there are always Borda counts, which weigh individual votes according to the order they were submitted by voters. For example, if a voter submits his top five picks, the top game receives more corresponding points than lower picks. This way, priority of picks elects the “best” games, not necessarily the majority of number one picks. “We actually used Borda counts for the last couple of years that I was at EGM,” says Shoe. Crispy Gamer, whom both Shoe and I contribute to, does the same in lieu of requiring that voters play every nominated game.
Is this thing on?
Not everyone thinks the lack of standardization in game of the year awards is a problem, though. “Whence came the outcry to change anything?” asks Bill Kunkel, the “grandfather” of video game journalism. “I’m not sure game of the year awards need improving,” seconds Semel. “Unlike movies and music, I don’t recall the game industry having the equivalent of Jethro Tull winning the Best Hard Rock/Metal Grammy.”
Still, the majority of critics I spoke to think a little internal auditing can go along way towards improving the quality of awards, even if an industry-wide voting system is several years out. “I think more openness about the procedure (vote breakdowns, eligibility, etc) is only a good thing,” Shuman concludes.
If it’s our duty as critics to ensure players stay informed about the best available games, then there’s no shame in evolving how our highest recommendation is named.
About the author: Blake has been a full-time feature writer, columnist, and video game critic since 2005. Armed with a degree in business, he learned the Jedi ways of journalism while working under a former Bloomberg editor while blogging for Joystiq (hi, Vlad). In addition to inspired instruction found in “The AP Guide to News Writing,” he’s been on the receiving end of constructive feedback from senior editors at GamePro, MSNBC, and Crispy Gamer, where he is a contributor. Snow lives in beautiful Utah with his wife and two toddlers.
