Metacritic and the Legitimacy of Video Game Journalism, Part I

I recently bought an Xbox 360 since I was interested in playing a couple of recent games. Some of my buying choices were influenced by reviews on websites. Games like Red Dead Redemption were great, but others didn’t quite live up to my expectations. I’ve now had about a handful of experiences were the glowing reception of the games by reviewers didn’t at all match my subjective experience. Thus, I wondered how legitimate video game journalism really is.

I also noticed that some games I tremendously enjoyed, such as Vanquish, received a rather lukewarm reception. Also, I couldn’t help but notice that gaming websites are plastered with ads, so the obvious assumption is that those sites don’t want to bite the hand that feeds them and therefore promote “AAA titles”, while they spend little attention to games that cater to a niche audience. It’s also quite obvious that many mainstream reviewers don’t understand particular genres or, well, just plain suck at playing video games.

Here is a prime example: the Destructoid review of Vanquish by Platinum Games, written by Jim Sterling. He gave the game a 5 out of 10, and every single word he wrote indicates that he just didn’t get how to play this game. Vanquish isn’t a “cover-based shooter” in the vein of Gears of War but instead puts heavy emphasis on offensive play. The game is a bit more complex than in, say, Call of Duty, but you’ll get amply rewarded if you spend a few minutes to learn the controls. This was seemingly too much effort for Jim Sterling, so he wrote:

Sam [the protagonist] actually needs energy to punch his opponents, and once he’s landed a single successful punch, he can’t glide away since the energy meter completely drains. Several times, I punched an enemy, failed to kill it thanks to Sam’s inability to aim his punches properly, and was killed because I could neither defend myself or swiftly escape.

The issue is that the energy meter that enables you to perform more powerful attacks depletes as you do your little tricks. However, there is a risk/reward mechanism built in. If you completely replete the energy meter, your combat suit overheats. You then have to allow it to cool down, which makes you vulnerable to enemy attacks since you can neither defend yourself properly nor quickly evade. The core mechanic is therefore to find a rhythm for your attacks. This is not as bad as it may sound since the energy meter replenishes very quickly. I found the game mechanic to be highly satisfying, and with some practice, it’s quite easy to get into a state of flow. Frankly, I thought that Vanquish was absolutely fantastic and that it reaches a high-water mark for action games. Enthusiastic reception of this game by actual players, like on NeoGAF, seems to indicate that I’m not the only one who had been very impressed by it.

Jim Sterling’s review of Vanquish may be an egregious example, but the average games journalist is hardly an expert. Especially when it comes to niche games they don’t seem know what they are looking at. For instance, one of my favorite genres are shooting games (STGs). No, not the Call of Duty kind, but the modern descendants of Space Invaders. While most games strive to be “entertainment”, and therefore offer at best a moderate challenge, STGs are designed for repeated play throughs, with the goal being mastery so that you can eventually clear the entire game on just one credit. Depending on your skill level, this may take many months, and with the harder games you may never even get there because you’re just not good enough. This can be a humbling experience, but if you master a game like that, you’ll feel a sense of achievement which you just don’t get from games that hold your hand all the way through. Sure, it’s not for everyone, but those games have enthusiastic fans. Yet, the typical mainstream reviewer is quick to dismiss those games because you can just hit “start” again and “see everything in 15 minutes”. The thread Amusingly bad reviews on shmups.com collects statements like that. You can only shake your head.

I’ve now mentioned some examples of games or genres that tend to get short shrift by mainstream reviewers. Now let’s look at games that receive lavish praise, and whose faults either get ignored or justified. One prime example is one of the greatest commercial successes in recent years: Grand Theft Auto IV. It sold 25 million copies, and it’s the top rated Xbox 360 game on Metacritic. I think it’s a decent game, but it has its flaws, like a repetitive mission structure, poor driving, and clunky weapon mechanics. It’s not a bad game, but hardly the masterpiece it is claimed to be. Less than a quarter of the people who bought the game finished it. The other three quarters probably got bored or frustrated.

Another example is Resident Evil 5. It’s probably not a bad game once you get into it, but it doesn’t make it easy for you to like it. My main gripe is that your character controls like a tank. One of the very first scenes has you enter a shack. Then zombies start attacking you from two and then three sides. They first come running at you, but before they reach you, they seem to hit an invisible wall that makes them stop. From this point onward, they take turns attacking you, which results in incredibly awkward gameplay. This isn’t my idea of having fun, so I have yet to return to this game. When Resident Evil 5 came out, reviewers were defending the controls as “traditional Resident Evil gameplay”, and hordes of obnoxious gaming fanboys were eager to tell anybody who dared to criticize their favorite franchise a variation of “the controls are fine, maybe you just suck at the game.”

Seeing that some of the biggest games have quite startling flaws, I ended up wondering whether “AAA games” that are backed by multi-million advertising campaigns get much more praise than they deserve, and not because they are so great, but because “money hatting” buys good review scores. The big games normally don’t dare to be challenging, so even your average video game reviewer can play them. On the other hand, it seems that many journalist lack the knowledge and skills to appraise niche games, and are therefore quick to dismiss them. A glance at review scores on Metacritc, which contrasts user and “expert” opinions seemed to support this hypothesis. Looking for hard facts, I then made use of my programming skills and analyzed their data. I will share the results with you in the next post.

4 thoughts on “Metacritic and the Legitimacy of Video Game Journalism, Part I

  1. Jack Sprat

    I thought it was pretty well-known that a lot of ratings from “professional” critics are inflated for whatever reasons, if not conformity. How else does Diablo 3, CoH 2, Dragon Age 2, etc. get such comically high marks. As far as video games are concerned, general consensus from consumers is always going to be more trustworthy than paid critics.

    Reply
    1. Gregor Ulm Post author

      When I bought my Xbox 360 it didn’t have a sticker on it that said “beware of professional video game reviews”. Of course, if you are an avid video gamer and have been gaming non-stop since the tender age of 4, you might just know about the untrustworthiness of game reviews because you’ve learnt through experience. This doesn’t apply to me, though. To put the article above in perspective: as a kid I got an SNES and, obviously, did not have a lot of disposable income. So, I got a few games from my parents over the years, and then I borrowed games from friends from school. I don’t think any of us paid much attention to video game journals, except Nintendo’s Club Nintendo Magazine, the European equivalent to Nintendo Power in the US. Back then, you could buy basically any game by Nintendo and expect it to be at least pretty good, and even the bigger games from third parties were good. The only game I didn’t enjoy was Mortal Kombat, but the other 20 or 25 I had access to were high quality.

      I skipped the PlayStation generation, and the next machine I had was a Sega Dreamcast. I primarily bought Sega games, and again, could just rely on first-party titles being very good: Crazy Taxi, Shenmue I & II, Skies of Arcadia, Jet Set Radio, Rez, Sonic Adventure, oh, and then there was Soul Calibur. I had fun with that console, but then life happened and I just didn’t have any time for video games anymore for about a decade, with the exception of Doom 3, of which I had heard of, bought a new graphics card for my PC for, and liked quite a bit. Since then I have hardly been playing games, except some random 2 player matches at friends’ places. I had a very vague idea of what has been happening in gaming, like the transition to more movie-like experiences, games becoming more “accessible”, i.e. really easy, and first-person shooters being the main genre. All this didn’t appeal to me.

      Eventually I discovered MAME, and found out about modern STGs, like Cave’s DoDonPachi. This was the kind of game I liked, but since I don’t like playing in front of a computer that much, I bought an Xbox 360, picked up most Cave games since that’s the only console you can play them on, and I’ve been having a great time with them. But I was also curious about other genres, so I bought a couple of Call of Duty games, some Xbox exclusives like Gears of War and Alan Wake, and certainly didn’t enjoy them as much as first-party Nintendo or Sega games back in the days. Nowadays I read “Microsoft Game Studios” on the box as “stay away”. Oh, and the less said about Call of Duty (except the very good CoD 4), the better. Sony does seem to have a lot of high-quality first-party games and exclusives on PlayStation 3, though, so it might just be Microsoft who doesn’t care about the quality of their games that much, as long as they make money. After all, they know how to peddle inferior products thanks to MS-DOS, Windows, and Office.

      The mainstream games I tried, based on reviews on gaming websites like IGN or Destructoid, were often hit or miss. What quite annoyed me was that the reviewers did not mention the glaring flaws some of those supposed masterpieces have. Why is it that I have to google for “why game X sucks” and skim through fanboy arguments on forums to learn about the actual flaws of those big budget games? I took a trip down nostalgia lane and looked up reviews of SNES games. Those were mostly scanned from magazines, and based on my memories I agree with them, like the perception that Super Mario World is one of the best platforming games ever created, Tetris Attack being highly addictive, or that the third Donkey Kong Country was a big step back from the second one. Back then reviewers seemed to have been a bit more objective and you could rely on them as guides for your purchasing decisions. Today this isn’t the case at all anymore.

      Reply
      1. Jack Sprat

        I agree, reviewers of then, compared to the reviewers of today, were in a very different atmosphere. Was it even was a problem? Gaming was a very niche thing when I started on stuff like the Atari and Nintendo, I was about 2 when I started, so I can’t really tell you how legitimate my experiences would be, I would’ve probably loved anything – but gaming was clearly not the huge consumer market it is today. Ever since the advent of the internet, online advertising, and video games becoming a mainstream thing; inflated scores feel really prevalent, especially from those big gaming websites like IGN or Giantbomb – GB gives out 100’s like free candy. If the website has Mass Effect 3 adverts plastered around, you could imagine how legitimate the coming review will actually be. If reviewers want to keep getting review copies for future games, you can imagine how hesitant they’d be to point out anything but minor flaws in a game. Media blacklisting happens. It’s the gross truth and it’s not just Microsoft. In any case, it was a really nice write-up.

        Your part two was also pretty spot on, a lot of people dismiss the user ratings on metacritic because of how polarizing or knee-jerk they can be, but I argue that they balance out when you get enough of them and they’re almost always a more honest reflection on game quality (in my experience anyways). Just look at Modern Warfare 3 (Xbox), which has an 88/100 metascore, but 8300+ people rated it to an average of 3.3! Are the critics this out of touch with the consumer or is something else going on? Personally, the user score has better reflected my experience more often than not.

        But it makes sense! The metascore is an aggregate score of a handful of “professional” critics who are influenced by more than their opinion, while the user score is the average from thousands of CONSUMERS. You. Me. People who buy the game and aren’t handed a review copy along with whatever quid pro quo BS that goes on.

        Reply
  2. Pingback: Metacritic and the Legitimacy of Video Game Journalism, Part II | Gregor Ulm

Leave a Reply

Your email address will not be published. Required fields are marked *

Spammer prevention; the answer is an integer: * Time limit is exhausted. Please reload CAPTCHA.

This site uses Akismet to reduce spam. Learn how your comment data is processed.