Reviews in the Media - Flawed and Inaccurate?

[Rayder]

I get the feeling that they don't play the whole game in a lot of cases. They write their review based on only a couple hours of playing. Like a review of their preview.

At any rate, I never read just ONE review for any game, I read many. I like to read the lowest ratings first, it helps temper me against the glowing "gamegasm" reviews. You start to see through the holes in the reviews as you go from low to high ratings. Some of the things mentioned in low reviews aren't mentioned in high reviews, and vice versa. By using the low to high review method, I am rarely disappointed with a game I actually buy. The diversity of the reviews gave me a much better idea of what I'm buying into than going from high to low.

But that's commercial reviews. If I really want to know if a game is my type of game or not is to read GAMER reviews, go to forums dedicated to that game and read-around the site, and don't forget to carefully read the "support" section.

 

[Ryukouki]

I get the feeling that they don't play the whole game in a lot of cases. They write their review based on only a couple hours of playing. Like a review of their preview.

At any rate, I never read just ONE review for any game, I read many. I like to read the lowest ratings first, it helps temper me against the glowing "gamegasm" reviews. You start to see through the holes in the reviews as you go from low to high ratings. Some of the things mentioned in low reviews aren't mentioned in high reviews, and vice versa. By using the low to high review method, I am rarely disappointed with a game I actually buy. The diversity of the reviews gave me a much better idea of what I'm buying into than going from high to low.

But that's commercial reviews. If I really want to know if a game is my type of game or not is to read GAMER reviews, go to forums dedicated to that game and read-around the site, and don't forget to carefully read the "support" section.

Wow, here's a voice I haven't seen in a long time, good to see you posting, Rayder! I definitely feel like reviewers don't get the entire game completed when they post it out, either. I think I do your low to high method, but I just don't have a name for it. But yeah, it works!

 

[DiscostewSM]

I think more money corruption ruined it, just look at the Kane & Lynch Gamespot controversy.

Oh wow....and to think, I thought people on other sites saying things like "IGN gave a Nintendo game a low score because Nintendo's check didn't clear" were just causing trouble when in fact it could very well be true.

 

[Mariko]

One instance of this that has burned into my mind is the GameSpot review of The Legend of Zelda: Skyward Sword. This game, popular among most sites, received a score of 7.5/10, which by the "standard," was appalling for a Zelda game. Fans reacted with fierce negativity against McShea, of course, but McShea came out later and ended up defending his point of view. This, to me, is absolutely horrible to look at, and with reviews like this out in the wild it is clear that something needs to be changed (...)

At this point I stopped reading. Why wouldn't it be ok to give a 7.5 score to a Zelda game? You state that's "by the standard that score is appalling". If that's true, we might as well have automated scripts in place that rate every Zelda game between 9 and 10.

You can't expect every single reviewer to score a game in the exact same way, even if their reviews aren't subjective. This doesn't change the fact that I myself am not a fan of scores, since it's far too difficult to accurately convert all of the game's aspects into a number. Reviews shouldn't have scores at all, but then again, the majority of gamers only cares about the scores, not the justifications behind them. You often hear people complaining about a score, but you rarely hear someone argument against a certain point brought up in a review. Then again, since almost everything is subjective, what's the point? One person might consider a given gameplay aspect to be the best thing ever, while another might be sick and tired of it, since they've played far too many games utilizing similar or identical gameplay features.

Reviews are losing their relevance mainly due to the fact that gaming community is changing. We want scores, and we want them now. That's why things like Metacritic exist. I'm not surprised video reviews took off like they did, though they're still combined with a score.

As for developers and publishers trying to influence review scores, it's nothing new, and the Kane & Lynch example isn't the only one. Publishers will often submit suggestions and guidelines on how their game should be scored. This has been brought up on several occasions, but happens far more often. It's a "I'll scratch your back, you'll scratch mine" type of deal. After all, both the developers and video game websites are trying to make money.

In the end, it all comes down to one's own judgement. I'm not going to miss out on a game simply because someone didn't like it. Games aren't that expensive these days, and if a game appears interesting enough, I want to check it out on my own. I've bought my share of games with bad reviews, and I don't regret it. Why? Because they weren't bad to me.

 

[grossaffe]

Another good topic. I think there are a number of problems when it comes to game reviews, especially with this past console generation. It seems to me that games that get hyped big time wind up getting amazing scores and reviews regardless of how much they disappoint. Somehow, GTA IV, for example, received a perfect 10/10 from IGN. Perhaps the most disappointing game in the series since it went 3D (I'm not counting handheld, as I don't know if they sucked) managed to get a perfect score and win game of the year, and it was destined to happen before the game ever came out because of the hype that surrounded it. AAA games are just expected to get 9s and 10s, so they do. Personally, I'd like to see average game scores drop... a lot. Whatever happened to the days when getting a 10/10 was nearly unheard of? Getting one 10/10 in a generation was an achievement, now you expect to see three of them a year. Have games really become that amazing? Gaming media seems driven too much by hype, and hype seems through the roof these days thus game scores are through the roof.

There's definitely the aspect of companies/reviewers getting paid off for reviews that you have to be mindful of, or punished for poor reviews, as evidenced by the Kane and Lynch fiasco.

Another issue I've seen is when a company has a reviewer doing a game that they will be pre-disposed to give a poor review. I'll cite IGN giving Greg "the super Sony fanboy" Miller the task of reviewing ZombiU, a Nintendo exclusive launch title for the Wii U. To nobody's surprise, the guy who slobbers over everything Sony does and bashes Nintendo gave the game a poor review (6.3, which by today's rating standards means you may as well take the disc out of the case and drop it directly into an incinerator) well below the average score the game received from other sites. Don't get me wrong, here, I'm not trying to profess that the game was a masterpiece, but rather that it was given a review by a self-confessed Sony fanboy who loves to bash Nintendo and thus should not be expected to deliver an impartial review.

On the topic of impartiality, you have the question of whether the reviewer is right for the game, and to me it's actually a pretty complicated subject. For example, a few months back after I had started trying to get myself into the JRPG genre, I started giving thoughts on my experiences with the games. I'm relatively new to the genre (I'd played a couple in the past like Pokemon, and Mario RPG when they were new, and went back and played Chrono Trigger as I remembered a good friend of mine being high on that game when I was younger), so it was important to me that I prefaced my "reviews" by letting people know that I was pretty green to the genre, so they would be following me on a journey into the genre, rather than getting the opinion of somebody who lives and breathes JRPG. I also had to make it clear in the beginning that I was not a fan of random encounters or grinding which were often staples of the genre. Some may say that because of that, someone like me would have no business reviewing JRPGs with my limited experience and my disdain for certain tropes that the genre has seen a lot of. But maybe instead, my outsider view will be more accessible to other greenhorns of the genre and I can help give them a place to start if they want to branch out. That being said, I don't think someone like me should be reviewing those types of games on a big review site because scores actually matter there, and the scores seen by that many people should really be given by someone who represents the majority of the audience who is likely to want to play the game rather than an outsider. I think an outsider type review is better off on an independent review site where people follow the reviewer and get to know how well they relate to the reviewer's opinions and thus have better idea of how a review translates to their own taste.

Overall, I think it's best to avoid big review sites if you want to get a legit review of a big game as they're all aboard the hype train. Maybe you can find a smaller independent review site with a reviewer with whom you identify and you know you can trust or at least you understand how his opinions would relate to your enjoyment of a game.

 

[aiat_gamer]

​

Games nowadays will receive on average an eight out of ten. Is the number eight considered the "new bad score?" It is rare to find anything in the media that goes lower than an eight unless the game is downright appalling.

​

I dont think I agree with that, at least the sites I check have a wide range of scores (GS for example). I kind of like the review score, it tells me at a glance if I should even bother with the game, so when the game scores 7+ I read the review and do some research to see if the game deserves my time and money.
But for me, it has become harder and harder to have a go-to source for reading review. It used to be that I trusted GS reviews completely and very rarely I was disappointed but ever since the main guy (Jeff Gretsmann, Vinny Caravella, Brad Shoemaker and Alex Navaro who I think is one the best the reviewers around) left GS, it just turned to crap. The reviews became a place for people to just force their political and social views on people , reviews became widely contradictory and other stuff.

I never believed the conspiracy that sometimes the review are bought, but unfortunately I have seen more than enough examples that are really main me believe that. The prime example that comes to my mind is this:
http://www.egmnow.com/articles/reviews/egm-review-aliens-colonial-marines/#/
Who in their right mind can rate that game a 9?!
And lately one incident let me to lose all my respect for GS after 11 years of being a loyal fan: The 6 score for Batman Arkham Origins and and later, the 8 for COD:Ghosts. For Batman review all they did was moan that the game is too much like the other entries in the series and how the lack of the innovation has led to the game becoming stale...fast forward to the COD review and no where in the review they complained about the repetitiveness of the game. or when it came to the graphics, in which the game is seriously lacking, in a desperate attempt to find something, anything positive to say about the game they came up with this:
""
Or IGN for example, they are basically Nintendo fanboys and rate Nintendo games, ALWAYS, higher than other sites. They went as far as rating the Legend of Zelda: Windwaker HD higher than the original game on GC! They gave Skyward Sword a perfect 10!
Overall it has become harder and harder for us gamers to find a outlet we can trust to make our game buying decisions which is very sad...no wonder many pirate their games first to try them out and then decide on buying them.

 

[reprep]

i completely stopped reading professional reviews for the reasons mentioned above. What i do is:

1) watch the gameplay videos on youtube (not the official trailers, or official gameplay blabla. real gameplay by amateur gamers)
2) read a few amateur game reviews. i usually read gamefaqs' user reviews and reading a few of them give me a fine idea.

 

[MarioFanatic64]

Are reviews flawed?

Short answer: Yes.
Long answer: Yes, reviews are flawed.

 

[T-hug]

The scoring is a huge problem and week after week we hear of big name developers bashing metacritic and even having their pay bonus based on how they average on metacritic after release. In some cases I've read of how games are developed just to check off review points.
Imo Kotaku have it right with the YES/NO review system, but even that is flawed when it comes to MP games at launch, which are often not working as they should day 1.

Then we have embargos which 99% of the time give the review to one huge site like IGN which will come a day early and never bash a game as they have the exclusive.
I also hate it when someone reviews a game and marks it down because they don't like the genre, like gamespots recent Batman Backgate review, the reviewer expected a port of the console game, that shouldn't even factor or be mentioned. Same with gametrailers review of Killzone Mercenary, they gave it a 7.7 but based that score on in being a FPS and comparable to console FPS. Imo it should be rated as a FPS on a handheld, because thats what it is!

I could talk about this all day but the only way things will change is if you don't go to the sites that are guilty of biased numeric review scores, which for a lot of people is the first port of call before buying a game.

 

[Mariko]

(...) the guy who slobbers over everything Sony does and bashes Nintendo gave the game a poor review (6.3, which by today's rating standards means you may as well take the disc out of the case and drop it directly into an incinerator)

That's another problem right there. If a score of 6.3, which by definition is above average on a 10 point scale, is considered too low to pick up a game, we might just as well not have scores at all, since people clearly have no idea how to operate within such scoring systems. If someone asks: "What do you think, is this game good, poor, playable?", and I answer with "it's not great, but definitely above the average", it effectively means it's a 6 ~ 7 point game. That's why expressing an opinion is much more effective than simply coming up with a number, especially today, when people aren't able to score games below 5 out of 10.

 

[emigre]

More on that for another day, however. What surprises me is that there are people who call themselves reviewers, yet they provide completely botched review score numbers and claim that a popular and hyped title is terrible. One instance of this that has burned into my mind is the GameSpot review of The Legend of Zelda: Skyward Sword. This game, popular among most sites, received a score of 7.5/10, which by the "standard," was appalling for a Zelda game. Fans reacted with fierce negativity against McShea, of course, but McShea came out later and ended up defending his point of view. This, to me, is absolutely horrible to look at, and with reviews like this out in the wild it is clear that something needs to be changed, but what can be changed in the current system without radicalizing the entire system?​

The Gamepot Skyward Sword debacle was just an utterly pathetic and embarrassing response from the fanbase. There's a few threads on this very forum which were just hilarious as fans were in uproar.

 

[Issac]

I voted "others", because I actually feel that subjectivity is a good thing if done right. If someone reviews a game and personally doesn't like it, and comes with good explanations of WHY he doesn't like it; that's fine! Maybe I feel that his views are in the same vein as mine. Maybe I feel, "Hey, I actually like those things he dislikes, so maybe this game is for me anyway". Subjectivity isn't bad in itself.

However, in a review of The Last of Us (Some high scoring game) in a Swedish newspaper, the reviewer went full retard. The game got a 2/5 (a really bad) score, and the review only talked about how bad the game was, because it was sexist, because the main character wasn't female, that the female was just a side character. sexism sexism sexism! men are pigs! the men are portrayed as strong fighters, the women are portrayed as weak. sexism sexism bad game!

THAT is a bad review.

EDIT:

The Last of Us got 2/5 because it wasn't The Walking Dead.

 

[Dragonlord]

Reviews are a joke. If reviewers don't give too high scores for crappy AAA games they don't get the early-review-releases any more so they praise garbage to keep this priviledge. I've not seen anymore since years an honest AAA game review which gives a mediocre or crappy games (and these are the norm since years now) the score it deserves. On the other hand if somebody gives a game the crappy score it deserves (as I do) then fanboys are send to down-rate them. Just a huge amount of joke and people too stupid to actually value "real" good games (granted it's a problem if you get next to no good games since years).

 

[DinohScene]

Reviewers getting paid to review is pretty old news.
What saddens me is that they also review games and give it a bad score cause it doesn't have Multiplayer.
Some reviewer site did that to Bioshock Infinite.

I always say: "The best reviewer, is you"
COD is getting high reviews but I personally hate it.
Hello Kitty for the 3DS isn't even reviewed, yet I'm pretty sure I'd love it.
And Sheep for the GBA gets a 6.5 whilst I personally love it (also it's one of me fav games all time)

I never cared about review scores at all.
Early on I knew that to like a game, you should play it yourself and not play it based on what some wannabe gamer aka a reviewer gives it.

 

[Dork]

There is no video game journalism, it's video game tabloids.

 

[w!!]

I can see the bias of money being an issue...but business is business. I think that we have to remember that most reviewers have a tough job of reviewing a game with the most general gamer in mind. It is a fairly subjective issue, especially when a reviewer doesn't like a franchise or genre.

I prefer to ignore the number score and actually read what the reviewer wrote. A lot more truth comes out there than in a simple number. But I agree with an earlier post, the best way to get a good honest review is to read "user comments" and make an educated decision by watching videos of the game play. I think all reviews should have comments enabled and that would ensure that overly good or bad reviews are called out by the community. In the end, at least here in the US, renting it for a buck or two before buying is a great way to see if you personally like it.

 

[Zeliga]

Sometimes the review scores suck, I swear. Did they ever play some GOOD games!

 

[zachtheninja]

-Societal pressure from the community

I've seen this before. Reviewers afraid to criticize a massively popular game because of the community. I think this happens more than "bribed" reviews.

 

[Hop2089]

I've seen this before. Reviewers afraid to criticize a massively popular game because of the community. I think this happens more than "bribed" reviews.

It does, look how reviewers were treated when GTA V was reviewed and the reviewers didn't give it a 10. Bribed reviews are way more common in Japan than the US which is why I don't like Famitsu reviews.

 

[Gahars]

Something else worth mentioning is the merit of reviewers. I mean, just compare reviewers for film and games. Film had people like Siskel and Ebert (whatever you may think of his opinion on other subjects). These were guys deeply versed in the art of filmmaking, with a passion for the art. They were able to make insightful analyses of films while keeping things accessible for a general audience; everyone, from the average joe to fellow film buffs, could appreciate what they had to say.

Gaming, though? Not even close. There's dirty money and tricks getting passed around, sure, but I think it's more than that. I feel like most game reviewers aren't really versed in the language of games the same way folks like Siskel and Ebert were versed in the language of cinema. These reviewers only scratch the surface and stop there. I also think we're being stunted by a weird, industry-wide inferiority complex. It feels like a lot of game reviewers are insecure about their profession, and so they try to justify gaming as a medium at every turn. Bioshock: Infinite? "It's the Citizen Kane of gaming, guys, you can take us seriously now!" Gone Home? "10/10, see, we're topical, really!" etc. etc.

You know game reviewing is downright farcical when Yahtzee is probably one of the best out there, and his "reviews" are made for comedy.

(Now, don't get me wrong - I know film reviewing has plenty of hacks all its own. Believe me, it does. Still, they seem to be quite the majority when it comes to game reviews.)

Also: Video Game "Journalism"