Reviews in the Media - Flawed and Inaccurate?

Discussion in 'General Gaming Discussion' started by Ryukouki, Nov 9, 2013.

?

What aspect(s) of media reviewing are contributing to the decline of game reviews?

  1. The review score

    25 vote(s)
    32.1%
  2. The subjective value behind the score

    24 vote(s)
    30.8%
  3. Media biases

    55 vote(s)
    70.5%
  4. Inexperience in reviewing a specific genre

    31 vote(s)
    39.7%
  5. Other (please specify)

    13 vote(s)
    16.7%
Multiple votes are allowed.
  1. Ryukouki
    OP

    Ryukouki See you later, guys.

    Member
    4,949
    2,614
    Jan 31, 2008
    United States
    [​IMG]
    As a video gaming site, we have all seen reviews in the media trying to talk about the newest video game releases. This also applies to other mediums, such as films or books. As GBAtemp is a website focused on video gaming, I will stick with the video gaming field of media reviews. In terms of reviews for video games, several groups of reviewers pop up. You can get the honest, straight shooting reviewer, you can get the reviewer who may have liked the game but will purposefully botch the score for the website views, or you can get the reviewer who gives too high of a score when it was not warranted, leading some to believe that the reviewer was bribed. The reviewing industry is filled with 8/10 or 9/10 type scores, so my question is this: are these numbers still relevant? Is there an unofficial guideline that gamers should follow when choosing their games? Are media reviews themselves still relevant?​
    There is a lot of distrust towards media reviewers right now. It is a bit of a shame, although I only say this as I have been in these shoes for the past several years. More on that for another day, however. What surprises me is that there are people who call themselves reviewers, yet they provide completely botched review score numbers and claim that a popular and hyped title is terrible. One instance of this that has burned into my mind is the GameSpot review of The Legend of Zelda: Skyward Sword. This game, popular among most sites, received a score of 7.5/10, which by the "standard," was appalling for a Zelda game. Fans reacted with fierce negativity against McShea, of course, but McShea came out later and ended up defending his point of view. This, to me, is absolutely horrible to look at, and with reviews like this out in the wild it is clear that something needs to be changed, but what can be changed in the current system without radicalizing the entire system?​
    One aspect of reviews that even has me questioning myself is the score number. I absolutely hate placing scores on reviews as the number is purely subjective, and does not have significant meaning to the overall context. Anyone that sees my recent reviews not posted in Review Center can see that I will include a "Should You Buy It" type of section. I do this to prevent the backlash that comes with placing a score. All in all, the number ends up fettering my own voice and leaves me to try and wrap my voice around the numbers, which on occasion will not match. Could it be a better solution to simply have a section of text that asks the user whether the game is good, based on presented evidence?​
    The review score nicely opens the next issue, which is the number itself. What do these numbers even mean nowadays? What is the new standard? Games nowadays will receive on average an eight out of ten. Is the number eight considered the "new bad score?" It is rare to find anything in the media that goes lower than an eight unless the game is downright appalling. In regards to the number, what could it mean? Say, for instance, I scored a game at 9/10. Does that 9 compare against anything? Does it compare itself to another game in the same console? Does it work with the same franchise in mind? Does it simply exist to cater to its audience? These questions are constantly plaguing me whenever I place my own reviews out. With games becoming more like cinematic movies with flashy effects, is there a new standard of scoring that needs to be adopted? Should games even be graded nowadays with the current framework of story, music, gameplay, replayability, and functionality? Where does the line need to be drawn? What could be something new that can revamp media reviews and bring back the trust?​
    Another issue is media bias. Media bias is present in some of these reviews, and it is evident when reading the meat of the review. In the media you can get a fantastic indie game that brings innovation out to the next level receiving a lousy score, and then you can get a game that is a hot seller time after time, completely losing its innovation, getting perfect scores by media sites and being declared innovative. A perfect example of this is the Kane and Lynch controversy, information courtesy of The Catboy. Is this reviewer being pressured by the developer to put out a false score? This could be possible, I have seen it happen myself. Most often, these threats come with the fact that should the reviewer place out a bad score, they will never receive review units or copies again. This paradox is something I would love to hear feedback from. Is it better to bow to the demands of a developer and place out a score that you do not agree with, so long as you receive your just reward? Or is it better to take the risk, going against the masses and placing out a review that is truthful? What could be done to remove these biases in writing? Could it be that the writer simply does not have experience reviewing games of a specific subtype?​
    [​IMG]
    I find it interesting to find critics balking at RPGs being too grind-heavy...
    Sometimes, in media reviews, reviewers are often inexperienced with writing about the specific game they are provided. A broad instance of this could be in the case of a reviewer who specializes in shooting games reviewing a role-playing game. In these cases, the role-playing game is criticized harshly on its fundamental elements, elements which clearly define the genre. I have seen reviews criticize role-playing games for being too heavy in terms of grinding. I find that instead of complaining about a fundamental aspect, could the reviewer establish a possible middle ground to present to the developers for future reference?
    So, there we have it. If you have followed me up until this point, that is awesome, and I am very thankful for that. If you are just tuning into the article, the basic outline is as follows: With video gaming reviews in the media - could they be changed? Some of the fundamental problems include scoring, the subjective meaning of the number itself, media biases, and inexperience within reviewing games of a specific genre. I also mentioned the paradox of bowing to the demands of the developer or being truthful about the product. I would love to hear feedback as to whether or not my points are valid, or whether I am missing something. Where could the system be improved, and better yet, what can be kept the same? I would love to be able to expand my own horizons as a reviewer of sorts to see how I can incorporate better tactics into my own articles in the future!​
     
    MegaBassBX and T-hug like this.


  2. Lilith Valentine

    Lilith Valentine GBATemp's Wolfdog™ Spooky cute.

    Member
    20,084
    21,225
    Sep 13, 2009
    Antarctica
    Between insane and insecure
    I think more money corruption ruined it, just look at the Kane & Lynch Gamespot controversy.
    Which pretty much leans on the bias issue, since once a game company starts paying for ads on a reviewing site, it's almost expected that the game will get a high review, regardless of the content. Mixed with the fact that no one seems to focus on the reviews anymore, looking at game covers nowadays it's just plastered with the game stores and maybe like, "Wow, such game, so amazing!" but no real content, like they do with movies.
     
  3. Ryukouki
    OP

    Ryukouki See you later, guys.

    Member
    4,949
    2,614
    Jan 31, 2008
    United States

    Oh yikes, I never even read about the Kane and Lynch incident. I think I heard of it in an off hand conversation, but DAMN. That's interesting to see.

    EDIT: Wait, what someone got FIRED from a poor review score? Wow, that's interesting and surprising, to say the least. :O I hope to God that I never end up in a space like that. That's awful.
     
  4. frogboy

    frogboy lacking both style and grace

    Member
    2,378
    1,414
    Dec 6, 2011
    United States
    You always seem to write the most interesting articles.
     
    T Link7, Shicky256, Xexyz and 2 others like this.
  5. Ryukouki
    OP

    Ryukouki See you later, guys.

    Member
    4,949
    2,614
    Jan 31, 2008
    United States

    Thank you! You know, just a typical Friday night on the keyboard.... ;)

    EDIT: Also, more seriously, you get a bunch of gamers who take things for granted. When I write these articles, I want to open a debate as wide as possible to make gamers take a second to rethink about an issue. It's also compliments like this that fuel my passion to continue writing, and for that I am very grateful. :)
     
    gamefan5 likes this.
  6. Lilith Valentine

    Lilith Valentine GBATemp's Wolfdog™ Spooky cute.

    Member
    20,084
    21,225
    Sep 13, 2009
    Antarctica
    Between insane and insecure
    This the reason I don't trust most big gaming reviewing sites like Gamespot and IGN because I can't trust that their reviews are actually legit anymore. Once game sites start plastering ads for upcoming games, it means they already saw money for the game and the review, thus meaning they already show a bias for this game. It also means they can't afford to give the game a bad review because they could either lose that sponsor or end up like what happened on Gamespot.
     
  7. Ryukouki
    OP

    Ryukouki See you later, guys.

    Member
    4,949
    2,614
    Jan 31, 2008
    United States

    That is definitely a very good point. Personally, when I am offered review samples that come with an additional payout, I will never accept it, and if I do choose to for whatever reason, I state very firmly in the review and to the developing party that my opinion was not swayed by the offered bonus cash. I think it's a disgusting practice to take in money, only to have the developer use that as a guillotine to garner positive praise. It definitely takes away from the "authenticity" of the review in question if money has been offered.
     
  8. Lilith Valentine

    Lilith Valentine GBATemp's Wolfdog™ Spooky cute.

    Member
    20,084
    21,225
    Sep 13, 2009
    Antarctica
    Between insane and insecure
    It's really one thing to get paid for doing reviews as a job, like Ben "Yahtzee" Croshaw does, it's another when someone is paid for the review. Although I glad to hear you don't allow such bias to control your reviews. ^_^
     
  9. Ryukouki
    OP

    Ryukouki See you later, guys.

    Member
    4,949
    2,614
    Jan 31, 2008
    United States

    I've had my arms twisted by many a Chinese dealer, and the result is always the same. The usual empty threats of "We won't send you review units anymore," only for me to reply coldly, "that's fine, I can always find another." Which is irksome because it isn't even the fault of the reseller, as it is more of a manufacturing issue. :angry: I'm absolutely against being paid for a review. I always tell my client that because it completely goes against the principles of honesty that I strive to uphold. That's kind of what a review is supposed to do, present a picture of sorts in an unbiased manner so that people can make decisions.
     
  10. elunesgrace

    elunesgrace GBAtemp Regular

    Member
    282
    62
    Dec 21, 2011
    Canada
    Good thread.

    I believe we can say there are minor issues and fundamental issues.

    Minor issues are things that I would call differences of opinion on the matter. These would include:

    -Using a number to score a review
    -Personality bias of the reviewer
    -Subjectivity of the scoring
    -etc

    I call these minor because people will differ on this issue, and there are different sides to the discussion. Some might say we should have multiple reviewers, others will say one is enough and makes more sense to avoid info overload, etc.

    What we should be concerned with are fundamental issues that invalidate the whole review process itself. These would be like:
    -Monetary bribes
    -External pressure from big companies
    -Societal pressure from the community
    -etc

    These break the review because it defeats the purpose of 'reviewing a game.' What value is a review if it simply gives people what they already want (as if to validate their purchase) or if it is in a conflict of interest due to money or pressure.

    Unfortunately this is the cycle of what happens:

    Stage 1: Sincere gamer starts reviewing game
    Stage 2: People like authenticity so he gets popular
    Stage 3: Starts getting influential so publishers give him money and free games to review
    Stage 4: He start turning his reviews into a money-maker
    Stage 5: Tries an honest review and gets pressure from publishers if he negatively reviews them
    Stage 6: Has to make a decision, usually chooses to sells out his review to keep his money-maker.

    What's truly sad is that I think some of the people who make an outcry about this, if they were in the same position would probably sell-out as well. This shows a problem with society, and how they view money vs integrity. My aunt (PhD in immunology) used to tell me that when I go to the doctor not to trust the young ambitious doctor and instead trust the old doctor. Because the young guy probably just wants to make money, and the old guy likely has already fulfilled his ambitions and just wants to help people.

    Anyways there you go!
     
    MegaBassBX and Ryukouki like this.
  11. Ryukouki
    OP

    Ryukouki See you later, guys.

    Member
    4,949
    2,614
    Jan 31, 2008
    United States

    Those are really good points, especially the fundamental issue division portion of your post. Thats kind of the driving force I was hoping to drive at here. ;) And I totally agree with the doctor portion. The young ones are a mixed bag.
     
  12. Transdude1996

    Transdude1996 GBAtemp Regular

    Member
    208
    62
    Dec 28, 2011
    United States
    Colorado Springs, CO
    Sorry I didn't read through your opening post TC, I just saw a wall of text and my TL;DR function just turned on. So, sorry if I address some things you already stated.

    IMO, it seems like reviews have become very skewed during the 7th Gen consoles and it looks like it may get even worse. Take a look at Jimquisition's Hate Out Of Ten and Dragon's Frown videos, and look at the backlash the Escapist and GameSpot recieved after not giving GTA V a 10/10. That's how bad things have gotten. And with the fact that a 5 year old can now access the internet, things are only going to get worse.

    One of the modern problems I've seen happening these days is that nearly every reveiwer treats the rating scale like their still in school (anything below 60% is a fail, see Rooster Teeth's Game On PSA, and I know it's a parody). The problem with this is that, if a game receives a score lower than about 70%, it's considered a failure (E.G. Sonic games delisted if they have an average Metacritic score). The problem is that with the increase in this mindset is that people pass-over a lot of good games just because of their score. And the are a lot of good games out there that receive a 5 or a 4 as a rating. Personally, my solution to the problem was to create my own rating scale (10-8: the game is worth owning, 7-5: the game is worth looking into, 4-2: You can find better games but at least give it a try sometime, 1-0: Don't even bother).

    As for the solution to gaming bias, I just try to find reviewers who I consider to be unbiased, or, at least, answer questions I want to ask about the game. Just saying, but I think that the most unbiased reviewer on the internet of CGR because all he really does is answer the simple question, "Is this game fun?". Even though I trust him a lot, I'm also subscribed to some other reviewers as well because, not only do I want a second opinion, but I also want to hear what are some problems with the game.
     
  13. Ryukouki
    OP

    Ryukouki See you later, guys.

    Member
    4,949
    2,614
    Jan 31, 2008
    United States

    Heh, those are fair points. You hit a lot of repeats, but that really drove my point home! If you want the real meat, I did a TL;DR sorta thing in the last paragraph. I'm not obviously going to write, DURR THIS IS THE TLDR part, but hey, it's worded to know that the meat of the article can be quickly summarized. :D Glad for the feedback either way. Most of my articles will appear to be text walls at times, but I try to make the content engaging to varying degrees of success. Your points solidify my argument, though, so kudos on that! :) If you get the chance though, take a minute or fifty and give it a quick read! It may be fun!
     
  14. Transdude1996

    Transdude1996 GBAtemp Regular

    Member
    208
    62
    Dec 28, 2011
    United States
    Colorado Springs, CO
    Thanks for the reply, did a quick scan, and I'd like to talk about two points I didn't mention in my previous post, or see anyone else talk about at the moment.

    The first is the inexperience in reviewing a specific genre. Technically, this door can swing both way when it comes to reviewers. The reason it can go both ways is because a reviewer my be tasked with reviewing a game for a genre that they haven't tried before, and they can either play the game and grow to hate it, or play it and fall in love with it within minutes. This aspect comes down to nothing but personal preference, which is why I like listening to some specific reviewers because I know their fans of a game genre, but, at the same time, they're also willing to try something new.

    Though, inexperience in a specific genre is really bad when it comes to making professional reviews, which brings about my second point. Professional reviewers don't actually have a lot of time to really enjoy a game. They're basically tasked with getting a game, playing through it within 1 or 2 days, giving it a score and moving on. This is the main reason why games like Tomb Raider generally receive a high score than games like The Wonderful 101. Because reviewers are on the clock, they don't really have the time to learn how a game really works, they have to pick, beat it, and move on.

    EDIT: One more point I just realized I didn't touch on. The whole idea on using reviews as the almighty Bible to whether a game was good or not. I believe I covered some of this in my previous points, but this will go into more detail. It seems like whenever someone sees a game receiving X/10, they immediately think that the game is that score despite what everyone else says. But, what people easily miss is that a review is the authors opinion on the game. Since I don't really want to type anymore, I think I'll just let The Big Picture take this one away (if I linked the right video that is).

    EDITEDIT: Yep, link the right video.
     
    Ryukouki likes this.
  15. Ryukouki
    OP

    Ryukouki See you later, guys.

    Member
    4,949
    2,614
    Jan 31, 2008
    United States

    Ahh, I see where you're coming from here. Very interesting points, by the way. And reviewers being on the clock is a great point to add, something that I should probably consider adding to strengthen the overall argument here. ;) I think of reviews as guidelines of sorts, I'm not gonna follow it religiously, but I'm going to definitely use some of them as a source of, say, whether or not something was executed well. A review may be an opinion, but we never let it appear so in the review. If we do, we make it really sly. :D
     
  16. Gahars

    Gahars Bakayaro Banzai

    Member
    10,254
    17,403
    Aug 5, 2011
    United States
    New Jersey
    11/10 - "It's okay, I guess."

    I can't remember where exactly I found it, but there was an article discussing some of the problems inherent in video game "journalism" (there are no quotation marks big enough). One of the biggest problems is that there really aren't any video game "journalists" with actual degrees in journalism. The people with a passion for the field and a deep knowledge of integrity, accountability, etc. avoid the games industry like the plague. When you think about it, this makes a depressing amount of sense.

    Journalism has produced men like Woodward and Bernstein. Gaming has the Doritos Pope.

    Also, glad that I have an excuse to post this...

    Warning: Spoilers inside!
     
  17. Ryukouki
    OP

    Ryukouki See you later, guys.

    Member
    4,949
    2,614
    Jan 31, 2008
    United States

    Oh man, link me if you find it. I'd love to give it a read. :) And yeah, it makes perfect sense. ._.
     
  18. Taleweaver

    Taleweaver Storywriter

    Member
    5,606
    1,644
    Dec 23, 2009
    Belgium
    Belgium
    Let's see...I voted for Media biases, inexperience in reviewing a specific genre, and the subjective value behind the score. In that order.

    Media biases are by far the worst, and I'm glad it's leading this poll. With good reasons: gaming magazines get sponsored by game publishers, so it's just stupid to expect anything other than a bias towards these games. And since there are no standards for objectivity and a lot of things pretty much HAVE to be opinion, it's hard to catch anyone on a "wrong review" (the defense of 'what do you mean, wrong? It's my opinion, damnit!' always works). So even without controversies...do you really trust those guys? It's like having fox news news anchors telling you who to vote for. They're paid to say that...so they do. And of course they believe what they say: they were hired because of their bias towards these games in the first place.

    The inexperience in reviewing a specific genre is a tough one. How do you review a game that isn't the first in its genre? Do you grade it based on their predecessors or by itself? And how many comparable games do you need to have played?
    IMHO, this is something where game magazines and sites need to differentiate. Games like AC3 should have a wildly different score depending if it is your first game or whether you've experienced the previous ones (read: if it's totally different than the previous ones, it's stupid to expect everyone to like it the same). But rather than sites catering to the newcomers in the series or even genre, everyone seems to want to cover every ground.

    And the subjective value...yeah. But we've got to be honest: all readers expect it. Of course you can't boil thousands of man-hours, gameplay, visuals, innovation, the entire experience and all that down to a single number. You can't compare an RTS to a FPS, but we want it nonetheless. It's a flaw that everyone endorses. Flawed as it may be...THAT will never change.

    I know Angry Joe did that at one point in his reviews. First he talked a bit (with arguments) on why the game was mediocre and bland, and then showed some review quotes (who were probably on the box as well). He couldn't figure out why those other sites did that, so he slapped this quote on it himself.
     
  19. Another World

    Another World Emulate the Planet!

    Former Staff
    10,499
    1,673
    Jan 3, 2008
    Colombia
    From Where???
    it seems like these days bias in the media is just a tool to stir the pot. i always picture them picking their angle, good or bad, and then driving home those points with purpose. its really rare to find a reviewer who will hit on both points and then sum it up by leaving it open to the reader to make up their own mind. sadly, those type of reviews don't sell the product. there is also an inherent belief that everything must be force fed to the reader, something that causes even more problems.

    i've experienced some of these issues first hand. i had one of my review scores changed by an editor due to their fear for the backlash that would follow. i've also been flat out told by one specific manufacturer that my honest review ruined their company and all chances of future sales. i had to go on the defensive in both instances to stand up for what i believe in, and what i wrote. this should never be the case. you shouldn't change the score of a review or the facts about a product. that bias kills your integrity and just leads consumes down the wrong path.

    -another world
     
  20. Puppy_Washer

    Puppy_Washer GBAtemp Advanced Fan

    Member
    920
    657
    Jul 18, 2009
    Sydney
    Warning: Spoilers inside!
    What was stopping them from successfully talking about the newest video game releases?

    Warning: Spoilers inside!
    What defines the "honest, straight shooting reviewer", the "reviewer who may have liked the game but will purposefully botch the score for the website views", and the "reviewer who gives too high of a score when it was not warranted"? Aren't reviews simply opinions? Couldn't the honest, straight shooting reviewer deliver a review that appears to have the "traits" of the other reviewers you listed? I wouldn't categorize reviewers like this, there are many, many shades and variations of reviewers.

    Warning: Spoilers inside!
    What needs to be changed? Shit, a reviewer can give a game a 7.5 if they want. Props to them if they back it up with further explanation too. Reviews are opinions. And of course there will be distrust against reviewers. Every single review will piss off a certain demographic. High score? You'll piss off the haters. Low score? You'll piss off the fans. Reviews, by nature, will constantly piss off every single type of gamer. Reviewers will never be loved.

    Warning: Spoilers inside!
    The numerical system works for a lot of reviewers, and for a lot of readers of reviews. It's a quick summary of the overall quality of the game. Granted, it has flaws, but it's widely adopted for a reason. Nothing is stopping other reviewers from adopting different summary techniques though.

    Warning: Spoilers inside!
    Varies from website to website. There is no standard. You have to read a website/author a few times to get a feel for their grading logic.

    Warning: Spoilers inside!
    Correct me if I'm wrong, but Gerstmanngate was less about Eidos pressuring Gamespot for a particular score, but rather about Eidos having the power (due to their advertising contract with Gamespot) to coerce Gamespot into firing their editorial director over a bad review. I do agree, however, that there is a growing issue with blacklisting websites from review copies due to scores. Also, review embargoes are an additional issue.

    Warning: Spoilers inside!
    Grinding is not a fundamental aspect of RPG games, although it is a common element to them. There is such thing as too much grinding, or dull grinding. Criticism of this is not indicative that the reviewer is inexperienced with RPGs.

    Warning: Spoilers inside!
    I don't think there's any realistic "change" you could make to the industry. If you think it's flawed, then you'll need to lead by example, as the industry is not a single entity that can be altered. Perhaps you could even ask the GBATemp community what style of reviewing they enjoy most? Hell, how about writing multiple short reviews for a game (with different approaches), and asking people for feedback? That could be fun.