Reviewers and objectivity

Discussion in 'General Off-Topic Chat' started by cenotaph, Nov 26, 2006.

Nov 26, 2006

Reviewers and objectivity by cenotaph at 4:52 AM (1,834 Views / 0 Likes) 22 replies

  1. cenotaph
    OP

    Member cenotaph GBAtemp Fan

    Joined:
    Oct 24, 2006
    Messages:
    341
    Country:
    United States
    Please note that most of this post is speculation, before going ballistic about me making "baseless accusations".

    I figure most of you read reviews of games you're going to buy on major gaming sites (and in printed papers (?)) before actually making purchases. I'd also say reviews in general exercise at least some influence in what games end up being commercial successes as they make it easy for the customer to avoid buying duds and get the most out of the capital put up. As a result, corporations strive to please reviewers by developing products in accordance with their preferences. After all, as regular customers, shouldn't they reflect what the general customer would want in a certain product? And even though it might cost developers more to make products conform, wouldn't it pay for itself in the end as favorable reviews provide boosts in sales?

    Though the answer to these questions may seem obvious, I'd say there is a backside to this; when reviews become mere tools in marketing and publicity. The outfits the reviewers write for (and this is especially true for Internet sites) usually depend on advertising to make ends meet, and as a result, they'd naturally want a good deal of visitors. The easiest way of accomplishing this within this certain field (i.e. gaming) would be to provide the most reviews of new games as those are what a large portion of the consumer base is considering to buy and wants advice on. Now, step back for a moment and think about how reviewing outfits get ahold of products to review. That's right, they're supplied by the vendors, which is all fair and good, right? After all, why should reviewers have to pay for the products if their reviews end up providing hefty profits for the manufacturers?

    I'd say there is a glaring error in this as it creates a conflict of interest.The outfits end up being dependent on vendors supplying them new material to review in order to attract visitors and get advertising revenue. And as we all know, all humans are susceptible to greed. By favoring products from certain vendor, reviewing outfits will end up getting favored by the vendor in question (as in being offered exclusive previews and other things generating high volumes of visitors and, in turn, more money) and vendors will gain customers -- a win-win situation. Why be satisfied with less when there's more?

    Why be objective when there's more personal profit in not being so?

    I'd very much like some input on this or on subjectivity in general concerning reviewers (and in particular game reviewers).

    ... lastly, I'd like to excuse any grammatical error(s) I might've made while typing this up. By the time I'm done, I usually end up detesting my writing, which results in it not being checked (I s'pose it has something to do with my non-existent self-confidence, but that has nothing to do with this).
     


  2. Opium

    Former Staff Opium PogoShell it to me ™

    Joined:
    Dec 22, 2002
    Messages:
    8,185
    Location:
    Australia
    Country:
    Australia
    What's the point you're trying to make?

    You learn to avoid game websites and journalists that put commercial interests ahead of content.

    Of course it goes on, but if you use your own judgemental mind I'm sure everything will be fine [​IMG]
     
  3. cenotaph
    OP

    Member cenotaph GBAtemp Fan

    Joined:
    Oct 24, 2006
    Messages:
    341
    Country:
    United States
    The point was that reviewers declaring themselves (or at least which are supposed to be) independent and objective are most likely far from it. From what I've managed to gather by reading these boards, people generally seem to pay a lot of attention to the scores products get, so I was interested in knowing what thoughts they'd have on the conflict of interest detailed above.
     
  4. Veho

    Global Moderator Veho The man who cried "Ni".

    Joined:
    Apr 4, 2006
    Messages:
    8,447
    Location:
    Zagreb
    Country:
    Croatia
    Here's a solution: every release should have a poll to rate that release. 47 thousand forum members can't help but be objective.

    As for games reviewers, they try to be objective, and I believe most of them think they are, but saying things like "this game offers nothing new" (-30% from the end score), or "not a major breakthrough in graphics" (-10% from the end score), and let's not forget, reviewers were once gamers, and had a preferred genre, and still tend to give games of that genre higher ratings (+10% end score), well, that makes reviews sliiightly biased.
     
  5. .TakaM

    Member .TakaM .II

    Joined:
    Oct 8, 2004
    Messages:
    3,525
    Country:
    New Zealand
    I enjoy GBAtemp's reviews of flash carts and hardware, because theyre always informative and interesting.

    but when anyone reviews a game, I almost always cant stand it, absolutely every aspect comes down to personal opinion and most reviewers are complete and utter retards
     
  6. Hitto

    Member Hitto MKDS Tournament Winner

    Joined:
    Nov 29, 2005
    Messages:
    1,450
    Location:
    Nice, France
    Country:
    France
    You wanna know if a review in a gaming rag is biased? You wanna know if ANY newspaper is biased?

    See any advertisements in it? Bingo, then it's biased.
     
  7. Veho

    Global Moderator Veho The man who cried "Ni".

    Joined:
    Apr 4, 2006
    Messages:
    8,447
    Location:
    Zagreb
    Country:
    Croatia
    LOL [​IMG]

    I read a laptop computer parallel test in a computer magazine recently, and get this:

    Laptop number one:

    1.4 GHz
    512 MB RAM
    60 GB HDD 5600 rpm
    4*USB
    DVD/RW, DVD writing 4X

    $1333

    Laptop number two

    1.2 GHz
    512 MB RAM
    60 GB HDD 4200 rpm
    2*USB
    DVD/RW, DVD writing 2X

    $2100

    Everything else identical.
    Guess which one got the "Best Buy" title? Guess! Guess!
    Yeeees! The second one!

    Naturally, I was like [​IMG] ...

    Untill I saw the light:
    Manufacturer of Laptop Number Two had 4 pages of ads in that month's issue...
    Manufacturer 1 had none.

    [​IMG]

    Review objectivity is a myth.
     
  8. Lily

    Former Staff Lily One Scary Lady

    Joined:
    Jun 18, 2004
    Messages:
    5,727
    Location:
    British Columbia
    Country:
    Canada
    Objectivity always goes out the window, even in places that start off objective. Needing advertising to finance the costs of production, down to wanting to continue receiving complimentary review copies..

    The only way you'll find objectivity is from a place that has no advertising, and that purchases all their own material.
     
  9. Veho

    Global Moderator Veho The man who cried "Ni".

    Joined:
    Apr 4, 2006
    Messages:
    8,447
    Location:
    Zagreb
    Country:
    Croatia
    ...or plays ROMs and ISOs and therefore doesn't depend on any financial support in a "you scratch my back" fashion [​IMG] [​IMG]
     
  10. Kamgusta

    Newcomer Kamgusta Advanced Member

    Joined:
    Aug 11, 2006
    Messages:
    58
    Location:
    Guess it
    Country:
    Italy
    You get objectivity by reading A LOT of reviews from different people.
    Then you make your own idea.

    Even places that have no advertising could be not-objective: never heard of fanboys?
     
  11. enoughrope

    Member enoughrope GBAtemp Regular

    Joined:
    May 22, 2006
    Messages:
    108
    Country:
    Japan
    Well I suppose those sort of thing is possible, but I'd figure the review sites has to maintain credibility... or they'll no longer be deemed credible and they'll lose the visitors that they need.
     
  12. Tamyu

    Member Tamyu GBAtemp Regular

    Joined:
    Nov 11, 2006
    Messages:
    223
    Location:
    Just outside Nagoya, Japan
    Country:
    Japan
    I have to agree with this. I did magazine stuff for a while, and let me assure you, readership is FAR more important than advertisers.
    Advertisers pay to, well, advertise in a magazine so that people will see the ads. They accept the risks that they may receive a poor review. Seriously, it`s the way things work.

    Here -
    Imagine that a magazine is totally biased, only printing what the advertisers *want* them to say. It would be pretty obvious if the reviews were overly skewed, and readers would quickly lose faith in the magazine. That would be the LAST thing an advertiser would want because it would cause a large drop in readership, which would make their investment in ads totally worthless. They`d stop advertising in that magazine, and the magazine would die out.

    How things usually work is:

    An advertiser sends a copy for review along with a list of terms they feel represents their product, and a list of words that they do not want associated with their product. Neither of these lists are set in stone, but as magazines do want to please the advertisers to an extent they are usually followed. The magazine will do a totally honest review, which is then edited and shaped to fit within the limits - which is why instead of a blatant "terrible", you`ll see less damaging terms like "It had such amazing potential" etc. You just have to know how to read the reviews in order to receive the true information. If it is bad, the magazine will not say it`s good. They`ll just be more gentle with their words by following the "Words we don`t want you to use" list. But as there are a hundred ways to say the same thing, they can still manage to be relatively accurate.

    The same goes for a 1 to 10 scale. A 1 or 2 would leave no hope, and there is always the chance that SOMEONE out there might love the game... So you`ll almost never see anything below a 3 or 4. The same goes for a perfect score - There will always be a reader that feels a game is TOTAL trash, so a perfect 10 is a pretty rare event even in a wonderful game.
     
  13. cruddybuddy

    Banned cruddybuddy Group: Banned!

    Joined:
    Aug 27, 2003
    Messages:
    2,864
    Location:
    California
    Country:
    United States
    Advertising is irrelevant. The source of a review (i.e. magazine, newspaper, website) is irrelevant. Individuality is irrelevant. You will be assim- … no, wait. Strike that last part. The point is that everyone is biased. Everyone. You? Yes. Me? Yes. GameSpot, NY Times, People, National Geographic, CNN, ign.com, Playboy, Stephen King, EVERYONE. It’s fine. It’s okay. We’re all biased! And to make things even more interesting, when reviewing videogames, everyone has different tastes. You don’t like the rating Gamespot.com gave to Zelda: TP? Big whoop. Don’t get mad. That is like getting mad because Baskin-Robbins gave a bad review on chocolate-mint ice cream. Everyone has different likes and dislikes. Deal with it. Better yet, celebrate it and start renting games before you buy them.
     
  14. bajibbles

    Member bajibbles Ragnafreak

    Joined:
    Nov 4, 2002
    Messages:
    784
    Location:
    Sydney
    Country:
    Australia
    to be honest, in my opinion there isnt a big emphasis on the actual number at the end of the review. The reviews on GBATemp are always factual and backed up by evidence, whether photographic or through testing (unless you believe they blatantly lie). The Slot-1 comparison chart is a good example; you can go through every field and it will TELL you the specifics (game compatability, size, etc.) They dont say, game compatability, 10/10; that's being more biased than their technique of explaining it in non-jargon; all games tested have compatability, low compatability, decent compatability except a few games (of which there is a list). You are obviously a critical thinker, and obviously the end score is much more objective, but if you don't concentrate so hard on that you will find that these reviews give the average gamer who will likely only buy one flash cart some of the best insight and facts into the various carts.
     
  15. Hitto

    Member Hitto MKDS Tournament Winner

    Joined:
    Nov 29, 2005
    Messages:
    1,450
    Location:
    Nice, France
    Country:
    France
    I don't know about the rest of the world, but it made the news in France, Paper Press IS dying. The readership IS sick and tired of so-called technical rags that only give you as much advice as a salesman...
    Online rags will do the same, and it will be way easier, as they only need to sell hits, not even credibility, to gulilble advertising partners. 8.8, anyone?

    The point is, if being honest worked, we'd know.
     
  16. Tamyu

    Member Tamyu GBAtemp Regular

    Joined:
    Nov 11, 2006
    Messages:
    223
    Location:
    Just outside Nagoya, Japan
    Country:
    Japan
    Paper press is dying because it costs *money*, while the same information is available in higher quantities online.
    It`s not an issue of bias - it`s more who is going to *buy* a magazine when they can go online and get the same content for free.
    If the main issue was honesty, then certainly no newspaper/magazines who were dying out in print have popular sites.

    I in no way meant to imply that all publications were honest - everything is biased to an extent - but more that if you know how to read between the lines you can find the honest opinion of the reviewer. (Who is human, and will be biased in some direction, though not necessarily in the direction the advertisers desire.)

    That`s why you find a publication or site that is biased in the same direction you are. Their reviews and opinions will match your own, and therefore will seem more "true".
     
  17. Veho

    Global Moderator Veho The man who cried "Ni".

    Joined:
    Apr 4, 2006
    Messages:
    8,447
    Location:
    Zagreb
    Country:
    Croatia
    Of course any magazine will try to make a name for themselves by being as objective and unbiased as possible, but whenever a single person is writing a review, it will be biased. Many reviews nowadays have a "second opinion" somewhere in there, a short note by another reviewer. Not surprisingly, the second opinion is often quite different from the "main" one. And yet, they are both supposed to be objective, and therefore generally similar.

    The only way to ensure objectivity would be to have several people review every piece of software or hardware, compare notes, and then summarize all their comments, critiques or praises in a single, definite text, that the publication (magazine, website, etc.) will stand behind entirely.

    Unfortunately, nobody has enough resources for that, be it people or time or hardware, and here's where sponsors come into play, and objectivity goes out the window.

    I'm not saying no review is trustworthy, but take everything you read with a grain of salt.
     
  18. stinkingbob

    Newcomer stinkingbob Advanced Member

    Joined:
    Dec 11, 2005
    Messages:
    69
    Country:
    THe problem with several people reviewing the same thing is that it would lead to great confusion for the reader. If 5 people like a product and 5 people don't, then how is the reader supposed to determine if he/she should buy it or not?
    The best way to determine something is to do the research yourself. Not sure what computer to get? Well, get a book, read up on the terms and then do a side by side comparison of the features of each product and make your decision from there. Or, you could browse the forums and see what other people have to say.

    It's all objective. For example, SHaun gave the Acekard a 7.5 score. For me, that was way to high for something that was too complicated to run. A 5 would have been better. He gave the ds-x a 9.5 and people almost shitted in their pants calling Shaun biased. Go figure. You can't please all the people all of the time. It's all objective.

    And also, nothing is more powerful than word of mouth. If an advertiser bribes a magazine or website to post positive things about its product, even though it may be shitty and poeple buy it and aren't satisfied with it, you bet that they will raise hell and will badmouth the product and the reviewer. Loss of money to both sides.
    So, if Shaun had given ds-x a 5 and Acekard a 9, I would never believe anything that he would write about again. Luckily, that is not the case. That is why I like gbatemp.
     
  19. Veho

    Global Moderator Veho The man who cried "Ni".

    Joined:
    Apr 4, 2006
    Messages:
    8,447
    Location:
    Zagreb
    Country:
    Croatia
    "Objective" means "unbiased". You mean "subjective".

    If several people review a piece of hardware, the ones that are negatively biased will point out the negative sides, and those that are positively biased will point out the good sides, and the end review will have all the pros and cons listed, and the reader just has to see if the pros outweigh the cons. That part depends solely on the reader.

    For example, if the hardware is complicated to use, it will prevent some people from using it, because they are looking for ease-of-use; others won't mind at all, because the quality is far more important to them than ease of use.

    But a real review has to mention both the good and the bad sides, and that means being objective.
     
  20. trebulator

    Newcomer trebulator Advanced Member

    Joined:
    Nov 27, 2006
    Messages:
    57
    Location:
    Knoxville
    Country:
    United States
    Moral of the story: do some research, maybe watch a gameplay video, and rent the game to develop your own opinion. Otherwise reviews are quite helpful IMO. [​IMG]
     

Share This Page