Samsung Tweaked OS to Cheat Benchmarks on Galaxy S4

Discussion in 'User Submitted News' started by IBNobody, Aug 1, 2013.

  1. IBNobody
    OP

    IBNobody I try to keep myself amused.

    Member
    1,127
    189
    Nov 16, 2006
    United States
    Texas, Hang 'Em High
    http://www.anandtech.com/show/7187/...ght.com&utm_medium=referral&utm_campaign=Feed

    The folks at AnandTech caught Samsung red-handed in tweaking the Android OS to run certain benchmarks better. They found the behavior on both the Snapdragon and the Exynos S4 models, but the Exynos models cheated even worse.

    Basically, Samsung added a "Benchmark Booster" feature to cheat benchmarks. (It's actually called that in the code!)

    If a certain benchmark app was detected as being run, the S4 Exynos would overclock its GPU from 480MHz to 532MHz and both models would set the CPU governer to the maximum CPU clock speed. While the CPU clock speed was available to all apps, the overclocked GPU mode was only made available to these benchmarks.

    Have a cool 3D high-end android game? The S4 Exynos's GPU will run at 480MHz.
    Have a meaningless benchmark? The S4 Exynos's GPU will run at 532MHz.

    UPDATE:

    Samsung admits to tweaking the S4.
    http://global.samsungtomorrow.com/?p=26314



    Now, what they are saying is a bit duplicitous. The only apps that can use the 532/533 MHz mode are ones that Samsung approves and white-lists. Samsung's apps can use it. Benchmarks can use it. Everything else can't. What's the point of benching at 532MHz when your real world performance is artificially capped at 480MHz?


    TL;DR:
    Don't trust benchmarks!
     


  2. Joe88

    Joe88 [λ]

    Member
    11,598
    2,825
    Jan 6, 2008
    United States
    NYC
    people actually care about cell phone benchmarks?
     
  3. IBNobody
    OP

    IBNobody I try to keep myself amused.

    Member
    1,127
    189
    Nov 16, 2006
    United States
    Texas, Hang 'Em High

    People care enough for Samsung to want to cheat.
     
    Rydian likes this.
  4. Rockhoundhigh

    Rockhoundhigh Man with a mission

    Member
    437
    210
    Apr 2, 2011
    United States
    Southern California
    Yeah, if there's anything I've learned from tinkering with my Android phone it's that benchmark scores mean nothing. Judge a phone only on its real world performance. In this case though Samsung was just being sleazy to get even more people to sing their praises.
     
  5. DinohScene

    DinohScene Capture the Dino

    Member
    GBAtemp Patron
    DinohScene is a Patron of GBAtemp and is helping us stay independent!

    Our Patreon
    15,843
    12,301
    Oct 11, 2011
    Antarctica
    В небо
    Oh Nokia 3310.
    Why are you so amazing!
     
    Joe88 likes this.
  6. Joe88

    Joe88 [λ]

    Member
    11,598
    2,825
    Jan 6, 2008
    United States
    NYC
    well its also the various review sites who mainly judge phones phones on benchmarks alone ignoring real world usage
     
  7. Lastly

    Lastly Honorable UltimateAdmiin - Highest Rank

    Member
    261
    41
    Aug 8, 2008
    United States
    Haha! I can actually imagine that:

    Code:
    // TOP SECRET - Do not let anyone see this code --- even your friend, wife, son, dog, cat, or your second part-time mean boss!
    // All who fail to comply will be persecuted to the full extent of law!
    // If this info is leaked, do not tell that the CEO, that's me, or head director explicitly instruct you to add this code. Blame it on the IDE you were using. Or that you accidentally multiplied it. Whatever it is, don't tattle on me.
     
    realScore = getScore(); // The real score! Too bad no one will see it now!
    fakeMultiplier = 2.5f; // This multiplier will increase the score by 2.5 times the real score.
    benchmarkBooster = realScore * fakeMultiplier; // The final score our "benchmarkphiles" will see! This will convince them, which ultimately increase our sales by 5,000 units!
    printf(@"benchmarkBooster %i", benchmarkBooster);
     
    IBNobody and Slamicite like this.
  8. jonthedit

    jonthedit GBAtemp Advanced Maniac

    Member
    1,691
    438
    May 30, 2011
    Bangladesh

    Even without the overclocking the phone is still great for N64 emulation :P
     
    1stClassZackFair likes this.
  9. IBNobody
    OP

    IBNobody I try to keep myself amused.

    Member
    1,127
    189
    Nov 16, 2006
    United States
    Texas, Hang 'Em High

    Their formula is wrong. It didn't take into account the competitor's benchmark score. :)

    benchmarkBooster = realScore * fakeMultiplier + 0.1f * competitorBenchmark;
     
  10. Lastly

    Lastly Honorable UltimateAdmiin - Highest Rank

    Member
    261
    41
    Aug 8, 2008
    United States
    A Galaxy S2 have no problem running most games, too. Don't get the wrong idea. I think the Galaxy S4 is great.
     
  11. Thanatos Telos

    Thanatos Telos random stuff

    Member
    842
    187
    Sep 13, 2009
    United States
    Still more honest than apple.
     
  12. Lastly

    Lastly Honorable UltimateAdmiin - Highest Rank

    Member
    261
    41
    Aug 8, 2008
    United States
    [​IMG]

    As expected from Thanatos.

    I remember reading that faking the benchmark, actually slow down the device. At least from what I remember reading a year ago.
     
  13. Thanatos Telos

    Thanatos Telos random stuff

    Member
    842
    187
    Sep 13, 2009
    United States
    Feel the cold treachery!
     
  14. shakirmoledina

    shakirmoledina Legend

    Member
    6,611
    218
    Oct 23, 2004
    Tanzania
    Dar es Salaam
    Welcome to the world of marketing.

    Samsung will just release a new iteration with no 'tampering'.
     
  15. DarkSpace

    DarkSpace Custom member title

    Member
    551
    0
    Aug 23, 2007
    United States
    Remember it's not unethical until someone catches you doing cheating the system.
     
  16. FAST6191

    FAST6191 Techromancer

    pip Reporter
    23,366
    9,169
    Nov 21, 2005
    I am going instead with if the system is meaningless then cheating the meaningless system is itself meaningless and thus outside ethics.
     
  17. Ethevion

    Ethevion Wannabe Artist

    Member
    1,783
    802
    Sep 23, 2010
    Canada
    Considering they go head to head with Apple in the smart phone segment, I'd say yes.
     
  18. Foxi4

    Foxi4 On the hunt...

    pip Reporter
    23,537
    21,496
    Sep 13, 2009
    Poland
    Gaming Grotto
    I don't think it's cheating per-se and here's my reasoning:
    • The benchmarking process is supposed to measure the maximum possible performance of hardware
    • Normally portable handsets run on underclocked values to preserve battery life and prevent overheating which may damage your handset in the long run
    • While running a benchmark, the hardware goes full-blast. Those values are not overclocked - they're the maximum possible values for the chips
    • If you're skilled enough, you can manually select the higher stepping, however it's locked out for normal operation to prevent the forementioned overheating and draining the battery
    Now, what Samsung should've done to be 100% "clean" was to embed a so-called "High Performance" mode which the user could switch to after agreeing (by checking a checkbox) that doing so may cause damage to the handset in high temperature/low air circulation conditions as well as severely lower the battery life of the overall setup. That way it'd be up to to the user to use those values and risk damaging the handset if it does go into total meltdown. The way it's been handled makes it seem fishy but I can see the logic behind it.
     
  19. IBNobody
    OP

    IBNobody I try to keep myself amused.

    Member
    1,127
    189
    Nov 16, 2006
    United States
    Texas, Hang 'Em High
    I've updated the OP with Samsung's response.


    To be fair, everybody would be jumping on Apple if this were them.
     
    The Milkman and nando like this.
  20. Gahars

    Gahars Bakayaro Banzai

    Member
    10,254
    17,403
    Aug 5, 2011
    United States
    New Jersey
    I always thought Samsung was a bench warmer, but this is ridiculous.