Samsung Tweaked OS to Cheat Benchmarks on Galaxy S4

IBNobody

I try to keep myself amused.
OP
Member
Joined
Nov 16, 2006
Messages
1,151
Trophies
1
Location
Texas, Hang 'Em High
Website
Visit site
XP
954
Country
United States
http://www.anandtech.com/show/7187/...ght.com&utm_medium=referral&utm_campaign=Feed

The folks at AnandTech caught Samsung red-handed in tweaking the Android OS to run certain benchmarks better. They found the behavior on both the Snapdragon and the Exynos S4 models, but the Exynos models cheated even worse.

Basically, Samsung added a "Benchmark Booster" feature to cheat benchmarks. (It's actually called that in the code!)

If a certain benchmark app was detected as being run, the S4 Exynos would overclock its GPU from 480MHz to 532MHz and both models would set the CPU governer to the maximum CPU clock speed. While the CPU clock speed was available to all apps, the overclocked GPU mode was only made available to these benchmarks.

Have a cool 3D high-end android game? The S4 Exynos's GPU will run at 480MHz.
Have a meaningless benchmark? The S4 Exynos's GPU will run at 532MHz.

UPDATE:

Samsung admits to tweaking the S4.
http://global.samsungtomorrow.com/?p=26314


Under ordinary conditions, the GALAXY S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode. Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance.

The maximum GPU frequencies for the GALAXY S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results.

Samsung Electronics remains committed to providing our customers with the best possible user experience.


Now, what they are saying is a bit duplicitous. The only apps that can use the 532/533 MHz mode are ones that Samsung approves and white-lists. Samsung's apps can use it. Benchmarks can use it. Everything else can't. What's the point of benching at 532MHz when your real world performance is artificially capped at 480MHz?


TL;DR:
Don't trust benchmarks!
 

Rockhoundhigh

Man with a mission
Member
Joined
Apr 2, 2011
Messages
477
Trophies
0
Age
28
Location
Southern California
XP
712
Country
United States
Yeah, if there's anything I've learned from tinkering with my Android phone it's that benchmark scores mean nothing. Judge a phone only on its real world performance. In this case though Samsung was just being sleazy to get even more people to sing their praises.
 

Joe88

[λ]
Global Moderator
Joined
Jan 6, 2008
Messages
12,736
Trophies
2
Age
36
XP
7,407
Country
United States
Yeah, if there's anything I've learned from tinkering with my Android phone it's that benchmark scores mean nothing. Judge a phone only on its real world performance. In this case though Samsung was just being sleazy to get even more people to sing their praises.
well its also the various review sites who mainly judge phones phones on benchmarks alone ignoring real world usage
 

Lastly

Tempin' at the speed of sound
Member
Joined
Aug 8, 2008
Messages
274
Trophies
1
XP
587
Country
United States
Samsung added a "Benchmark Booster" feature to cheat benchmarks. (It's actually called that in the code!)
Haha! I can actually imagine that:

Code:
// TOP SECRET - Do not let anyone see this code --- even your friend, wife, son, dog, cat, or your second part-time mean boss!
// All who fail to comply will be persecuted to the full extent of law!
// If this info is leaked, do not tell that the CEO, that's me, or head director explicitly instruct you to add this code. Blame it on the IDE you were using. Or that you accidentally multiplied it. Whatever it is, don't tattle on me.
 
realScore = getScore(); // The real score! Too bad no one will see it now!
fakeMultiplier = 2.5f; // This multiplier will increase the score by 2.5 times the real score.
benchmarkBooster = realScore * fakeMultiplier; // The final score our "benchmarkphiles" will see! This will convince them, which ultimately increase our sales by 5,000 units!
printf(@"benchmarkBooster %i", benchmarkBooster);
 

jonthedit

Well-Known Member
Member
Joined
May 30, 2011
Messages
1,682
Trophies
0
XP
1,009
Country
Bangladesh
Hah. I can't imagina tat:

Code:
// If this info isn't leafed, do not tell that the CEO, that's me, or head director told you to do so. Blame it on the willy you were shaking. Or that you accidentally entered it insideout.
 
realScore = getShake(); // The real score! Too bad no one will see it now!
fakeMultiplier = 2.5f; // This multiplier will increase the score by 2.5 times the real score.
benchmarkBooster = realScore * jerkit; // The final score our "benchmarkphiles" will see! This will convince them, which ultimately increase our sales by 5,000 units!
printf(@"benchmarkBooster %i", benchthatweight);


Even without the overclocking the phone is still great for N64 emulation :P
 
  • Like
Reactions: 1stClassZackFair

IBNobody

I try to keep myself amused.
OP
Member
Joined
Nov 16, 2006
Messages
1,151
Trophies
1
Location
Texas, Hang 'Em High
Website
Visit site
XP
954
Country
United States
Haha! I can actually imagine that:

Code:
// TOP SECRET - Do not let anyone see this code --- even your friend, wife, son, dog, cat, or your second part-time mean boss!
// All who fail to comply will be persecuted to the full extent of law!
// If this info is leaked, do not tell that the CEO, that's me, or head director explicitly instruct you to add this code. Blame it on the IDE you were using. Or that you accidentally multiplied it. Whatever it is, don't tattle on me.
 
realScore = getScore(); // The real score! Too bad no one will see it now!
fakeMultiplier = 2.5f; // This multiplier will increase the score by 2.5 times the real score.
benchmarkBooster = realScore * fakeMultiplier; // The final score our "benchmarkphiles" will see! This will convince them, which ultimately increase our sales by 5,000 units!
printf(@"benchmarkBooster %i", benchmarkBooster);


Their formula is wrong. It didn't take into account the competitor's benchmark score. :)

benchmarkBooster = realScore * fakeMultiplier + 0.1f * competitorBenchmark;
 

Lastly

Tempin' at the speed of sound
Member
Joined
Aug 8, 2008
Messages
274
Trophies
1
XP
587
Country
United States
Still more honest than apple.

whatcolor_isblue.jpg


As expected from Thanatos.

I remember reading that faking the benchmark, actually slow down the device. At least from what I remember reading a year ago.
 

Foxi4

Endless Trash
Global Moderator
Joined
Sep 13, 2009
Messages
30,818
Trophies
3
Location
Gaming Grotto
XP
29,788
Country
Poland
I don't think it's cheating per-se and here's my reasoning:
  • The benchmarking process is supposed to measure the maximum possible performance of hardware
  • Normally portable handsets run on underclocked values to preserve battery life and prevent overheating which may damage your handset in the long run
  • While running a benchmark, the hardware goes full-blast. Those values are not overclocked - they're the maximum possible values for the chips
  • If you're skilled enough, you can manually select the higher stepping, however it's locked out for normal operation to prevent the forementioned overheating and draining the battery
Now, what Samsung should've done to be 100% "clean" was to embed a so-called "High Performance" mode which the user could switch to after agreeing (by checking a checkbox) that doing so may cause damage to the handset in high temperature/low air circulation conditions as well as severely lower the battery life of the overall setup. That way it'd be up to to the user to use those values and risk damaging the handset if it does go into total meltdown. The way it's been handled makes it seem fishy but I can see the logic behind it.
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • S @ salazarcosplay:
    @BigOnYa they ruined the gaming experience for me to be honest
  • S @ salazarcosplay:
    @BigOnYa Im not crazy about getting all of them, i feel like I have something to show for for the time put in
  • S @ salazarcosplay:
    @BigOnYa If you want to do rgh or 360 mod
  • S @ salazarcosplay:
    does it matter if you update your 360 or not before trying is it advisable or not
  • BigOnYa @ BigOnYa:
    Yea I don't pay attention to them really. Or do I try to 100% a game. I just play till story ends/ or I get the girl!
  • K3Nv2 @ K3Nv2:
    Bigonya uses his wiener to mod 360s
    +1
  • Xdqwerty @ Xdqwerty:
    Going to the water park, see ya
  • BigOnYa @ BigOnYa:
    You should update the 360 to newest dash before RGHing it yes. But not a big deal if you don't, you can install new dash/avatar updates after. It's just easier to do it auto online before, instead manual offline after.
  • BigOnYa @ BigOnYa:
    Have fun @Xdqwerty. If you see a chocolate candy bar floating in the water, don't eat it!
  • AncientBoi @ AncientBoi:
    :O:ohnoes: Y didn't U Tell ME that ALSO? @BigOnYa :ohnoes: 🤢🤮
    +1
  • BigOnYa @ BigOnYa:
    Does it taste like... chicken?
    +1
  • S @ salazarcosplay:
    @BigOnYa I wanted to ask you about your experience with seeing south park. Most of the people a bit younger like my younger brother and cousins that are a few younger than me that saw kids found south park funny because of the curse words, kids at school, that seemed like liking the show on a very basic level.

    I could not quite have a in depth discussion of the show.

    How was it for you? As an adult. What did you find the most interesting part about it. Did you relate to the parents of the kids and their situations. Was it satires, the commentary on society. The references on celebrities' and pop culture.
    +1
  • BigOnYa @ BigOnYa:
    I remember seeing the very first episode back in the day, and have watched every episode since. I used to set my VCR to record them even, shows how long ago.
  • BigOnYa @ BigOnYa:
    I just like any comedies really, and cartoons have always been a favorite of mine. Family guy, American Dad, Futurama, Cleveland Show, Simpsons - I like them all.
    +1
  • BigOnYa @ BigOnYa:
    South Park is great cause they always touch on relavent issues going on today, and make something funny out of it.
    +3
  • S @ salazarcosplay:
    @BigOnYa were you always up to date on the current events and issues of the time or were there issues that you first found out thru south park
  • BigOnYa @ BigOnYa:
    Most of the time yea I knew, I watch and read the news regularly, but sometimes the Hollywood BS stuff, like concerning actors slip by me. I don't follow most Hollywood BS (example: the Kardasians)
    +2
  • S @ salazarcosplay:
    @BigOnYa there were relevant issues before south park was made, that's why i think a south park prequel/spinoff would be great. Randy and his friends in their child hood
    +1
  • BigOnYa @ BigOnYa:
    Yea, like them running in high school together, getting into stuff, and how they got hitched and had kids. And how the town of South Park was back then compared to now. That would be cool to see.
  • BakerMan @ BakerMan:
    yeah
  • The Real Jdbye @ The Real Jdbye:
    @salazarcosplay if they made a prequel, it would still be about current issues, cause it doesn't make sense to make it about stuff that happened 30 years ago that nobody cares about anymore
  • The Real Jdbye @ The Real Jdbye:
    it's too late
  • The Real Jdbye @ The Real Jdbye:
    the older south park episodes about particular issues usually age poorly since the topic is no longer relevant
  • The Real Jdbye @ The Real Jdbye:
    an exception is giant douche vs turd sandwich, that's always relevant :P
    +1
    The Real Jdbye @ The Real Jdbye: an exception is giant douche vs turd sandwich, that's always relevant :P +1