Samsung Tweaked OS to Cheat Benchmarks on Galaxy S4

IBNobody

I try to keep myself amused.
OP
Member
Joined
Nov 16, 2006
Messages
1,136
Trophies
0
Location
Texas, Hang 'Em High
Website
Visit site
XP
901
Country
United States
http://www.anandtech.com/show/7187/...ght.com&utm_medium=referral&utm_campaign=Feed

The folks at AnandTech caught Samsung red-handed in tweaking the Android OS to run certain benchmarks better. They found the behavior on both the Snapdragon and the Exynos S4 models, but the Exynos models cheated even worse.

Basically, Samsung added a "Benchmark Booster" feature to cheat benchmarks. (It's actually called that in the code!)

If a certain benchmark app was detected as being run, the S4 Exynos would overclock its GPU from 480MHz to 532MHz and both models would set the CPU governer to the maximum CPU clock speed. While the CPU clock speed was available to all apps, the overclocked GPU mode was only made available to these benchmarks.

Have a cool 3D high-end android game? The S4 Exynos's GPU will run at 480MHz.
Have a meaningless benchmark? The S4 Exynos's GPU will run at 532MHz.

UPDATE:

Samsung admits to tweaking the S4.
http://global.samsungtomorrow.com/?p=26314


Under ordinary conditions, the GALAXY S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode. Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance.

The maximum GPU frequencies for the GALAXY S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results.

Samsung Electronics remains committed to providing our customers with the best possible user experience.


Now, what they are saying is a bit duplicitous. The only apps that can use the 532/533 MHz mode are ones that Samsung approves and white-lists. Samsung's apps can use it. Benchmarks can use it. Everything else can't. What's the point of benching at 532MHz when your real world performance is artificially capped at 480MHz?


TL;DR:
Don't trust benchmarks!
 

Rockhoundhigh

Man with a mission
Member
Joined
Apr 2, 2011
Messages
477
Trophies
0
Age
25
Location
Southern California
XP
682
Country
United States
Yeah, if there's anything I've learned from tinkering with my Android phone it's that benchmark scores mean nothing. Judge a phone only on its real world performance. In this case though Samsung was just being sleazy to get even more people to sing their praises.
 

Joe88

[λ]
Global Moderator
Joined
Jan 6, 2008
Messages
12,640
Trophies
1
Age
33
XP
6,199
Country
United States
Yeah, if there's anything I've learned from tinkering with my Android phone it's that benchmark scores mean nothing. Judge a phone only on its real world performance. In this case though Samsung was just being sleazy to get even more people to sing their praises.
well its also the various review sites who mainly judge phones phones on benchmarks alone ignoring real world usage
 

Lastly

[Insert Title Here]
Member
Joined
Aug 8, 2008
Messages
263
Trophies
0
XP
277
Country
United States
Samsung added a "Benchmark Booster" feature to cheat benchmarks. (It's actually called that in the code!)
Haha! I can actually imagine that:

Code:
// TOP SECRET - Do not let anyone see this code --- even your friend, wife, son, dog, cat, or your second part-time mean boss!
// All who fail to comply will be persecuted to the full extent of law!
// If this info is leaked, do not tell that the CEO, that's me, or head director explicitly instruct you to add this code. Blame it on the IDE you were using. Or that you accidentally multiplied it. Whatever it is, don't tattle on me.
 
realScore = getScore(); // The real score! Too bad no one will see it now!
fakeMultiplier = 2.5f; // This multiplier will increase the score by 2.5 times the real score.
benchmarkBooster = realScore * fakeMultiplier; // The final score our "benchmarkphiles" will see! This will convince them, which ultimately increase our sales by 5,000 units!
printf(@"benchmarkBooster %i", benchmarkBooster);
 

jonthedit

Well-Known Member
Member
Joined
May 30, 2011
Messages
1,682
Trophies
0
XP
969
Country
Bangladesh
Hah. I can't imagina tat:

Code:
// If this info isn't leafed, do not tell that the CEO, that's me, or head director told you to do so. Blame it on the willy you were shaking. Or that you accidentally entered it insideout.
 
realScore = getShake(); // The real score! Too bad no one will see it now!
fakeMultiplier = 2.5f; // This multiplier will increase the score by 2.5 times the real score.
benchmarkBooster = realScore * jerkit; // The final score our "benchmarkphiles" will see! This will convince them, which ultimately increase our sales by 5,000 units!
printf(@"benchmarkBooster %i", benchthatweight);


Even without the overclocking the phone is still great for N64 emulation :P
 
  • Like
Reactions: 1stClassZackFair

IBNobody

I try to keep myself amused.
OP
Member
Joined
Nov 16, 2006
Messages
1,136
Trophies
0
Location
Texas, Hang 'Em High
Website
Visit site
XP
901
Country
United States
Haha! I can actually imagine that:

Code:
// TOP SECRET - Do not let anyone see this code --- even your friend, wife, son, dog, cat, or your second part-time mean boss!
// All who fail to comply will be persecuted to the full extent of law!
// If this info is leaked, do not tell that the CEO, that's me, or head director explicitly instruct you to add this code. Blame it on the IDE you were using. Or that you accidentally multiplied it. Whatever it is, don't tattle on me.
 
realScore = getScore(); // The real score! Too bad no one will see it now!
fakeMultiplier = 2.5f; // This multiplier will increase the score by 2.5 times the real score.
benchmarkBooster = realScore * fakeMultiplier; // The final score our "benchmarkphiles" will see! This will convince them, which ultimately increase our sales by 5,000 units!
printf(@"benchmarkBooster %i", benchmarkBooster);


Their formula is wrong. It didn't take into account the competitor's benchmark score. :)

benchmarkBooster = realScore * fakeMultiplier + 0.1f * competitorBenchmark;
 

Lastly

[Insert Title Here]
Member
Joined
Aug 8, 2008
Messages
263
Trophies
0
XP
277
Country
United States
Still more honest than apple.

whatcolor_isblue.jpg


As expected from Thanatos.

I remember reading that faking the benchmark, actually slow down the device. At least from what I remember reading a year ago.
 

Foxi4

Endless Trash
Global Moderator
Joined
Sep 13, 2009
Messages
28,273
Trophies
2
Location
Gaming Grotto
XP
25,683
Country
Poland
I don't think it's cheating per-se and here's my reasoning:
  • The benchmarking process is supposed to measure the maximum possible performance of hardware
  • Normally portable handsets run on underclocked values to preserve battery life and prevent overheating which may damage your handset in the long run
  • While running a benchmark, the hardware goes full-blast. Those values are not overclocked - they're the maximum possible values for the chips
  • If you're skilled enough, you can manually select the higher stepping, however it's locked out for normal operation to prevent the forementioned overheating and draining the battery
Now, what Samsung should've done to be 100% "clean" was to embed a so-called "High Performance" mode which the user could switch to after agreeing (by checking a checkbox) that doing so may cause damage to the handset in high temperature/low air circulation conditions as well as severely lower the battery life of the overall setup. That way it'd be up to to the user to use those values and risk damaging the handset if it does go into total meltdown. The way it's been handled makes it seem fishy but I can see the logic behind it.
 
General chit-chat
Help Users
    AkiraKurusu @ AkiraKurusu: I've got a 2017 MacBook Pro, the one with the Touch Bar thing above the keyboard; well, today I...