Why Samsung Cheats at Benchmarks (Allegedly)

Ron Amadeo, over at Ars Technica: 

The smoking gun here is CPU idle speeds, which can be viewed with a system monitor app while using the phone. The above picture shows how differently the CPU treats a benchmarking app from a normal app. Normally, while the Note 3 is idling, three of the four cores shut off to conserve power; the remaining core drops down to a low-power 300MHz mode. However, if you load up just about any popular CPU benchmarking app, the Note 3 CPU locks into 2.3GHz mode, the fastest speed possible, and none of the cores ever shut off. Stopping the CPU from idling shouldn't in and of itself affect the benchmark scores a whole lot, so this was our first sign that something was wrong. Benchmarks exist to measure the performance of a phone during normal usage, and a device should never treat a benchmark app differently than a normal app.

As mentioned in the article, this isn't the first time that Samsung has been caught with its hand in the benchmark cookie jar. Months ago, it was determined that the Galaxy S4 was guilty of the same sort of shenanigans. To be fair, almost every Android manufacturer does this too. My focus, though, is on Samsung since they were the ones most recently caught 'juicing' their benchmark results. Why does Samsung (allegedly) cheat at benchmarks?

The answer is quite simple: because cheating has very many positive benefits and very few negative effects.

Consider the benefits of hypothetical headlines that proclaim Samsung Releases Most Powerful Phone or Galaxy Note 3 is Fast. Those kind of headlines help Samsung cultivate a particular brand image, specifically that their phones are better than their competitors' phones. Certainly, all the tech enthusiasts would have seen such a headline, and with any luck the story would be picked up by the traditional media outlets. Some tech sites are obsessed with benchmarks, so anything that can be done to boost those benchmarks helps Samsung's cause.

Samsung, by the way, is fully aware of the fact that news story retractions are never as impactful as the original story.  Once a particular impression has been made (e.g. that Samsung makes fast phones), it is difficult for that impression to be changed in consumers' minds.

Look at Ars Technica's headline. Note 3’s benchmarking “adjustments” inflate scores by up to 20%. That certainly doesn't have as much oomph as Galaxy Note 3 is Fast . It simply isn't catchy enough. In fact, the headline loses impact by stating the facts indirectly. They could have written a different headline such as Galaxy Note 3 Cheats at Benchmark Tests  which would have been just as true. If they were worried about potential legal implications, then they could have written a headline such as Galaxy Note 3 Artificially Inflates Benchmark Scores . The positive headlines are much easier to remember and much more likely to be propagated among the general public. The negative headlines are much more likely to stay out of the public consciousness.

Folks, manufacturers shouldn't be rewarded for their deception. Consumers should demand honesty (and reward honesty).