An explanation of why benchmarking is only relevant to determining if a hardware can perform as well or better than the expectations expressed by the hardware developer.
Episode #9-15 released on November 25, 2018
I am definitely guilty of this, getting a new piece of hardware and then using a benchmark application to it just to see its potential. And, yes, it does give an idea of what the maximum potential of the hardware is, but is it accurate?
There are several conditions that can make hardware perform worse, or better. These include thermal output, cooling, and power, to say the least. There are, also, many other factors that can come into play related to compatibility, bottlenecking, drivers, etc. You, also, have the silicon lottery to consider in all this. Then, we, also, have real world use case versus extremely rare use case scenarios. So, how about we start picking apart benchmarks and leave them where they belong.
What does benchmark software do to begin with?
Hardware companies like to publish specifications to go with their hardware. Benchmark software can be used to determine if the hardware is technically capable of achieving the goals of the specifications released for that piece of hardware.
Benchmark software allows us to determine if it meets or exceeds the goals of the specifications by pushing the hardware to perform at its maximum potential. The result is higher heat output, more power required, and better cooling required. Some people, also, overclock their system to see how much more their system can perform which results in even more heat, and power required. Although, provided adequate cooling, overclocking does in fact net you more performance, during everyday tasks and benchmarking.
Benchmarking software demonstrates whether the hardware product actually performs by pushing the system to its limits. Depending on the type of benchmark software, because there are several, it can be pushing the CPU and or the GPU, and other components at its maximum limits, which is why the power used during a benchmark is higher than convention and results in more heat output and better cooling being required. If the cooling isn't adequate, it will result in thermal limits being attained, resulting thermal throttling and negatively impact the score received by the benchmark test. The result of being tested in this manner is that the CPU, GPU, and or other hardware being benchmarked will be running at or just below the maximum sustainable level, what we normally call 100% usage in Windows and other operating systems. Having various components not running at 100% usually indicates that the system is being bottlenecked, a condition where a piece of hardware in the system is slowing down the entire system.
Why does benchmarking a system not reflect the daily reality of using a given computer that has been tested?
You will rarely be able to push your computer, in the same way as a benchmarking application, and thus never experience the same performance issues, rarely have the same thermal issues, and thermal throttling may actually occur less than a benchmark application may indicate. It is therefore only accurate to demonstrate the system specifications, but not accurate in testing everyday usage. This is notably why a few video games also come with benchmarking software which is far more accurate in testing the system for that given game. Although, if you are used to running other applications in background, a more accurate result would be to run them, too, while doing the benchmark to see what is going on.
To be very frank, benchmarking is important, and useful, but does not illustrate the realities of the everyday user. Many people do not have monitors that can even benefit from high frame counts, and some people do not have the budget for higher end mainboards, processors or graphics cards. The majority of public benchmarking still leans towards the higher end, too. While, it may be accurate in determining how a system will be pushed at its maximum, comparing the results to benchmark to reality isn't accurate, to say the least. There is, also, the issue that a benchmark seen online may not result in the same results given that you may not have the same mainboard, ram, drives, processor or GPU, and even if you did, the silicon lottery, a term used to explain why two processors in the same series may not perform the same, is a thing, and a you may end up with a better or worse silicon component than those you see online. This is why I will tell you now, take all benchmarks with a grain of salt, and look at as many before you buy remembering, you don't have the same system, it doesn't reflect reality, and your mileage will always vary, wildly, too.
Host : Steve Smith | Music : | Editor : Steve Smith | Producer : Zed Axis Dot Net