Understanding CPU Benchmarks

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

balnazzar

Active Member
Mar 6, 2019
221
30
28
I'm trying to understand how to interpret benchmark results. Look at this:


We have two 8-core processors compared. The overall (multithreaded) score in hugely in favour of the Epyc Rome, as we should expect, since it's much newer. However, if we look at the single-threaded result, the W-2145 wins by a substantial 25% (2500 vs 2000).

I truly don't understand how single- and multi-thread result should relate to each other..

Thanks.
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
The W-2145, as a "workstation" marque, is more akin to a desktop processor and thus has a much higher clock speed - 4.5GHz under boost (which will be what a single-threaded benchmark sees), 3.7GHz when all cores are loaded. The Epyc however, as a server-class chip where scaling out efficiently is usually more important than scaling up, only has a single core boost of 3.4GHz.

Beyond that, you've got differences in the benchmark itself. Some things run faster on AMD architecture, some on Intel - it's completely workload-dependent. This is why synthetic benchmarks should only be taken as a ballpark figure full of salt for comparative performance; real-world benches are where it's at.
 

balnazzar

Active Member
Mar 6, 2019
221
30
28
What you are looking are benchmark values reported by users. The Epyc CPU has exactly one sample.
Unless this benchmark suite is heavily geared towards something the Epyc CPU excels at, I would just dismiss the "CPU Mark" score as an outlier. And the single-thread result might be an indicator that this set of benchmarks does not favor Zen2.

Any particular reason why you are interested in these two CPUs?
I need a workstation processor in the 300-400eur range, with a good blend of sigle and multi threaded performance, and modest power needs. I can find the used 2145 at the same price (roughly) of a new 8-core epyc rome. Data analysis and Deep Learning is the use case.

Coming to the benchmarks, all you said is reasonable, but the, look at this: AMD Ryzen 9 3950X vs AMD EPYC 7302P vs Intel Core i9-9940X @ 3.30GHz vs AMD Ryzen 9 3900X [cpubenchmark.net] by PassMark Software

We have two almost identical amd processors (16 cores, same arch but different frequencies), and an Intel CPU having 14 cores. It seems to be the best counterpart to the AMD cpus. I added the 3900X as well.

Observations:
1. The two amd 16-core cpus have an almost identical cpumark, but their single-thread rating is hugely different.
2. The 3950X draws a lot less power than the epyc, despite being clocked way higher.
3. The Intel cpu lags waaay behind the amd ones despite having roughly the same 1-thread performance of the 3950X and being much better at that than the epyc. It lags behind the 3900X too, despite having two more cores.
 

balnazzar

Active Member
Mar 6, 2019
221
30
28
The W-2145, as a "workstation" marque, is more akin to a desktop processor and thus has a much higher clock speed - 4.5GHz under boost (which will be what a single-threaded benchmark sees), 3.7GHz when all cores are loaded. The Epyc however, as a server-class chip where scaling out efficiently is usually more important than scaling up, only has a single core boost of 3.4GHz.

Beyond that, you've got differences in the benchmark itself. Some things run faster on AMD architecture, some on Intel - it's completely workload-dependent. This is why synthetic benchmarks should only be taken as a ballpark figure full of salt for comparative performance; real-world benches are where it's at.
Which one would you choose for a typical data analysis workstation workload? Thanks.
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
Which one would you choose for a typical data analysis workstation workload? Thanks.
Define "typical" :) There's thousands of different types of data and thousands of different ways to analyse them so it all depends on the operations in question. Personally I'd choose the Epyc, since the type of work I do scales out relatively easily well over lots of threads and never really gets bottlenecked on a single core (and IPC is higher for me on Epyc than Skylake anyway so it'd only be Skylake's clock advantage and potential lower cost that'd be going for it).

If some or part of your workflow is easily bottlnecked on relatively few threads then you're probably better off going with the Intel, but it would be worthwhile investigating whether AM4 or threaripper might be viable for your use case (I'm running a 3700X in my home server myself).
 

68k-dude

Member
Jan 2, 2020
50
18
8
Hi @balnazzar

With just one core at full load the Intel part it will boost that core to 4.5GHz the AMD part will boost to 3.4GHz. That is where the difference comes from.


Xeon_W-2145_Frequency.png
 
Last edited:

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
Higher clock speed on the Xeon is a plausible explanation for the higher single-core result.
But it also contradicts the multi-core result, where the Epyc gets a much higher score, despite lower clock speed.
We don't know how the benchmark works
We don't know how the benchmark scales across cores/SMT
We don't know how the benchmark scales with memory bandwidth/latency
We don't know how the benchmark deals with ALU/FPU/SIMD differences across either architecture

Some further digging showed that a recent update to passmark made a colossal difference scores (again, we don't really know how or what)

Given that the results don't really make much sense and there seem to be shenanigans occurrin', at the moment my advice would be to ignore these numbers completely and concentrate on more reputable tests that resemble your actual workload and see how they compare.

"There are three types of lies: lies, damned lies, and synthetic benchmarks"
 
  • Like
Reactions: balnazzar

68k-dude

Member
Jan 2, 2020
50
18
8
Hello @EffrafaxOfWug o/

I agree that this Passmark result is potentially meaningless. The sample size of one brings the figure into doubt.

You could always buy one and make it a sample size of two if you are feeling adventurous. :)

Anyway, I've liked this thread. Much thinking and digging into specs.


-68k
 
  • Like
Reactions: balnazzar

balnazzar

Active Member
Mar 6, 2019
221
30
28
We don't know how the benchmark works
We don't know how the benchmark scales across cores/SMT
We don't know how the benchmark scales with memory bandwidth/latency
We don't know how the benchmark deals with ALU/FPU/SIMD differences across either architecture

[...]

"There are three types of lies: lies, damned lies, and synthetic benchmarks"
:)
 

i386

Well-Known Member
Mar 18, 2016
4,221
1,540
113
34
Germany
Let's assume that both cpus use the same instructions and require the same amount of clocks per instruction for simplification :D
The intel cpu will win the single threaded workloads because of the higher turbo clock.
In multithreaded benchmarks it will get a little bit more complex. If the cpu is only crunching numbers with data in the registers the intel cpu should win because of the higher clocks on all cores. But if you're loading a lot of data from ram the epyc cpu can hold more data in the l3 cache (128 MByte epyc = 16MByte per core vs 11MByte xeon = 1.375MByte per core) and requires less "slow" ram access for the same work. (Actually the same logic should apply to the single threaded benchmark :thinking: )

My thought's, please don't quote me on that :p
 
  • Like
Reactions: balnazzar