Workstation: 5120T vs. W-2140B, and mobos.

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

balnazzar

Active Member
Mar 6, 2019
221
30
28
I'm building a (cheap) deep learning / data analysis workstation. Staying within my budget limits, I've found the option I mentioned in the thread headline. I can buy them for the same price, more or less. They are from the same generation (Skylake), and I expect them to draw the same amount of power.

The 5120T should be much better, but let's see what passmark says:


1. They have the same CPUmark, despite the 5120T having almost twice the number of cores (14 vs. 8).

2. Importantly, the single core score for the 2140B is 2500 vs. 1800 for the 5120T, wich is *a lot*.

Should I trust these numbers?

Additional question about the mobos:
the 2140B would go upon a Supermicro X11SRA, a board that is very well tested and reviewed, while for the 5120T I'd go for the Gigabyte MU71-SU0. It is the only xeon scalable board that is ATX (not CEB/EEB) and with a sufficient number of 16X slots (I need at least three of them). But I know nothing about this board, and was unable to find a review.

Thanks.
 

blinkenlights

Active Member
May 24, 2019
157
65
28
Should I trust these numbers?
I would not trust anything that is not a benchmark of your specific application. I am not being difficult; it is just a fact that synthetic benchmarks will only take you so far in terms of real-world performance.

Additional question about the mobos:
the 2140B would go upon a Supermicro X11SRA, a board that is very well tested and reviewed, while for the 5120T I'd go for the Gigabyte MU71-SU0. It is the only xeon scalable board that is ATX (not CEB/EEB) and with a sufficient number of 16X slots (I need at least three of them). But I know nothing about this board, and was unable to find a review.
So, choosing between a Scalable and Xeon-W build involves a lot of variables. The Scalable price will be higher, but it gives you more options (higher core count, multiple sockets, six memory channels) in the future as prices drop. If you are only planning to keep the system a couple of years, it might make more sense from a budget perspective to go the fewer core, higher clock Xeon-W route until you really need more resources.

I see you mentioned the X11SRA. Well, I have one of those sitting on my workbench because it got replaced almost immediately with an Asus WS C422 PRO/SE. The Supermicro board was great in concept but poorly executed, in my opinion. Look at the reviews on the Asus board - I think they have a "SAGE/10G" version with dual 10 GigE if you need it. Note that there are now two revisions of the X11SRA board - the newer one has "bugs" fixed and more (should have been there to begin with) power delivery to the processor.
 

bayleyw

Active Member
Jan 8, 2014
292
95
28
The 5120T is not a good workstation processor; it is very expensive and has bad turbo. In addition, it only has one AVX-512 unit and no Optane DCPM support, which are key advantages Intel has over its competitors. Being a Skylake-generation part, it also lacks hardware Spectre mitigations.

To really advise on this we'd need to know:

-Are you 100% married to using an Intel solution? (For example, Intel's profiling tools are far superior to AMD's, enough that I use Intel knowing fully AMD has the lead at the top end right now).
-What CPU-intensive applications are you running? How important is 1T performance?
-What's your budget?
-Are you willing to settle with a used CPU? The reason I ask is that while Intel's list prices are uncompetitive, the actual prices that large customers pay is much lower and this is reflected in the used market.
 

balnazzar

Active Member
Mar 6, 2019
221
30
28
Thanks for your feedback. Let me address your observations.

1. I can buy both of them (used) at a very reasonable price (~300eur) so don't take the price factor into account.
2. Power consumption is important since I have some limitations about it. So I selected the 5120T and its lower all-core turbo frequency.
3. I'd go for intel because of the MKL "issues" with AMD. I'll make heavy use of Intel MKL (OpenCV, PIL, Pytorch, Tensorflow).
4. I know, even the worst Epyc Rome (the 7252 I think) would easily beat both of them at general computing.
5. I owned an Epyc 7282 with an asrock EPYCD8 and was unable to get sleep or hibernation to work (ubuntu) . I absolutely need at least one of them. On the other hand, the 7282 was lightning fast compared to my dual e5-2690v3, drawing a third of the electricity.
6. Has any of you been successful in putting the supermicro Epyc board to sleep with linux?
7. I have 128gb (4x32gb) of samsung ram (M393A4K40BB0-CPB) I have to use. The new board has to be compatible.
8.The GPUs will be fed by just one thread. So, good single thread performance is somewhat recommended, even if real-world experiments show that the impact should be minimal as long as you stay above some 2ghz (tested with some Xeon e5-v3).
9. We won't find any cascade lake at 300eur, that's for sure.
10. If you can point me towards an epyc rome board that you verified having sleep/hiber working with linux, I think I can go for AMD even with some of these MKL limitations.
11. What you say about the X11SRA worries me. How is the Asus one? Did you encounter any issue with that board? I decided for the Supermicrosince the Asus is badly reviewed on newegg: ASUS WS C422 PRO/SE ATX Server Motherboard - Newegg.com
12. On the other hand, the gigabyte board is completely unreviewed, and this worries me even more.

Thanks!
 

68k-dude

Member
Jan 2, 2020
50
18
8
Hello @balnazzar o/

I'm thinking aloud here. What about Threadripper? I see some good deals on ebay from time to time.

Lots of PCIe 16x slots ATX form factor

As for Threadripper and Ubuntu I can't answer that, maybe others who have experience can.


-68k
 

balnazzar

Active Member
Mar 6, 2019
221
30
28
Hello @balnazzar o/

I'm thinking aloud here. What about Threadripper? I see some good deals on ebay from time to time.

Lots of PCIe 16x slots ATX form factor

As for Threadripper and Ubuntu I can't answer that, maybe others who have experience can.


-68k
I cannot go for the TR. Reasons:
- Monstrous power draw.
- Inability to use RDIMMs.
- Cost.
 

68k-dude

Member
Jan 2, 2020
50
18
8
Oh yes.

I hadn't looked into that memory you listed.

That's a shame, €500 would have got you a nice board and probably a 12 core chip to stick in it.

Hmmm, I guess you are in the experimental boat then. At the forefront of discovery you could say.

I like these challenges, it adds some interest to the build.

Are you stuck with the choice between those Intel parts?
 

blinkenlights

Active Member
May 24, 2019
157
65
28
6. Has any of you been successful in putting the supermicro Epyc board to sleep with linux?
10. If you can point me towards an epyc rome board that you verified having sleep/hiber working with linux, I think I can go for AMD even with some of these MKL limitations.
Hey, speaking of reasons that an X11SRA is sitting on my workbench......... no sleep under CentOS, Fedora, and even Ubuntu live. Sorry, not a green energy freak, but the wife's "desktop" (tower workstation) must be able to suspend and resume properly. After years of teaching the family to suspend the computer, telling them to just leave it running does not fly.

11. What you say about the X11SRA worries me. How is the Asus one? Did you encounter any issue with that board? I decided for the Supermicrosince the Asus is badly reviewed on newegg: ASUS WS C422 PRO/SE ATX Server Motherboard - Newegg.com
Had some early glitches that were resolved with newer BIOS revisions. If you are buying new, chances are it comes loaded with a recent BIOS and any early release issues are resolved - unlike my X11SRA paperweight with the latest BIOS that will not boot properly. To be fair, SMC produced a second revision of the board and I could probably get mine replaced.. my last (consumer) support interaction was not excellent. They claimed "a" pin was bent and wanted to charge nearly $100 for "repairs and diagnostics"... on a board that I paid about $150 for, in order to diagnose the original problem.

Let's break down those three reviews - creeping up on a year old:

The manufacturer responded to the first reviewer, pointing out that the Xeon-W requires an appropriate VROC key to do RAID on the M.2 drives (this is an Intel thing, not Asus). That truly is user error, and the user should have updated the rating to reflect it.

The second review says they cannot get 512GB to work in their WS C422 PRO/SE board, but it works fine in a WS C422 SAGE/10GE - essentially the same board, but with a 10 GigE chipset instead of 1 GigE. My guess? Bent pins. Probably (also) user error.

The third review is typical early production run/BIOS stuff - run into a problem, report it to the manufacturer, they give you a new BIOS, you test it and it either works or does not. Not unique to that manufacturer. Saying that the company has gone down the toilet based on rear I/O shield design seems a bit much.

I say this as someone who has used Supermicro boards and is prepping to sell boards from the X9, X10, and X11 series - they do not always work well as workstations. Won't boot with a soundcard, won't suspend with an NVIDIA graphics card installed, spins up fans with black screen but won't even give POST codes with graphics card installed, BMC keeps messing with fan speeds because they are not 15k RPM server fans, etc. Confident in their products for servers, not as much for desktop/workstation applications.
 

balnazzar

Active Member
Mar 6, 2019
221
30
28
Ok @blinkenlights, your advice is truly useful for me. In fact, two essential requirements for me are the ability to sleep properly and decent workstation usability *with* nvidia cards.

I reckon this takes out the SMC boards for Epyc, leaving as my only options:
1. The Asus WS C422 andthe 8-core Xeon W. It will sleep for sure.
2. The Gigabyte board and the 5120T. Would it sleep? Probably.
3. Maybe a ROMED8 with some cheapo 7252. My guess is that it won't sleep though.

EDIT: I just checked. The Asus board has a peculiar layout such that the last 16x slot won't support a double width card, at least not in my case.
Any other viable c422 option?
 

Styp

Member
Aug 1, 2018
69
21
8
I had a look into that topic recently too. ASUS WS C422 is the one to go for a workstation, you can checkout benchmarks on Puget Systems. I think its the best option for a workstation.

But keep in mind that the platform is old. Depending on your workload a Broadwell E-26xx v4 could be an option too.
What is your workload? How does your load profile look like?
 

balnazzar

Active Member
Mar 6, 2019
221
30
28
Mh.. The platform shouldn't be so old. After all, W-22xx is the current cpu for workstations, and it goes on c422 as well.

I'd avoid broadwell with 2011-3 platforms precisely due to lack of future expansion options.

My typical workload consists in large pandas df processing, data augmentation for vision, and similar.
 

Styp

Member
Aug 1, 2018
69
21
8
I think C422 is as old as X299 - release early 2017. The CPUs are just incremental improvements.

In the end, C422 is a good platform if you really need AVX512 and many PCIe lanes. Costs and technology is another topic.
 

balnazzar

Active Member
Mar 6, 2019
221
30
28
I need sufficient lanes for 3 GPUs. I also need a layout that allows me to install those 3 GPUs without physical interference.
I do not need (at least for now) AVX512, but I really need sleep/hibernation. Furthermore, I need support for RDIMMs, decent (not top-tier) performance in both multi- and single- threaded use cases, and decent MKL performance.

As you may observe, mine are not impossible requisites to satisfy, and still I'm having troubles in selecting the right combination of cpu-mb:
1. No lga3647 ATX mb with three mechanical 16x slots except for the Gigabyte, which is totally unreviewed.
2. No guarantee for sleep/hyber except perhaps for the c422 asus board. The X11sra is also a workstation board but the user above reports it doesn't sleep. Also, the xeons have 48 lanes, which means one GPU will run at 8X.
3. Epyc Rome is the newest platform with stellar price-perf ratio, and plenty of lanes, but such platform doesn't sleep and may have troubles with MKL-intensive tasks.

What the hell... :(
 

Styp

Member
Aug 1, 2018
69
21
8
Just one input, that Patrick gave me a long time ago.
Do you train on multiple GPUs at the same time? Then look maybe into the PCIe topology, I think Broadwell is better in terms of inter pci-e latency and might be beneficial for your use-case.

Nevertheless, if you need 3 GPUs --> WS C422 SAGE/10G | Servers & Workstations | ASUS Global could be an option too.

I am in the same boat, need a new workstation and I am a little undecided which option is the best...
 

blinkenlights

Active Member
May 24, 2019
157
65
28
2. No guarantee for sleep/hyber except perhaps for the c422 asus board. The X11sra is also a workstation board but the user above reports it doesn't sleep. Also, the xeons have 48 lanes, which means one GPU will run at 8X.
And just to be clear, that was my experience shortly before swapping out the X11SRA for the Asus board. I had reached a point of frustration with moving cards around, pulling and re-seating DIMMs, clear the settings, try to boot, clear the settings again, try to boot, figured out the sound card was the reason it did not POST, clear the settings, try the sound card in another slot, get unhelpful advice from Supermicro customer support, etc. and the inability to sleep once I got it running was just the icing on the cake.

The second revision motherboard combined with the recent BIOS might resolve many of these issues. I just know that, for me, the experience was night and day - the Asus board booted on the first try, even though I had to tweak some things to make it completely useful.
 

balnazzar

Active Member
Mar 6, 2019
221
30
28
Just one input, that Patrick gave me a long time ago.
Do you train on multiple GPUs at the same time? Then look maybe into the PCIe topology, I think Broadwell is better in terms of inter pci-e latency and might be beneficial for your use-case.

Nevertheless, if you need 3 GPUs --> WS C422 SAGE/10G | Servers & Workstations | ASUS Global could be an option too.
Yes, I plan to use at least data parallelisation (in not model parallelisation) with Pytorch.. May you elaborate a bit more? It's incredibly hard to find valuable information about such stuff nowdays.
Note also that the sageis a CEB board, which would force me to ditch my case. I'd rather avoid that.

I am in the same boat, need a new workstation and I am a little undecided which option is the best...
Please keep me posted about that, if you want :)
Thanks!
 

balnazzar

Active Member
Mar 6, 2019
221
30
28
And just to be clear, that was my experience shortly before swapping out the X11SRA for the Asus board. I had reached a point of frustration with moving cards around, pulling and re-seating DIMMs, clear the settings, try to boot, clear the settings again, try to boot, figured out the sound card was the reason it did not POST, clear the settings, try the sound card in another slot, get unhelpful advice from Supermicro customer support, etc. and the inability to sleep once I got it running was just the icing on the cake.
Believe me, I know what you are talking about. It's exhausting.