My September 2017 Workstation Build

Patrick

Administrator
Staff member
Dec 21, 2010
12,003
4,987
113
Over the labor day weekend I have been working on a new project: building a new workstation. This is the first time in years that I will be moving to a single socket system. For this edition, AMD Threadripper. I think it is important that if I am going to publicly recommend AMD products, that I actually use them.

Another change is going to be that I will be moving away from the massive internal SSD array on the previous generation system. As an aside, there are going to be 2x Intel Xeon E5-2690 V3's and 2x Intel Xeon E5-2698 V4's freed up in the lab soon, so there was a valid question regarding whether the upgrade should just be updating CPUs again.

Historical Accounts and Comments on the May 2015 Build

Here are the previous builds:
The E5-2650 V3's gave way to the Intel Xeon E5-2690 V3's as faster chips became an option. I sold a pair of Intel Xeon E5-2690 V3's and ended up pulling them from the May 2015 build a few months ago. As a result, I simply pulled a pair of Xeon E5-2658 V3's that were free and started using them. I have not noticed much single threaded performance hit. The GPU was upgraded from the GTX 970 to the GTX 1070 to a GTX 1080. Cards like the Flashmax gave way to simply using a Samsung 960 Pro in an add-on card.

While both CPUs and GPUs went through major upgrades, the platform has remained an extremely quiet build. Frankly, it is nice to have sockets with a massive range of CPUs. Also, I am going to miss being able to use 8x DIMMs. My workstation has become the graveyard for 8GB DDR4 ECC RDIMMs as 16GB has become the lab's standard small capacity.

Introducing the September 2017 Build

This time, I decided to build a machine that is more of a "high-end consumer" workstation.
The $1000 Threadripper was tough to swallow. 16 cores with only 4 DDR4 channels and dealing with NUMA may seem great for someone coming from a mid-range build, but in 2013 using 16 cores that were lower clocked had 4 DDR4 channels per 8 core CPU/ NUMA node.

DeepLearning10 may be decommissioned this week. If so, there is a good chance I will pull the Gigabyte 1080 Ti in as the GPU, although the GTX 1080 is doing well.

Storage is a major change. Moving most of the storage to a NAS that is also on a ConnectX-3 segment. Now that we have ~100 nodes in the main STH lab at any given time, I am using local storage less. If you are wondering why I am using those SSDs it is simple, I had them on hand. 512GB seemed too small for what I wanted to use.

I did want to thank @William for helping me on some of the components. I knew he was building a Ryzen Threadripper system, so his advice on parts like the Thermaltake cooler was invaluable.

For those wondering, the #1 reason for doing this is that I did not feel comfortable recommending AMD gear if I was not willing to make the switch myself. I actually built myself a really nice Ryzen 7 system but it did not have enough power and expansion for what I need. I am hoping the Threadripper remedies that.

The September 2017 Build


September 2, 2017 Progress
I thought I had a Corsair H100i V2 around. Turned out it was a H80i. Gave up and just bought a giant Thermaltake to arrive on September 3 in the evening.


September 3, 2017 Progress
Morning is here. System is ready to go aside from the CPU cooler. I am going to try the Dell PCIe 3.0 x16 to 4x SFF-8643 card in this system to power the two Samsung XS1715 SSDs.



Just waiting for the Amazon Prime delivery man. Tried to source the cooler or the H115i at Fry's to no avail.

Update: Amazon came. Thermaltake installed. Booted first try! The Dell NVMe card did not work so switched to the two port Supermicro. The entire setup is not silent, but very quiet. I still need to work on the LED lighting. Going to get RAM to DDR4-2400 then try some benchmarks for a burn-in.

September 4, 2017 Progress
Linux benchmarking underway
Windows 10 initial installation.

September 5, 2017 Progress
Applications installed. Was able to use 6x 16GB DIMMs, not of the same type successfully for 96GB.


Cutover this evening. Will swap video cards and such then.
 
Last edited:

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,053
1,598
113
CA
Are you running VMs on your desktop or why do you need so much power in a desktop when you have 100+ nodes in the lab?
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,003
4,987
113
Are you running VMs on your desktop or why do you need so much power in a desktop when you have 100+ nodes in the lab?
I do run a lot of VMs, and Docker containers on the desktop. It does just about everything from Adobe Premiere Pro CC to hosting VMs. Frankly, moving down to 64GB RAM at the start is going to be my biggest issue/ worry.
 
  • Like
Reactions: William

William

Well-Known Member
May 7, 2015
785
250
63
63
Nice Patrick !

I am also moving to a TR system as soon as reviews are done.
My old system which has been running well for years is 2x E5-2699v3's, ASUS Z10PE-D16 WS, 256GB RAM etc., will be parted out after. This is a beast of a system and truly nothing wrong with it. But I do like the features X399 platform has.

I am still testing things on a new board but I love this platform. As far as cooling... the Thermaltake Water 3 Ultimate seems to handle TR just fine. Until we get full TR blocks on actual water coolers get a Triple Rad AIO for best results. Air coolers do not fair well on TR under loads.

If you are using the board I sent you Patrick, its very good. Oh I see you are using that one :)

I am not a RGB guy, but everything is RGB now so I am going to try and embrace it LOL

I am undecided on a case. I tend to prefer the huge monsters like a Thermaltake Core WP100 Super Tower Chassis but a Core X5 ATX Cube Chassis will work just as well.
 

Davewolfs

Active Member
Aug 6, 2015
337
31
28
Nice Patrick !

I am also moving to a TR system as soon as reviews are done.
My old system which has been running well for years is 2x E5-2699v3's, ASUS Z10PE-D16 WS, 256GB RAM etc., will be parted out after. This is a beast of a system and truly nothing wrong with it. But I do like the features X399 platform has.

I am still testing things on a new board but I love this platform. As far as cooling... the Thermaltake Water 3 Ultimate seems to handle TR just fine. Until we get full TR blocks on actual water coolers get a Triple Rad AIO for best results. Air coolers do not fair well on TR under loads.

If you are using the board I sent you Patrick, its very good. Oh I see you are using that one :)

I am not a RGB guy, but everything is RGB now so I am going to try and embrace it LOL

I am undecided on a case. I tend to prefer the huge monsters like a Thermaltake Core WP100 Super Tower Chassis but a Core X5 ATX Cube Chassis will work just as well.
Why move from E5-2699 V3s to TR? Isn't that a step backwards?
 

Stephan

IT Professional
Apr 21, 2017
150
61
28
Bavaria, Germany
A bit daring to call this a "workstation build" without ECC. ;-) But I can feel the pain about memory support. In Europe we call systems like this "bananaware": Delivered green, ripens at customer site.
 
  • Like
Reactions: Stux

Davewolfs

Active Member
Aug 6, 2015
337
31
28
Dual socket E5 V3/ V4 has a minimum of 8 DIMM slots on the board and can take ECC RDIMMs. I have a TON of ECC RDIMMs but memory support on X399 is dumfounding at best. If I could put 8x 32GB RDIMMs I have around in it, I would be extremely happy.
I'm surprised that DDR4 prices haven't come down - there is a large premium for Unbuffered ECC DDR4 vs what I paid for my DDR3.
 

William

Well-Known Member
May 7, 2015
785
250
63
63
Dual socket E5 V3/ V4 has a minimum of 8 DIMM slots on the board and can take ECC RDIMMs. I have a TON of ECC RDIMMs but memory support on X399 is dumfounding at best. If I could put 8x 32GB RDIMMs I have around in it, I would be extremely happy.
I know what you mean, I have a bunch of 16GB sticks I would love to use :(
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,003
4,987
113
RAM and NAND prices are not coming down. @Stephan agreed on the lack of ECC front.

While waiting for the cooler, I have been considering swapping everything into a NZXt s340. I have a Corsair H80i V2 but I am a bit worried about the noise using that.
 

wildpig1234

Well-Known Member
Aug 22, 2016
1,833
273
83
45
Why move from E5-2699 V3s to TR? Isn't that a step backwards?
That's the only reason why my current build is dual 2686 v3 since this configuration would still beat 1950x for full multicore use. You would also have to be willing to use QS/ES cpu.

So the current situation right now is that just about the only affordable solution to possibly equal 1950x in multithread for much cheaper is by using dual 2696 v2 (cost of 2 cpu around $620 with much cheaper memory) but you have really have no other cpu to upgrade your 2011 for faster speed..... Additionally 2696 v2 does not work with intel S2600 MB so you also have to buy another MB from supermicron, tyan

If you want to have the ability to go faster than 1950x for multithread, you have to use 2011 v3. The cost of the ram are significantly more. The cpu cost also increase significantly. MB price is same or possibly slightly more than 2011. My current build when finished will have dual 2686 v3 which will definitely beat 1950x multithread by a significant margin. the cost of the 2 cpu is the same as 1950x. so you are paying the same and getting maybe 15-20% better multithread performance.

So yes, I do think that moving from dual 2699 v3 to TR is a downgrade in total multithread performance. The only reason is that because you can have more than 1 cpu with 2011 v3 whereas 1950x is strictly single cpu.

But if you are using just about any intel cpu in single cpu configuration, 1950x is actually an upgrade. even the most expensive e5-26xx v4 is still slower in single cpu than 1950x for multithread.