My December 2019 Workstation Build

[Patrick]

Making a quick plug here.

Historical Accounts and Comments on the September 2017 Build

Here are the previous builds:

• My September 2017 Workstation build
• My May 2015 Workstation Build
• My December 2013 Workstation Build

Overall, the Threadripper 1950X platform has worked great. I have no complaints about the setup. The reason for upgrading is that there is an opportunity for 2x the cores and a single NUMA node. I also want PCIe Gen4. At some point, it is just time.

Looking back on my notes from two years ago, this one made me smile as HEDT CPU prices have increased: "The $1000 Threadripper was tough to swallow." The core wars are on.

Introducing the December 2019 Build

I decided to stick with the HEDT segment for this one.

• CPU - AMD Ryzen Threadripper 3970X
• Motherboard - MSI TR40 Creator
• RAM: 128GB (4x 32GB although this may change)
• Chassis - RAIJINTEK ASTERION PLUS
• Cooling - CORSAIR HYDRO Series H115i PRO RGB
• GPU - Gigabyte 1080 Ti to be upgraded
• NIC - Mellanox ConnectX-3 VPI w/ QSFP+ to SFP+ adapter / to be upgraded to 25GbE ConnectX-4 Lx later
• Storage - 1x Sabrent 1TB PCIe Gen4 NVMe SSD - hoping to get 4x4TB NVMe U.2 drives for data, but may continue with everything on the NAS
• PSU - EVGA 1000W G3 PSU

There are still a few bits that I need to nail down but this seems about right. Although I added more NVMe storage in my current workstation, realistically I launch programs from my OS drive then all of the data sits on NAS devices so I think it is time to get rid of local storage.

I am also continuing the "eat your own dog food" tradition. If we recommend EPYC Rome, I am going to use the desktop derivative daily.

More coming.

 

[Laugh|nGMan]

The unanswered questions for me is......:
1) does this build is intended for local or remote storage?

2) imho E-ATX MB form factor with ONLY 4 slots is lame, then better own ATX MB with 7 slots and for example - Raijintek THETIS case. Much smaller case with bit less cooling for sure.

3)Can u use 3 VGA(one primary and two for compute needs) + 1 NIC + 1 SAS card......probably not. Not enough slots

4) does this board have SLI certified mb with all sli quadro bios certificate bullshit

5) how much RAM bandwidth MB have with all channels populated? Does that enough to push full traffic with 100GBE MLX5 or MLX4 nics? CPU to 100GBE NIC traffic.....bandwidth, maybe there is some bottleneck? Or we need, still, to wait for sTRX8 for that?

6) what about NVMeOF.......can we push in future 100GBE or 200GBE thru this MB to remote storage in home?

7) RAM CAPACITY....128GB....how much is optimal for RAM drive purpose nowadays in short term/long term? For Example, after few years i want to put RedDeadRedemption3 on my RAMdrive to toying with it, its my homelab right....oh, then i probably need RAM twice as much- 256 GB....damn all slots populated and ram downclocks losing bw.

8) if mb+cpu combo is top-notch, then lifetime for this combo is 6-8 years for "old farts" (like me). Typing on sandy-ivy bridge xeon now.

No offense.

 

[Patrick]

That is a good list:

1. Noted above, but usually OS/ applications locally, then data on the NAS. It makes upgrading very easy.
2. I like having the x16's plus M.2 on the motherboard actually. On the 280W TDP CPUs and with a big GPU, I want more airflow. Plus, I am re-using the Asterion.
3. I usually do single GPU builds these days. More GPUs in a build means it ends up in another box that is not within earshot. NIC is important to me. SAS is useless to me in a workstation these days. Everything has been SSD for years and now it is all NVMe SSD. I actually would not mind losing SATA even.
4. I pulled a Titan RTX today. Still deciding if this is one Titan RTX or just a RTX 2070. For Adobe and what I use it for, this is fine.
5. Plenty for 100GbE.
6. Yes.
7. 128GB I am not a huge fan of. I have 8x 16GB but I am going to use a 4x 32GB kit in the event I want to upgrade later.
8. Likely the case. At the same time, I hit some days where I simply need more productivity. An example is the other day when I shot the MikroTik video, export/ encoded it, then realized I needed a fix and had to do it again. It is not bad on the 1950X, but I expect it to be better on the 3970X.

 

[William]

Nice setup Patrick

Been using that NZXT Kracken X62 on the 3960X. It does get hot.
Thinking of moving up to a higher-end cooler, triple rad AIO or even an actual water cooling setup but that's a bit too costly for me, I don't know.

 

[Laugh|nGMan]

Tnx Patrick for reply.
3. How about multiple vga (quadro p1000 with pair quadro P4000) ....mine preference to use single slot vga cards, that's why I mentioned 7 slot mb. With silicon shrinking even more in future, single-slot vga should do pretty good and have the decent performance for home lab. Enough horsepower for average editing in Davinci Resolve in future.

 

[T_Minus]

Nice new build @Patrick

I've decided my next build this month for my folks will be AMD, it's been almost 20 years since my last AMD build... rather excited

 

[Patrick]

This board can do 2x dual width GPUs. Frankly, with the Titan RTX, you get a ton of performance per slot.

I may move down to the RTX 2070 just for lower power.

@William not that Water 3.0 right?

 

[William]

Thermaltake Water 3.0 would work, but I would want something with a bit more punch to it... err cooling capacity
The Tripple Rad Kracken would be better I think.

A full-blown EKWB setup is like $536 ! ouch.

On my Kracken X62 I have to set it to Performance mode and then I adjusted the curves for more fan action. If I leave it on silent mode an alarm goes off after a bit, I believe it means the 3960X is running too hot when that happens.

I drool over that Titan RTX

 

[Patrick]

Ok. It is alive. Rough build.

A few notes:

1. Corsair H115i Pro does not use the same waterblock/ pump module so you need an extra bracket. Fry's did not have it, Central Computers did not have it and no one day Amazon. I ended up buying a NZXT Kraken X62 since it was there.
2. 8TB Intel DC P4510 was not working. The 4x 4TB came in, but I decided to just use 2x 1.92TB Samsung drives for local scratch. That is still 3.49TB formatted with NTFS striped in Windows 10. The Sabrent 1TB was DOA. Rough storage day for sure!
3. The Oloy (Yolo backward?) 4x 32GB kit is working great at DDR4-3200 XMP. They are not RGB, they do not look particularly great, but they work and were inexpensive.
4. Added a Supermicro PCIe to SFF-8643 card to get two connections for U.2 drives. I also changed to a M.2 to U.2 converter for the Samsung 800GB OS boot drive.

 

[William]

I like that U.2 expansion card. I picked up a single port U.2 card which works fine.
Do you have a link to the one you used ?

 

[TXAG26]

I like that snazzy STH sticker!

 

[frogtech]

I like that U.2 expansion card. I picked up a single port U.2 card which works fine.
Do you have a link to the one you used ?

It looks like a Supermicro AOC-SLG3-2E4.

 

[William]

Yeah I was just google that and came up with the same link.

 

[Patrick]

I like that snazzy STH sticker!

STH logo magnet!

 

[msvirtualguy]

I need to ditch the older PCI-E AIC 750 and replace with that U2 card and grab two of those DC P3520's off of ebay.

I was thinking of grabbing the Asus or Asrock M.2 NVMe 4 slot add-in to replace it but the cost of the M.2 NVMe drives, especially in the Samsung variant are a little steep. The above might be a better option.

 

[Patrick]

So there is a bit to this @msvirtualguy that is "what I had lying around". I would have used the non-PCIe switch version if I had it since the board can bifurcate to x4x4 in that slot. Also, you can see I am using a M.2 adapter plus cable. It is ~$35 per port for those but higher-end boards today have 3+ M.2 slots so it is fairly easy to swap.

The big question I have is whether I keep 2x 1.92TB SSDs or I just move up to 3.84TB/4TB SSDs and have a lot more local scratch storage.

 

[jpmomo]

The current build I am working on took the epyc route vs threadripper. It is based on the sm h12ssw-in mb that has 3 x 16 pcie 4.0 slots and 2 x m.2 pcie 4.0 slots. I have a specific requirement to keep the form factor as small as possible and don't need any discreet gpu so a bit different than your build goals. I am using the 7452 with special sw that runs the cpus at 100% during certain modes. it is surprising easy to keep cool (just using a noctua a12x25 on top of a dynantron copper heatsink.) Did you consider an epyc rome vs the threadripper? AMD has sure changed our options for building these devices!

 

[Patrick]

The current build I am working on took the epyc route vs threadripper. It is based on the sm h12ssw-in mb that has 3 x 16 pcie 4.0 slots and 2 x m.2 pcie 4.0 slots. I have a specific requirement to keep the form factor as small as possible and don't need any discreet gpu so a bit different than your build goals. I am using the 7452 with special sw that runs the cpus at 100% during certain modes. it is surprising easy to keep cool (just using a noctua a12x25 on top of a dynantron copper heatsink.) Did you consider an epyc rome vs the threadripper? AMD has sure changed our options for building these devices!

Yes. There are a few major considerations (actually this was going to be a weekend article.)

• TR is has higher clock speeds. For a workstation, that still matters.
• The motherboard options for TR have features such as Wi-Fi, Bluetooth, and Type-C connectors built-in
• I like having audio, although I am usually on HDMI or USB headsets/ mics

For me at least, TR nailed what I need. We have plenty of EPYC chips, but the higher single/ few core clock speeds were what pushed me to TR.

 

[SRussell]

Very clean cabling.

 

[Patrick]

@SRussell compared to many on here I think I do a poor job at cabling, especially in a 2 year old system.

This thread just made a cameo on the main site https://www.servethehome.com/amd-epyc-7002-rome-v-threadripper-for-workstations/ and in the associated YouTube video: