Over the labor day weekend I have been working on a new project: building a new workstation. This is the first time in years that I will be moving to a single socket system. For this edition, AMD Threadripper. I think it is important that if I am going to publicly recommend AMD products, that I actually use them.
Another change is going to be that I will be moving away from the massive internal SSD array on the previous generation system. As an aside, there are going to be 2x Intel Xeon E5-2690 V3's and 2x Intel Xeon E5-2698 V4's freed up in the lab soon, so there was a valid question regarding whether the upgrade should just be updating CPUs again.
Historical Accounts and Comments on the May 2015 Build
Here are the previous builds:
The E5-2650 V3's gave way to the Intel Xeon E5-2690 V3's as faster chips became an option. I sold a pair of Intel Xeon E5-2690 V3's and ended up pulling them from the May 2015 build a few months ago. As a result, I simply pulled a pair of Xeon E5-2658 V3's that were free and started using them. I have not noticed much single threaded performance hit. The GPU was upgraded from the GTX 970 to the GTX 1070 to a GTX 1080. Cards like the Flashmax gave way to simply using a Samsung 960 Pro in an add-on card.
While both CPUs and GPUs went through major upgrades, the platform has remained an extremely quiet build. Frankly, it is nice to have sockets with a massive range of CPUs. Also, I am going to miss being able to use 8x DIMMs. My workstation has become the graveyard for 8GB DDR4 ECC RDIMMs as 16GB has become the lab's standard small capacity.
Introducing the September 2017 Build
This time, I decided to build a machine that is more of a "high-end consumer" workstation.
DeepLearning10 may be decommissioned this week. If so, there is a good chance I will pull the Gigabyte 1080 Ti in as the GPU, although the GTX 1080 is doing well.
Storage is a major change. Moving most of the storage to a NAS that is also on a ConnectX-3 segment. Now that we have ~100 nodes in the main STH lab at any given time, I am using local storage less. If you are wondering why I am using those SSDs it is simple, I had them on hand. 512GB seemed too small for what I wanted to use.
I did want to thank @William for helping me on some of the components. I knew he was building a Ryzen Threadripper system, so his advice on parts like the Thermaltake cooler was invaluable.
For those wondering, the #1 reason for doing this is that I did not feel comfortable recommending AMD gear if I was not willing to make the switch myself. I actually built myself a really nice Ryzen 7 system but it did not have enough power and expansion for what I need. I am hoping the Threadripper remedies that.
The September 2017 Build
September 2, 2017 Progress
I thought I had a Corsair H100i V2 around. Turned out it was a H80i. Gave up and just bought a giant Thermaltake to arrive on September 3 in the evening.
September 3, 2017 Progress
Morning is here. System is ready to go aside from the CPU cooler. I am going to try the Dell PCIe 3.0 x16 to 4x SFF-8643 card in this system to power the two Samsung XS1715 SSDs.
Just waiting for the Amazon Prime delivery man. Tried to source the cooler or the H115i at Fry's to no avail.
Update: Amazon came. Thermaltake installed. Booted first try! The Dell NVMe card did not work so switched to the two port Supermicro. The entire setup is not silent, but very quiet. I still need to work on the LED lighting. Going to get RAM to DDR4-2400 then try some benchmarks for a burn-in.
September 4, 2017 Progress
Linux benchmarking underway
Windows 10 initial installation.
September 5, 2017 Progress
Applications installed. Was able to use 6x 16GB DIMMs, not of the same type successfully for 96GB.
Cutover this evening. Will swap video cards and such then.
Another change is going to be that I will be moving away from the massive internal SSD array on the previous generation system. As an aside, there are going to be 2x Intel Xeon E5-2690 V3's and 2x Intel Xeon E5-2698 V4's freed up in the lab soon, so there was a valid question regarding whether the upgrade should just be updating CPUs again.
Historical Accounts and Comments on the May 2015 Build
Here are the previous builds:
The E5-2650 V3's gave way to the Intel Xeon E5-2690 V3's as faster chips became an option. I sold a pair of Intel Xeon E5-2690 V3's and ended up pulling them from the May 2015 build a few months ago. As a result, I simply pulled a pair of Xeon E5-2658 V3's that were free and started using them. I have not noticed much single threaded performance hit. The GPU was upgraded from the GTX 970 to the GTX 1070 to a GTX 1080. Cards like the Flashmax gave way to simply using a Samsung 960 Pro in an add-on card.
While both CPUs and GPUs went through major upgrades, the platform has remained an extremely quiet build. Frankly, it is nice to have sockets with a massive range of CPUs. Also, I am going to miss being able to use 8x DIMMs. My workstation has become the graveyard for 8GB DDR4 ECC RDIMMs as 16GB has become the lab's standard small capacity.
Introducing the September 2017 Build
This time, I decided to build a machine that is more of a "high-end consumer" workstation.
- CPU - AMD Ryzen Threadripper 1950X
- Motherboard - MSI X399 Gaming Pro Carbon
- RAM: 112GB (4x 16GB Patriot, 2x 16GB Corsair, 2x 8GB Patriot) aka what I had laying around
- Chassis - RAIJINTEK ASTERION PLUS
- Cooling - Thermaltake WATER 3.0 Triple Riing RGB
- GPU - Gigabyte GTX 1080
- NIC - Mellanox ConnectX-3 VPI w/ QSFP+ to SFP+ adapter in one slot for 10Gb SFP+ connectivity
- Storage - 2x Samsung PM1725 800GB SSDs, Samsung 950 Pro 512GB
- PSU - EVGA 1000W G3 PSU
DeepLearning10 may be decommissioned this week. If so, there is a good chance I will pull the Gigabyte 1080 Ti in as the GPU, although the GTX 1080 is doing well.
Storage is a major change. Moving most of the storage to a NAS that is also on a ConnectX-3 segment. Now that we have ~100 nodes in the main STH lab at any given time, I am using local storage less. If you are wondering why I am using those SSDs it is simple, I had them on hand. 512GB seemed too small for what I wanted to use.
I did want to thank @William for helping me on some of the components. I knew he was building a Ryzen Threadripper system, so his advice on parts like the Thermaltake cooler was invaluable.
For those wondering, the #1 reason for doing this is that I did not feel comfortable recommending AMD gear if I was not willing to make the switch myself. I actually built myself a really nice Ryzen 7 system but it did not have enough power and expansion for what I need. I am hoping the Threadripper remedies that.
The September 2017 Build
September 2, 2017 Progress
I thought I had a Corsair H100i V2 around. Turned out it was a H80i. Gave up and just bought a giant Thermaltake to arrive on September 3 in the evening.
September 3, 2017 Progress
Morning is here. System is ready to go aside from the CPU cooler. I am going to try the Dell PCIe 3.0 x16 to 4x SFF-8643 card in this system to power the two Samsung XS1715 SSDs.
Just waiting for the Amazon Prime delivery man. Tried to source the cooler or the H115i at Fry's to no avail.
Update: Amazon came. Thermaltake installed. Booted first try! The Dell NVMe card did not work so switched to the two port Supermicro. The entire setup is not silent, but very quiet. I still need to work on the LED lighting. Going to get RAM to DDR4-2400 then try some benchmarks for a burn-in.
September 4, 2017 Progress
Linux benchmarking underway
Windows 10 initial installation.
September 5, 2017 Progress
Applications installed. Was able to use 6x 16GB DIMMs, not of the same type successfully for 96GB.
Cutover this evening. Will swap video cards and such then.
Last edited:
