Threadripper or EPYC for a gentoo workstation?

Larco

New Member
Jun 7, 2020
4
0
1
I've been planning for a new computer build for a couple months now. With the new 3990x and EPYC server processors launching, I've been wanting to move away from my current X299 build as my chip has been really tough to cool and my needs have shifted. With this, I've kind of planned out two different builds and choose one of them to go forward with. Here's the current stuff that's confirmed and that I'm NOT changing (Mostly because I already have most of these parts):
Case: Fractal Design 7 XL
Fans: Noctua's A14 Industrial PPC 2k RPM fans
Power Supply: EVGA SuperNOVA T2 1600 W
Cooler: Noctua NH-U14S TR4-SP3.
GPU: AMD Radeon 5700 XT (Still waiting on big navi)
Alright, now here's comes the tough choice. I've been wanting to move to the 3990x 64 core processor for those amazing 128 threads and 256 MB of cache for a gentoo workstation. Not to mention the clocks are pretty dang amazing. However this is where the limitations start to come in. I'm only allowed to put in up to 256 GB of RAM thanks to AMD not allowing for RDIMMs and LRDIMMs on TR. Additionally with TR, I'd only get 4 PCI-E slots (Not enough IMO) and quad channel memory. This is where I started looking into EPYC (Specifically a 7702P) as an alternative. I could go up to 4TB of RAM (More than I'd ever need), 8 channel memory, and with an Asrock Rack board I've been eyeing (Specifically the ROMED8-2T), I could have 7 x16 PCI-E slots. Amazing! The only concerns I have is that I'm not so sure how well pwm fan control works on server mobos since I'm pretty picky when it comes to noise. Additionally, the rear IO is pretty lackluster (That can be solved with expansion cards however), and since practically all epyc mobos lay the socket horizontally, my noctua would pull hot air from my GPU. In theory it should be fine since EPYC is going to be cooler and there's still an exhaust fan, but I'm still somewhat concerned about it. With all of that said, would you guys recommend EPYC or Threadripper for me? Also I'd really appreciate any answers regarding my concerns too such as pwm fan control on server boards.
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,267
428
83
The only concerns I have is that I'm not so sure how well pwm fan control works on server mobos since I'm pretty picky when it comes to noise. ... Also I'd really appreciate any answers regarding my concerns too such as pwm fan control on server boards.
If the IPMI implementation is the same as the other recent ASRock boards, you'll be able to control fan control to a relatively good degree using ipmitool - you can set a custom duty cycle on each fan header (or at least up to six of them).

I don't know whether it's true of your Noctua or not, but mine at least could be oriented N-S or E-W but I suspect that'd be dependent on the motherboard.

Personally I'd go threadripper myself for the higher clocks which are generally better suited for most workstation workloads, but I don't have any tasks myself that would need more than 128GB of RAM and I don't know what your intended workload is.
 

Larco

New Member
Jun 7, 2020
4
0
1
If the IPMI implementation is the same as the other recent ASRock boards, you'll be able to control fan control to a relatively good degree using ipmitool - you can set a custom duty cycle on each fan header (or at least up to six of them).
Good to know, I did a little research beforehand and saw something regarding ipmitool, but wasn't sure if asrock would be similar or the same.
Personally I'd go threadripper myself for the higher clocks which are generally better suited for most workstation workloads, but I don't have any tasks myself that would need more than 128GB of RAM and I don't know what your intended workload is.
My workload is mostly compiling programs on gentoo linux. Having large amounts of high speed ram helps too as I tend to put my compile jobs onto a ram disk which cuts a decent amount of time from compiling vs using SSD cycles. It's also helpful because I want to futureproof a bit and 512GB (Which is the amount I plan to go with if I'm using EPYC) should last me a long time. Like I said before I'm currently on X299 and TR kind of feels like a sidegrade as it's still quad channel, though with the benefit of double the ram and nearly 4 times the cores (I'd still get the same amount of cores on EPYC, just reduced clock speed).
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,267
428
83
Are you compiling just for yourself or are you running a build farm for other people/projects? I've also been a gentoo dabbler for about 15 years and... well, I think a 64P chip would be overkill for just one person. We're just trialling an Epyc 64P in the build lab at work and it's more than the developers can keep up with. When I was testing my new 3650X I did a gentoo install for kicks... I didn't time it, but let's just say it's a far cry from my first install on an Athlon X2. But unless you've got some other task in mind I'm not sure what the purpose of spending the money is.

In terms of keeping writes to a minimum, IIRC keeping /var/tmp/portage on ramdisc/tmpfs (or even better, zram) was all you really needed, and even large projects would fit in to 8GB worth (and likely far less if you used zram).

Don't worry too much about memory channels; not only is Zen 2 much more forgiving of memory speeds, but compiling in general isn't terribly memory sensitive (depending on the code and the compiler of course).

In terms of the ASRock IPMI, on the boards I've used at least you can use the following raws to respectively read and set the duty cycles:
Code:
ipmitool raw 0x3a 0x02
ipmitool raw 0x3a 0x01 0xAA 0xBB 0xCC 0xDD 0xEE 0xFF 0xGG 0xHH
(Where AA-FF represent the first six fan headers in hex from 00 meaning 0% and 64 meaning 100% duty cycle... still not entirely sure what the last two are for)
 

Larco

New Member
Jun 7, 2020
4
0
1
Are you compiling just for yourself or are you running a build farm for other people/projects? I've also been a gentoo dabbler for about 15 years and... well, I think a 64P chip would be overkill for just one person.
At the moment it's pretty much all for myself. Yes I know it's pretty overkill, but my current build was overkill anyway for its time. I've also wanted to dabble in other things such as rendering and video editing, so it's pretty much an all purpose build to handle anything I throw at it. Also I can always drop down to 32 cores on either EPYC or Threadripper if I decide having clock speed is more important.

In terms of keeping writes to a minimum, IIRC keeping /var/tmp/portage on ramdisc/tmpfs (or even better, zram) was all you really needed, and even large projects would fit in to 8GB worth (and likely far less if you used zram).
I already have a tmpfs on portage's build directory, but I haven't heard of zram, I might give that a shot in the future.

Don't worry too much about memory channels; not only is Zen 2 much more forgiving of memory speeds, but compiling in general isn't terribly memory sensitive (depending on the code and the compiler of course).
I initially passed up on 1st gen TR for pickiness on memory, I figured they fixed it for 2nd/3rd gen. Glad to hear it's less picky when it comes to RAM.

In terms of the ASRock IPMI, on the boards I've used at least you can use the following raws to respectively read and set the duty cycles:
Code:
ipmitool raw 0x3a 0x02
ipmitool raw 0x3a 0x01 0xAA 0xBB 0xCC 0xDD 0xEE 0xFF 0xGG 0xHH
(Where AA-FF represent the first six fan headers in hex from 00 meaning 0% and 64 meaning 100% duty cycle... still not entirely sure what the last two are for)
Good to know. I'll try this if I decide to go EPYC. I currently run asrock rack boards on a couple of v2 xeons, and I've been pretty happy with their server boards overall.

As it stands after having this conversation, the main reason for going EPYC would be again the extra memory capacity (Not really needed but would be nice) and extra expansion cards as I currently have my X299 build filled with 2 GPUs, a 40 gigabit network card, and a USB expansion card. And as mentioned before with all TRX boards, they only give 4 PCI-E slots at the max, and I'd be filling all of those ports easily. I've also wanted to step the storage up in my computer as I am a data hoarder who likes fast nvme ssds, and I'm pretty sure EPYC would make that easier because I could add more nvme drives with either expansion cards or slots on the mobo. All of this together, it seems to be coming down to raw expansion vs raw performance, with EPYC meeting raw expansion and Threadripper meeting raw performance.
 

Larco

New Member
Jun 7, 2020
4
0
1
Have you seen the recently announced AsRock Rack TRX40D8-2N2T? If you're going the TR route, consider it. I saw its specs and inmediately fell in love, it has almost everything in the right place.
No I haven't seen it. I was considering going with a Gigabyte AORUS XTREME TRX40 board (iirc) if I went TR because of the 4 slots being separated as well as the 3 M.2 slots, though it is really dang large. I'll be honest I'm not too impressed by that ASRock TRX board, but I do LOVE this Asrock EPYC board that I listed in my original post.