Is building an EPYC 2 Rome workstation as simple as building a PC?

[TacticalCoder]

Hey everybody,

software dev here. I've been building my PCs myself since the 90s but never tried to build a server or a high-end workstation so I'm asking the pros here...

Is building, say, an EPYC 2 Rome workstation as "simple" as finding the list of compatible parts and assembling it, then putting (Debian) Linux on it? Can this basically work on the first try or is this something requiring some wizardry which I may not have?

As a bonus question: if I buy a mobo for a single CPU, like an ASRock RomeD8-2T, do I need a CPU ref specifically meant for one socket configuration? For example would a 7402P work where a 7402 would fail? Or are the 7402 and 7402P basically exactly the same CPU working both in single and dual sockets boards?

Thanks for any help,

 

[i386]

Can this basically work on the first try or is this something requiring some wizardry which I may not have?

It's the (basically) same process
Choose a mainboard + cpu(s), heatsink(s), ram, storage etc, assemble the parts and install your favorite os.

As a bonus question: if I buy a mobo for a single CPU, like an ASRock RomeD8-2T, do I need a CPU ref specifically meant for one socket configuration? For example would a 7402P work where a 7402 would fail? Or are the 7402 and 7402P basically exactly the same CPU working both in single and dual sockets boards?

The dual socket cpus (without the P suffix) work also in single socket configurations. Single socket cpus (with the P suffix) do not work in dual socket systems.

 

[TacticalCoder]

Thanks a lot.

So for example I've never done any IPMI stuff: it's not mandatory to get running? Say to configure the BIOS the first time?

Re- the CPUs: so the single socket CPUs (the ones which do have the 'P' suffix) are usually cheaper? (as they're "less capable", in a way)

 

[i386]

So for example I've never done any IPMI stuff: it's not mandatory to get running? Say to configure the BIOS the first time?

It's not necessary, but it makes the management a lot easier (installing the os via iso, remote bios updates, starting/shutting down the system, system healt data etc.)

Re- the CPUs: so the single socket CPUs (the ones which do have the 'P' suffix) are usually cheaper? (as they're "less capable", in a way)

That probably depends on your source of the cpu.
Here in germany the cheapest epyc single and dual socket rome cpus are on pair pricewise.

 

[TacticalCoder]

Ah interesting. I'm in the EU so I take it my source would be the biggest economy there: so germany too

 

[alex_stief]

For single socket, building such a workstation and getting an OS installed is exactly the same process as for any other PC. It can be a bit more exciting because the components cost a bit more. But since you are not a first-timer, I see no reason why you could not pull it off. If there are any specific questions along the way, you are already in the right forum to get help.
The only thing slightly different is the CPU retention mechanism of SP3. But there are quite a few video tutorials for it.

Some people report problems with memory channels or PCIe slots not showing up first try, which has to be fixed by mounting the CPU again with the "correct" torque settings. I never had that issue, despite hand-tightening the socket screws. So in my opinion, that torque wrench is optional. I did not get one, even when buying "retail" CPUs.
Since you are in the EU, particularly Germany, this site might help you find the cheapest retailers: AMD mit Segment: Server, Architektur: Zen 2 Preisvergleich Geizhals Deutschland

 

[TacticalCoder]

Ah interesting information about the torque and thanks for the vid, bookmarked.

I have already seen "Geizhals" mentioned several time here (I lurked a lot before opening an account) and it looks like a nice price comparator. But what about the individual shops Geizhals shows? Are they all to be trusted? For example I see often "Mindfactory" and "PC King": can I trust these?

 

[alex_stief]

They also have a rating system for the shops listed. It's not perfect. But at least you can read some reviews, and find out why people were not satisfied.
Mindfactory is one of the largest retailer for PC components in Germany. Bought there plenty of times. PC King is not as big, but I would still consider them trustworthy in the sense that you won't get scammed out of your money. Yet from reading a few reviews, it seems like they have some problems managing their inventory.
These are the shops I use most of the time: Mindfactory (usually cheapest), Alternate (more expensive, but more customer-friendly return policy), Jacob Elektronik (sometimes the only choice for more server-y stuff), Aquatuning (water cooling and cables) and if there is really no other option: Amazon.

 

[Wolvez]

One thing to take note of is that most server have limited external I/O. For example, the Romed8-2t only has 3 rear usb ports (2 type A & 1 type C) and 2 front type A usb ports. There is also no sound. IPMI is great for a headless server but isn't as useful for a workstation that has a video card and monitor attached full time.
If you are building a workstation with KB, mouse, sound, video, and monitor you may be better served by the Threadripper offerings. I haven't tried stock Debian on my Epyc build but Proxmox (based on Debian 10 Buster) worked just fine. With stock Buster you may need to run one of the 5.x kernels from backports.

 

[TacticalCoder]

No need for sound: it'd be a workstation on which I wouldn't need sound at all (current PC actually doesn't have any speakers hooked to it). I'd need a GPU though: as you say, indeed, for a monitor attached full time. No need for many USB ports either (I'm basically using only one USB port: for U2F / webauthn authentication).

TR seems to draw a lot of power and the ECC story with TR seems... Complicated? But yes it's an option. From the look of it though it looks it's possible to build an EPYC 2 workstation that is cheaper and quieter than a TR? (TR having higher clockrates)

TR is an option that said. But so far I never had a "real" workstation and I'm kinda interested in building one.

 

[TacticalCoder]

These are the shops I use most of the time: Mindfactory (usually cheapest), Alternate (more expensive, but more customer-friendly return policy), Jacob Elektronik (sometimes the only choice for more server-y stuff), Aquatuning (water cooling and cables) and if there is really no other option: Amazon.

Thanks a lot for that feedback, taking notes!

 

[sovking]

An EPYC build for a workstation how deal with suspension and wake up ? From a previous experience with server hardware, I found that some typical functionality used on workstation may be missing because usually these functionality are not required for servers.

 

[TacticalCoder]

An EPYC build for a workstation how deal with suspension and wake up ? From a previous experience with server hardware, I found that some typical functionality used on workstation may be missing because usually these functionality are not required for servers.

Which functionality in addition to suspend and wake up would be missing? No sound by default ain't an issue in my case: I don't have and don't need sound on the computer on which I work. Suspend and wake up doesn't seem like showblocker to me. FWIW it's intriguing you mention waking up and sleeping because one year ago I actually was the first to solve a 20 years old cryptographic problem (created by Ron Rivest) by basically ... never turning my everyday desktop computer off for four years! (it was literally computing day and night towards the solution).

Several vendors are branding EPYC-based workstation (and there are even specific versions of coolers/fans for AMD EPYC CPUs which are quieter, branded for workstation use of those CPUs). There's an article/vid posted on this very site about "EPYC vs TR for workstations": https://www.servethehome.com/amd-epyc-7002-rome-v-threadripper-for-workstations/

Are AMD EPYC CPUs really not meant to be used in workstations?

 

[alex_stief]

Just out of curiosity, I clicked on "suspend" on my dual Epyc 7551 workstation running OpenSUSE. Worked like a charm, and after pressing the power button, I was greeted with the desktop as I left it just a few seconds later.

Looking at the CPUs alone, they are very similar. After all, it's almost the same silicon in the same package. The pros and cons for each of them are:
Threadripper:
+Higher clock speed
+Overclockable, both cores and memory
-only 4 memory channels
-no support for RDIMM and LRDIMM, effectively limiting maximum memory support to 256GB
-single socket only

Epyc:
+full 8 memory channels, apart from some low-end SKUs
+support for RDIMM and LRDIMM up to several TB of memory
-lower clock speed

There is nothing that outright disqualifies Epycs for "workstation" usage. I am sure quite a few people on this forum are doing exactly that. The only downside is that you don't get the highest possible clock speeds with Epyc. For some, a workstation CPU has to have high clock speeds. But that depends on your use-case.
In theory, Threadripper Pro could fill that gap for single-socket systems. It retains all features of the Epyc lineup, while delivering CPU clock speeds similar to regular Threadripper CPUs. But AMD screwed that up for DIY enthusiasts like us, by releasing them much too late, and only in OEM machines from Lenovo.

Long story short: maybe don't focus too much on labels like "workstation". Better check what you actually need your system to do. Also, don't conflate clock speeds, TDP and cooling. The noise level from a CPU cooling solution depends more on the amount of effort (and to some extent: money) that went into it.

 

[mirrormax]

Just out of curiosity, I clicked on "suspend" on my dual Epyc 7551 workstation running OpenSUSE. Worked like a charm, and after pressing the power button, I was greeted with the desktop as I left it just a few seconds later.

Looking at the CPUs alone, they are very similar. After all, it's almost the same silicon in the same package. The pros and cons for each of them are:
Threadripper:
+Higher clock speed
+Overclockable, both cores and memory
-only 4 memory channels
-no support for RDIMM and LRDIMM, effectively limiting maximum memory support to 256GB
-single socket only

Epyc:
+full 8 memory channels, apart from some low-end SKUs
+support for RDIMM and LRDIMM up to several TB of memory
-lower clock speed

There is nothing that outright disqualifies Epycs for "workstation" usage. I am sure quite a few people on this forum are doing exactly that. The only downside is that you don't get the highest possible clock speeds with Epyc. For some, a workstation CPU has to have high clock speeds. But that depends on your use-case.
In theory, Threadripper Pro could fill that gap for single-socket systems. It retains all features of the Epyc lineup, while delivering CPU clock speeds similar to regular Threadripper CPUs. But AMD screwed that up for DIY enthusiasts like us, by releasing them much too late, and only in OEM machines from Lenovo.

Long story short: maybe don't focus too much on labels like "workstation". Better check what you actually need your system to do. Also, don't conflate clock speeds, TDP and cooling. The noise level from a CPU cooling solution depends more on the amount of effort (and to some extent: money) that went into it.

could argue the lack of motherboards kinda disqualified epyc(i say that running an epyc workstation myself) but the asrock boards seem decent, wouldnt recommend the supermicro ones although they do work(terrible placement of IO headers conflicting with GPU, lack of usb3 headers etc)

 

[alex_stief]

One COULD argue that. That's why I prefaced this section by "looking at the CPUs alone". I am well aware of the shitshow that is motherboards for AMD Epyc, or lack thereof.
But then again, your platform choice should depend more on what you actually need, not on what someone else arbitrarily defined as must-haves for your use case.
For now, I am fine with not having front USB 3.0. And If I ever need it: PCIe expansion card. That might not work for other users who e.g. need lots of GPUs in their workstation. Which brings me back to my initial conclusion: look at what you actually need, not at what others consider necessary.

 

[Wolvez]

Are AMD EPYC CPUs really not meant to be used in workstations?

From the way you phrased your original question I wasn't sure if you knew what you were getting into. It seems like you know what your requirements are and the downsides of EPYC. EPYC will work just fine for you.

 

[TacticalCoder]

could argue the lack of motherboards kinda disqualified epyc(i say that running an epyc workstation myself) but the asrock boards seem decent, wouldnt recommend the supermicro ones although they do work(terrible placement of IO headers conflicting with GPU, lack of usb3 headers etc)

A thanks for jumping in. Do you have a link to a thread or could you post here the exact parts you used for your build?

 

[mirrormax]

A thanks for jumping in. Do you have a link to a thread or could you post here the exact parts you used for your build?

Been meaning to make a build log but here's the basics parts for now.

supermicro H11DSi-nt
16*16gb 3200cl22 Samsung dual rank M393A2K43DB3-CWE
2x NH-U14S TR4-SP3
Fractal Design Define 7 XL
Corsair 1600w

 

[alex_stief]

I'd love to see that build log too. Especially how the board fits into that case, and how much space is left to build around it