Epyc Homeserver

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
N

[Nobody]

Member
Nov 1, 2020
36
4
8
Hi,
I want to upgrade my R710 with two L5640s and 144 GB of RAM. I'm running some webservices, but I want to run some Windows VMs for gaming and video editing as well.
I want to use a Quadro P620 and (for now) a GTX 1650. The Quadro is for Cuda encoding in LXC (pytorch, jellyfin), while the GTX 1650 will be passthroughed to a Windows VM. The host-OS is Proxmox. The Case is able to host ATX and E-ATX (Rosewill l4500).

I'm planning to buy:
Epyc 7252 8c/16t
(at first) 2*32 GB DDR4 3200 Reg ECC of Kingston
but with the motherboard I'm not sure.
Should I go for
ASRock Rack ROMED8-2T (tons of PCI-E 4 16x, Oculinks, 2* NVME, 2* 10 GbE)
or
ASUS KRPA-U16 (16x RAM)
or
Supermicro H11SSL-NC? (???)

For the Asrock I read about some problems (with NVME, SATA, IOMMU and Fan-Controls), the Asus seams to be limiting the length of the graphics cards (somewhen I upgrade to something bigger GPU like a RTX3060 or such thing). And about the Supermicro I read about some BIOS problems.
I prefer the ASRock Rack, but I'm afraid about these problems. Which one would you recommend?

And what do you think about the CPU? Most of the time the server will run at 3-5 % load, only so 1-2 times a week I need high computing power for gaming or video editing. And the Epyc will be 4x faster than my dual L5640 with less power usage.
 
C

csp-guy

Active Member
Jun 26, 2019
372
153
43
Hungary, Budapest
Hi.

I have a 7301 for sale, if you are interested, 16 cores, 32 threads. :)

I am looking for ASRock Rack ROMED8-2T too, in my opinion it is a very well designed motherboard, 620EUR price. The seven PCIE slots are very impressive.

But I want to use as workstation, insead my pc, equipped with lot of nvme drives.

You can not use over 225W tdp Epyc cpus, but who cares?
 
Last edited:
H

hmw

Active Member
Apr 29, 2019
581
231
43
[Nobody] said:
And the Epyc will be 4x faster than my dual L5640 with less power usage.
Click to expand...
Are your L5640's loaded all the time? Intel has some amazing idle power figures. EPYC boards idle quite high. My 7302P/S8030 combo idles at 100W
 
H

hmw

Active Member
Apr 29, 2019
581
231
43
I'm running ESXi 7U1. My build is here: https://forums.servethehome.com/index.php?threads/tyan-s8030gm2ne.28914/

Keep in mind that the ECC RAM, Mellanox cards, LSI HBA - all are relatively power hungry. And EPYC has far higher idle figures compared to Intel. Where it shines is when it is under 100% load. You will see Intel CPUs pulling insane amounts of wattage - but the AMD chip will not go above the 180W it is meant for. If you're running the machines at 50%-70% load all the time, EPYC is fantastic. If the machines are lightly loaded and you're interested in having lower power draw, wait for the 7xx3 chips or then look at Ryzen/ThreadRipper - as these have lower idle figures than EPYC (but still far higher than Intel)
 
N

[Nobody]

Member
Nov 1, 2020
36
4
8
Yeah I saw the 7262, but it is 180 € more than a 7252.
ESxi isn't known for it's power savings.
Does anyone here have a Epyc 7002 Single CPU system running and may tell me about the power consumption?
 
I

i386

Well-Known Member
Mar 18, 2016
4,243
1,546
113
34
Germany
[Nobody] said:
ASRock Rack ROMED8-2T (tons of PCI-E 4 16x, Oculinks, 2* NVME, 2* 10 GbE)
or
ASUS KRPA-U16 (16x RAM)
or
Supermicro H11SSL-NC? (???)
Click to expand...
Of these 3 choices I would go supermicro > asus > asrock

Why not asrock? Too many problems reported here and other forums about their epyc boards.
Why not asus? To be fair I don't know if their enterprise stuff has better quality, but their consumer stuff has many problems with broken sensors causing alerts, weird default bios settings (eg fans are too slow and go straight to jet mode under medium load) and bad customer service.
 
H

hmw

Active Member
Apr 29, 2019
581
231
43
[Nobody] said:
ESxi isn't known for it's power savings.
Click to expand...
I beg to differ. You can turn on power management in ESXi to get it to match the idle usage you see in Windows for example. As with any hypervisor you will take a hit in performance but if lowering idle usage is your goal, it is certainly possible with ESXi and well documented

docs.vmware.com

Host Power Management Policies

You can apply several power management features in ESXi that the host hardware provides to adjust the balance between performance and power. You can control how ESXi uses these features by selecting a power management policy.
docs.vmware.com docs.vmware.com
 
Dolpa

Dolpa

Member
Apr 27, 2017
54
30
18
Hungary
Hi Folks,

I'm new here, but I have some experience with ASRock Rack ROMED8-2T.
I have been using it on a daily basis for about 2 months as my primary home server and I love it. I don't know what type of problems has peoples with this MB but yes its a different style, not a Supermicro MB (I had it before)

What's I like:
- Lot of 7 PCIE x16 slots
- 2x10GB X550 lan
- onboard 2x4 SATA port, 2x NVME, 2X OCulink for U2
- very simple and nice HTML5 IPMI web interface

What's I dislike:
- missing fans controlling in web interface (bur working nice with ipmitools in linux)

That's it, working NVME, SATA ports, IOMMU and X550 SR-IOV (2x64 virtual X550 ethernet ). I have an Asus HYPER M.2 X16 Gen 4 4x M.2 working PCIE 4x4 bifurcation in all slots. Working PCIE passthrough GPU to my windows and linux desktop.
 
  • Like
Reactions: Erzdorf, vkEnthusiast and Magic8Ball
Erzdorf

Erzdorf

Member
Apr 11, 2017
30
3
8
Dolpa said:
Hi Folks,

I'm new here, but I have some experience with ASRock Rack ROMED8-2T.
I have been using it on a daily basis for about 2 months as my primary home server and I love it. I don't know what type of problems has peoples with this MB but yes its a different style, not a Supermicro MB (I had it before)

What's I like:
- Lot of 7 PCIE x16 slots
- 2x10GB X550 lan
- onboard 2x4 SATA port, 2x NVME, 2X OCulink for U2
- very simple and nice HTML5 IPMI web interface

What's I dislike:
- missing fans controlling in web interface (bur working nice with ipmitools in linux)

That's it, working NVME, SATA ports, IOMMU and X550 SR-IOV (2x64 virtual X550 ethernet ). I have an Asus HYPER M.2 X16 Gen 4 4x M.2 working PCIE 4x4 bifurcation in all slots. Working PCIE passthrough GPU to my windows and linux desktop.
Click to expand...
Hi @Dolpa,
Hope your ROMED8-2T build is still going strong.
Could you tell me how you set your Asus HYPER M.2 X16 Gen 4 4x M.2 up? - I have it installed and bifurcated, having now 4 NVMe drives in BIOS but cannot see the RAID Mode to set up the PCI slot.
As I went through the BIOS settings I have couple of questions:

1. Has "PCI Subsystem Settings>SR-IOV Support" anything to do with it, cos it's DISABLED?

2. Should "PCIE SLOT OpROM" be in UEFI or AUTO? - As it says, AUTO disables NVMe devices and is LEGACY with other devices.

I was expecting to see the PCI RAID Mode option in subOS level in BIOS but just can't find it.

Köszönöm Szépen!
 
D

Dreece

Active Member
Jan 22, 2019
503
161
43
You won't see a raid mode per-se in the bios on that board. Once you set the pcie slot assigned to the hyper card to x4x4x4x4 (bifurcate), then you need to define your raid at os software level, be it linux or windows.
 
  • Like
Reactions: Erzdorf
Erzdorf

Erzdorf

Member
Apr 11, 2017
30
3
8
Thank you @Dreece,
I found out online about that, I just didn't know ROMED8-2T didn't have PCIE RAID Mode on h/w level.
Is AMD s/w RAID any good for this particular case?
Many thanks.
 
D

Dreece

Active Member
Jan 22, 2019
503
161
43
There is no AMD advantage whatsoever with 'raid', it is all software. I would stick to the OS provided software-stack for software raid configuration and management. AMD drivers in general leave a lot to be desired.
 
  • Like
Reactions: Erzdorf
Erzdorf

Erzdorf

Member
Apr 11, 2017
30
3
8
Dreece said:
There is no AMD advantage whatsoever with 'raid', it is all software. I would stick to the OS provided software-stack for software raid configuration and management. AMD drivers in general leave a lot to be desired.
Click to expand...
Thank you.
 
Dolpa

Dolpa

Member
Apr 27, 2017
54
30
18
Hungary
Erzdorf said:
Hi @Dolpa,
Hope your ROMED8-2T build is still going strong.
Could you tell me how you set your Asus HYPER M.2 X16 Gen 4 4x M.2 up? - I have it installed and bifurcated, having now 4 NVMe drives in BIOS but cannot see the RAID Mode to set up the PCI slot.
As I went through the BIOS settings I have couple of questions:

1. Has "PCI Subsystem Settings>SR-IOV Support" anything to do with it, cos it's DISABLED?

2. Should "PCIE SLOT OpROM" be in UEFI or AUTO? - As it says, AUTO disables NVMe devices and is LEGACY with other devices.

I was expecting to see the PCI RAID Mode option in subOS level in BIOS but just can't find it.

Köszönöm Szépen!
Click to expand...
Hi @Erzdorf ,

As @Dreece said there is now hw raid mode in the bios, I use the NVME in linux with zfs softraid and it's working fine. Anyway the mobo has not any hw/sw raid options.
 
  • Like
Reactions: Erzdorf
You must log in or register to reply here.
Top Bottom