Xeon Scalable vs EPYC idle power consumption

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

i386

Well-Known Member
Mar 18, 2016
4,250
1,548
113
34
Germany
h12ssl, 7443p, 128gb ram, 17x 16tb hdds in a sm 846 with sas expander backplane and raid controller: idle ~190 watt* (windows server 2022 login screen, large movie file being read from the array)

*measured via ups: system under load - system was powerded off

@graczunia how do you define "idle"?
 
  • Like
Reactions: ano

Markess

Well-Known Member
May 19, 2018
1,166
784
113
Northern California
IMO if you're sweating 100 watts Epyc and Xeon SP might not be the best route.

Maybe look to upgrading the Xeons? I have a dual 2696 v3 setup that peaks at 370w with 8x8gb DDR3 installed so DDR4 should be lower power still. RAM type and quantity makes a difference here too. I use it when I have a bucket of threads I need to throw at something and not huge IPC.
I don't think the OP, @graczunia, is specifically sweating 100w. That came into the discussion later. Reading their post, I think their main question was, could they replace a dual E5 v2 system with something newer and get both more compute and more efficiency? They specifically mention Scalable and Epyc but mention they are open to other ideas.

Since Sandy/Ivy Bridge are a dead end upgrade wise, a system upgrade is going to require replacing motherbaord, CPU, and RAM. @graczunia , I'm not sure if your main criteria is reducing consumption or replacing a system that's on an almost 10 year old platform?

Reading everyone's replies, the consensus (which I agree with) is that it's going to be the disks, HBA(s) , 10G networking, maybe PSUs if they are innefficient, and everyting else that's attached, that drive the higher consumption. If you replace a dual E5-26xx V2 motherboard and CPUs with a single early generation Scalable or EPYC, but transfer over/reuse your other components with the same general configuration, you may get more compute/easier management, but it won't save you much powerwise.

There also seems to be a consensus that, for the early generations, EPYC drew more power. I don't have either one, so can't comment on that.

Since you'll need Motherboard, CPU and RAM as a minimum, the 500 euro budget for the base components is going to be tight. You'll get newer, but not much better in the efficiency area.
 
Last edited:

i386

Well-Known Member
Mar 18, 2016
4,250
1,548
113
34
Germany
is not idle

idle(not sleep / power saving):
  • unemployed,
  • unoccupied,
  • free to start a job
Saw another post of yours in another thread and remembered that I wanted to treply to a certain post :D

In my case even when nothing was read from the array the differences were between 5-10watt (usually closer to 5 watt)
 

ano

Well-Known Member
Nov 7, 2022
656
273
63
h12ssl, 7443p, 128gb ram, 17x 16tb hdds in a sm 846 with sas expander backplane and raid controller: idle ~190 watt* (windows server 2022 login screen, large movie file being read from the array)

*measured via ups: system under load - system was powerded off

@graczunia how do you define "idle"?
no bios tuning Im guessing?
 

i386

Well-Known Member
Mar 18, 2016
4,250
1,548
113
34
Germany
moestly default settings, only things changed are related to secureboot and efi (vs csm and legacy stuff)
 

ano

Well-Known Member
Nov 7, 2022
656
273
63
they are very good at default settings, only all flash(faster flash, not sata) ZFS and ceph requires the bios tuning imho, but for most stuck stuff default is plenty good.

its quite easy to get it to use 270w vs the 200w you were at and marginal performance... and across servers and time this is $$$. I've been tuning 100G and ceph stuff for hours today... an excersise in futility, I'm now back at what the previous baseline we use for ZFS systems... (and really all performance systems)
 

graczunia

Member
Jul 11, 2022
45
21
8
IMO if you're sweating 100 watts Epyc and Xeon SP might not be the best route.

Maybe look to upgrading the Xeons? I have a dual 2696 v3 setup that peaks at 370w with 8x8gb DDR3 installed so DDR4 should be lower power still. RAM type and quantity makes a difference here too. I use it when I have a bucket of threads I need to throw at something and not huge IPC.
I do have a spare ASUS Z10PE-D16 WS, might play around with it and see how it goes - would fit perfectly in my Supermicro chassis.


@graczunia how do you define "idle"?
For the sake of the argument let's assume system is booted into Proxmox with a TrueNAS VM running, no reads/writes on the disks/network activity.


I don't think the OP, @graczunia, is specifically sweating 100w. That came into the discussion later. Reading their post, I think their main question was, could they replace a dual E5 v2 system with something newer and get both more compute and more efficiency? They specifically mention Scalable and Epyc but mention they are open to other ideas.

Since Sandy/Ivy Bridge are a dead end upgrade wise, a system upgrade is going to require replacing motherbaord, CPU, and RAM. @graczunia , I'm not sure if your main criteria is reducing consumption or replacing a system that's on an almost 10 year old platform?

Reading everyone's replies, the consensus (which I agree with) is that it's going to be the disks, HBA(s) , 10G networking, maybe PSUs if they are innefficient, and everyting else that's attached, that drive the higher consumption. If you replace a dual E5-26xx V2 motherboard and CPUs with a single early generation Scalable or EPYC, but transfer over/reuse your other components with the same general configuration, you may get more compute/easier management, but it won't save you much powerwise.

There also seems to be a consensus that, for the early generations, EPYC drew more power. I don't have either one, so can't comment on that.

Since you'll need Motherboard, CPU and RAM as a minimum, the 500 euro budget for the base components is going to be tight. You'll get newer, but not much better in the efficiency area.
Spot on - I'm looking for a replacement for the powerhog that the dual E5 v2 system is; not really considering how to reduce it's power draw as it doesn't seem to be worth the effort. Something that is more 'worthy' to run 24/7 on a student budget for a lack of better word :D
Not really sweating the 100w indeed, just trying to find something that would give me better performance per watt, although I am trying to reduce my lab's overall power consumption - and replacing this system would be a huge step towards that. One way or another, the system has to go and if I'm going to shell out some money I'd rather do it right and pick up something that would last me a couple good years - and while LGA2011-3 might be the way to go, I figured I'd might as well look into something newer as I'm rather unfamiliar with anything beyond E5 V4's.

Maybe I should specify more, here are the current system specs:

2x E5-2630L v2
Supermicro X9DRH-7F
12x8GB DDR3 ECC
2x Supermicro 750W Platinum
Intel 82599 (x520) NIC
2x 120GB SATA SSD
2x Samsung PM983 0.96TB
8x HGST 2TB

According to IPMI, it seems to idle in the neighborhood of 150W, with a very light load putting it up to over 200W. If I could come up with something that gives me say, twice the performance while cutting the idle power in half I'd see that as a win. I might stretch out the budget a little by selling off the CPU/MB/RAM combo, but the rest would ideally stay. There are for sure some further optimizations to be made (drive spindown, bigger but less drives etc), but the E5 v2 system doesn't seem to be a good base for it as the hardware itself is simply inefficient compared to modern offerings.