AMD EPYC Milan Idle Power Consumption

[vincococka]

Just for comparison with similar setup /w Intel Xeon 3rd:

- SuperMicro X12SPL-F BIOS:1.1c BMC:1.0.21
- CPU Intel Xeon 3rd 6326 16c 2.9GHz
- 4x 32GB Micron DDR4 DualRank 3200MT/s
- 4x 128GB Intel Optane PMEM 200-series
- 1x Chelsio T6225
- 1x Mellanox CX5 EN 100GbE single-port
- 2x SSD Kioxia CD6-R 1.92TB
- FibreChannel 8Gbps dual-port
- 3x 120mm FAN, CPU cooler SuperMicro LGA4189, 2x 40mm Noctua tiny fans for PMEM
- PSU Seasonic Focus GX 650W
- OS RHEL 8.5 /w kernel 4.18.0-348.20.1

Measured consumptions:
- OFF /w IPMI management active: 4w
- idle OS: ~145W
- loaded /w prime95 AVX512: ~310-350W
- loaded /w prime95 FMA3: ~320W

 

[[Nobody]]

Wow, so my full equipped R620 (2* e5-2630l V2, 6*16 GB DDR3 1.35V, broadcom sfp+ nic, Quadro p620, 4* SSD, 4* 2.5" HDD, H310p) is with 98W @2 % CPU load not so bad at power consumption. Thus, it would not make any sense to upgrade just because of power consumption.

 

[vincococka]

@[Nobody]
What if I had improper BIOS settings that resulted in increased idle power consumption ?
What if OS does not know how to save power because platform is still somehow "new"?

 

[vincococka]

I've put PCIe cards + NVMe U.2 disks out of system to measure almost pure server consumption

Configuration

- SuperMicro X12SPL-F BIOS:1.2 BMC:1.1.17
- CPU Intel Xeon 3rd 6326 16c 2.9GHz
- 4x 32GB Micron DDR4 DualRank 3200MHz
- 4x 128GB Intel Optane PMEM 200-series
- 2x 120mm FAN, CPU cooler SuperMicro LGA4189, 2x 40mm Noctua tiny fans for PMEM/DDR4 slots
- PSU Seasonic Focus GX 650W
- OS RHEL 8.6 /w kernel 4.18.0-372.13.1

Consumption:
idle OS: ~115W
loaded /w prime95 AVX512: ~270W
loaded /w prime95 FMA3: ~270W

 

[redeamon]

hwinfo is good for this if you're running windows. My 73F3 isn't too bad (around 75w idle) with performance mode on, cTDP of 280w and all memory channels populated @ 3200. I'll try to take a few screenshots later.

https://forums.servethehome.com/index.php?threads/tyan-s8030gm2ne.28914/page-9#post-341855

 

[RolloZ170]

My 73F3 isn't too bad (around 75w idle) with performance mode on, cTDP of 280w and all memory channels populated @ 3200

help me if i am wrong: with 3200 memory the IF runs at 1600Mhz right ?

 

[redeamon]

help me if i am wrong: with 3200 memory the IF runs at 1600Mhz right ?

Yes

 

[Stephan]

Consumption:
idle OS: ~115W
loaded /w prime95 AVX512: ~270W
loaded /w prime95 FMA3: ~270W

Try removing Optane because 115W idle is really really high, no? Set OS or hypervisor to "power-save" or "balanced".

 

[vincococka]

Try removing Optane because 115W idle is really really high, no? Set OS or hypervisor to "power-save" or "balanced".

Tried to measure minimal configuration containing only:
- X12SPL-F with 1.2 BIOS and 1.1.17 IPMI
- Xeon 6326
- SM Cooler SNK-0080AP4
- front and rear FAN 120x120 slow RPM
- 1x 2280 NVMe x4 in m.2 slot on board

Measured consumptions in idle with following ram stick/s = Micron DDR4 RDIMM 32GB 3200 MT/s:
1x RAM stick ~82W in RHEL 8.6 idle (powersave CPU governor)
8x RAM sticks ~105W in RHEL 8.6 idle (powersave CPU governor)

 

[Stephan]

@vincococka Check Linux with recent "powertop" if CPU package sleeps in C6. If it does, I am all out of options. If it doesn't, as a really last resort I can think of, boot from SATA and remove the NVME. The delta at idle between 1 and 8 sticks is really high. Instead of 3 watts per additional stick it should be more like 1 watt, doing refresh and not much else.

 

[Vader]

• PSU: 2U Redundant 800 Watts (ASPOWER R2A-DV0800-N) - labeled with 94% efficiency

Hi! As I see you use the ASPOWER supplies from inter-tech. How would you describe the noise level with medium loads about 70 Watt? The fans are controlled, aren't they? I am interested in ASPOWER R2A-DV0800-N or even the weaker one (but also redundant).