Ideas needed for a Low Power system around DDR4 ECC RDIMM
- Thread starter DHCPdude
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Careful, Xeon D can be really power hungry. Somebody forgot to design it for low power. Or disabled such features, because of bugs.
If you want DDR4 ECC RDIMM I recommend a 3647 board in ATX form factor and a x2xx Xeon CPU. Like X11SPL or similar.
Lower would be some Core i3 9100 and UDIMMs. Get a Coffee Lake Refresh or later. And some C246 board. X11SCA X11SCM and similar. Big breathable case will allow for silent 120mm fans at < 1000 rpm and make the machine basically silent.
If you want DDR4 ECC RDIMM I recommend a 3647 board in ATX form factor and a x2xx Xeon CPU. Like X11SPL or similar.
Lower would be some Core i3 9100 and UDIMMs. Get a Coffee Lake Refresh or later. And some C246 board. X11SCA X11SCM and similar. Big breathable case will allow for silent 120mm fans at < 1000 rpm and make the machine basically silent.
@DHCPdude , do you already have some DDR4 RDIMMs, and/or are you planning to put a lot of RAM in the system and want RDIMM because its more affordable? Any other constraints, like motherboard form factor, number of drives you plan to use, etc.?
@DHCPdude, energy efficient and affordable are relative things. There's not a lot of choice for really low power gear that uses RDIMMs. If you don't mind getting used gear, a board with Atom C3xxx would work (assuming you don't have too many drives or need lots of computing power) This series of Atom works with RDIMMs. Supermicro, for example, makes a whole range of these boards with various models of these System on a Chip (SoC) CPUs. This one is in the middle of the range A2SDi-4C-HLN4F | Motherboards | Products | Supermicro The CPU in this, Atom C3558, is a 25w TDP SoC. Idle power isn't too bad either, although most motherboards with these Atom CPUs will have a BMC/IPMI for remote management, which can add 3-7 watts depending on the brand. These motherboards are still available new as well, but cost more of course. ASRock Rack also makes boards with the Atom C3xxx series, but they tend to cost more. I'm not sure about other brands.
For more power while being "energy efficient" you're going to be more limited if you need to have it work with RDIMMs. Xeon D is a step up powerwise, but draw more power. With a little fine tuning, Haswell/Broadwell E5-26xxx v3 or v4 based systems (which are older, and therefore cost less) can have a relatively low idle power. My home NAS has a Xeon E5-2630 v4 CPU in a Fujitsu Workstation Motherboard and 11 SSDs. Idles just over 30 watts.
Yes, for this I agree, Atom (specifically the C3xxx series) would work well.
Yes, Xeon D isn't always energy efficient, but I don't really think LGA 3647 is either. There's "low power" LGA 3647 chips, but they are only "low power" in relation to other LGA 3647. And they're still pretty expensive once you get a motherboard as well. i3 and C246 are going to be energy efficient of course, but don't work with RDIMM which, I suspect, the OP already has and is why they asked the question.
@DHCPdude, energy efficient and affordable are relative things. There's not a lot of choice for really low power gear that uses RDIMMs. If you don't mind getting used gear, a board with Atom C3xxx would work (assuming you don't have too many drives or need lots of computing power) This series of Atom works with RDIMMs. Supermicro, for example, makes a whole range of these boards with various models of these System on a Chip (SoC) CPUs. This one is in the middle of the range A2SDi-4C-HLN4F | Motherboards | Products | Supermicro The CPU in this, Atom C3558, is a 25w TDP SoC. Idle power isn't too bad either, although most motherboards with these Atom CPUs will have a BMC/IPMI for remote management, which can add 3-7 watts depending on the brand. These motherboards are still available new as well, but cost more of course. ASRock Rack also makes boards with the Atom C3xxx series, but they tend to cost more. I'm not sure about other brands.
For more power while being "energy efficient" you're going to be more limited if you need to have it work with RDIMMs. Xeon D is a step up powerwise, but draw more power. With a little fine tuning, Haswell/Broadwell E5-26xxx v3 or v4 based systems (which are older, and therefore cost less) can have a relatively low idle power. My home NAS has a Xeon E5-2630 v4 CPU in a Fujitsu Workstation Motherboard and 11 SSDs. Idles just over 30 watts.
Last edited:
I already have the RAMs (16x 16GB DDR4 ECC RDIMM, quite overkill I know), they are in my dual Xeon E5 2630L v3 machine which is now low power and is now offline since a while ago! It eats 80-100W in idle mode. I want to swap the CPU and motherboard only. The rest of the system is in place (SSDs, NIC, efficient PSU, rack case, etc.). Looks like your "same machine" is using one third of mine, which is inspiring! Can you elaborate a bit on your config?
Going back to Atom sounds like a good idea, although the processing power seems to be significantly lower.
Used gear is good, I am not running mission critical business at home.
Is it true that i3-9100T and i3-9100 both can use ECC RDIMM?
Am I going to gain any power saving by going from 2530l v3 to v4?
Last edited:
Honestly, consolidating your RAM and eliminating a CPU would probably be a non-trivial savings even if you kept the board. Check the idle power without the second CPU and with only 32GB of RAM. If that's not low enough for you, swapping over to an V4 CPU will probably cut your idle power consumption by even more. If you're thinking about swapping over to an Atom board I suspect the performance would still be fine in that configuration.
Yes, what @reasonsandreasons and @T_Minus said! Pull one CPU and the corresponding memory and see how much it saves. Switching to a v4 CPU will give you a bit better idle behavior, but it won't get you a ton of savings. But, if you do decide to try a v4 CPU, E5 -26xx of that generation had two different dies: up through 10 Cores had a smaller die than the 12-20 core models and idle on a bit less power. Although again, not a big difference.
"L" series processors just help at the top end, not with idle really. In fact, there's folks that believe that most processors that get binned as an "L" are because they aren't stable at the higher frequencies, and many aren't as stable at lower end either so they tend to clock higher at idle and ramp up a bit for no reason. So, folks in that camp will argue that an "L" series CPU will actually draw more power at idle than a standard one. Who knows? But, the reality is that in a home server that's on 24/7 it will spend a lot more time at idle than not, so a non-L, standard, CPU is usually going to be fine.
My NAS: has a single processor workstation board, so no BMC/IPMI, which saves a small amount on the power draw. Its in a desktop case with two super slow 140mm fans and a "big chunk of metal" CPU Cooler with its own slow moving fans for the reason @Stephan mentions above...mostly silent. But between that and the 80+ Platinum partial zero RPM PSU, there's not a lot of power draw from the cooling solution. 32GB of RAM currently in two 16GB sticks (recently reduced from 64).
Even with only one CPU installed, your dual CPU motherboard is going to draw a bit more power than a UP board simply because there's more mass to push power through. But it won't be much more really. Probably not enough difference to go out and buy something else if you are sticking with E5s. IPMI, which I assume you have with a DP system, will be a bigger factor. Supermicro are kind of powerhungry, and I understand IPMI adds 6-7 watts for the generation board you have. ASRock is a bit less, and I don't know about other brands. Again, not enough to run out and buy something else if you're sticking with a 2011 platform. If your system is rackmount and you're running redundant PSUs, you can save a bit by pulling one. There's an article on it on the main site about it. Of course, you'd loose redundancy. And if you're rackmount, the much more capable cooling solution will draw more power than a couple big consumer fans like I'm running.
For now, try pulling a CPU and (at least) half the RAM. I bet you see an immediate difference.
"L" series processors just help at the top end, not with idle really. In fact, there's folks that believe that most processors that get binned as an "L" are because they aren't stable at the higher frequencies, and many aren't as stable at lower end either so they tend to clock higher at idle and ramp up a bit for no reason. So, folks in that camp will argue that an "L" series CPU will actually draw more power at idle than a standard one. Who knows? But, the reality is that in a home server that's on 24/7 it will spend a lot more time at idle than not, so a non-L, standard, CPU is usually going to be fine.
My NAS: has a single processor workstation board, so no BMC/IPMI, which saves a small amount on the power draw. Its in a desktop case with two super slow 140mm fans and a "big chunk of metal" CPU Cooler with its own slow moving fans for the reason @Stephan mentions above...mostly silent. But between that and the 80+ Platinum partial zero RPM PSU, there's not a lot of power draw from the cooling solution. 32GB of RAM currently in two 16GB sticks (recently reduced from 64).
Even with only one CPU installed, your dual CPU motherboard is going to draw a bit more power than a UP board simply because there's more mass to push power through. But it won't be much more really. Probably not enough difference to go out and buy something else if you are sticking with E5s. IPMI, which I assume you have with a DP system, will be a bigger factor. Supermicro are kind of powerhungry, and I understand IPMI adds 6-7 watts for the generation board you have. ASRock is a bit less, and I don't know about other brands. Again, not enough to run out and buy something else if you're sticking with a 2011 platform. If your system is rackmount and you're running redundant PSUs, you can save a bit by pulling one. There's an article on it on the main site about it. Of course, you'd loose redundancy. And if you're rackmount, the much more capable cooling solution will draw more power than a couple big consumer fans like I'm running.
For now, try pulling a CPU and (at least) half the RAM. I bet you see an immediate difference.
Does Epyc have lower idle power consumption than E5 V4? I've always assumed the I/O die would cancel out any savings at peak for most homelab applications, absent system consolidation.
Oh, just noticed that you mention you have a NIC. What do you have? Depending on the brand, number of ports, and generation, 10G NICs, for example, can draw a decent chunk of power.
Gut says to get significantly lower than 80-100W, say below 50W, you'd need to dump everything and move to C246 with two UDIMMs for dual channel, Core i3 (no AMT, thanks, Intel!) or some desktop Xeon, and a lowish power board. Board without IPMI, only onboard 1 Gbps (chipset + i210 etc.), no fancy PCI/PCIe bridges, no SAS. Also important, for disks only SATA to raise chances to get the CPU package into C6/C7. Use ALPM on SATA SSD links to lower power even more. Might not work with disk drives, beware.
Any SAS card +10W.
Your run of the mill 10 Gbps card +5W.
Mellanox ConnectX3 56 Gbps card +10W.
IPMI +5W.
No "super-premium" PSU (Seasonic fanless etc) only "Bronze" with 70% not 90% at 50W +10W.
If you have a Ferraris utility meter, get a Hoymiles HM-600 and two 410W solar panels and put those out on the balcony. Will pretty much deliver year over year what a 60-70W system consumes. Or a HM-1600 and four 410W panels to cover the fridge and a bunch of other stuff as well.
Any SAS card +10W.
Your run of the mill 10 Gbps card +5W.
Mellanox ConnectX3 56 Gbps card +10W.
IPMI +5W.
No "super-premium" PSU (Seasonic fanless etc) only "Bronze" with 70% not 90% at 50W +10W.
If you have a Ferraris utility meter, get a Hoymiles HM-600 and two 410W solar panels and put those out on the balcony. Will pretty much deliver year over year what a 60-70W system consumes. Or a HM-1600 and four 410W panels to cover the fridge and a bunch of other stuff as well.
May be true. If he cuts it all the way back to 32GB of RAM for a test, any idea how much power 14 x 16GB sticks draws at idle? The second CPU won't be a ton by itself. But maybe between that and the RAM he can get some decent savings without having to replace the system?
As I mentioned above, I'm ~30w at idle for my 2011-3 NAS System, but I don't have any of the power demanding features/hardware you mention above:
- No 10G (or faster) networking. I wasn't making use of the speed, so I went back to onboard 1g NICs a couple years ago.
- My NAS Motherboard (Fujitsu D3348-B2) has no IPMI (it does have AMT, but with no iGPU its more like "WOL+")
- No SAS or Mechanical drives, so more power savings there.
- The SSDs are pretty low power, a mix of NVMe and SATA m.2 (PM953 & Micron 5100 Pro) plus some Samsung SV843.
i350 is generally well behaved and doesn't suck up a ton of power in the 2 or even most 4 port configurations. 6 port (and higher if they exist?) cards tend to have a PLX chip in them and the power draw & heat goes way up, even at idle.
Edit: If you try the single CPU config test, you'll want to make sure your card(s) are in PCIe slots connected to CPU1, as the slots connected to CPU2 won't work anymore once the CPU is removed.
I went through this with my Blue Iris box that runs my security cams. Low power / low performance just didn't cut it. I was running with a i7 4790 and 24GB of ram and the recording were herky jerky and the cpu always ran hot. Moved it to a SM X9SRH-7F with a E5 2690v2 and 64GB of ECC RDIMM's. Recordings are butter smooth and fps stay pegged at max. I also added 2 heavy duty high volume fans to the SM 732 case and the system runs cool and hardly breaks a sweat. Some applications just won't let you get by with low power gear.