Upgrading My NAS from an X10SRL-F?

[reasonsandreasons]

I'm shuffling some things around in my setup, which currently centers around a TrueNAS Core machine based on a X10SRL-F with an E5-2640v3. I'm spinning up a second TrueNAS Scale box to handle backups, Plex, and some other tasks that are easier on Linux. This is happening alongside a 10G networking upgrade for both servers and my workstation. In the new arrangement, the TrueNAS Core box will handle storage with other services being offloaded to the Scale machine.

As newer hardware has been getting cheaper and the X10 generation has been appreciating slightly, I've been mulling moving to a different platform, mostly to minimize idle power consumption. I know I could get significantly more performance and PCIe lanes with Epyc, but I doubt my workload would benefit from it and I worry about the IO die's power draw. I could drop down to a consumer board, but I run a NVMe pool alongside my HDDs and appreciate the extra lanes and option for registered memory. I've thought about X11 or X12 to play around with Optane DIMMs, but unless it offers lower power consumption there's probably not much reason to jump ship there, either. The option that seems most attractive is jumping to a low-end E5-16XXv4 part for the lowest possible power consumption in the same socket, but I don't want to double-down on a ten-year-old board if there are better options.

Any first-hand experience out there?

 

[PigLover]

If you are concerned about IDLE power then going to the E5-16xxV4 will have little/no impact compared to your current E5-2640. Within the same CPU family Intel’s idle is very consistent. It’s the top-end power that varies. I don’t think you’d be very satisfied with the outcome. To significantly reduce idle power you need to get to newer processor families.

 

[unwind-protect]

Yeah. My E5-2697A v4 with 4 RDIMMS (1 CPU in a dual board) took 71 watt idle in Linux, and that is in a board with 4x 10 GbE. It just doesn't get much lower with many PCIe lanes, except maybe using a Xeon-D.

Xeon-D systems are expensive, it will be hard to make the money back on the power bill.

 

[reasonsandreasons]

I've looked at Xeon D, especially now that Plex is off the machine, but it seems fairly pricey for what it's offering and you do have meaningfully fewer lanes on X10SDV boards, even if it's partially offset by more on-board capability. I was hoping Haswell to Broadwell would be a meaningful difference, but it makes sense that a one-generation hop is pretty minor in the grand scheme of things.

To contextualize a bit, I'm not worried about my idle power consumption as much as looking for ways to optimize this machine for its new role and perhaps offset a bit of the new power consumption from the Scale box. If it's not going to go much lower for the same capability even with newer hardware I might just pick up a cheap low-end E5 V4 processor to eek out whatever savings I can and keep using it until it dies. I don't have much of an issue with the current performance and I'll take the excuse to not spend more on this until I have to.

 

[T_Minus]

If you want to get as much power saving as possible here's some more ideas...

- check IPMI fan profile (see next item on list too)
- set FAN threshold based on your ambient temp and cooling needs, will save a few watts for sure unless you're in the tropics w\out AC
- reduce # of HDD you have... ie: go from 10TB to 20TB
- go from 7200rpm to 5400rpm if you have the faster\higher power drive
- schedule the system to shut down for 1-2 hours every night
- if you have dual PSU (and don't need it) pop one out
- now that DDR4 RDIMM is cheap-cheap look for the lowest power memory and compare 2x8GB vs 1x16GB
- go direct-attach backplane and eliminate the expander if you can\needed (will depend on if you need more HBA, ports etc if this is a true benefit)
- go into BIOS and disable functionality not needed \ make sure power config is what you want (research this, you don't want to disable power saving modes)