E5 v3 -> v4 upgrade - idle power draw of -L and non-L variants?

[sko]

Thanks to the prices for used hardware being somewhat "normal" again, I'm back to looking into upgrading the E5-2630L v3 CPUs in my main homelab server to v4s.
The 2560s are available at a bargain now (~50EUR/pc), and hence seem to offer a very cheap upgrade path to broadwell while also substantially increasing core count (8 -> 12 or 14).

I usually only need a lot of compute power ~once a week or every two weeks for my poudriere package builds. Most of the time that server mostly idles with very low load (~0.4 avg).
So I'm looking for a good compromise between low idle power consumption (and heat/noise) and decent performance to get the package builds finished in reasonable time. Especially in terms of heat and noise I'm a bit more drawn to the non-L variant, because it has a considerably higher TCASEmax and hence could run a bit hotter whereas the low TCASEmax of the -L variant occasionally triggers the BMC alarm, especially because its readings are 7° off, so I have to keep the CPU temps even lower...
I usually run package builds when I'm not at home - so a higher noise level due to higher power consumption and heat dissipation during builds is not that big of an issue.

I've already browsed the documentations, thermal guides etc that are (freely) available by intel, but apart from TDP and some theoretical maximum power consumption there's nothing to be found like e.g. the base and minimum assured power values that can be found for newer generation processors.

Can anyone share or point me at some real-life figures or comparisons of 'same-model'-L and non-L Xeon v4 variants? The 2650 vs 2560L would be perfect, but any other model that has an -L counterpart would be sufficient to get an idea if there is actually any difference in idle power consumption.

Edit: Also what's the lowest clock rate the non-L can go? Are they the same as for the -L? Intel only publishes the base frequency, not the range/steps that the clock rate can be set below that base frequency (e.g. my 2630L v3 are specified with 1.8GHz base but can go as low as 1.2GHz)

 

[matt_garman]

I haven't looked into this in detail in several years now, but last I looked at this, those lower-TDP CPU variants essentially just have a lower heat/power ceiling, but the floor is the same (assuming you have all the power-saving features enabled). The target for the lower-TDP variants is when you have heat/power/cooling limitations, and simply cannot support the higher TDP in your application for whatever reason. But I think in general, you can expect the idle power consumption to be the same.

Many years ago, I had a regular Xeon E3-12xx CPU (I forget which version, probably v1 or v2). I was using the original U-NAS chassis, mini-ITX only, and I was concerned about being able to cool the CPU adequately if it were to max out (due to very limited space around the CPU and general airflow concerns). I didn't need the full compute power, so I disabled two of the cores in the BIOS. I measured the at-the-wall power consumption using a Kill-a-Watt, before and after the change. There was literally zero change in idle power consumption (though the at-load power draw did go down considerably, as expected). My conclusion was that they've optimized the core least-power idle states enough that they are all but removed from the circuit when truly idle.

In other words, I think that as long as you can adequately cool the regular CPU when it maxes out during your builds, there's likely no reason to get the L variant.

 

[T_Minus]

I'm testing this myself, but in a single CPU configuration.

I just got a E5-2650L v4, E5-2667 v4, and have some misc other V4 and V3 (and E5-16xx v3\4 too)... I think the 16xx may save around 5w from at-a-glance, but will know more when I do head-to-head comparison.

Idle power consumption FOR SURE changed on E5 v1\v2 with the "L" CPU, at-least SuperMicro really really slowed the fan speed down with the L v1\v2... on v3\v4 so far I don't notice on the SM Chassis I've tested on.

 

[sko]

I haven't looked into this in detail in several years now, but last I looked at this, those lower-TDP CPU variants essentially just have a lower heat/power ceiling, but the floor is the same (assuming you have all the power-saving features enabled). The target for the lower-TDP variants is when you have heat/power/cooling limitations, and simply cannot support the higher TDP in your application for whatever reason. But I think in general, you can expect the idle power consumption to be the same.

Thanks for your insights. This is what I also suspected/expected from reading between the lines of all the marketing gibberish. But somehow I hope(d) the "L" variants might also have some advantages at idle like e.g. the U variants for desktop/mobile CPUs that always shaved some Watts off at all states compared to their standard counterparts.

In other words, I think that as long as you can adequately cool the regular CPU when it maxes out during your builds, there's likely no reason to get the L variant.

Cooling isn't an issue - there are no spinning 3.5" heaters that block the airflow in that server anymore, so that CSE-829 now has a quite good airflow and the 2.5" SAS SSDs aren't seeing any load from package builds (all done one NVMes in the back). At normal load the fans are spinning at ~10% with CPU temps below 40°C, so there's still plenty of headroom.

I just got a E5-2650L v4, E5-2667 v4, and have some misc other V4 and V3 (and E5-16xx v3\4 too)... I think the 16xx may save around 5w from at-a-glance, but will know more when I do head-to-head comparison.

This sounds interesting. It would be great if you could share some of your findings!

Idle power consumption FOR SURE changed on E5 v1\v2 with the "L" CPU, at-least SuperMicro really really slowed the fan speed down with the L v1\v2... on v3\v4 so far I don't notice on the SM Chassis I've tested on.

TBH the v1/v2 were space heaters compared to even v3. Especially for v1 the L variants were heavily underclocked and capped to be able to run in something like small/embedded appliances without going up in smoke. For v3/4 the specs (except base clock) and performance are much closer to other non-L variants, so I don't think they were still crippling those as much as the v1 L's

 

[sko]

small update:

I got a pair of E5-2660v4 at a bargain (77EUR for both) and installed them today.

Power draw at normal load with the 2630Lv3s hovered around ~170W +/-2W
Now with those 2660v4s the server is sitting at ~180W +/-4W

Fans are still running at ~2100-2300RPM at normal load and CPU temps still well below 40°C, so no difference here - BUT, thanks to the TJmax of 92°C vs 70°C they don't ramp up immediately with some light load and I can allow the CPUs to run at ~60-70° and still have some headroom.
So in terms of noise it is a big win.

In terms of power consumption:
Both CPUs have a lowest frequency of 1200MHz, so no change here at normal loads. The RAM is now working at full 2400MHz instead of 1866MHz with the 2630Lv3s and I increased core/thread count by 75% with that swap.
Given all that, I can very happily live with the ~10W increase in power draw.

Currently the first poudriere bulk job is running. After a few of them I might be able to compare how much time (and hence time with high power consumption) I might have saved for that workload.

EDIT:
First round of pkg builds showed peak power consumption of ~380W under high load, with 5min averages at 270W; which was previously at ~280/200W, so an overall increase of roughly 30%.
I compared the build times of some long-running ports and overall they were basically halved - some even more and some a bit less (e.g. gcc12 from 3-4hrs to 1:40; firefox-esr from ~1-1:30 to 0:33). Of course those times are always heavily dependent on what else is being built in parallel, but overall it looks like at least a 50% increase in build performance.