Is it worth to replace E5 2670 with 2630L ?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

_alex

Active Member
Jan 28, 2016
866
97
28
Bavaria / Germany
Hello,
i`m currently in the state of finishing / optimizing a 3-Node Proxmox Cluster.
Its 3x Sc826, each dual 2670 with 128GB Ram (think of going to 192 as i have some 8GB Modules left). On PCI there are two ConnectX-2 dual-port (40 + 10Gb) cards and a LSI 9207 in each node.

Currently the nodes all idle in the range of 150-170W, but said without anything that isn't necessary turned of in the BIOS yet.

The third node is mainly for Quorum and Backup-Tasks and assembly and provisioning of shared storage (NFS + iSer) and secondary service's VM (Backup MX, mysql fan-in replication etc.).


So, i wonder if it`s worth to downgrade the CPU`s in this node to 2x 2630L for Power-Saving reasons.
Power in Europe is quite expensive, to say about 22 Euro per 100Watt in the DC.

An E5 2630L can be found for about 150 Euro, so a saving of 50W / CPU would payout in about 14 Months (without taking into account that the replaced 2670 can be sold ...)

I`ve read the thread about CPU Power Consumption, there is stated that idle consumption wouldn't differ much between the L and non-L Versions. On the other Hand the Voltage on the L-CPU`s should be lower, what in my understanding should result in powersavings.

So, does anyone have sort of a number how much (if at all) a 2670 differs from a 2630L in situations with a high quota of idle-time ?
 
  • Like
Reactions: Aestr

Evan

Well-Known Member
Jan 6, 2016
3,346
598
113
CPU's of the same generation generally idle very similar, older CPU the number of cores does make a difference but new ones it's nothing as they very efficiently shutdown cores.

When the 'L' or low TDP CPU's do show a big difference is under load. But generally that's really just a maximum load so the low TDP may still use the same energy for the same workload simply over a longer time.

The low TDP or 'L' are perfect where you have absolute thermal or power limits.

Having said all this I love low power and the challenge to manage it :) low power usually means low heat and low noise.
 
  • Like
Reactions: pgh5278

_alex

Active Member
Jan 28, 2016
866
97
28
Bavaria / Germany
Hi,
this is what i observed by disabling cores in the BIOS - absolutely nothing changed in idle-consumption.

The two things that made me think it could be worth to go for 2630L on this node are:
- Lower count of Cores, 6 vs. 8 - could save 25% in theory

- here, at 'typical power consumption', there is a difference of nearly 50 watts (52% of the 2670).
So, i wonder if this is true and where these numbers come from:

Intel Xeon E5-2670 vs E5-2630L

Alex
 

Evan

Well-Known Member
Jan 6, 2016
3,346
598
113
As far as I know cpuboss makes that up based on TDP or something like that, certainly I have see a lot of crazy incorrect statements in their comparisons. Again I assume under load and not idle.
If you don't need so much power and if you don't need the pcie lanes or memory channels then just try disabling a CPU and better still remove it.
 

Evan

Well-Known Member
Jan 6, 2016
3,346
598
113
150-170 you say your seeing is about what I would expect if your wondering if that value is typical, I would say yes.
 

_alex

Active Member
Jan 28, 2016
866
97
28
Bavaria / Germany
Yes, i`m not too concerned about 150-170w with this setup - i expected it to be in this range.

Disabling / removing a CPU is something i consider, need to have a close look at the block-diagram if i can go with the PCI-lanes left then. Memory should be fine, have 8x 16GB DDR3 that can be handled by a single CPU.

I also have two 2603v2 but uncertain if they would be enough for the load / worth the missing cores/frequency for the savings in power.
 

Evan

Well-Known Member
Jan 6, 2016
3,346
598
113
I have some systems with single L5640 and X5670, if I have find a few still in use on Tuesday I will see what power they are pulling but from memory at idle they were similar depending on 1 or 2 CPU, that made a difference.

Never really saw the dual x5670's loaded, they were all ESX or DB servers but relative light loaded. Suppose I could fire up prime95 if you can't get the info elsewhere.
 

_alex

Active Member
Jan 28, 2016
866
97
28
Bavaria / Germany
Hi, i have two dual 5405 Systems but not measured yet. These are mainly Lab / try some configuration - boxes and not running a lot until the 2670`s are deployed to the DC.
Also have 4x 5640 (non-L) to replace the 5405 but didn't find time yet.
But i have the feeling those are about the same in idle like the 2670 boxes, at least by judging on the amount of heat they blow out ....

Did prime95 on the 2670`s where they went up to about 330w - what was not a big surprise.

Seeing what the difference between X5670 and L5640 at idle really is would be interesting, as it`s comparsion of similar L vs. non-L CPU.

In the end, it`s a bit hard to accept that there is no saving on power with the L - labeled CPU in Idle / low-load situations.

Maybe i just should get two 2630L to get own numbers and re-sell / re-purpose if they don't pay out ;)
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
First off, I am really excited to see what you do with the cluster!

Second, what kind of loading do you expect? Are these web workloads with mostly read from memory (nginx + redis/ memcached)? Are they DB heavy or number crunching heavy?

There are a few important vectors here. The goal is to figure out exactly how much CPU performance you need and then optimize around that power profile. The idle power consumption will be better on the E5-2670 V1 but not to a great amount even on three servers.

With the backup/ quorum node, have you ever thought about running a single CPU node? It saves more power than disabling cores, but you will lose PCIe lanes. Given what you have I might start there and if that works, use the extra CPU + a low cost LGA2011 platform and replace once of the 5405 systems.
 

_alex

Active Member
Jan 28, 2016
866
97
28
Bavaria / Germany
Hi Patrick,
there will be mixed-workload, typical web-centric. Say: Apache, php, mysql (clustered), ldap, postfix, solr etc. etc.
We recently started to use influxdb and will make more use of it for all sort of mainly time-series based information (i.e. Stats / Analytics) as it fits way better than traditional rdms. Also, we plan to go towards predicitve analytics what will be more cpu-consuming than the initial workload. This is one of the reason the whole thing is a bit overt the top, just to have options / cpu-power at hand.

For the quorum/backup node: yes, i think i might just remove 2nd cpu. The board is a x9dr7-ln4f, what has feed 3 x8 pci-lanes by cpu1 - so this is maybe the way to go for now. Basically, i also considered a Xeon-D based plattform for this node, but they all lacked of pci-lanes and/or availability in europe. So, that`s maybe something that will happen in 2018 or so.

For the labs, i will get a 1u / 1 Socket 2011 / 8x 2.5 Bay System from OemXS soon that should be ok to replace one of the older Westmere-Nodes.
There is also a R2000 from Oemxs with 2x 2670 running in the Labs / Office 24/7 i will re-consider if there`s really need for 2 CPU in it.
But basically, this system will be deployed to another DC for fast accessible off-site backups and replaced with the 1u single-Socket.

When the Cluster/Setup, in special the parts regarding the IB-Setup and Storage, is finished i certainly will make a longer post, there is nice things going on with SRP and mdraid. In Short, nodes1 and 2 provide multiple SSD + HDD via SRP, node3 assembles them with mdraid and exports as NFS / iSer Targets back to the Cluster as shared storage. In case of Failure of node3 also node1 or 2 could assemble/export, what means become the active Storage-Target via a Service-IP ;)
This was/is a bit of a fight with multipathd, mdraid, scsi in general, nfs etc.
The last tests show the failover looks stable and fast, also with VM`s running fio during a hard power-down.
 

Marsh

Moderator
May 12, 2013
2,644
1,496
113
I have both CPU E5-2630L and E5-2670v1 chips, the idle power saving is not that great ( approx 5-7w ). The LSI card consume more watts.

In US, the E5-2650v1 or E5-2660v1 is pretty cheap, 95w vs 135w. It may be worth a consideration to replace the E5-2670 to E5-2650 or E5-2660
 
  • Like
Reactions: _alex

_alex

Active Member
Jan 28, 2016
866
97
28
Bavaria / Germany
I have both CPU E5-2630L and E5-2670v1 chips, the idle power saving is not that great ( approx 5-7w ). The LSI card consume more watts.

In US, the E5-2650v1 or E5-2660v1 is pretty cheap, 95w vs 135w. It may be worth a consideration to replace the E5-2670 to E5-2650 or E5-2660
Thanks, this is exactly the number i was interested in. So, a replacement with 2630L is in fact not worth.
 

Jerry Renwick

Active Member
Aug 7, 2014
200
36
28
43
The Intel E5 2670 obvious has the better performance than 2630L, but costs a little higher than 2630L. As far as I am concerned, it is not worthy replacing it.
 

cheezehead

Active Member
Sep 23, 2012
723
175
43
Midwest, US
As you already have the 2670's, try dropping to a single 2670. No cost and saves power. If you don't have enough power with a single proc, what about a single larger proc? I've upgraded procs in the past to reduce from dual to a UP setup and saved 30w at idle.
 

_alex

Active Member
Jan 28, 2016
866
97
28
Bavaria / Germany
The host serves at least 2 raid 10 Arrays assembled from remote disks, either via nfs or iser.

One array is 8x m500dc. On this nfs currently drives me crazy, about 80% of my iops compared to iser vanishes in a kvm guest and also does a noticable amount of CPU usage.
I really consider to sacrifice the ability of snapshots on it and go iser. with iser i managed to have about 90% of raw performance in the kvm and also CPU usage is much lower. If i do so removing the 2nd CPU shouldn't have any impact, for NFS on this Volume it might be worth to pay the bill for a 2nd CPU...

The whole thing is still work in progress and mainly done between other jobs.
Hope to find some time for further testing/benchmarking of the diffrent options during the next few days.