Low power servers - does socket matter?

[fractal]

I have been putting off upgrading to LGA2011 for a while. I love the older LGA1151 systems for low power servers and appliances. 40 watts for an idle file server is pretty attractive. But, I do feel cramped with only 32GB of memory.

I have also been collecting a few of the LGA1366 systems for number crunchers when I needed more compute power in a small space. Dual LGA1366 systems compete very well against LGA2011 in the budget cruncher space.

But, I have always been curious how much extra I was spending to power my LGA1366 workstation compared to a LGA2011 workstation. Several threads on this, and other forums suggest it should be noticable. My workstation, much like my ESX server, is idle most of the time.

Then I spot a HP Z420 on eBay without memory or video for 200 bucks delivered.. So I buy it.

I then set up two almost identical systems. A HP Z400 with a 130W TDP W3690 and a HP Z420 with a 130W TDP E5-1620. Both are single socket 32 nm workstation processors.

I added a Quadro FX 1800 to both systems along with a single 8GB stick of DDR3-1600 unbuffered ECC memory and a 40GB Intel SSD. I used the same memory and hard drive in both systems. The hard drive had Ubuntu 14.04 installed.

The results were a bit unexpected..

HP Z420 / LGA2011 E5-1620 : 61W / 62VA idle
HP Z400 / LGA1366 W3690 : 61W / 63VA idle.

Compare this to a HP ML10V2 which is just under 40 watts idle with 16G RAM and a SSD.

So, to quote an old commercial "where's the beef?" Other than a slightly better power supply, the chipset in the Z420 is no better than the chipset in a Z400. I can get more cores in LGA2011 but I can get more crunch for the buck in LGA1366.

So, why does everyone give LGA1366 such a bad rep? It doesn't look like it is because of power.

 

[T_Minus]

6 Cores 9500 score but 1500 single
PassMark - Intel Xeon W3690 @ 3.47GHz - Price performance comparison

4 Cores 9000 score but 1900 single
PassMark - Intel Xeon E5-1620 @ 3.60GHz - Price performance comparison


Other than your example practically all others including my own tests the 1366 idle higher, and perform less overall and per-core. This starts to add up when you go to 2 CPUs or compare 8 core 2011 to 8 core 1366 performance... not to mention the 2011 space has a lot higher core CPUs as well as "V2" which will surely idle lower, and give you more power per-watt as well as an updated chip that may have things you need.

 

[Jb boin]

Also note that low power CPU even if they have similar specs except the letter on their model are more efficient at idle (and probably at other speeds too), for example a server with a single L5640 was use about 15w less than the same one with a E5645 if i recall correctly the numbers.

L5630 and L5640 are easy to find and really cheap, you can have a complete server with a pair for less than 100€ while low power LGA1151 are still quite expensive.

In the end, it really depends on your needs and budget.

 

[nk215]

I don't know about 1366 HP board but my dual 1366 socket board uses around 80 watts by itself. I've never seen my dual 1366 idle at less than 280 watts. The equivalent setup (HDD, GPU, HBA etc) for a a single E3-2670v2 idles at 150 watts (approximately equivalent CPU power as a dual X5650).

TS140 also idles around 40 watts with 32G and a dual DC3700 HD. It seems like white-box home built system uses more power (Power supply has much more headroom so not very efficient at light load is also part of the reason, way more fans etc)

 

[Jb boin]

From some notes i found back, comparing on identical HP GEN6 servers (except that the one with the E5645 had an additional HP P812 RAID card) with only one PSUs plugged, 48Gb of RAM, an HP P410 RAID card, only one CPU, according to iLO :

• With one E5645 (and the additional P812) : 177w
• With one L5640 (almost the same specs) : 122w
• In both, plugging the second PSU added around 10-11w according to iLO

Other tests i did with a cheap wattmeter (but the same one was used for all these tests) :

• A Dell C6100 with 4 nodes (with only one PSU plugged) having each 2*L5630, 24Gb or RAM and one 7200RPM SATA disk, each booted on Debian Jessie, idling was consuming around 260W, so 65W per nodes.
  Removing one of the CPU should lower around 15W per node i think.
  
  The 4 nodes shut down but with the IPMI working, it was still using 50W for the whole chassis (so 12.5W per nodes).

• A Supermicro server (X9SCM with IPMI) with a Xeon E3 1230v1 (or v2 maybe) with 32Gb of RAM was doing around 30W in idle but still 22W with the server stopped.

• An idling Supermicro motherboard (X8STi) with one Core i7 920, one disk and 12Gb of RAM was doing arond 80W so more than the C6100 that is of the same generation with dual L5640 and twice the RAM.

• An idling Supermicro motherboard without IPMI with a Core2Duo E6320, one disk and 3Gb or RAM... did 60W in idle.

 

[fractal]

Ok, so my recollection was wrong.

HP ML10V2 / LGA1156 i3-4150 : 24W / 26VA with the same memory and SSD as the above tests.

As has been said, it does depend on needs. There is no point in recommending a LGA1156 i3 if you need 100G of RAM or 16 cores. But, there is also no point in upgrading your LGA1156 i3 with 32GB of RAM to a 4 core LGA2011 with 32GB of RAM. You are only wasting electricity.

LGA2011 wins the top end if you need high core counts but LGA1366 is still competitive if you need modest core counts with decent memory. Nothing beats LGA1156 for the price if you needs are met with 4c/8t and 32GB of RAM and don't need PCIv3 or USB3 or ...

I was tempted to replace my i3 ESXi server with the Z420 if I can find a low power E5 that will get me under 50 watts idle, but that will have to wait until I find an affordable low power e5 on ebay. v1's start at a hundred bucks and v2's are closer to 200. And I still don't know if that will get me to under twice the power of the current i3 server that is still doing the job.

 

[T_Minus]

I don't think I've seen anyone ever upgrade in recent months to 2011 4 core setup from 1366 or 1156 for that matter... most are going for 2P, and minimum 8 core.

Kind of hard to beat E5-2660 for $49 each...

16c is def. going to idle more than 4c of most any other generation too though LOL

If all you NEED is 2-4C and not a lot of RAM then 2011 likely will never 'fit' budget-wise... it's just "wrong" for what you're going after.

For insanely low power 'DIY' system (non-SOC) I really like the E3-1220L V2 chip!

 

[Laugh|nGMan]

Few months ago i double checked idle watts on my setup, yeah i know, this more suits for workstation usage, anyway:
Windows 10/server 2016 Supermicro x9srl-f old chipset c602 + 128Gb DDR3 1866 ECC + E5-1650V2 6 core LGA2011 chip at Balanced power plan (idles 1.2GHz when on desktop) + SATA DOM 64Gb = 47W
When i add NVIDIA Quadro K1200 +1-2W when idle (for Hyper-V RemoteFX scenario).
When i swap for Quadro M4000 +13W when idle (for Hyper-V DDA passthru).
Samsung 950 256Mb Pro thru m.2 pci-e adapter +1-2W when idle for HomeLAB Hyper-V storage needs.
Storage raid adapter Intel rs2bl080 (LSI2108) + old firmware that spin down hdd drives +3-4W when idle (for backup or NL storage needs). If i remember correctly hdd spin down occurs after 30min.
So there is room maybe for one or two PCI-E devices .... NIC ..Mellanox ConnectX-3 .. or sacrafice slot to external bracket for external LTO tape drive + external storage.

So i can stretch idle watt usage pretty fast from bare minimum ~50W, to the 70W range with pair HDD's. Not to mention Performance power plan that skyrocket idle watts even more.

 

[Toysrme]

1366 is the budget bargain for CPU power.

A pair of $100usd x56## 6c/12t westmere-EP's is going to be right there with a stock clock $1,000usd i7-5960x in apps like cinebench.



It's a bit of a joke tho power consumption wise. You're doubling the power consumption to equal the workload.
Not a big deal when used in occasion. Big deal if you run all the time!


Sent from my iPhone using Tapatalk