Building a Server: Is LGA1366 Still sweet spot today for Power Consumption vs Cost?

[cgtechuk]

Hi All,

I currently own 4 x HP ML310e G8 Servers that are decked out with the max ram that that supports 32GB and they all have the E3-1220 V2 CPU in them.

I am a VM ESXi junkie and memory is what I really need and a good amount of it, but at the same time I need a sensible trade off of cost for running vs CPU power.

The reason I am no longer a real fan of the current HP Servers I have:

• 
  
  
  • Ram is very expensive for it (Only supports UDIMM and a max of 32GB Total)
  • HP G8 Servers hate expansion cards that arent HP and ramp the fans up
  • Because of the RAM requirement I need to run a few machines.

My question is because of this I have decided to build a custom server in a case same as I would a PC to fit my needs exactly. So High DDR3/4 (RDIMM) Memory, Decent Multicore CPU, Low power and lots of Storage capacity.

So anyway what I am asking is where does the LGA1366 range fit in here? I can see lots of Supermicro boards around this age / range and all look pretty decent spec wise with some Lower Power Xeons I can get to fit in them. I used to own a ML350 G6 with 2x5650s running 96GB RAM and this thing was a flying machine but I had to sell it due to the spiralling CPU costs.

Is LGA1366 still a good budget alternative? My current gripe is anything that is not E5 wont support RDIMMs and only UDIMM so memory becomes very very expensive. I basically want a motherboard and CPU combo for around £400 / $600 with lots of RAM that is reasonable on electricity consumption like I dunno under 100W Idle ? e.g a Supermicro dual socket board with like a L5640 in it then I can load up the RAM and if I want to expand I can double it . I just need to find a sweet spot

TLR - Want to build a custom Server as Im sick of HP / Rack stuff, Is LGA1366 still cheap enough for power consumption vs cost given the cost of modern CPUs and memory

 

[T_Minus]

The 'sweet' spot IMHO is the E5-2600 generation V1 or V2 if you're future proofing a bit and want even more power savings and perf. boost.

The CPUs (v1) are cheap, V2 price jumps a lot but still not bad for a couple 10C CPUs! Checkout the E5-2660 and E5-2660 v2 a lot of people here got the E5-2670 when they were $49-$59 now they're $75-100$+

These take DDR3 not 4.

Another option, but as you mentioned cost of UDIMM is $$ but the latest gen Intel E3 can handle 64GB.

 

[cgtechuk]

What socket is the E5-2660 ? can reasonably priced Supermicro motherboards be had for them?

I would probably opt for a single CPU in a dual socket board for now but I would ideally like to have low CPU consumption as the VMs wont do much most of the time but when they are use they are being used properly lol. I have offloaded the true 24/7 VMs off to another basic ITX 10Watt Server and at the moment powering off the other server which I dont need it but I want an All in one solution I can use that I can fit in a case.

 

[Evan]

2011 socket for E5's
Define reasonably priced ? I don't think so, I think SM boards are a bit pricy for the models you need. Still if the cpu is cheap then can work well.

 

[cgtechuk]

Thanks for your reply Evan, I class reasonably priced as between £100 - £200 / ($250- $350) maybe cheaper would obviously be nice but I can get a dual socket board for a LGA1366 SuperMicro for under £/$100 at the moment which is why I was asking..

Other than supermicro what other options are there? I find consumer boards a bit of a mind field as they are all labeled as gaming this and that and the ECC support is a mine field. lol

 

[Rapturoso]

E5 ES2 / QS CPUs if you can get hold of them for a reasonable price. They are reliable and nothing like the standard ES.

I have 2 x E5 2630 v4 QK3G and they were VERY cheap new. They are pretty amazing and I've had no trouble with instability or any issues. They are pretty much identical to retail so there's no point in buying retail processors new or used.

If budget is a major concern this can shave a fair amount off the cost of a system allowing spending more on RAM and system board.

You'll benefit from system longevity and not have to worry about upgrading for quite some time if you go this route, plus you'll save BIG on energy costs as the memory and CPU will be slashing your kWh usage.

Idle consumption is exceptionally tiny on my system and at full load the CPUs run relatively cool, further maximising their life. I have been able to go completely overkill on my cooling keeping core and die package temperatures between 20°C and 45°C even with quiet and relatively slow fan speeds on air coolers.

If you want something that's going to perform like you say and is going to last, it might be worth increasing your budget slightly to accommodate a newer architecture and the masses of benefits that come with it. These far outweigh the initial cost of the hardware and you'll be glad that you added just a few hundred £/$ to your new build. I for one will never look back.

 

[cgtechuk]

What board do you run them with and is DDR4 cost pretty decent compared to DDR3?
Am I reading that right that the CPUs are nearly $1k each for a v4!! WOW , little out my price range

 

[T_Minus]

You can easily find a 2P SuperMicro E5 v1/v2 board for $350 or less, I have one (smaller one) for sale here in the low 200$ range even.

The E5-2660 are $40 each hard to beat them for the price!

DDr3 and DDR4 are both 'cheap' compared to 2015 but the prices have gone up in the last few months... DDR3 is still cheaper and you can find 16GB RDIMMs very affordable.

 

[Evan]

Thanks for your reply Evan, I class reasonably priced as between £100 - £200 / ($250- $350) maybe cheaper would obviously be nice but I can get a dual socket board for a LGA1366 SuperMicro for under £/$100 at the moment which is why I was asking..

Other than supermicro what other options are there? I find consumer boards a bit of a mind field as they are all labeled as gaming this and that and the ECC support is a mine field. lol

Ok that price you will be ok, but won't find much cheaper unless stumble on a good used deal.

ASRock rack, Asus, gigabyte, tyan, Intel, all have server boards. SM is probably the most popular but nothing wrong with the others. And there is lots lesser well known Brand’s but I would stick to what's better known in terms of being able to get community support easily.

 

[zer0sum]

Thanks for your reply Evan, I class reasonably priced as between £100 - £200 / ($250- $350) maybe cheaper would obviously be nice but I can get a dual socket board for a LGA1366 SuperMicro for under £/$100 at the moment which is why I was asking..

Other than supermicro what other options are there? I find consumer boards a bit of a mind field as they are all labeled as gaming this and that and the ECC support is a mine field. lol

The advice here is spot on already

There are a range of E5-26XX CPU's that you can find for under $100 each.
The E5-2650L v1/v2 for instance has a 70w TDP and is 8/16 or 10/20 cores/threads respectively
Slower core clocks compared to a non L series chip, but much lower TDP

Then you need a board like Supermicro X9D series, Gigabyte GA-7PESH2, Intel S2600CP2J and as much DDR3 ram as will fit into your budget.

I actually have a Gigabyte GA-7PESH2 for sale for $259, and it is an awesome board, but of course I'm biased

 

[cgtechuk]

Thanks
T_Minus
, Can you send me a link to your post? I am struggling to find the For sale one you had? Also would you ship it to UK

 

[Fritz]

My current workstation is running dual E5 2670 v1's. I'd love to update to v2's but they're still way overpriced for what they are and not worth it to me. as for power consumption, I don't see much difference between my fully decked out workstation and my LGA 1366 based servers. My next move will be to Xeon D's LGA 2011 is just too overpriced as far as I'm concerned.

 

[cgtechuk]

My current workstation is running dual E5 2670 v1's. I'd love to update to v2's but they're still way overpriced for what they are and not worth it to me. as for power consumption, I don't see much difference between my fully decked out workstation and my LGA 1366 based servers. My next move will be to Xeon D's LGA 2011 is just too overpriced as far as I'm concerned.

This is very interesting, so from your experience there is little Wattage difference idle and load between 1366 and 2011?

 

[Evan]

This is very interesting, so from your experience there is little Wattage difference idle and load between 1366 and 2011?

A lots of people have said this but I generally see a big difference between HP DG380 G7 (1366) and Gen8 (2011)
Very busy this week but maybe it's something to do with the memory load in the systems or whatever, I may be able to have a play if I still have any waiting for disposal G7's to do a test.

What I have not seen is much of a big difference between the 2011 systems, sure v4 probably is better but not so much as for it to massive stand out to me.

 

[cgtechuk]

A lots of people have said this but I generally see a big difference between HP DG380 G7 (1366) and Gen8 (2011)
Very busy this week but maybe it's something to do with the memory load in the systems or whatever, I may be able to have a play if I still have any waiting for disposal G7's to do a test.

What I have not seen is much of a big difference between the 2011 systems, sure v4 probably is better but not so much as for it to massive stand out to me.

Wow Evan I would really appreciate that, Im just torn between buying 1366 with cheap CPU and RAM and the 2011 socket stuff that was quoted above. The biggest issue I have with getting any of these boards is I am in the UK and all the reasonable prices and range / variety is in the USA which is a bit of a pain so all help appreciated!

Ill get there lol I am looking forward to fitting it in a standard PC case and making it look nice as well , Part of me also wants to consolidate a few of the smaller drives I have for bigger ones but then RAID rebuild time becomes an issue over power usage too.

 

[Fritz]

I was going by what my metered PDU was saying and it's not as accurate as a power meter but yea, no difference that made a difference. When I get a chance I'll put a power meter on them and report back.

 

[Evan]

I was using the iLo and looking at the power info so I hope it's accurate. I suspect since my figures differ from others is something else different that what most people use. Maybe it's the fiber channel cards or the memory config (systems I were looking at both had big memory, with 1366 also using all 3 banks, e.g. 24 dimms LR active)

Almost out of 1366 servers, just a few left in odd roles but i do have some in the disposal queue. See what I can do, got me curious to also see the E5 differences (I have v1,v2,v3,v4 systems so I have some in very similar config I am thinking to compare myself)

 

[T_Minus]

I just moved a SM846 saw a board was in it... opened it up and bam 1366 SM 2P board with a stick-it note that says "working".
I think I have some 4C L cpus someplace I'll throw in there and power it up and get a reading, I wanted to think with 8x dimms it was around 180W idle with x5670s back when I ran it, def. was not low power idle like v2/v3/v4 systems do.

 

[cgtechuk]

I just moved a SM846 saw a board was in it... opened it up and bam 1366 SM 2P board with a stick-it note that says "working".
I think I have some 4C L cpus someplace I'll throw in there and power it up and get a reading, I wanted to think with 8x dimms it was around 180W idle with x5670s back when I ran it, def. was not low power idle like v2/v3/v4 systems do.

Again that would be really helpful if you could, Just wish electricity wasnt so dear in the UK lol.

Incidentally did you find the for sale post for the motherboard you had for sale the one mentioned in one of your other posts above? T_Minus?

 

[Evan]

DL380 G7 – 2 x L5640 (6-core @ 2.26GHz, 60W), 24gb (6 x 4gb DDR3), 6 x 2.5” SAS, 1 x HBA

- 24hr Avg usage 236W (Power regulator mode : Dynamic power)



DL380 Gen8 – 2 x E5-5670 (8-core @ 2.60GHz, 115W), 256gb (16 x 16gb DDR3) 2 x 2.5” SAS, 2 x HBA, 1 x 4-port 1G NIC

- 24hr Avg usage 178W (Power regulator mode : MAX (Static high))


This is a reasonably idle workload (G7 is stand alone running windows, Gen8 is running ESX) and last 24hrs is Sunday including evening until midday or so Monday.

Look at the extra memory in the 2011 system, that also adds some watts. Also windows is generally a better power saver than an ESX server.

Note that is the HP hardware , I really can't comment on white box stuff.
I do have blades all with 512gb ram on E5 v1,v3,v4 (no v2) config and all with same dual port 10g + 2 x dual port HBA i will check on when I get some time out of interest since that power usage will exclude fans and only the 2 x 300gb disks included so is about the closest you can get to same/same config.