Dual E5 or Single?

TechIsCool

Active Member
Feb 8, 2012
263
117
43
Clinton, WA
techiscool.com
Hey everyone I am working on building a new all in one server. What I would like to know what your opinions are on running dual E5-2609's or a single E5-2630

A few questions if I run a dual processor's is my ram split in half?
Is the offset in running two processors in power worth it?


What does everyone think?
 

sotech

New Member
Jul 13, 2011
303
0
0
Australia
What exactly is the server for? How much RAM/many expansion slots will you need?

If you're going for a single E5 you may want to look at the 1xxx range rather than the 2xxx range - you'll see higher clocks in a similar price range, if I remember correctly - there's a chart around here somewhere. No idea when they're due to be released, however.
 

TechIsCool

Active Member
Feb 8, 2012
263
117
43
Clinton, WA
techiscool.com
Its going to be an all in one server that will be virtualizing 3 Dell SC1425's that each have 4GB of ram. I will be passing through 2 M1015's for the hard drives and running ZFS for the file system structure.

I would like to have 32GB of ram just so I can actually allocate more ram to the servers since one the servers has 4GB and is running 4 virtual's on it.
 

sotech

New Member
Jul 13, 2011
303
0
0
Australia
OK, have a look at:

http://www.cpu-world.com/news_2012/2012030701_Intel_rolls_out_Xeon_E5-1600_and_E5-2600_CPUs.html

E5-1650, 3.2ghz hexacore around the same price as the E5-2630 2.3 hexacore. On paper it looks a better choice for a single CPU system...

Our all-in-one is currently running an i7-3820 with the intent of upgrading to the 1650 or similar once they're released in this country and it's not struggling for CPU power, even running a handful of VMs. We're running 32GB of RAM in an Asus workstation motherboard at the moment - you definitely don't need two sockets for only 32GB.
 

TechIsCool

Active Member
Feb 8, 2012
263
117
43
Clinton, WA
techiscool.com
Alright that make sense I just want to buy this once and not deal with it again for a few years but still have room for upgrading. The sc1425's have been running for 6 years so far. They work alright but are slowing down and wasting power running.

It would seem that intel has yet to release the E5-1650.
 

sotech

New Member
Jul 13, 2011
303
0
0
Australia
Alright that make sense I just want to buy this once and not deal with it again for a few years but still have room for upgrading. The sc1425's have been running for 6 years so far. They work alright but are slowing down and wasting power running.

It would seem that intel has yet to release the E5-1650.
Yeah, of the 1, 2 and 4- product lines in the E5 family only the 2 has been released so far - I haven't seen an ETA on the others, one would hope they weren't too far away though. I'm waiting eagerly to pick up a 1650 myself.

I'd very much like a dual-socket system but I don't think I'd utilize the additional cores enough to make the additional expense (both upfront and power usage over the years we intend on keeping the system) worth it.
 

TechIsCool

Active Member
Feb 8, 2012
263
117
43
Clinton, WA
techiscool.com
Yeah, of the 1, 2 and 4- product lines in the E5 family only the 2 has been released so far - I haven't seen an ETA on the others, one would hope they weren't too far away though. I'm waiting eagerly to pick up a 1650 myself.
It really would be nice since they were originally where supposed to launch Q4 of last year. I will give it a few more weeks but need to start building this. Thanks for the input.

If anyone else wants to add input go for it always love other peoples thoughts.
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,966
4,931
113
I am really mixed on the E5 series. First off, they are absolutely great CPUs. On the other hand, they are really expensive. Some of the CPU tasks I have been running (will post updates soon) I'm seeing the dual E5-2690 show similar performance to a quad Opteron 6166HE system. Yikes! The performance is great, but it comes at a cost. Just to give you an idea, 100% load on both CPUs and I am pulling <400w with dual E5-2690's.

On the memory side, if you have 4x 4GB DIMMs per CPU, you will see 32GB in ESXi/ Linux/ Windows Server 2008 R2 and etc. You do not split the RAM.

Did you ever look at the AMD side? You could buy two eight core CPUs now, then upgrade them to 12/16 cores later. Also, the G34 platform still has one more CPU generation coming. CPU speed wise not comparable, but they are less expensive. Even two 2GHz+ eight core Opterons will be faster than the three SC1425.

Another thought is looking at the E5-2400 series. Likely you will see lower motherboard costs there because you have fewer DIMM channels to trace out.
 

TechIsCool

Active Member
Feb 8, 2012
263
117
43
Clinton, WA
techiscool.com
Thanks Patrick for the input really I have not actually looked at the AMD side not sure why maybe because I don't understand the technology on pass-through and I have always stayed with Intel for servers. I will look into AMD and see if I can find a comparable system that will last.
 

sotech

New Member
Jul 13, 2011
303
0
0
Australia
FYI - The 1xxx E5 Xeons have gone on sale in Australia which I assume means they're also available in the US... we usually get stuff after you guys.
 

dba

Moderator
Feb 20, 2012
1,478
181
63
San Francisco Bay Area, California, USA
It is very difficult to beat the AMD magny-cours CPUs for price/performance in threaded workloads, including virtualization. You can pickup 8-core AMD 6128 CPUs for $100 to $150 on eBay - 12 core 6172 CPUs can be found for $350 to $400. A pair of either of those in a workstation or server motherboard is an incredible combination. 16 cores for $200 anyone?

And if you are interested in a 4 CPU system (e.g. 32-64 cores) then do check out eBay item 130651945077. The seller will accept $900 - at least he did for me. Add $400 for 32 cores of AMD server power are you are only at $1,300 - plus RAM and drives of course.

I have heard somewhere that the AMD CPUs aren't as power efficient as the Intel CPUs when running big VMWare workloads. If power consumption is critical then do your homework first.
 
Last edited:

Patrick

Administrator
Staff member
Dec 21, 2010
11,966
4,931
113
I will have a main site post this week or next on a 2P 6128 system I built. One of the big issues today is that going with ESXi 5.0 free limits you to 32GB of RAM. Once you start paying for higher-end licenses the hardware cost becomes a smaller proportion of the overall cost.
 

TechIsCool

Active Member
Feb 8, 2012
263
117
43
Clinton, WA
techiscool.com
I will have a main site post this week or next on a 2P 6128 system I built. One of the big issues today is that going with ESXi 5.0 free limits you to 32GB of RAM. Once you start paying for higher-end licenses the hardware cost becomes a smaller proportion of the overall cost.
Its sad that they can't give things away for free but they have to make there money someplace right. I just wish one of the other bare metal virtualization companies supported pass through as well as VMware does.

Does AMD-V 2.0 support pass-through?
 

TechIsCool

Active Member
Feb 8, 2012
263
117
43
Clinton, WA
techiscool.com
Alright I am bumping this up since well everything has been played with I have a maximum of a E-ATX board size and would like to find a motherboard to drop in the case. Do you guys think dual g34's would be better than a single e5 for running a small 8 person exchange, dc, and some ubuntu clients?
 

sotech

New Member
Jul 13, 2011
303
0
0
Australia
Alright I am bumping this up since well everything has been played with I have a maximum of a E-ATX board size and would like to find a motherboard to drop in the case. Do you guys think dual g34's would be better than a single e5 for running a small 8 person exchange, dc, and some ubuntu clients?
Power usage will be quite a bit higher on the G34 box both at idle and under load; in terms of performance, though - which CPUs would you be using for either? There's a reasonable range of threads and clocks in both G34 and S2011...
 

TechIsCool

Active Member
Feb 8, 2012
263
117
43
Clinton, WA
techiscool.com
Power usage will be quite a bit higher on the G34 box both at idle and under load; in terms of performance, though - which CPUs would you be using for either? There's a reasonable range of threads and clocks in both G34 and S2011...
for a single e5 I was thinking the E5-2630 but if I went dual I would go down in price and stay with something like two of the E5-2609's. If I was to go AMD It most likely would be two of the 6128's I think maybe something different but most likely that.

Board right now is undecided but I need at least space for two M1015's since I will be doing zfs internally with this box as an all in one.


I know power is a factor and I would like to keep it down but just as a fyi I have 3 servers with dual P4's that are drawing a ton of power I would like to recover the price of hardware with power draw but also don't want to trade performance for energy usage.

Draw is about 1000w worth of power.
 
Last edited:

sotech

New Member
Jul 13, 2011
303
0
0
Australia
We've got a 2630 running two internal wikis, a small mailserver, an inventory database VM, another VM for Subsonic, an apt-cache VM, a WHS2011 VM and an OI+napp-it fileserver VM... everything but the latter two are Ubuntu installs. Going off ESXi's reporting the most we've pegged the CPU in the last 24 hours is a single spike to around 93%, where it normally sits around 25-40% usage. Peak usage times see 60-75% here and there. The ZFS VM has 24GB of RAM and around 20 green HDDs to play with and the remainder is spread around the others with a couple of GB spare for various VM projects.

All of the VMs with the exception of the WHS VM are on SSD mirrors inside the ZFS VM, which I'm sure helps the responsiveness quite a bit. (the OI VM itself is on a separate ESXi SSD datastore just for it and a couple of others which aren't currently in production).

Edit: 6C/12T vs. 16C/16T, 2.3 vs 2.0ghz... how many threads do you think you'll be able to take advantage of before you start seeing diminishing returns?
 
Last edited:

sotech

New Member
Jul 13, 2011
303
0
0
Australia
My power meter has turned out to be innacurate and I have yet to buy another; going off the power rating of the various components (M1015s, NICs, motherboard chipset+motherboard,CPUs,drives,ram,etc.etc.) it's around 350-400W most of the time if I recall - it was running just fine on a Seasonic X-560 560W PSU up until last week when we replaced that PSU with a Platinum 1000W PSU; I needed the 560W for another computer and wanted more efficiency for the server since it was on 24/7 and it's around 93% at the 400W mark, so it seemed to fit the bill pretty well.

I have to pick up a proper power meter in the next couple of weeks and we'll see how accurate my numbers were then :p I might re-do them today if I get time just to double check.