take my money and build it!

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

sotech

Member
Jul 13, 2011
305
1
18
Australia
Just a quick comment on the CPU - for not all that much more cash you can get a hyperthreaded model, effectively doubling the threads in your machine from 8 to 16, and also getting 50% more cache (15MB vs. 10MB). Even in Australia the cost difference between the 2609 and 2620 is down to $120, which I feel is well worth it considering the overall cost of the investment and how long you expect it to last. I think for 5x VMs (potentially more/different ones in the future) the additional threads will be more beneficial than the slight clockspeed advantage of the cheaper processor.
 

sotech

Member
Jul 13, 2011
305
1
18
Australia
Just a quick comment on the CPU - for not all that much more cash you can get a hex-core, hyperthreaded model rather than a 4 core, 4 thread model - effectively tripling the threads in your machine from 8 to 24 and also getting 50% more cache (15MB vs. 10MB). Even in Australia the cost difference between the 2609 and 2620 is down to $120, which I feel is well worth it considering the overall cost of the investment and how long you expect it to last.
 
Last edited:

wookienz

Member
Apr 2, 2012
98
4
8
does the diff in Ghz, 2.0 vs 2.4 make much difference these days with multi cores and hyper threading?


cheers.
 

mobilenvidia

Moderator
Sep 25, 2011
1,956
213
63
New Zealand
Apps that use few Threads can benefit from CPU with few cores and high clock speed.

Apps that can use multiple threads (ie Virtual Machines) benefit from more cores and speed is not as critical.

But getting a Dual CPU Mobo chances are you are after more Core/threads than a single CPU can deliver.
As our Aussie friend just said the 2620 will be a better choice than the 2609 mainly due to the extra threads with HT enabled and more cache is always better.
And also very important is RAM with 5x VMs and the boot OS, you're looking at 32GB as a starter to give them all room to do their thing.
 

wookienz

Member
Apr 2, 2012
98
4
8
Updated Build to date:

The dual cpu board i had in mind wont fit in the norco 4224, i needed to cut costs so i will stay with single cpu.

Case: norco4224
Mobo: SUPERMICRO X9SRA ATX Server Motherboard LGA 2011 DDR3 1600
CPU: Intel Xeon E5-2620 Sandy Bridge-EP 2.0GHz 15MB L3 Cache LGA 2011 95W Six-Core Server Processor BX80621E52620
Fans: Corsair H100
PSU: SeaSonic X750 Gold 750W ATX12V V2.3/EPS 12V V2.91 SLI Ready 80 PLUS GOLD Certified Full Modular Active PFC Power Supply
RAM: 8x4Gb ECC of some brand (suggestions?)
HDD: HGST Deskstar 7K3000 HDS723020BLA642 (0f12115) 2TB 7200 RPM SATA 6.0Gb/s 3.5" Internal Hard Drive -Bare Drive
HDD OS: RAID 1 SSD (unsure which yet)
RAID: IBM Serveraid x3 (one will be in a x4 slot)

So... what holes have i missed?

cheers.
 

sotech

Member
Jul 13, 2011
305
1
18
Australia
I'm sure you've got this down but just in case - you'll need a third fan for the fan wall (and don't forget to buy the 120mm fan wall itself). I have good success with Kingston RAM and they have 1600mhz ECC out now - I've no experience with their compatibility (or otherwise) with Supermicro boards, though. Depending on how much extra it is for you locally you might also consider the Platinum 860w PSU for higher efficiency.

Crucial or Intel are good choices for the SSDs, imho. We have had good experiences with both brands over a large number of SSDs.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,519
5,828
113
The Intel RAID Expander Card RES2SV240 is very popular SAS expander because it can also take power input from a 4-pin molex connector. The HP SAS Expander on the other hand needs PCIe power.
 

Mike

Member
May 29, 2012
482
16
18
EU
Is there a particular reason for going with the Xeon E5 26XX series instead of the 16XX series? The UP xeon's have higher clock speeds although the max tdp is rated at some 130 watts or so, idle shouldn't be higher than a DP Xeon.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,519
5,828
113
Good point Mike. If you are going single CPU, the E5-1600 series might be a better bet and odds are at least some of the cores are going to spend a good amount of time at idle.
 

sotech

Member
Jul 13, 2011
305
1
18
Australia
Is there a particular reason for going with the Xeon E5 26XX series instead of the 16XX series? The UP xeon's have higher clock speeds although the max tdp is rated at some 130 watts or so, idle shouldn't be higher than a DP Xeon.
Oh awesome - they're finally out in the US. The last time wookienz posted specs up only the E5-1660 was available... the 1650 looks like a very good choice here for a single-S2011 setup.

Sadly Aus distributors only have the 1660 still for $1300 and no others :(
 

wookienz

Member
Apr 2, 2012
98
4
8
I am struggling to find a mobo for the E51650 that can give me at least 4 x8 slots, 3 for m1015 and one spare for expansion. 3 x8 with 4 nics on board.

Ideas?
 

mobilenvidia

Moderator
Sep 25, 2011
1,956
213
63
New Zealand
ASRock Fatal1ty X79 Professional

Supports:
2011 Xeon E5-1660(C2) Sandy Bridge-EP 3.3GHz BLK 100MHz 15MB 130W P1.20
2011 Xeon E5-1660(C1) Sandy Bridge-EP 3.3GHz BLK 100MHz 15MB 130W All
2011 Xeon E5-1650(C2) Sandy Bridge-EP 3.2GHz BLK 100MHz 12MB 130W P1.20
2011 Xeon E5-1650(C1) Sandy Bridge-EP 3.2GHz BLK 100MHz 12MB 130W All
2011 Xeon E5-1620(M1) Sandy Bridge-EP 3.6GHz BLK 100MHz 10MB 130W P1.20
2011 Xeon E5-1620(M0) Sandy Bridge-EP 3.6GHz BLK 100MHz 10MB 130W All
2011 Xeon E5-1607(M1) Sandy Bridge-EP 3.0GHz BLK 100MHz 10MB 130W P1.20
2011 Xeon E5-1607(M0) Sandy Bridge-EP 3.0GHz BLK 100MHz 10MB 130W All
2011 Xeon E5-1603(M1) Sandy Bridge-EP 2.8GHz BLK 100MHz 10MB 130W P1.20
2011 Xeon E5-1603(M0) Sandy Bridge-EP 2.8GHz BLK 100MHz 10MB 130W All
Slots:
Slots - 4 x PCI Express 3.0 x16 slots (PCIE1/PCIE2/PCIE4/PCIE5: x16/8/16/0 mode or x16/8/8/8 mode)
- 1 x PCI Express 2.0 x1 slot
- 2 x PCI slots
- Supports AMD Quad CrossFireX™, 4-Way CrossFireX™, 3-Way CrossFireX™ and CrossFireX™
- Supports NVIDIA® Quad SLI™, 4-Way SLI™, 3-Way SLI™ and SLI™
ASRock Fatal1ty X79 Champion
Same CPU support as the professional above, different PCIe slot layout

Slots
- 5 x PCI Express 3.0 x16 slots (PCIE1/PCIE5: x16/16 mode; PCIE1/PCIE3/PCIE5: x16/8/8 mode; PCIE1/PCIE4/PCIE7: x16/8/8 mode; PCIE1/PCIE3/PCIE5/PCIE7: x16/8/8/8 mode)
- 2 x PCI Express 2.0 x1 slots
- Supports AMD Quad CrossFireX™, 4-Way CrossFireX™, 3-Way CrossFireX™ and CrossFireX™
- Supports NVIDIA® Quad SLI™, 4-Way SLI™, 3-Way SLI™ and SLI™
ASUS P9X79 WS

Supports the E5-1660 so next BIOS upgrade should have newest 1600s added

Expansion Slots
4 x PCIe 3.0/2.0 x16 (dual x16, triple x16/8/8, quad x8, black+blue) *1
2 x PCIe 3.0/2.0 x16 (x4 mode, white) *1
I see the 1600's have an upgraded stepping now, must have been issues dogging their release.
 
Last edited:

sotech

Member
Jul 13, 2011
305
1
18
Australia
We are using the P9X79 WS and have been since it was released... it has handled 4 M1015s + GFX + 4-port NIC fine. Highly recommended. Now if only I could get an E5-1650 to replace the 2630...
 

wookienz

Member
Apr 2, 2012
98
4
8
i was initially looking for a server grade board, am i being a bit precious not considering these?
 

sotech

Member
Jul 13, 2011
305
1
18
Australia
i was initially looking for a server grade board, am i being a bit precious not considering these?
Asus' workstation boards seem to perform well in server roles; we have had a P8B WS that was a fileserver/VM box for ~12 months and the chap we sold it on to once we upgraded is still using it in the same capacity - uptime of months wasn't an issue and with ECC RAM support they seem pretty well up to spec.

You lose IPMI and an onboard graphics chip, most notably to my mind - IPMI I can live with but I do wish I wasn't sacrificing a x4 PCI-E slot to graphics. Given how few other single-CPU S2011 options are available in AU, though, that's a sacrifice I'm more than willing to make for the expansion slots/8DIMM slots/etc.

To be honest I would have gone with something that's more typically "server-grade" if it were available originally; given my good experiences with 4 of the Asus workstation boards now, though, I've relaxed a little about that and happily put together customer builds with the Asus boards without worrying if their needs dictate that.
 

mobilenvidia

Moderator
Sep 25, 2011
1,956
213
63
New Zealand
ASRock mainstream boards officially support Xeon processors, one of the few vendors that do.

My Z77 board will be my second ASRock build, first with a Xeon and new one with a Ivy Bridge Core i5 supporting ECC
I'm very impressed with ASRock, the new mobo uses top notch components (review to follow)

You'll be hard pressed to find a Server Mobo with 4x PCIe 8x slots
You could wait for the ASRock Extreme11 motherboard as this is going to be a true beauty.
 

wookienz

Member
Apr 2, 2012
98
4
8
UPDATED LIST:

CPU: Intel E 1650 CPU or similar
MOBO: Asus P9X79 WS
RAM: 32 Gb (4x8gb ) ECC DIMM 1600 (not sure what brand)
Data: Caviar green Seagate 2TB x 12 initially.
VMS: Intel 330 series 120Gb SSD x 2

Case: Norco 4224
RAID: M1015 x3
Cooling: Corsair H100 cooling, 120mm fan bracket, 1x Noctua 120MM fan, 2x Noctua 80mm fans
1m Cables

Will run ESXi 5 with Vm's on SSD's (Raid1)

Thoughts..
 
Last edited:

sotech

Member
Jul 13, 2011
305
1
18
Australia
Might be a typo - 3x 80mm fans for that chassis? What's the plan for the third?

Which controller are you going to use to raid the ESXi datastore? With two M1015s and the onboard controller available I would guess one of the M1015s in raid1, the other in IT mode and using a 4-1 reverse breakout for the other 4 of the 12 drives on the onboard controller?

Looks very close to our setup - which we have zero complaints about.