2xE5-2670 / Intel S2600CP2J Natex Build Check (x-post from /r/homelab)

[mmm_dat_data]

First time poster here, hope I'm doing this right. Found this forum through /r/homelab where I didn't get as much attention as I was hoping (Though some helpful folks did comment). Looking through older build threads on here with natex stuff, it looks like I'm not the first one to query both forums... thanks for looking!

Build’s Name: Hydra
Operating System/ Storage Platform: ESXi
CPU: 2xE5-2670
Motherboard: S2600CP2J
Chassis: Rosewill RSV-L4500 (I realize I will need to mod as this is an EATX case)
Drives: TBD
RAM: 64GB (8 x 8GB 64GB PC3-12800R ECC Registered
Add-in Cards: H310 (IT mode) (maybe add small video card down the line)
Power Supply: EVGA SuperNOVA 750 B1

Other Bits:
Noctua i4 CPU Coolers and more Noctua Fans for Chassis (Quiet build intended)

Usage Profile: ESXi box (Primary/Initial VM will be ubuntu based (mdadm) NAS) EDIT: Another thing I forgot to mention is I expect to have a plex server running on it too, which will probably be the most intensive VM...

Other information…
I realize I'm a bit behind as a lot of you guys have done this build last year. I wanted to post up and see if I've missed anything in the threads I've read through. This box will be an effort to consolidate current and future hardware (computers) with two interests in mind, maintaining/improving current power consumption within my "lab" and giving me somewhat of a digital playground for future interests with learning networking services and general noobie experiments in VM environments.

This is the Natex link for SR0KX Xeonds that support PCI passthrough
Intel S2600CP Motherboard Dual E5-2670 SR0KX, 64Gb. PC3-12800R RAM, Natex.us

These are some of the lengthier STH threads I've read through:

Intel s2600cp Dual Xeon 2670 3D / VFX Rendering Node

Dual E5-2670 build in the smallest case NZXT S340

Some Links I've copied (and expect to use) through these posts:

I expect to need to do a BIOS update
Download Intel® Server Board S2600CP Firmware Update Package for EFI

Latest version is 02.06.0005 which apparently has sleep (s3 mode) issues, although I don't intend to make use of sleep as an ESXi box, and would prefer to not discover any pci-e issues that I read were resolved witht he latest bios version.

Motherboard Service Guide
http://www.intel.com/content/dam/su...ver/sb/g54899001_s2600cp_p4000cp_sg_rev14.pdf


Questions/Concerns I have:

What kind of idle power consumption can I expect?
Ideally I'd like to be below 180w, at least that's what would save me money, since the 2 computers that would be virtualized first are currently idling at about 180w... Not sure how reasonable this is.

Where are people getting the heatsinks for this motherboard? I dont see them on Natex's site.

Am I missing/forgetting anything?

Thanks for looking!

EDIT: here's an imgur album, almost finished another natex build

 

[fractal]

That looks like a good build if you can use all the cores of a dual 2670. All my lab vm boxes run out of memory before CPU. My current lab box has a single E5-2650 with 128G RAM and barely wiggle the cpu meter with ram half full. I based that build on a HP Z420 I got for 150.

Now, there is nothing wrong with WANTING a dual 2670 just because you can, but I wanna ask if you really need the heat / power / noise of such a box.

 

[marcoi]

Here is my build out thread. IT might have some useful info
Intel S2600CP w/ Intel P4000 Case

my average idle, but i have alot of HDD, 3 raid cards and full 128gb ram. I have about 10-15 vms running.

System Power Statistics value over the last : 444.21 Hours
Minimum: 126W
Current: 196W
Maximum: 338W
Average: 197W

 

[mmm_dat_data]

@fractal- thanks for the response! and yea I think you make a real good point. do I need this much? probably not... as a matter of fact I should say definitely not. I guess I'm not well informed of a broad spectrum of options...

I considered going the low power route and looking into an avoton build, but I don't think it supports vt-d for pci pass-through (to vitualize my mdadm raid) and I read something that small struggles past one transcoded plex stream. Any suggestions on an inbetween? maybe you're suggesting a single e5-2670 build? should I shop for a cheaper motherboard?

Maybe I should blab about how I arrived at this natex site?

I originally started to run VMs on my main workstation (i7 2700k), and even wasted some money maxing it out with RAM only to find out that some of the things I wanted to do didn't run well on a type 2 hypervisor. I imagine partly because The host system was doing things like transcoding for plex and other workstation things while trying to run multiple VMs.

This build would be my third attempt at getting a usable ESXi box. I got a hold of a 2950 for free only to realize it's unbearable loud and can't operate in a bedroom. So I shopped c-list till I found a dual L5420 (iirc) xeon 1u supermicro and paid 130 for it, again, only to realize it was way too loud. I spent 60+ bucks trying to outfit it with noctua fans and while putting it back together realized the PSU died on me. I gave up. (I whined about this on /r/homelab, lesson learned. the 1u SM parts are now on /r/homelabsales and no one is interested).

These 2 failure may have me a bit overzealous on making sure this next attempt works out.

Someone in one of my homelab posts linked to the natex site, and welp, here I am.

@marcoi, thanks for the numbers and link! exactly what I was looking for!


Thanks again for reading and you opinion, I appreciate it as I don't work in IT, but love this stuff as a hobby.

 

[jrdnlc]

I have dual E5-2680 CPU's 128GB of ram, evga 1060 gpu, and 10 hard drives. My server idles at 190-200watts.

Also have the same Rosewill case with a Supermicro X9dr7 mb and it fit right in. I didnt have to modify it at all.
Using two Noctua NH-U9DXi4 CPU coolers

 

[mmm_dat_data]

I have dual E5-2680 CPU's 128GB of ram, evga 1060 gpu, and 10 hard drives. My server idles at 190-200watts.

Also have the same Rosewill case with a Supermicro X9dr7 mb and it fit right in. I didnt have to modify it at all

thanks! that sounds like a decent idle number considering what you've got powered up... I pulled the trigger and ordered the natex combo with 128GB of ram. Plan to purchase the case soon. I googled that mopbo, looks like it's EATX on SM's site, so wouldnt expect the need for mods on that case. The 2600 intal mobo isnt eatx, it's ssi-eeb which apparently only lines up on the outer corners...

 

[jrdnlc]

Ah, gotcha! Sorry missed that it was ssi-eeb.

 

[jrdnlc]

Actually, E-ATX and SSI-EEB are exactly the same in size. The only thing that will be an issue is that 3 holes won't line up on the case. Which should be an easy fix. @mmm_dat_data

Doubt you'll need those 3 holes if the MB will be laying down flat in that rackmount case.

 

[mmm_dat_data]

Actually, E-ATX and SSI-EEB are exactly the same in size. The only thing that will be an issue is that 3 holes won't line up on the case. Which should be an easy fix. @mmm_dat_data

Doubt you'll need those 3 holes if the MB will be laying down flat in that rackmount case.

yea, I think I heard somewhere that some folks didn't even mod the case, they just put plastic standoffs in the locations where there's no mounting holes to hold it up... I wonder about the difference in functionality regarding grounding and whatnot of mounting points... /parnoid haha

 

[Red_Devil_24]

It's not listed in the spec but it does have ssi eeb mounting hole

 

[mmm_dat_data]

welp, that's awesome to see... thanks!

 

[jrdnlc]

@mmm_dat_data Plastic standoffs sound like a great idea so the motherboard doesn't get damaged with the extra weight. You should be fine on the grounding part, lol.

 

[mmm_dat_data]

Welp, dumb of me I suppose, I ordered the noctua coolers after seeing them in another build with the same case only to learn the hard way there's several versions of the model I got... It's just BARELY too tall. (probably ~2mm too tall) The cover on the case fits on but is in contact with the cpu cooler. I opted for return since I figure I dont want the top of the case to be applying force on the mobo...

fan removed, and cooler placed for clearance check...


Also, this is my third RSV case and I just realized that the HDD cages come out of the front like this.. what a dummy

I'm realy hoping that I don't have any hardware issues from the natex stuff, I'm eating into my return period here...

thx for following

 

[jrdnlc]

You need the Noctua NH-U9DXi4 coolers. Those fit perfectly in that case.

 

[mmm_dat_data]

You need the Noctua NH-U9DXi4 coolers. Those fit perfectly in that case.

I have these things in my 2 other RSV boxes, and it keeps my i7 2700k under 70 while being pegged on all cores transcoding for plex... the other box never runs much of a cpu load so it's got an easy job to do.

 

[mmm_dat_data]

added imgur link in post: another natex build

 

[Bill1950]

Get a couple of Supermicro SNK-P0050AP4 and stop worrying. They do a great job and fit comfortably in a 4U case. SNK-P0050AP4. They do a very effective job of cooling and produce little noise.