Home AIO & ESXi/Dev

[T_Minus]

Build’s Name: Minus_AIO
Operating System/ Storage Platform: ESXi + WIN7/CENTOS/NAS VMs
CPU: 2x E5-2620(v1)
CPU PASTE: Gelid GC-Extreme (previous AS5 user, this stuff is great!)
CPU HSF: SuperMicro 4U Active
Motherboard: Super Micro X9DRH7- 7TF-0
Chassis: Rosewill 4U w/120mm fans!!!
Drives: 4xS3700, 5x5TB WD RED, Possibly 5x300gb 15k RPM SAS RAID10
RAM: 128gb
Add-in Cards:
HighPoint PCIE USB 3.0 HBA (Possibly add a 2nd if I want USB for TV -- They have one that's eSATA + USB, that may be nice to have at the TV for portable drives.)
2x RADEON HD 6450 (USB&VID over IP -- For TV, & VM "station")
Possibly adding LSI RAID card
Power Supply: Seasonic SSR550 Watt Active PFC
Other Bits:

Usage Profile: RAID10 5x300gb SAS NFS share, RAID10 4xS3700 NFS share for 'fast' storage managed by NAS, RAID6 backup/NAS duty (5x5TB) plex/etc, NFS via NAS I believe for file access on LAN & possibly remote "cloud" type access via NAS Software/OS/Plugin, Win7 VM host for video/usb over IP workstation "family room PC", Win7 (or8) VM for Blue Iris security camera monitoring (around 6 cameras) -- plan to record to the SAS or RAID6 array share, CentOS VMs for dev/test work. Motherboard has on-board LSI RAID & 1GB Cache, also have additional RAID cards to add to get configuration I want with the drives.

Other information:
- Sandisk Extreme USB 3.0 16gb FLASH Drive (SSD on a stick) for ESXI.
- Currently w/out any PCIE cards & 1 SSD for Linux OS testing:
-- 95-100watt Idle on Kill-A-Watt

I could probably drop 5-10 watts if I didn't have a 550w power supply I'd imagine, but the spinners may change that need so we'll see. 5-10w won't kill me



Questions/Concerns:
How close am I coming to the limits of the CPU for general VM usage as planned? (I haven't run Blue Iris with 6 cams yet so I'm not sure on usage, same with NAS + Media Server, I've only run my NAS as a NFS shares & cloud (synology), will NAS + MEDIA server start to tax the CPUs? (Only streaming to 1 maybe 2 TVs, rarely at the same time, 1080p, we also want to serve music to our stereo receiver)

24/7 VMs would be:
- Win7: Blue Iris (most important) (3 cores?)
- NAS (OS, NAS Software, MEdai Stuff -- Plex, & Other nas add-ons possibly like "cloud" storage for remote access, etc...) (3 Cores?)
- Win7: Family PC (2Core?)

Any concern with other limits? NIC is 10Gig but currently only have 1Gig switches/hardware... hope to fix that soon, 1 other ESXi host has 10Gig so may link them together depending on setup.


This system (mobo/cpu/ram) I got from another member here at ServeTheHome forums!! Thanks to the for sale threads. I put this together yesterday and have pics and more to share. Right now it's running various monitors in Linux Mint Cinnamon as that's what I had to boot with fast For some reason the LSI MegaRaid randomly will be detected, and then not on boot... previous owner (member on this forum) hadn't been able to test so that's #1 priority is to hookup some drives to both of my 8087 fan outs and see what I can do

 

[T_Minus]

Saved For Pics

 

[T_Minus]

Saved For Pics #2

 

[rubylaser]

I'm looking forward to the pictures

 

[T_Minus]

Sorry for lack of updates, I had another build for a E5-1650 going and got stuck testing the SSD array I have pics of both and wll get htem posted

 

[T_Minus]

Updates:

The 5x5TB WD RED arrived so I can test the LSI onboard RAID controller and verify it's working so the seller and I can get it RMA'd if not. Hoping I don't need a BIOS update for the 5TB, may be another e-mail into SuperMicro

I also received:
2x50ft CAT7 S-STP (does this mean it's shielded + each wire is foil wrapped, or only shielded as a whole?, it's rather thin compared to my direct burial cat5e!! LOL)
HDMI over Ethernet Adapters (what the Cat7 shielded is for)
75ft HDMI w/built in booster
2x RADEON Fanless Video Cards
And the USB HBA came the other day, not sure I posted on that.

Plan to setup 1 VM on Win7 to use the HDMI over Ethernet & USB HUB, and then for the TV the HDMI cable & usb cable SHOULD reach from where my rack/gear is located not requiring money/conversions potential quality loss. Debating returning the HDMI cable as I think a 25' would actually work, and be a hell of a lot cheaper... then again the HDMI could reach where I plan to put the 2x50ft CAT7 over HDMI so maybe I'll keep it for that usage and use the others for something else

 

[T_Minus]

Motherboard / Onboard RAID something funky going on.

On cold boot detects LSI card.
On reboot randomly (most often) does not detect LSI card, randomly will freeze, other times just shows an error (I have screenshots w/phone) and continues to boot. While using the LSI utility/gui to play around when it "worked" I gave up after 5 attempts as it would randomly freeze if I clicked the wrong Icon. It even froze once clicking the adapter then the "OK" to get into the adapter, 100% locked up.

So, as of now build is on hold so I can contact the seller and figure out how to RMA the motherboard / what to do next.

This is the 2nd SuperMicro motherboard I've had to RMA in 2 months.

 

[T_Minus]

I was thinking, and, well, if the seller (of the items i got from the forum) approves I'd rather just use a PCI-Express RAID card from ebay, and split the cost with the seller. That should be ~same price it would cost him to send it to him or RMA, and back since the ebay raid cards ~$150, and save me 2 weeks down time, and having to rebuild the entire server.

Fingers Crossed.

 

[chinesestunna]

Motherboard / Onboard RAID something funky going on.

On cold boot detects LSI card.
On reboot randomly (most often) does not detect LSI card, randomly will freeze, other times just shows an error (I have screenshots w/phone) and continues to boot. While using the LSI utility/gui to play around when it "worked" I gave up after 5 attempts as it would randomly freeze if I clicked the wrong Icon. It even froze once clicking the adapter then the "OK" to get into the adapter, 100% locked up.

So, as of now build is on hold so I can contact the seller and figure out how to RMA the motherboard / what to do next.

This is the 2nd SuperMicro motherboard I've had to RMA in 2 months.

I've had the same stupid LSI gui freezing with wrong mouse clicks or moves, updating the firmware seems to have helped in that case. Have you checked the firmware version that's onboard? Some of those SM boards and RAID cards have super old firmware still included.
Another possibility is cooling of the RAID chip, the 2 LSI cards I have get smoking hot without good airflow over them. I'm thinking this because you said it always works during cold boot which could mean temperatures haven't gone critical yet

 

[T_Minus]

Funny you mention that, I was talking to a friend about how some have fans and some don't and 100*C seems hot I may put a fan on it, and now you say it... I think I will try that too + updating firmware today on the cards.

 

[chinesestunna]

Funny you mention that, I was talking to a friend about how some have fans and some don't and 100*C seems hot I may put a fan on it, and now you say it... I think I will try that too + updating firmware today on the cards.

Yea I didn't think they'd get that hot as most don't include fans, but it seems fanless ones are intended for rack mount or systems with specific cooling paths so they get adequate cooling.
Testing my SM LSI9260 on the bench while idling it hits 80C+ before I plugged in a case fan to move some air over it

 

[T_Minus]

I will try the fan + updating MSM within next 20min, and report back.

 

[gigatexal]

Dude, that's a sick 4U case.

 

[T_Minus]

Are you talking about my Rosewill RSV-L4412

For the money I couldn't find a higher quality, hot-swap, 120mm fans room for SSD w/normal PSU. I have 2 other 4U here that I got then didn't use they're all nice for the $$, but this one was about 3x the $ but about 5x the features/quality in my opinion.

I'm going to swap to a superMicro 3u from a member here as I'm running out of space.

If you're interested in the Rosewill I just got it last week, and have just been using it for this AIO build / test setup. Since I'm yanking mobo/cpu combo as one I don't mind stripping the rest of the gear and selling it. Optionally with upgraded fans if you want

 

[T_Minus]

PS: I have tons of pics but 6am to midnight for 3 weeks between work and projects; I have a LOT of picture/writing/unloading to do still!! They're coming.

 

[markarr]

I had that case as well, it works great, the 120mm fans are quiet and move a decent amount of air. The trays work with 3.5 and 2.5 hard drives. Two quirks i found was the molex requirement rather than sata power on the bays, and the indicator lights are opposite they are solid till there is activity, but the front cover has a foam filter that dims the lights pretty well.

 

[T_Minus]

I actually just unscrewed the front cover/lock as I found it wasn't really needed, and I went for the "better" air flow.

The MOLEX is no surprsie in 1U,2U,3U,4U, etc chassis it's actually rather common from my experience with a handful of various brands. I try to keep a half dozen or so adapters on hand too, sata<->molex and back and forth + expanders (1->4) they're SO CHEAP from Amazon and others it's not worth the frustration for me not to have them on hand, then have to wait 2-4 days to finish a project... LOL!!

FWIW: All my seaSonic & corsair PSU have had molex cables so it really was just a matter of adding more where needed. The 4U w/out the "4u dual psu" leaves a TON of room for probably 8-12 SSD if mounted cleanly behind the PSU. I planned to put 4 there myself if I kept her.

 

[gigatexal]

I've got plenty of case fans to recycle. Mostly noctuas. But if the price is right then that changes things. I like the idea of positive airflow case. 5 intake fans and two outflow 92mm fans is a sure win for positive airflow. If I can sell my 900D locally I'll probably be looking for one of these 4U cases.

 

[chinesestunna]

@T_Minus how's the raid card working? Hope you're having better luck than me recently

 

[T_Minus]

It's on my list to get done today, at the bottom... got other business stuff that's more important. LIKE invoicing clients