Luna

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

sotech

Member
Jul 13, 2011
305
1
18
Australia
This is our home and small business fileserver and virtualization machine, replacing a similar system which was housed in a Fractal Design Define XL chassis.

Current spec:

i7-3820 (waiting on E5 Xeons to hit our shores then will be swapping one in)
32GB Kingston ECC RAM (1333)
Asus P9X79 WS
Video card
M1015 (IR)
M1015 (IT)
M1015 (IT)
M1015 (IT)
Intel i350-T4 Quad Gigabit Ethernet
5 Crucial M4 SSDs
Mixture of WD20 and WD10EARS/X
Seasonic X-560
Corsair H-100

It runs ESXi 5.0.

The main VM is openIndiana+napp-it and has the IT M1015s passed through. Other VMs include a Ubuntu VM for photo uploading, a couple of work-related wiki VMs, a WHS VM, 2-3 constantly-changing *nix VMs for learning/experimentation/trials and probably a couple of others I'm forgetting. There are plans to virtualize more of the functions of other computers floating around the office/home but that's happening only slowly due to time constraints.

I will do some proper photos once I finish sleeving it but here are a couple of phone pics from the construction over the weekend:





Ready to be racked:



Cables are a mess at this stage and I need to turn the H100's fans down as they're keeping the drives a little too cool, which is good as I'll appreciate the noise levels being a bit lower. I'm super keen to clean up the spot in the office where the old server sat - so many less cables in here now it's not funny, going for a rack was a very good idea.

Update 3/12:

Turned the H100 down to the minimum settings and the HDDs are being kept at 35-40 degrees in ~23 to 25 degree ambient temps. Pretty happy with that. The third fan on the fan wall is now a Scythe Slipstream 1900rpm and seems to match up fairly well to the Corsair fans in terms of noise and performance. Work room is now a ton cleaner now that the switchgear/server is gone, which is a huge relief. We have a large number of HDDs on order and the 24 bays should hopefully be close to full this week or next and we can start offloading data from the other systems onto here :)

Also disposed of the Norco 50cm SAS cables and replaced them with 75cm cables to lessen the strain on the connectors - while they did (just) fit between the backplanes and the rear M1015 ports it was too much tension for my liking.
 
Last edited:

sotech

Member
Jul 13, 2011
305
1
18
Australia
Alright, updated - got an i350-t4 card incoming to replace the PT quad with and have an update on drive temps. Will see how the temps go when it's full of drives...
 

sotech

Member
Jul 13, 2011
305
1
18
Australia
Vanity shot in rack prior to the addition of the new drives:



i350-t4 arrived today:





It's certainly a lot slimmer than the quad PT it's replacing.

Out with the old, in with the new...




Loaded the drivers into ESXi 5.0 and the card was recognised first go.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
sotech you love playing with depth of field more than I do :)

Looks really good.
 

sotech

Member
Jul 13, 2011
305
1
18
Australia
sotech you love playing with depth of field more than I do :)

Looks really good.
Haha... Thanks. I can't seem to find it in me to stop down unless absolutely necessary... :S

I always treated our servers in desktop cases like any of my computers and put a lot of effort into their aesthetics and while it does seem reasonably pointless to spend time and money doing things like doing custom length sleeved cables for the interior here I'm having fun so I figure why not. Not many white Norcos out there... I quite like the look of the little tabs on the drive trays being white and it only took a few mins and some paint.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
Did you paint it yourself? I think it looks awesome.
 

sotech

Member
Jul 13, 2011
305
1
18
Australia
Did you paint it yourself? I think it looks awesome.
I did - it started out as a standard 4224. Took about a week of undercoating, drying, then multiple coats - gloss white on the inside and matte black on the outside. Apart from the drying the longest part was masking off every single hole so that the inside and outside stayed separate colours... It was worth it in the end, though, as the spraying itself only took a few minutes per coat and I had the free garage space to let it sit. The little tabs on the front only took five minutes by comparison :p
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
sotech: I showcased your awesome work in the Corsair Hydro Series H100 review because this looks great! congratulations. If you ever do another and want to make a how-to guide I would be happy to give it main-site exposure.
 

sotech

Member
Jul 13, 2011
305
1
18
Australia
sotech: I showcased your awesome work in the Corsair Hydro Series H100 review because this looks great! congratulations. If you ever do another and want to make a how-to guide I would be happy to give it main-site exposure.
Aw, that was a nice surprise to wake up to :)

I intend on doing a proper how-to and taking some actual photos in the studio of the setup next time I unrack it - which will be to put an E5-1650 in once they're released... the supplier reckons a week or two yet until they hit AU but we'll see. If they don't turn up soon I'll just stick a 2xxx in there and re-use it in another build later on. I'll get you a nicer non-phone shot of the setup for the review you just posted, too ;)

I stuck another 6 HDDs in there and temps are unchanged - despite there only being a total of 3 fans in the system the CPU, motherboard and drive temps are just lovely.
 

sotech

Member
Jul 13, 2011
305
1
18
Australia
I experimented a little with the H100 speeds and drive temps today - this is with 15 WD20EARX or similar and three SSDs in the bays.

Lowest speed - drive temps between 38-41 Celsius, noise levels are extremely low. Almost inaudible from the doorway of the server room.

Medium speed - drive temps are 36-38 degrees, noise level has gone up noticeably - would now be too noisy to work near and is audible outside the room.

High speed - Tried briefly but the fan noise was audible rooms away. I would call it unsuitable for home use unless you have a separate room with a closed door or the server in the garage etc. Will get temps for the sake of completeness another time.

I'm happy with the drive temps on the slowest setting and the HBAs seem to be getting plenty of airflow judging by my inaccurate hand-behind-the-slots-to-judge-airflow and thumb-pressed-into-heatsink tests, where there was plenty of airflow and the heatsinks were cooler to the touch than when I had a 140mm fan blowing over the slots in the old tower case. The SSDs are in the column directly in front of the slots to increase airflow to this area due to the fact that more air flows past a 2.5" drive in the caddies than a 3.5".

I am basing my ideal drive temps on my interpretation of the Google drive study.

All other heatsinks/ram/etc. in the case were pleasingly cool.

Ambient in here is 25-27 degrees at the time of these tests. Measurements were via SMART polling 60 mins into a very large transfer.
 
Last edited:

mobilenvidia

Moderator
Sep 25, 2011
1,956
212
63
New Zealand
Looking at picture above you've got the radiator blowing warm air into the motherboard space
So blowing warm air onto a CPU you are trying to keep cool (ie leaving fridge door open will it cool the whole house ;) )
I would swap the 2 fans on the back to where the H100 is, H100 to the back.
Suck air from the front push it all out the back.

The Noctua 140mm fans have 120mm screw holes, if you have room for the diameter of these Noctua fans another great way to get volume of air quietly.

But very nice setup, glad you are sharing your experiences with us
 

mobilenvidia

Moderator
Sep 25, 2011
1,956
212
63
New Zealand
OK, looking closer at the Norco case my idea won't fit, possibly could retrofit a single fan version.

On my 4u case I can fit a 120mm fan above the Mobo I/O shield
Would be an ideal place for the radiator, but a single 120mm version.

I still don't like blowing warm air into a space you want to keep cool.
 

sotech

Member
Jul 13, 2011
305
1
18
Australia
Looking at picture above you've got the radiator blowing warm air into the motherboard space
So blowing warm air onto a CPU you are trying to keep cool (ie leaving fridge door open will it cool the whole house ;) )
I would swap the 2 fans on the back to where the H100 is, H100 to the back.
Suck air from the front push it all out the back.

The Noctua 140mm fans have 120mm screw holes, if you have room for the diameter of these Noctua fans another great way to get volume of air quietly.

But very nice setup, glad you are sharing your experiences with us
OK, looking closer at the Norco case my idea won't fit, possibly could retrofit a single fan version.

On my 4u case I can fit a 120mm fan above the Mobo I/O shield
Would be an ideal place for the radiator, but a single 120mm version.

I still don't like blowing warm air into a space you want to keep cool.
I agree that blowing warm air over components which should ideally have cool air blown over them isn't the best idea; however, the H100 seems to be that much overkill for keeping a stock CPU cool that the air that blows through it is not even warm until you peg the CPU and HDDs, and even then I suspect that the HDDs contribute as much as the CPU. I think if I were to be running a 2687W or 2690 or similar it might be more of an issue but as it stands I'm pretty happy with the temperature of the air blowing over the motherboard. I would rather the radiator be where the air is exiting the case, really, but I suppose then it might be harder to pull the air in over the hard drives...

I did have a number of Noctua 140mm fans on-hand when building it and they didn't quite work out - though I didn't try staggering them on either side of the radiator, now that I think about it.

Which 4U case has room for a 120mm fan over the I/O shield? That would be awesome! A H80 would be more than enough for this setup...
 

mobilenvidia

Moderator
Sep 25, 2011
1,956
212
63
New Zealand
Just had a closer look at my case, I always thought it was a Tower/4u case, but it's 200mm high so more a 4.5u
The extra 1" height gives it enough room for a 120mm fan.

So yes a 4u case will not fit a 120mm fan

Asetek make very nice cooling for servers

Would a 95mm radiator fit above the I/O shield in a Norco 4224 ?
Like this one


Liquid cooling in a rack case is definitely the way to go, just need to get the radiator to the back of the case, there used to be 80mm fan versions but seems no more.
 

sotech

Member
Jul 13, 2011
305
1
18
Australia
Just had a closer look at my case, I always thought it was a Tower/4u case, but it's 200mm high so more a 4.5u
The extra 1" height gives it enough room for a 120mm fan.

So yes a 4u case will not fit a 120mm fan

Asetek make very nice cooling for servers

Would a 95mm radiator fit above the I/O shield in a Norco 4224 ?
Like this one


Liquid cooling in a rack case is definitely the way to go, just need to get the radiator to the back of the case, there used to be 80mm fan versions but seems no more.

Interesting... I've not seen an all-in-one rad smaller than 120mm before. I'll do some measurements next time I have the chassis out of the rack; I can't find any specific dimensions (or a model number) for the picture you have above - can you tell me which Asetek model that is?
 

mobilenvidia

Moderator
Sep 25, 2011
1,956
212
63
New Zealand
Last edited:

sotech

Member
Jul 13, 2011
305
1
18
Australia
Last edited:

sotech

Member
Jul 13, 2011
305
1
18
Australia
Well, the new PSU arrived today but I won't have time to put it in until next weekend :( I did have time to take a photo of it, though.



The server seems to pull around 350-400W a lot of the time which falls into where this PSU is most efficient - ~93%. That should hopefully make a small difference to our energy bill, though the environmental benefit is probably more of an appropriate focus since the additional cost of the PSU will take a long time to recoup through the energy savings.