My new Home Server Build [Build Log with Photos]

Pri

Active Member
Jul 30, 2014
122
50
28
It seems every other year I do a major build and in the interims I do upgrades. This year was a major change for me as it's the first time I'm using a rack chassis at home. Prior to this I used a Lian Li PC-343B case with six x 5.25" to 3.5" backplanes. Whilst that build was functional it was rattly. The backplanes were expensive but poorly manufactured. I eventually turned to stuffing bits of paper between the drive slots just to stop the case from vibrating to pieces. You can see a photo of that here: http://i.imgur.com/pJPxpNC.jpg

So the new server. Before I post the pictures lets just detail the specifications:
  1. Windows Server 2012 R2 Datacenter Edition, not pirated, legit license.
  2. Intel XEON E5-1650 (6 Cores, 12MB Cache, 3.2-3.8GHz) CPU
  3. 64GB (8x8GB) of Kingston ECC 1600MHz 1.35v UDIMM Memory
  4. Dual Socket Asrockrack EP2C602-4L/D16 Motherboard
  5. X-CASE 4U 24 Bay GEN II Rackmount Case
  6. Seven x SAS SFF-8087 to SFF-8087 Cables
  7. SuperMicro CPU Heatsink for LGA 2011
  8. Corsair AX1200i 1200 Watt Power Supply
  9. Artic Cooling 80mm silent fan (I use this to cool the RAID and SAS Expander Cards)
  10. Corsair 120GB Neutron GTX SSD (This is for the Operating System)
  11. 1TB Western Digital Black Hard Drive
  12. Samsung 840 Pro 128GB SSD
  13. 9x4TB Hitachis 8x2TB Samsungs 7x3TB Western Digitals
  14. LSI 9260-4i RAID Controller with 512MB DDR2 RAM + LSI CPU
  15. HP 32 Port SAS Expander Card
  16. 900 Watt CyberPower UPS Battery Backup (21 Minutes run time at current load)
You may have some questions about certain specifications, feel free to ask me why I chose what I did below! :)

Memory:


Power Supply:



Case Pictures:











Installation Pictures:

















Build Complete











Now I'm sure one question you're all thinking is, why on earth am I using an AX1200i in this build? Why am I using a 1200 Watt power supply in a system that is pulling right now 270 Watts from the wall socket?

Well there are a few reasons. Firstly I wanted a platinum rated power supply with a very good warranty and I've been impressed with Corsairs power supplies in the past. I've owned a HX1000 and two AX1200's which I used in desktop builds. The 7 year warranty and the platinum efficiency are nice but also the zero fan noise is also nice.

But still why not buy the AX760i or the AX860i ? both of which offer Platinum efficiency, low fan noise and the 7 year warranty. Well I thought the 860i would be more than enough but when it came time to buy it from my reseller the AX 1200i was only £40 more. And its fan doesn't even begin to turn until it reaches 360 Watts of load. So I figured what the hell. I did however do some research I made sure I would be in the "butter" zone of its efficiency. At the current load (270 Watts) it will be 92% efficient. Its peak is 94% which it reaches between 400-700 Watts load. This is still a huge step up from the Bronze rated 85% or so efficiency I was getting before.

In-fact this server consumes 100 Watts less than my old one while increasing memory from 24GB to 64GB, increasing CPU performance slightly and generally being a much better system.

If you have any questions at all feel free to ask. I know there isn't too many builds using the HP SAS Expander, I've been using that part in my old server for 2 years and moved it to this build feel free to ask about how thats been running. Also the case I'm using the X-Case 424 GEN II is brand new out this month so if you have any questions about that I'm happy to oblige :)
 
Last edited:

Pri

Active Member
Jul 30, 2014
122
50
28
I actually reached the picture limit of 20 in my original post so here is the 21st image I meant to include but was unable.

Specification shot from within Windows

 

rubylaser

Active Member
Jan 4, 2013
842
229
43
Michigan, USA
This looks like a very nice build and good hardware. With the mix of disk sizes and using hardware RAID, are you running three separate arrays to maximize your storage space? Also, is this just bulk storage for media or what are you storing on it?

Thanks for sharing!
 

Pri

Active Member
Jul 30, 2014
122
50
28
I'm using three separate arrays yes. I store media on the 9 x 4TB. I have that setup as a RAID6. Then on the other two arrays they are both in RAID6 also but only have backups on them. What I'll do eventually is swap out those older arrays for probably 6TB Disks and then the current 9 x 4TB will become a backup array.

I do also have a separate backup policy with Crashplan and Backblaze. So this is just more for time saving than critical backups etc.

The server is used for a whole host of things. Surveillance storage for our three CCTV cameras, backups of our notebooks and my desktop system. It stores our Movies, TV Shows, Music and family pictures. It runs Plex for media streaming, It runs some P2P services. I host three separate Minecraft servers on it and Bungee Cord (Reverse proxy for Minecraft). I run some virtual machines on it including XPEnology (for AFP/Time Machine support).

I get really good internet here at home (161Mb/s) so it's more affordable for me to build a server and keep it at home and host services from it than it is to colocate in a datacenter.
 

Pri

Active Member
Jul 30, 2014
122
50
28
Data Center allows you to run more than 2 VM's in HyperV. At the moment I'm using VMWare Workstation 10 as that is what I was using on my previous server when I ran Windows Server 2008 R2 but I do intend to switch that to HyperV in the future. Also I have an MSDN subscription so I get 5 licenses to each individual SKU of all the Microsoft software so the cost difference between Standard and Data Center for me was $0.
 

Marsh

Moderator
May 12, 2013
2,294
1,109
113
Nice ;)

Just wondering the reason installing a XEON E5-1650 in dual socket cpu board.

I do the same sometimes to take advantage off on-board LSI SAS controller, it may be cheaper to buy a dual socket board with LSI SAS controller, than single system board with addition LSI SAS controller.

One more question,
How do you backup?
 
Last edited:

Pri

Active Member
Jul 30, 2014
122
50
28
Quite a few reasons. Firstly I already had a 3930K in my old server. It is essentially identical to the E5-1650 except the XEON has ECC support and a higher RAM ceiling. The standard clock, turbo clock, cache, core count and architecture are identical between these processors.

So when it came time to upgrade I did not want a CPU that was slower than the 3930K I was already using on single threaded workloads due to Minecraft server software not being completely multi-threaded. Now you may think that is quite easy as the range of XEON processors available is quite extensive.

However on a purely core to core performance ratio the E5-1650, E5-1650v2 and E5-1660v2 are pretty much as fast as it gets. You've probably noted already that they are all E5-1xxx CPU's so you can only use one of them in a system and that's probably why you've asked this question.

So now you're wondering why I bothered to purchase a Dual Socket motherboard if I intended to use a CPU that only supports single socket usage.

Well the answer to this is simply money. This motherboard on Amazon at the time I purchased it was £163 brand new. Comparable single socket LGA 2011 boards that support ECC memory are £250-£350 and do not include IPMI or an onboard BMC chip for graphics.

That would mean not only would I need to spend around £85-£185 more but I'd also lose out of band management and a PCIe slot to a dedicated graphics card.

So for me it was an easy decision as the case I already intended to buy supported SSI-EEB sized motherboards.

As a side note the reason I didn't go LGA 1150 or 1155 with an Ivy Bridge or Haswell XEON E3 is I needed more than 32GB of RAM support and I do still need a minimum of six processor cores. I think it is likely in a year or two I will replace the E5-1650 with two E5-2xxx series processors once the prices fall due to people upgrading to the Haswell-E based XEON's on the LGA 2011-3 socket.

But the per core performance situation will be the deciding factor as too much of the software I use cannot make good use of multiple threads and I need them to perform as fast as possible within my budget.
 

Pri

Active Member
Jul 30, 2014
122
50
28
I backup to Crashplan online and I have 2 separate arrays in the same server that backup the main array. Very important things are also backed up to a 4TB External.

The noise level is quite low. This case features high quality dual ball bearing fans and the SuperMicro 4U CPU Cooler is PWM controlled by temperature of the CPU. It is actually quieter than my old server was while featuring much better cooling.

I'm unable to hear it running over a desk fan in the same room it has a very deep humming sound.
 

Chuckleb

Moderator
Mar 5, 2013
1,017
331
83
Minnesota
Ah, you have a similar backup model to me. CrashPlan online and CrashPlan local to a 4TB drive. Also a software raid that I rsync everything over to every so often (~2 weeks).
 

hagak

Member
Oct 22, 2012
85
0
6
I assume you know that without the second cpu populated that you will be limited to the memory slots and probably PCIe slots as well.
 

Pri

Active Member
Jul 30, 2014
122
50
28
I assume you know that without the second cpu populated that you will be limited to the memory slots and probably PCIe slots as well.
I think this goes without saying. You can clearly see in the pictures I populated the 8 DIMM slots surrounding the CPU socket I populated. If I were to populate both sockets only then would I be able to use all 16 DIMM slots.

Only one of the PCIe slots on the board goes to the AP (Application) processor. The rest are all wired to the BSP (Bootstrap Processor). I made sure of all of this before I bought anything, I'd have to be a fool otherwise.
 

TuxDude

Well-Known Member
Sep 17, 2011
615
336
63
Data Center allows you to run more than 2 VM's in HyperV. At the moment I'm using VMWare Workstation 10 as that is what I was using on my previous server when I ran Windows Server 2008 R2 but I do intend to switch that to HyperV in the future. Also I have an MSDN subscription so I get 5 licenses to each individual SKU of all the Microsoft software so the cost difference between Standard and Data Center for me was $0.
All editions of server 2012 allow running as many VMs as you like - even the free HyperV server allows unlimited VMs. What you get with Datacenter edition is free windows server licenses for the VMs on that host, but you need to be running quite a few of them before buying datacenter becomes cost effective. Unless of course you are using an MSDN license as you are, though that carries a large list of other restrictions.
 

Pri

Active Member
Jul 30, 2014
122
50
28
All editions of server 2012 allow running as many VMs as you like - even the free HyperV server allows unlimited VMs. What you get with Datacenter edition is free windows server licenses for the VMs on that host, but you need to be running quite a few of them before buying datacenter becomes cost effective. Unless of course you are using an MSDN license as you are, though that carries a large list of other restrictions.
I was not aware of this. Everytime I read about it it just said Standard was restricted to two virtual machines, I didn't realise it meant the licensing of Windows Server within those VM's I just figured it meant 2 VM's total. Thanks for the clarification.
 

TuxDude

Well-Known Member
Sep 17, 2011
615
336
63
I was not aware of this. Everytime I read about it it just said Standard was restricted to two virtual machines, I didn't realise it meant the licensing of Windows Server within those VM's I just figured it meant 2 VM's total. Thanks for the clarification.
It's all covered here in far too much detail: http://download.microsoft.com/downl...indows_Server_2012_R2_Licensing_Datasheet.pdf

Just the complexity of trying to figure out which and how many licenses I need for things from MS is part of the reason I've switched over to Linux for everything I possibly can. It doesn't help at all that they change what is allowed under different licenses every time a new version is released either.
 

Dennisy

New Member
Oct 9, 2016
2
0
1
59
Hello, I know this is an older thread but I'm hoping you will monitor it. I'm curious if there were any special steps you had to take in order to install Windows 2012R2 considering that the Intel 82574L isn't supported now. I have a total of 3 EP2C602-4L/D16 boards, and 4 pairs of Intel Xeon CPU E5-2670 SR0KX 2.60GHz Eight-Core LGA2011 20MB CPUs, and 4 sets of 128GB (16x8GB) DDR3 1333 MHz PC3L-10600 Low Voltage ECC Registered Ram. I have been fighting back and forth to get 2012R2 installed with mixed results.

The first setup worked well, it has Hyper-V Server 2012R2 installed and has a dozen or so guest VMs running one it. It has Bios 1.7.0 and BMC 0.13, IPMI connection never attempted.

The second board installed and was working with Hyper-V Server 2012R2 until either an update or when I attempted to use IPMI. In either case, the machine started to hang a few minutes after boot and after the OS was loaded. Tried flashing Bios to 1.8 and BMC to 0.18, same situation. Suspecting the board was bad, ordered another. This one has 2 bad RAM sockets so it is running with 14 sticks. Did not have luck with Hyper-V Server 2012R2 so tried the 180 eval of Full Windows 2012 R2. Have all updates installed, along with the hyper V role. It freezes when ever copying large (30g vhdx) from internal ssd to internal hd.

I'm about to go back and try the original board that was suspected bad and try a new install. All CPUs and RAM were bench tested on another EP2C602-4L/D16 without any issues. I either have 2 bad boards or I'm missing something in the Bios or a recent update is killing it.

Thanks for any suggestions you might offer
 

Pri

Active Member
Jul 30, 2014
122
50
28
Hi Dennisy, I'm happy to answer your questions.

Firstly the Intel 82574L that the board uses is still supported by Intel until 2018, they still manufacture and ship this chip to OEM's like Asrock to use in products. You can confirm this on Intels website here: Intel® 82574L Gigabit Ethernet Controller Specifications

It works perfectly in 2012 R2 for me. I didn't need to do anything special.

I've found my board works very well, I've not had any problems setting up a LAG using LACP with the built in 2012 R2 LAN system either.

I will say this though, I also had the random crashing a few minutes after booting and the culprit ended up being the Intel SATA controller software, RSTe which is available on the Asrock and Intel websites. This driver causes my system to crash every time it boots within a few minutes. Sometimes it will get past and work solidly but more often than not it will crash.

After I uninstalled this driver (after going through many many memory dumps to determine the driver was the issue) I now no longer have any boot up crashing at all. Not one single crash since I stopped using that driver almost two years ago.

I would of course advise you to update everything to the latest and contact Asrock if you feel your board is malfunctioning. Mine is really working quite perfectly to this day 24.7 with 2012 R2 loaded with the latest updates. I don't use HyperV on mine, but I do use VMWare Workstation on it in much the same roles as HyperV would be deployed and I have no problems there either.

If you have any more questions feel free to reply and I'll try to answer them :)
 

Dennisy

New Member
Oct 9, 2016
2
0
1
59
Thank you for the quick reply. I didn't expect it so soon. I'm going to go ahead and do another clean install of the 180day evaluation. I've already reset BIOS to default and am going to try with only 1 NIC connected. I'll let you know what happens. I don't want to burn any additional MDSN licenses until I know it works.

I have to be honest with you, since starting to use ASRock boards and then finding ASRockRack, I'll never go back to anything else. Both are some of the best boards I have used, and believe me, I've used a ton. I currently have 11 servers running as VM hosts to about 150 VMs. Even if it turns out that there is an issue with the 2 that I'm having issues with, I will not complain, I really like what I've seen on the one that is functioning. The EP2C602-4L/D16 were my first Xeon boards and I started out planning on having 2 running in failover but if I can get all running I'll end up setting them all up in failover.