New Home Server Build

[Scout255]

Hello All,

I'm looking for some advice on how to best configure my home server. It will be used mostly for storage and backup of files (To a ZFS share, either with solaris or ZoL), a media server, Linux desktops, a few windows VM's for hosting some gaming servers, etc.

My plan so far:

Dell C6145
Opteron 6348 (12 cores X 2.8ghz) X 4
64 GB Memory (16 X 4gb Dimms)

For the storage I was looking for feedback, as the chassis I will be getting is a 2.5" X 24 bays. I need about ~8TB after raid of useable space at the moment, though it will no doubt grow

Option 1: Use 2.5" consumer storage hard drives
Data -> Western Digital Green 2TB SATA2 2.5" drives X6 (Raid Z2) (8TB useable) (~$230 each, $1380 total cost)
Booting, VM storage -> Old OCZ Vertex 3 SSD (128 GB)
Read Cache (ZFS) ->Old OCZ Vertex 3 SSD (128 GB)

The disadvantage of this option is the large cost of consumer 2.5" drives. The ones listed above are ~$230 per disk. The advantage is that I can fit everything in the server "as is" with no add on HBA cards, external DAS, etc.
On a similar note, does anyone know of portable external hard drives that still have internal SATA connectors on them? It seems like WD changed their passports to have a native USB 3 interface so those are out. By doing this, price drops to 150 or so per disk, so significant savings could be realized.

Option 2: Use 3.5" consumer storage hard drives
Data -> Toshiba 3TB Drives SATA3 3.5" Drives X 6 (Raid Z2) (12TB Useable) (~$150 each, $900 total cost)
Booting / Cache - As Above
SAS Card -> 9207-8e (~$400)
External Enclosure -> Unknown (SE3016? Norco?)

The disadvantage of this would be the need to buy HBA / RAID card as well as an external enclosure to house the drives. Even without the cost of the external DAS enclosure, the cost is inline with the cost of the 2.5" option, though with more useable space and faster drives. Any thoughts on what the cheapest way to get an external rack mounted DAS would be? Would it be best to get a Norco case, or perhaps trying to get one of the SGI SE3016's?

Also, due to the amount of ram, ESXI is out for a hypervisor. Any thoughts on what would be a good alternative that isn't too hard to learn to use? I would going to try OmniOS, but it would appear the KVM does not work with non-Intel CPU's so that is out unfortunately.

 

[Patrick]

Why the 9207? I would look for a less expensive LSI SAS2008 solution. With the Opterons you will end up using PCIe 2.0 anyway and for mechanical disks you won't hit either a SAS2008's max capacity or the PCIe bus bandwidth limit.

Also, on the external chassis. There are some good guides on the main site. I would personally go the 3.5" route. If you did need more storage, 3.5" disks can be added less expensively since you need to provide fewer ports (e.g. use 4TB drives max right now in the 3.5" form factor so hitting 16GB is only 4 drives of raw storage v. 8 drives at 2TB.)

 

[dba]

With a Dell c6145 used lightly like you plan to, I would go with ZFS, a simple $100-150 LSISAS2008 HBA**, and as many $90 2.5" 1TB laptop hard drives as you need - or 2TB drives if you think you'll grow to over 20TB. I've had excellent luck with Samsung m8 drives for such purposes. Add an SSD or two for your VMs (and optionally for a ZIL if you need it) and you are done, all in one box. One really heavy and rather loud box, but one box.

Also, unless you have some heavy-duty folding planned, I suspect that you will be wasting around $1,600 or more by buying 48 cores of 63xx CPU - and driving up your power bill. I'd go for one or two $55 6128 HE CPUs, spending any budget excess on 1TB disks and SSDs. You can run an absolute ton of VMs on 8-16 61xx cores, but if you run out of steam - you won't - just add two more. And if that's not enough then you also have the second motherboard sitting there unused.

**LSI SAS9205 4I4E 6GB s PCI E SATA SAS Host Bus Adapter | eBay will give you 2GB/S to the 24 internal drives plus provide an external port for future expansion.

Hello All,

I'm looking for some advice on how to best configure my home server. It will be used mostly for storage and backup of files (To a ZFS share, either with solaris or ZoL), a media server, Linux desktops, a few windows VM's for hosting some gaming servers, etc.

My plan so far:

Dell C6145
Opteron 6348 (12 cores X 2.8ghz) X 4
64 GB Memory (16 X 4gb Dimms)

For the storage I was looking for feedback, as the chassis I will be getting is a 2.5" X 24 bays. I need about ~8TB after raid of useable space at the moment, though it will no doubt grow

Option 1: Use 2.5" consumer storage hard drives
Data -> Western Digital Green 2TB SATA2 2.5" drives X6 (Raid Z2) (8TB useable) (~$230 each, $1380 total cost)
Booting, VM storage -> Old OCZ Vertex 3 SSD (128 GB)
Read Cache (ZFS) ->Old OCZ Vertex 3 SSD (128 GB)

The disadvantage of this option is the large cost of consumer 2.5" drives. The ones listed above are ~$230 per disk. The advantage is that I can fit everything in the server "as is" with no add on HBA cards, external DAS, etc.
On a similar note, does anyone know of portable external hard drives that still have internal SATA connectors on them? It seems like WD changed their passports to have a native USB 3 interface so those are out. By doing this, price drops to 150 or so per disk, so significant savings could be realized.

Option 2: Use 3.5" consumer storage hard drives
Data -> Toshiba 3TB Drives SATA3 3.5" Drives X 6 (Raid Z2) (12TB Useable) (~$150 each, $900 total cost)
Booting / Cache - As Above
SAS Card -> 9207-8e (~$400)
External Enclosure -> Unknown (SE3016? Norco?)

The disadvantage of this would be the need to buy HBA / RAID card as well as an external enclosure to house the drives. Even without the cost of the external DAS enclosure, the cost is inline with the cost of the 2.5" option, though with more useable space and faster drives. Any thoughts on what the cheapest way to get an external rack mounted DAS would be? Would it be best to get a Norco case, or perhaps trying to get one of the SGI SE3016's?

Also, due to the amount of ram, ESXI is out for a hypervisor. Any thoughts on what would be a good alternative that isn't too hard to learn to use? I would going to try OmniOS, but it would appear the KVM does not work with non-Intel CPU's so that is out unfortunately.

 

[Scout255]

Why the 9207? I would look for a less expensive LSI SAS2008 solution. With the Opterons you will end up using PCIe 2.0 anyway and for mechanical disks you won't hit either a SAS2008's max capacity or the PCIe bus bandwidth limit.

Also, on the external chassis. There are some good guides on the main site. I would personally go the 3.5" route. If you did need more storage, 3.5" disks can be added less expensively since you need to provide fewer ports (e.g. use 4TB drives max right now in the 3.5" form factor so hitting 16GB is only 4 drives of raw storage v. 8 drives at 2TB.)

Not totally set on the HBA, I just chose that as it is a card available locally, as it would appear all of the 16 port used LSI cards have been sold on ebay, and the other 6G Sas HBA's with external ports are approximately the same price as new or they will not ship to Canada.

With a Dell c6145 used lightly like you plan to, I would go with ZFS, a simple $100-150 LSISAS2008 HBA**, and as many $90 2.5" 1TB laptop hard drives as you need - or 2TB drives if you think you'll grow to over 20TB. I've had excellent luck with Samsung m8 drives for such purposes. Add an SSD or two for your VMs (and optionally for a ZIL if you need it) and you are done, all in one box. One really heavy and rather loud box, but one box.

Also, unless you have some heavy-duty folding planned, I suspect that you will be wasting around $1,600 or more by buying 48 cores of 63xx CPU - and driving up your power bill. I'd go for one or two $55 6128 HE CPUs, spending any budget excess on 1TB disks and SSDs. You can run an absolute ton of VMs on 8-16 61xx cores, but if you run out of steam - you won't - just add two more. And if that's not enough then you also have the second motherboard sitting there unused.

**LSI SAS9205 4I4E 6GB s PCI E SATA SAS Host Bus Adapter | eBay will give you 2GB/S to the 24 internal drives plus provide an external port for future expansion.

I was looking at the 63XX's as their are a few used ones up on ebay (or atleast there were last week...) for around 350 or so a processor. Also, one of the games I am planning on hosting requires a very high clock speed in order to run a high player count server (needs 3.2 Ghz+ minimum) and as such the 6128's unfortunately won't work well. 'Though it would run great for everything else I have planned. Certainly tempting at that price indeed....

How does warranty work on the Samsung drives now that they no longer produce them? Are you basically out of luck if one fails? Also, do you have any experience pulling drives out of external hard drives for 2.5" drives?

Any thoughts on the hypervisor to run as well due to the ram limitation of ESXi?