All-In-One ESXI, OpenIndiana, Ubuntu, MythTV & Zoneminder....

[337Manni]

Hi all.

I’ve been reading into ESXI and VM’s and I have come to the conclusion to merge several long term projects into one. I originally set out to build a NAS, then later a media server then a CCTV DVR. I’m now looking to build an All-In-One Server/NAS. I’m looking into having the following setup:


Software

ESXI as the base layer

OpenIndiana VM for controlling my NAS
NAS - RAIDz2 using 4 or 6 x 2TB HDD (To start with)

Ubuntu VM
MythTV Installed for a backend Media Server
Zone Minder Installed for CCTV system (DVR)


Hardware

I have a 4U server case with 12 hot swap SATA drive bays at the front. 2 HDD bays inside and supports up to an EE-ATX sized motherboard.

PCIe Cards
NIC Card - Silicom PEG6i - 6 x GB Ports
DVR Capture Card - LinkDelight 4 Port
HBA - 8 Port Sata/SAS Controller
DVB-T2 HD Tuner - BlackGold BGT3650 Quad DVB-T2 / T

HDD’s
6 x 2TB Seagate ST2000DM001 HDD’s (RAIDz2)
1 or 2 x 60GB OCZ Agility 3 SSD (For VM’s)

Motherboard - Supermicro X9DR7-LN4F (More than likely this one….)
CPU’s - 2 x Intel Xeon E5-2603 - 4 Cores, 1.8Ghz, 10MB Cache, 80W
RAM - 4 x Kingston KVR1333D3S4R9SK2/8G (32GB Total)

What do you think? Can you see any problems I may have with the hardware…?
Any improvements / changes I could make?

I intend to put the VM’s on one or two SSD’s for performance and back them up to the RAIDz2.

I have a couple of questions though.

1. On the X9DR7-LN4F can the LSI 2308 SAS controller be put / flashed into IT mode?

2. If not is this a problem for OpenIndiana?

3. If I store my media on the RAIDz2 that OpenIndiana controls, and use Ubuntu with MythTV for a media backend to stream to other PC’s on my LAN, to which VM do I pass the NIC to, Ubuntu, OpenIndiana, both?

4. What size PSU would you recommend and any particular brand / model?

5. I’m sure I’ve read somewhere that with RAIDz2 6 or 10 disks are preferred. Is this true and why? I would like to start with 4 disks and add 4 at a time due to cost, but will use sets of 6 disks if there is a valid reason.

6. Anyone got any tips / hints or guides for my project?

 

[dswartz]

In general, you want the number of disks to be a power of two (I believe that was the deal.) Keep in mind the performance hit is pretty much negligible though (e.g. this tends to be over-hyped...) Why do you need to pass the NIC through? If you really need multiple NICs for various VMs and such, create multiple vswitches, assign the requisite NICs and map the various vnics for each VM to the proper place. This is much better, since you can give the VMs nice standard NIC types (I prefer e1000) and not have to worry about driver issues.

 

[PigLover]

From testing I did the performance degradation from using a non-ideal config is very small and probably doesn't matter outside of the most usage-intense enterprise configs.

However, the best practices document says that for ZFS RaidZ you want the number of drives to be a power of 2 plus the # of parity slices, e.g.,

For RaidZ1 you want 3, 5, 9, etc. (n^2 + 1 parity)
For RaidZ2 you want 4, 6, 10, etc. (n^2 + 2 parity)
For RaidZ3 you want 5, 7, 11, etc. (n^2 + 3 parity)

 

[Tim]

Hi

Using ZFS you'll want as much RAM as possible.
Given that you want only 32GB of RAM, I don't know why you're spending the extra money on the socket 2011 system instead of the socket 1155.
At least give each CPU 32GB of RAM and provide enough RAM to the VM running the ZFS filesystem.
Read up on ZFS and RAM usage to find out how much you need.
Unless you are planning on more RAM soon and really need the dual CPU (I can't see the need for the powerfull CPU that you list for this need, it would just idle).

I don't know what frontend you have for the MythTV backend you're planning. (I'm guessing MythTV frontend?)
But I see that you want to use a DVB-T2 PCIe card under vmware.
My setup is an Apple TV 2 with XBMC as frontend and (soon) the tvheadend with a DVB-S2 PCIe card (TBS6981) under vmware as a backend.
Just wanted to give you a heads up on the problems on virtualization of these cards under vmware if you don't already know about it.
Or, if you've found a solution, please share it.
In short, the problem is with the passthrough (intel vt-d/ vmware vmdirectpath) of the card and interrupts/realtime behaviour.
I think that your motherboard choise is good but google around to see if the DVB-T2 card is possible to use in passthrough.

Regarding performance on disk for the VM's.
I think I would boot the ESXi from USB or from one of the internal SATA ports. (USB port to free up a sata port as you'll want sata3 for the SSD)
And then take a look at this, on how to setup your SSD for best performance. And store/run your VM's on/from those SSD.
http://forza-it.co.uk/esxi-5-1-using-raw-device-mappings-rdm-on-an-hp-microserver/
Then using passthrough to forward the 8-port HBA to the VM running the ZFS based NAS.

You'll want to share your media files (pictures, movies, music) and other files you want to share (documents) via NFS. (NFS provides the best speed).
NFS should be configured on the NAS VM.
Your frontend/clients then just use NFS to get the files directly from the NAS, I don't see the need for a dedicated MythTV for sharing files.
On the other hand, your MythTV backend might be needed for sharing of the DVB-T2 card to your frontend that you're going to watch DVB-T2 signals on.
But no need to mix in the NAS stored files with the MythTV backend, that's just another layer of trouble.

I'm not sure what frontend needs you've got (for media) since you want to use mythtv backend to stream mediafiles.
NFS from the NAS should be enough.
I understand you need a backend for the DVB-T2, but take a look at tvheadend instaed, it might care for your needs.

Also, as dswartz comments, I don't see why you'll want dedicated NIC for each VM unless you need to dedicate speed and/or security.
Just do what dswartz tells you and you'll be fine.
I guess the multiple ports NIC is for the CCTV if you're using IP based cameras? (and that you'll want to passthrough that one aswell to a dedicated VM).

RaidZ and disks.
Quoted from here: (after choosing the raidz options and 2TB disk size and 4 hdd's)
http://www.servethehome.com/raid-calculator/
"RAID-Z uses one disk for Parity much like RAID5 and requires at least three drives to be used."
So 4 disk of 2TB will give you 5.5TB of usable storage.
"RAID-Z3 uses three disks for Parity and requires at least five drives to be used."
So 4 disk of 2TB will give you 1.8TB of usable storage.

My backup plan is simply to allow for media files to get lost (movies/tv-shows, music).
But use ZFS snapshots to increment backups of important datafiles (pictures/documents) to a spare HDD.
This let me sleep at night without robbing the bank for extra disks in raid systems (I don't see the for need for raid in home use since I've got ZFS snapshot instead).
I've no solution for backup of the VM's yet. Some sort of hardware mirroring would be the easy way to solve this I think, but here I'm open to suggestions.