HP DL320s ZFS PLEX server

[NEO Dan]

Operating System/ Storage Platform: looking at Ubuntu+ZFS
CPU: Dual-Core Intel Xeon Processor 3070 (2.67 GHz, 65 W, 1066 FSB)
Chassis: HP DL320S - 14 LFF bays
Drives: 5x WD RED 3TB + 1 or two SSD's for OS and logs
RAM: 6 GB
Add-in Cards: HP SmartArray P400 or P800 ---> swapping to IBM M1015
Power Supply: HP 575W
Other Bits: iLO 2, Windows 2003 Storage Server license built in. HP ProLiant DL320s Quick Restore DVD

Usage Profile: Home media and all around file storage box.

Notes:
iLO 2:
Great for remote headless installations where you would want to hide the server in an inaccessible area, default unlicensed configuration is sufficient to configure the server BIOS and issue power commands.
The built in remote console is not worth buying an "Advance" license for, good old remote desktop is much easier on the eyes IMO...
Catch 22: iLO requires a dedicated network connection and is meant to be powered continuously, the PSU fan will run even with the server off.


Embedded Intel ICH SATA:
To date I've not been able to get these ports working.

 

[NEO Dan]

So this is my attempt to get my media into one bucket and centralize my storage and backups.

I chose this box with hopes that it will be easy on power and yet have enough power to handle PLEX and ZFS.

I'm using this as a learning experience, as I've not had any in depth hands on experience with Ubuntu/nix or ZFS.

 

[NEO Dan]

I am waiting on some SAS cables for the new M1015 controller.
In the meantime I have made a test install of Ubuntu 10.04.4 Server 64bit.

UPDATE the FW on the box, BIOS, P800 RAID card, iLO 2 - it is available in one bootable package, just burn and boot the DVD. I used the unattended/automatic option, there was some delay at the end before the reboot, but it went without a hitch. Get the HP FW DVD here.

 

[Zuhkov]

Very interested to hear how your build goes. Are you planning 1 5-drive RAIDZ pool?

I use unRAID virtualized under ESXi for media storage (including Plex and SageTV) and backups, but I have been experimenting with ZFS (OI virtualized under ESXi) though just for VM storage.

 

[NEO Dan]

Hi Zuhkov,
It's going albet slowly... Thanks for voicing interest.

Interesting setup you have going there, what hardware do you run on?

I won't be fully up and running till next week, but today I did receive one of the SAS cables and so I was able to install the M1015 card.

The M1015 was not supported by the Ubuntu 10.04.4 64 bit Server LTS installer CD, no drives show so the installer stops to ask for storage drivers. The 12.04.3 64 bit Server LTS OTOH did work and installation goes without a hitch.

Plex is installed and ready.

 

[NEO Dan]

Time for an update...

The 5 disk RAIDZ scrubs at 230 megs/s and it has no problems serving Blu-rays simultaneously to multiple PLEX clients, including a transcode for an iPod touch 4g.

 

[mrkrad]

Maybe 33_viper_33 will see this and know if its inline with what MWR could use?

I've got two of these rotting but they had the DVD rom option in the rear. One of them I think I taped up the dvd-hole and force mounted 4 SLC ssd for boot.

 

[33_viper_33]

Maybe 33_viper_33 will see this and know if its inline with what MWR could use?

I've got two of these rotting but they had the DVD rom option in the rear. One of them I think I taped up the dvd-hole and force mounted 4 SLC ssd for boot.

I've been paying attention to this one. It is similar for what I was thinking for MWR, but more complex than I'm prepared to accept for MWR purposes. The hardware I think is perfect. My issue is I don't know who will be managing the server after I'm gone and need to apply the KISS principle. I'm thinking 2003 Server, JBOD storage (3TB drives maby), vmware server with pfsense as a VM. Remote access would be preferred since the system is going to be locked away in a storage room.

 

[NEO Dan]

Um... What's MWR?

 

[NEO Dan]

You can mount SSD's in these trays quite easily, one little snip and you are good to go...

 

[rnavarro]

I've gone one of these in a data center with an M1015 in it running ZFS. I've got 8x 1TB drives, 4 drives in a raidz1 striped into a single pool. It provides mass storage for my ESXi host.

Solid platform I think

 

[33_viper_33]

Moral, Welfare, Health.

I repurpose my old hardware and donate to our deployed troops instead of throwing it away. Generally, they use it for a router and movies but sometimes they add game servers. I've donated one complete machine belonging to me and I've pieced together another three from parts bins belonging to both friends and I.

The past servers have been old P4 and AMD systems and a 1 x core 2. I was thinking of using ESXi for the next one but fear it is too complicated for the average user as well as lacks support for virtualization on old hardware.

 

[NEO Dan]

Time for a small update...

I've now put >1000 movies in the pool and I've learnt an incredibly important lesson...

DO NOT try to write multiple files simultaneously, you will pay for that in array performance. I spent the better part of a week doing batch copies out and back in to clean up the mess. It really makes a difference for a modest system like this...

                                                  capacity     operations    bandwidth
pool                                           alloc   free   read  write   read  write
---------------------------------------------  -----  -----  -----  -----  -----  -----
zp00l                                          6.14T  7.48T  2.25K     25   281M  43.8K
  raidz1                                       6.14T  7.48T  2.25K     25   281M  43.8K
    scsi-SATA_WDC_WD30EFRX-68_WD-WMC4N0372100      -      -    903      8  70.6M  45.6K
    scsi-SATA_WDC_WD30EFRX-68_WD-WCC1T0701336      -      -    984      7  70.6M  45.6K
    scsi-SATA_WDC_WD30EFRX-68_WD-WCC1T0584402      -      -    972      8  70.7M  44.8K
    scsi-SATA_WDC_WD30EFRX-68_WD-WCC1T0704770      -      -    988      8  70.6M  44.8K
    scsi-SATA_WDC_WD30EFRX-68_WD-WCC1T1463196      -      -    978      8  70.6M  41.6K
---------------------------------------------  -----  -----  -----  -----  -----  -----

  pool: zp00l
 state: ONLINE
  scan: scrub in progress since Wed Nov 13 19:06:49 2013
    614G scanned out of 6.14T at 320M/s, 5h3m to go
    0 repaired, 9.76% done
config:

	NAME                                           STATE     READ WRITE CKSUM
	zp00l                                          ONLINE       0     0     0
	  raidz1-0                                     ONLINE       0     0     0
	    scsi-SATA_WDC_WD30EFRX-68_WD-WMC4N0372100  ONLINE       0     0     0
	    scsi-SATA_WDC_WD30EFRX-68_WD-WCC1T0701336  ONLINE       0     0     0
	    scsi-SATA_WDC_WD30EFRX-68_WD-WCC1T0584402  ONLINE       0     0     0
	    scsi-SATA_WDC_WD30EFRX-68_WD-WCC1T0704770  ONLINE       0     0     0
	    scsi-SATA_WDC_WD30EFRX-68_WD-WCC1T1463196  ONLINE       0     0     0

errors: No known data errors

ZFS not needing a defrag is totally dependent on your setup, average home systems may not have the resources to keep the writes to disk clean and orderly.

 

[Patrick]

Nice update Dan! Defrag is probably less relevant if you are doing write once and store movie files.

 

[NEO Dan]

Hi Patrick,
When things go bad IO drops to ~ 20-30 M/s and the system acts like it's having a stroke. I couldn't imagine using it as a primary data storage and backup box in that state.

Long story short: don't outrun the cache and life is good...

That is if you can stand the fan noise, it's way too loud for me. :banghead:

 

[MiniKnight]

Single parity penalty. Somewhat expected behaviour.