Need guidance on what OS/storage solution to choose for my media server

[zicoz]

Hi, I am currently mulling over what OS to run on my media server.

I have been trying out a variety of solutions over the years, starting out with Windows Home Server, then Freenas, and some other ZFS-sollutions like Solaris, OpenSolaris and NexentaCore.

With Windows 8 I moved back to Windows and Storage Spaces, but I later outgrew the 63TB limit on the pools.

Recently I've been going back to Freenas, but I've been having some issues which makes me want to look into what other options there are.

I am currently running RaidZ3 with 1 pool of 11 x 6TB and another of 11 x 4 TB, and I have 10 x 3TB drives ready to be added, but I feel is that I lose a lot of potential storage with RaidZ3, and it's very expensive to have to buy 11 drives if I want to expand the server.

I have 1:1 backups of all the content on the server, so a crash won't really cause any real issues other than having to copy the data over to the server, and the server being down for a few days.

Like I said earlier I used to use Storage Spaces, and I was running that without any sort of raid-like redundancy and I was fine with the risk of doing that, the reason why I moved away was because of the 63TB limit.

So basically I am looking for something that is easy to manage, that can give me a stable media server and a Plex-server. What would you suggest that I use for this? Should I stick with Freenas, or go for something else? I am looking for something that I can grow to at least 150TB.

 

[whitey]

Well for 'What OS to run your media server on' I would pretty much stick to Linux/Plex (Ubuntu LTS and latest plex) but I don't think that is really what you are after and you thread title may be a bit misleading. What it appears to me you are after is 'what OS/stg solution to use that can address your massive 100+ TB needs AND serve as a media server'? (assuming plex here)...To that end I would likely stick w/ FreeNAS and a jail of linux (can jails do this on freebsd...run Linux in jails?) If that is supported I would take that route otherwise stick to whatever stg platform meets your needs and are most comfortable with and run some sort of virtualization layer/setup (array to choose from from vSphere (w/ lics), ESXi, KVM (libvirt/oVirt, virt-manager), Xen, LXC/D (containers/docker tech), openvz, bhyve, OpenStack, etc.) and run a Linux VM/container w/ Plex media server on top of your hypervisor of choice.

Running a media server is drop dead simple w/ Plex...running a scalable NAS/SAN as easy...yeah NOT so much :-( (lots of choices/tech in this space, pick your poison/make your bed...lay in it/eat your own dog food) :-D

 

[rubylaser]

As Whitey said, there are tons of options here. I would ask a few questions as well. In a perfect world, would you feel most comfortable with this running in Windows, Linux, or FreeBSD? You mention easy to manage... Does that mean that you have to have a web GUI to manage files or are you comfortable occasionally using the CLI to add disks or make changes (partition disks, edit /etc/fstab, schedule cron jobs, etc.). What type of data is this (pictures, movies, tv shows, virtual machine hard drives, backups, etc.)

Just some things to consider. ZFS has an awesome feature set, but as you mentioned, it can get cost prohibitive to add disks 11 at a time to match your current config. I assume that when you say you are using RAIDZ3, you mean each vdev is losing 3 disks to parity (that's alot of space to lose in your case). Another thing you have to consider is the difficulty you might have to free up storage space if you decided to move to a new OS or storage scenario (you all already heavily invested in ZFS).

You could continue with ZFS or consider one of the snapshot RAID solutions that are more flexible, but have their own shortcomings. I can just tell you what I have done for my home media storage. All of my VMs sit on a ZFS RAID10 array with a S3700 log using ZFS on Linux (in Proxmox 4). All of my bulk home media sits on an Ubuntu 14.04.3 64-bit server and uses SnapRAID with mergerfs as the pooling solution. SnapRAID is open source, supports up to 6 parity disks, only one disk needs to be spun up for a read (great for a media server, so the whole array doesn't need to be spun up, saves energy), provides bitrot protection via scrubs, works great with disks of varying sizes, and allows you to add one disk at a time if you'd like.

Here's a table that compares most of the common solutions.

Comparison Chart

 

[whitey]

If you wanna get REALLY creative and live 'on the edge' (and you probably don't) you could try rockstor (btrfs goodness) out :-D They have bar none the slickest/cleanest/most functional web based GUI that I have seen for OpenStorage solutions yet w/ FreeNAS coming in at a close 2nd. I honestly feel they are 1-2 or maybe even 2-3 years out on a rock solid product w/ the issues I encountered (replication broke, gui snapshot button not working regression...both to be fixed in upcoming releases...sure there are others I am forgetting or that others know about) but all in all it looked like they had a solid foundation and were marching in the right direction.

 

[zicoz]

Thank you all for your replies, I guess there is a lot more to look into than I thought.

The machine is almost exclusively used for media (as in video and picture files). The only exception is that I might move some other stuff over for temporary storage if I need to reinstall one of my computers.

I'm pretty sure I want a GUI, I used command line back when I first started playing around with ZFS years ago, but I guess I've become a bit spoiled over the last few years.

Yes, I mean 3 parity drives per vdev, so I basically loose close to 18TB on my vdev with 6TB drives.

 

[gea]

Every solutions has its own advantages and every user has its own preferences based on experience and use case.
Let me give you my view of things.

If you look for a pure storage system with SMB, NFS and iSCSI with a focus to performance, simplicity and security, try a Solarish system like Oracle Solaris (most feature rich ZFS at the moment) or the free fork OmniOS. Media clients are often ok with a pure filer or build their own mediaserver upon.

If you want ZFS with Plex support, try something based on BSD like FreeNAS

While data security is unbeaten with ZFS as a filesystem, ZFS software raid is realtime raid what means that all disks are running on access. Backup solutions based on raid technology that build a parity disk on demand like snapraid can be an option for a pure mediaserver - on ZFS or any other filesystem as you do not need bitrot/ checksum protection from initial save, realtime protection or performance that is better than from a single disk.

 

[zicoz]

After reading a bit on the Rockstor site as well as the comparison chart on the it seems that I should be looking for something BTRFS-based. It seems to be ZFS-"like", but with the ability to grow pools, so instead of having to buy 10-11 drives when I want need more space, I can buy 3, and then just add them one by one after that up to a certain point and then buy 3 more when I need to start the next pool.

But does BTRFS support "pooling of pools"? As in can I put several BTRFS-pools into a single volume the same way I can pud several Vdevs into a single volume on ZFS?


Snapraid looks good as well, but the lack of GUI scares me quite a bit.

 

[rubylaser]

The only issue with brtfs is the very recent addition of RAID6 and a maximum of only two parity disks. I wouldn't trust my data to in in RAID6. In a few years once it is feature compete, it will be awesome.

 

[zicoz]

Ugh, there is always something isn't it?

But as long as I have everything backed up on "offline disks" that shouldn't be a problem should it? (Other than having to copy everything back to the file server if it crashes.)

 

[rubylaser]

Nope, as long as you have a backup, you can certainly give it a try. I would encourage you to set it up in a VM and see how it works. Fail some disks, add disks, etc.

 

[vl1969]

I have been using btrfs for the last 2+ years. Experience mostly positive. I did have a data loss of almost 2tb when a drive failed in an raid1 pool. It seams to be a bug in how btrfs uses raw devices. Now I prepare the drives by gparted or parted and create a partition which I than use in apool. This way I never had any issues so far and in all my testing I never loos any data as opposed to using raw devices.

 

[gea]

Sun invented ZFS on Solaris ten years ago with a revolutionary new philosophy of how a filesystem should work. In the meantime btrfs and ReFS adopted the basic principles like checksums, copy on write and versioning based on snaps. While they were wanna bees in the first, I expect that btrfs and ReFS can be a technical alternative, at least in the next years.

One aspect remains. ZFS ist the best of them as it evolves as well. It is 5 years ahead and it is available now on BSD, Linux, OSX and Solaris - everywhere beside Windows while btrfs is Linux only and ReFS is Windows only.

 

[zicoz]

Ok, I've been playing around with it for a little bit now, but I can't seem to find a way to "pool" two raid-arrays.

What I want is to create three raid6 arrays, one with 3 TB drives, one with 4TB and one with 6TB drives, and then have them show up as a single share like you can with vdevs on ZFS.

Am I missing something, or is this simply not possible?

Having 30-40 drives in a raid2 seems sort of "dangerous", and must take a lifetime to resilver if a drive fails.

 

[snakyjake]

Snapraid looks good as well, but the lack of GUI scares me quite a bit.

Check out OpenMediaVault and the plugings.

I really don't understand why everyone wants ZFS like RAID for home media storage. SnapRAID is the best for me.

 

[rubylaser]

Ok, I've been playing around with it for a little bit now, but I can't seem to find a way to "pool" two raid-arrays.

What I want is to create three raid6 arrays, one with 3 TB drives, one with 4TB and one with 6TB drives, and then have them show up as a single share like you can with vdevs on ZFS.

Am I missing something, or is this simply not possible?

Having 30-40 drives in a raid2 seems sort of "dangerous", and must take a lifetime to resilver if a drive fails.

This is one of the main reasons why I said above you have a maximum of only 2 parity disks. There aren't vdev's in btrfs. I would consider either staying with ZFS or looking at something like SnapRAID/FlexRAID that I previously mentioned. Maybe start with (3) 6TB disks as parity disks and all the rest for data. Replacing a disk is as simple as running a -e fix on the one missing disk and does not require a complete resilver like a ZFS array. The mention of OMV is another option. You could try out SnapRAID + mergerfs as the pooling solution. The downside of this is that you only get single disk speeds with this sort of setup (with modern disks, it's still fast enough to saturate gigabit).

 

[zicoz]

Ok, thank you once again for both your knowledge and your patience

 

[mjt5282]

I run a FreeNAS 9.3 media server (pair, primary & backup). my bluray and dvd rips are MKV and my audio CD rips are FLAC. I run FreeNas 9.3 + Plex in a jail. I find The scanner/transcoder/interface on various platforms just fine (Roku and Apple TV 4). I run the video files on a separate 8x6Tb raidz2 pool, the audio files and random other datasets are on a 6x3Tb raidz2 pool (which I am thinking of shrinking into a 5x3Tb or even 4x3Tb raidz2 pool if I can delete some largeish datasets). If you are familiar with the UNIX command line, it is pretty straightforward. I run the jails on a zfs mirror Intel S3700 SSD files. The jails pretty much run 24x7 so it allows the data drives to spin down 95% of the time.

My audio server backend is being ported to Linux - when FreeNas 10 comes out it will have virtualization technology to allow me to run FreeBSD or Linux jails so I can take my audio server (Roon) and run it on the jail on my server (headless). ZFS is very powerful and stable, performant on the FreeBSD platform. It is probably overkill ... but my ripped CD and BD/DVD collections are very large and would take forever to re-rip.

 

[snakyjake]

I also wanted to mention that SnapRAID is super simple and easy to experiment with. You can download it to linux/windows, setup some virtual drives, and play/test with it. The concept is so simply basic it doesn't really need a GUI. OpenMediaVault does have the web GUI plugin. Again, you can create a VM and play with it, and the learning curve is super simple.

Another mention...experimenting with OMV+SnapRAID+mergerfs gives you a better idea of what's going on with your storage. This is important because I think there are trade-offs with competing concepts.

 

[uOpt]

I think the attractive solutions these days are:

- Linux md + ext4fs. Only up to raid6. Very robust. No features
- Linux md + btrfs. Maybe less robust in the FS layer but still robust in the raid layer. Adds snapshots and offline dedup.
- Linux, FreeBSD or Solaris + ZFS. Theoretically better but maybe complicated raid layer. But you can go raid"7". Snapshot all you want. Working online dedup if you have tons of RAM.

If you have a full backup I think ZFS is doable. The code is literally the same in FreeBSD and Linux.