Looking for a new storage setup

[jseba]

Hey all,

My dad has a nearly full RAID5 array with 8x2TB hard disks attached to an HP P400. It's worked well for him for the past couple years, however it's getting to needing an upgrade. He's looking to purchase 6 4TB drives to migrate the current data set and sell off the old drives. I managed to convince him to switch to the RAID set from his pseudo RAID10 he was doing by using multiple external USB drives and copying everything around. The only problem I've had with the array is the fact that expanding the array is a scary ordeal, hoping that nothing breaks during the rebuild and it's been done two or three times as the array started with 4 drives.

I've talked him into holding off for a few months, as he's got about 2TB of space left and adds roughly 20 to 30GB per week, so he should be fine for a while. I plan on getting him set up with a 2U/3U expander chassis and using ZFS and NappIt/FreeNAS to manage the disks, a simple LSI HBA card and use one of the computers laying around as the host.

My only remaining question is how best to setup the array in software. The array will be regularly expanded, at the rate of either 1 or 2 drives at a time (unless a good deal shows up at some point ). I know using RAID5/RAID-Z1 is unsafe; even though it's worked okay in the past, I know it's only a matter of time before getting bit, especially with drives as large as 4TB. I'd like to know if there's a different filesystem/setup that will allow for pain-free expansion in groups of 2 drives and still handle one or more drive failures without data loss.

 

[TuxDude]

ZFS does not allow expansion or shrinking of a RAID-Z device. There is no way to slowly grow a RAID-Z by just a disk or two at a time - you can either add enough drives all at once to create a second RAID-Z device, which can be put into the same pool as the first one to expand the filesystem, or you can replace the disks in the existing device one at a time with larger ones and then grow the device at the end after all of the drives were swapped. Also keep in mind that there is no way to remove a device from a pool. ZFS has a lot of great features, but flexibility is not one of them.

Btrfs would be the perfect solution for what you are trying to do - supporting growing/shrinking a RAID5/6 pool by a single device at a time as well as using mixed-size devices within the pool efficiently. However it is not yet ready for production use, especially the RAID5/6 bits which should be mostly complete for kernel 3.19 but have virtually no testing yet.

If flexibility is important for you, then I suggest looking at either linux mdraid or snapraid. Mdraid is very mature and well tested, will do online capacity expansion and online migration between raid levels, but does not protect against bit-rot as ZFS/btrfs do. Snapraid does protect against bit-rot and is stupidly flexible in terms of adding/removing drives, but is not a real-time traditional RAID - it does not pool all of the space from all of the drives together, and you have to run a 'sync' job when you want new data to be protected (often done as an automated nightly job). Snapraid is great for things like media-libraries where files are mostly static, otherwise mdraid is probably a better fit.

 

[capn_pineapple]

Just use Xpenology. Will do everything you need and is simple to grow/change disks.

It's just Linux/MDADM, but has a sexy web management portal.

 

[Mike]

Md with or without LVM for the time being can be shrunk when the time comes to migrate to Btrfs. The flexibility in that area is pretty amazing. Most of the hate you see with Btrfs is by people that used it some time, years ago, when unstable meant about to explode. Opensuse is probably the most btrfs minded distro for now.

 

[smccloud]

Personally, I would do ZFS in striped mirrors and expand it 2 disks at a time. Not flexible if you want to use RAID-Z2 (i.e. 6) but bit rot protection is nice and since you said he will probably expand 1-2 disks at a time making it a two disk requirement would not be horrible.

 

[rubylaser]

Personally, I would do ZFS in striped mirrors and expand it 2 disks at a time. Not flexible if you want to use RAID-Z2 (i.e. 6) but bit rot protection is nice and since you said he will probably expand 1-2 disks at a time making it a two disk requirement would not be horrible.

Mirrors for large, bulk storage can become really expensive in a hurry because of the need to have twice as much storage space due to mirroring. Also, his spindle count will be larger, likely driving the need for a new case(s) in the future to expand, and ultimately a mirror does not guarantee maintaining the users data if two disks are lost at once like RAID6 or dual parity does.

For bulk home media storage, SnapRAID is free and is very flexible as previously mentioned. mdadm would be a good alternative though.

 

[smccloud]

Mirrors for large, bulk storage can become really expensive in a hurry because of the need to have twice as much storage space due to mirroring. Also, his spindle count will be larger, likely driving the need for a new case(s) in the future to expand, and ultimately a mirror does not guarantee maintaining the users data if two disks are lost at once like RAID6 or dual parity does.

For bulk home media storage, SnapRAID is free and is very flexible as previously mentioned. mdadm would be a good alternative though.

True, it is really expensive in terms of drives needed for space. But it is the easiest way to use ZFS for self healing file system and not needing 3+ disks to expand the array. Personally, I'm running HW RAID1 in a N40L at home right now and will expand it in the future once I need more than 2TB of space (i.e. once I get a house and can afford rack mount stuff).