SnapRAID is available for windows as well as linux. I am probably going with linux version so I posted it here, although that's not absolutely set in stone.
Short version of my project: storing lots and lots of digital video (up to 8k and multiple camera motion capture), designing a 32-300TB scaleable NAS to hold and work on that data, digital video data needs to go D2D2T (disk to disk to tape) finally ending up on Ultrium LTO6 tapes.
Things I like about SnapRAID:
- More large systems in existance than systems like FreeNAS (lots of people seem to have 70TB and up, 24 drives a not uncommon configuration, when I looked into freeNAS over 32tb was uncommon and RAM limits under ZFS were an issue)
- Very low system requirements (easy to scale up to those big arrays, even 8gigs seems enough, no "1 to 1" RAM to Terabyte match needed like with FreeNAS ZFS), a 300TB array is doable today if I could afford the drives apparently.
- Lower power use (spools up one drive at a time when needed), was considering a MAID array (not 100% required just contemplating if it's an always on PC)
- No proprietary filesystems (easier possible recover using existing tools if a drive partially dies - no "total pool loss" nightmares like i've heard of with ZFS), I like the fact that it works OVER the file system without changing, you can even use it with data already in place/nothing to migrate in. Or stop using it without having to migrate out data. The separate parity data is just discarded if no longer used.
- Provides robust file integrity verification (my main reason for originally wanting ZFS) checking the whole system with a single command
- Provides protection like RAID does for 1-6 hard drive failures, currently ZFS has nothing beyond triple parity
- Upgrade disks in place at any time. Just first mirror it then swap it. Less annoyance or workaround. Less rebuild time than it sounds like FreeNAS/ZFS will use.
- More easily scaleable. Start with disks of any size. Start with any number. Upgrade size or number whenever. Pretty sure I can downgrade too.
Downsides:
- Doesn't automatically run, I have to schedule or script the snapshots, integrity verification, and similar.
- No built in storage pooling.
- May be limited to single disk performance. (I dont know if anyone has set up RAID under SnapRAID but I dont know why it shouldnt work, though it's not built in. Certainly no state of the art SSD caching and such like i'm told FreeNAS has.)
- May be limited in multiuser scenarios.
So far i'm pretty sold. I can work around the downsides for now, especially since the primary NAS may become a higher performance NAS/SAN in the future, but then the secondary NAS/backup/mirror will still have SnapRAID, and is still used preparing data for backup by the Ultrium LTO drive. So i'm sure SnapRAID is what I need for the second NAS, and if I have to learn it for that I might as well start by using it on the first NAS, even if the performance bottleneck limits me in the future. At least I wont hit limits for data scale out since I need minimum overhead cost per drive and "pay as you go" drive upgrades to store new digital video data. Heavy editing/processing wont happen until notably later probably and can be separately funded.
THAT ALL SAID i'm here to learn and am wondering if there are even any viable alternatives concerning my needs.
Short version of my project: storing lots and lots of digital video (up to 8k and multiple camera motion capture), designing a 32-300TB scaleable NAS to hold and work on that data, digital video data needs to go D2D2T (disk to disk to tape) finally ending up on Ultrium LTO6 tapes.
Things I like about SnapRAID:
- More large systems in existance than systems like FreeNAS (lots of people seem to have 70TB and up, 24 drives a not uncommon configuration, when I looked into freeNAS over 32tb was uncommon and RAM limits under ZFS were an issue)
- Very low system requirements (easy to scale up to those big arrays, even 8gigs seems enough, no "1 to 1" RAM to Terabyte match needed like with FreeNAS ZFS), a 300TB array is doable today if I could afford the drives apparently.
- Lower power use (spools up one drive at a time when needed), was considering a MAID array (not 100% required just contemplating if it's an always on PC)
- No proprietary filesystems (easier possible recover using existing tools if a drive partially dies - no "total pool loss" nightmares like i've heard of with ZFS), I like the fact that it works OVER the file system without changing, you can even use it with data already in place/nothing to migrate in. Or stop using it without having to migrate out data. The separate parity data is just discarded if no longer used.
- Provides robust file integrity verification (my main reason for originally wanting ZFS) checking the whole system with a single command
- Provides protection like RAID does for 1-6 hard drive failures, currently ZFS has nothing beyond triple parity
- Upgrade disks in place at any time. Just first mirror it then swap it. Less annoyance or workaround. Less rebuild time than it sounds like FreeNAS/ZFS will use.
- More easily scaleable. Start with disks of any size. Start with any number. Upgrade size or number whenever. Pretty sure I can downgrade too.
Downsides:
- Doesn't automatically run, I have to schedule or script the snapshots, integrity verification, and similar.
- No built in storage pooling.
- May be limited to single disk performance. (I dont know if anyone has set up RAID under SnapRAID but I dont know why it shouldnt work, though it's not built in. Certainly no state of the art SSD caching and such like i'm told FreeNAS has.)
- May be limited in multiuser scenarios.
So far i'm pretty sold. I can work around the downsides for now, especially since the primary NAS may become a higher performance NAS/SAN in the future, but then the secondary NAS/backup/mirror will still have SnapRAID, and is still used preparing data for backup by the Ultrium LTO drive. So i'm sure SnapRAID is what I need for the second NAS, and if I have to learn it for that I might as well start by using it on the first NAS, even if the performance bottleneck limits me in the future. At least I wont hit limits for data scale out since I need minimum overhead cost per drive and "pay as you go" drive upgrades to store new digital video data. Heavy editing/processing wont happen until notably later probably and can be separately funded.
THAT ALL SAID i'm here to learn and am wondering if there are even any viable alternatives concerning my needs.