So I posted originally 4-5 years back I think with a project that nobody seemed to understand quite exactly what I was doing or why... which is fine, because I realized I was braindumping too much not fully baked information about something that was still forming in my head. I've posted on other boards at different times and nobody had any answers or quoted me $50,000 and up to start making a solution plus needing a sysadmin to run things which wasn't going to work for me.
Fast forward a bit between covid shutdowns and unexpected life threatening medical issues i'm still recovering through things have been a bit stalled including progress towards my film production degree, but I hope in the next year I might be able to explain and show what I was trying to do then - build a D2D2T Disk to Disk to RAIT Tape offline media archiving system designed for long term catastrophe level recoverability (designed from the ground up for maximum protection against silent bit rot, corruption of data AFTER it leaves the server for which ZFS cant protect you, loss of data, loss of volumes, etc) at absolute minimum possible cost per terabyte and absolute minimum overhead cost including power use.
A RAIT means if I have a stripe of 8 data tapes and 2 parity tapes, I can lose any two of the original 8 tapes and fully restore the data and remake replacement main or parity tapes. This means I don't have to store full 2way/3way mirrors in offsite locations (ie 16/24 tapes) but can use 2 or 3 parity tapes to insure against loss of a volume in a set. Combine something like a local 2 way mirror because its faster to recover from a mirrored tape than regenerate a parity tape, and offsite storage in 1+ other locations where it's only 1 way but with 2-3 parity tapes being the goal.
I'm also designing it as part of a bigger D2D2T system on purpose meaning I have a primary NAS for active files, a backup NAS, and a migration path to and from tapes. The goal being all data from the primary NAS is backed up to at least one other disk as soon as possible (maybe not realtime but could schedule several times per day), and nothing is erased from the primary NAS until it is verified written to tape (and not erased from the backup NAS until it's on TWO tapes) so there's never a single point of data loss.
I'm pretty sure places like r/Datahoarder will be interested but i'm curious how interested people here are in the rest of this project as I build it. I could do anything from saying nothing until i'm done to just show the final project to keeping people constantly updated I mean... I just felt I might be on my own to do it all anyway since my original post a few years back left people a bit confused it seems with how to respond.
Fast forward a bit between covid shutdowns and unexpected life threatening medical issues i'm still recovering through things have been a bit stalled including progress towards my film production degree, but I hope in the next year I might be able to explain and show what I was trying to do then - build a D2D2T Disk to Disk to RAIT Tape offline media archiving system designed for long term catastrophe level recoverability (designed from the ground up for maximum protection against silent bit rot, corruption of data AFTER it leaves the server for which ZFS cant protect you, loss of data, loss of volumes, etc) at absolute minimum possible cost per terabyte and absolute minimum overhead cost including power use.
A RAIT means if I have a stripe of 8 data tapes and 2 parity tapes, I can lose any two of the original 8 tapes and fully restore the data and remake replacement main or parity tapes. This means I don't have to store full 2way/3way mirrors in offsite locations (ie 16/24 tapes) but can use 2 or 3 parity tapes to insure against loss of a volume in a set. Combine something like a local 2 way mirror because its faster to recover from a mirrored tape than regenerate a parity tape, and offsite storage in 1+ other locations where it's only 1 way but with 2-3 parity tapes being the goal.
I'm also designing it as part of a bigger D2D2T system on purpose meaning I have a primary NAS for active files, a backup NAS, and a migration path to and from tapes. The goal being all data from the primary NAS is backed up to at least one other disk as soon as possible (maybe not realtime but could schedule several times per day), and nothing is erased from the primary NAS until it is verified written to tape (and not erased from the backup NAS until it's on TWO tapes) so there's never a single point of data loss.
I'm pretty sure places like r/Datahoarder will be interested but i'm curious how interested people here are in the rest of this project as I build it. I could do anything from saying nothing until i'm done to just show the final project to keeping people constantly updated I mean... I just felt I might be on my own to do it all anyway since my original post a few years back left people a bit confused it seems with how to respond.