backups restorable by non-experts?

[polyglot]

I have a quantity of data, mostly photos, amounting to about 1TB jpegs or 5TB including raws. Currently it lives in a zpool and gets backed up to an offsite zpool so all good for as long as I'm around.

If I get hit by a bus though, my family will have no hope of restoring that backup, and I would like for them to be able to access at least the photos, if not the rest of my zpool.

Problems I have:

• a simple HDD or two might last for 5 years, and I'm not sure I trust helium-filled drives to be reliable in the long term after sitting on the shelf, outgassing
• LTO should last 20 years but they will be as opaquely inaccessible as a box full of zpool parts to my family
• normal optical media can be trusted as far as you can throw it
• I'm trying 100GB BDXL M-DISC this week but failing to get usable outcomes (see below)
• single-layer BD M-DISC might work, but 40+ discs will be hella tedious and expensive
• it would take me a year to upload 1TB to S3 Glacier from my connection, and the $50/TB-yr plus $90/TB restore cost is not very nice

I guess LTO and "pay a recovery service to read them" is a valid answer but I don't like it. What obvious answer am I missing / what does everyone else do?


re the BDXL:

• first two burns are coasters due to a growisofs bug; this is like buffer-underruns on $10 CD-Rs in 1998
• third burn succeeded with cdrskin but it throws a bunch of medium error / L-EC uncorrectable error when trying to verify
• fourth is going now but I don't have high hopes

Got 15x$14 of media sitting on my desk here now and not sure it has any value to me at all

 

[rtech]

Why just not write detailed instructions how to do access your ZFS?
With ubiquitous howtos and even dark power of agentic AI I would not be too troubled about your progeny not being able to access your ZFS pool. I would rather have the best pictures printed out rather than stored on HDD. It it was me. Top 100 pics printed out seems like much better strategy than preserve everything in your ZFS.

 

[Stephan]

Difficult problem.

Forget LTO, that is special tech for true specialists, mostly greybeards at this point. Your family will not know what drive generation it was, have a machine to connect the drive to, have the exact software with exact settings, or still have the file database. Or know how to extract a listing from a tape, if you were smart to backup that too. Any single problem like wrong block size or unknown encryption settings will derail the entire endeavour.

Best bet for recoverability indeed is BD, because UDF and the form factor suggest easy use by almost everyone. I recommend only dual layer, higher layers burn with too many errors for my taste. Burn everything at 1x. Find a LG drive that can be flashed to mk-firmware-pack-20200720\MK\HL-DT-ST\WH16NS58\HL-DT-ST-BD-RE_WH16NS58-1.V5-NM00900-211802070957.bin because then you can use https://github.com/dhucul/qpxtool to scan the burn quality of your BDs.

Maybe you can tier the data so important stuff is burned first.

Other option is some disks formatted to GPT and NTFS (yeah, I know) and rsync from zpool to these regularly. Instructions would be to pull them out of a NAS and stuff them into a USB case for connection to a Windows laptop.

Finally there is the George Carlin-esque issue of "your stuff" vs "their stuff". Will they care about your photos? Care enough? Because their timelines might have tangential but different value attached to pictures. Most data will just die with a person. The truly good stuff will probably fit on a single BD-R. Births, birthdays, that one time you ran from a curious group of cows in the alps. Could have the five best raw files in 1x1 meter high quality durable art print and frame it.

TEMPVS EST IGNIS QVO VRIMVR

 

[polyglot]

Thanks. I guess I did kind of ignore AI but I still need to solve for longevity of media.

I have a BH16NS40; same series of Hitachi/LG drive but much earlier. Not sure if it will run that firmware but I will look into it; it doesn't support qpxtool out of the box.

I got one disc to burn and verify (at full speed, no retries) successfully last night so I am a little more hopeful about BDXL now, and somewhat resigned to 25GB BD if these discs don't work.

You are right about care-factor, and it is absolutely going to be a problem due to timescales. The kids won't give the tiniest **** about this content for the next 15 years at least but 10-20 years after that I think they might want their childhood photos and/or pictures of their (grand)parents, so there's a couple decades gulf with a good chance that nobody actively maintains/verifies/transfers the data to fresh media. My zpools, or ntfs drives in sealed boxes, will be dead by then even with perfect instructions recorded.

Yes I have prints... bulky and expensive though, and short-lived if my parents' photos are any guide. My silver darkroom prints will live on but they take 1-2 hours EACH to make, and I can't make them reliably from digital or C41. Data-hoarder tech seems more scalable

Maybe in 2060 nobody will recognise 120mm shiny discs as being media at all, and they'll be hung in the fruit-trees to scare away birds. Nobody knows what the Save Icon means anymore, right?

 

[BlueFox]

I think you're over-complicating this? If you aren't regularly verifying your backups, then they might as well be non-existent.

A single USB hard drive will do. It doesn't need to survive for years, just whatever the interval is that you happen to be verifying the backup, plus a bit extra. If that's monthly, means should something happen to you and your family takes a month to get around to dealing with it, that's 6 weeks on average since the last time it was tested or 2 months max. Pretty good odds I would say if nothing happening during that time.

 

[TrevorH]

it would take me a year to upload 1TB to S3

You really only have 275Kbps upload capacity? If you have a 10Mbps pipe up then flat out it would take just over 10 days. If you have a 100Mbps pipe then it'll be done in a day. You only have to copy the existing data once and then do incremental changes from then on.

 

[polyglot]

ADSL, it's still a thing. Link speed is 1Mbps up but in practise using even 400kbps results in the connection being borderline unusable for browsing.

I could probably monopolise a friend's connection for a few weeks but it will be annoying for them.

You really only have 275Kbps upload capacity? If you have a 10Mbps pipe up then flat out it would take just over 10 days. If you have a 100Mbps pipe then it'll be done in a day. You only have to copy the existing data once and then do incremental changes from then on.

 

[louie1961]

You are overthinking this. I got a Synology for this very purpose. Family photos and documents that will be important to my wife after I go sit on the Synology. My TrueNAS box is only for home lab apps and as a backup destination for Synology. I back the Synology up to their C2 cloud service, TrueNAS and a USB disk, using hyper backup. Almost anyone can do a hyper backup restore. BTRFS does scrubs, and I have RAID 5 on 4 disks. That is good enough for family photos.