What are people doing for backups?

[jcl333]

No matter how good your storage server is, of course you need to back it up.

The main things I need to backup are my family pictures and videos, and some important stuff like tax documents. I am inclined to do something for the Blu-ray ISOs, but that would basically require a second storage server to handle that much data, I can re-rip them, it is time consuming but I won't lose data that can't be brought back.

So, here are the options I can think of:

* Put an array in another computer in my house - just for daily/weekly backups, cheap solution but would do the job
* Setup another array on the same box - this would protect against array failure or data corruption, and backups would be fast
* Attach one or more external USB drives or similar - not that expensive, and you could bring some of them to another location
* Setup another storage server, maybe smaller - more expensive, but completely redundant
* Buy a QNAP or Synology - complete redundancy, low power, uses different technology, but expensive

And finally, there is the issue of off-site backups of the very most important data. I was thinking of locating some kind of a box at my parents house, possibly upgrading their internet connection to Comcast for business so there is no monthly data cap (which I currently have at my house). I could move the initial data over with an external USB drive, which could also work for the occasional large data dump. Other than that it is just a matter of a data replication solution.

Of course, you may all say go with one of the many cloud providers, there are issues with that as well but it is an option.

So, I would very much like to hear what others do about this issue.

-JCL

 

[cactus]

I use SpiderOak(encrypted) and Dropbox and I don't have backups of movies and music. My dad uses Carbonite, which is fine if all your data is on local storage of a single computer, and in my experience is slower than molasses when backing up and restoring. I have read OK things about Crashplan and it is cheap.

How many computers are accessing the data? If it is only 1, add a local array and backup to your server. If multiple, I like two arrays in a single box or USB drives for home environments where downtime does not lose money or make clients/bosses angry. A single server does leave you with a single point of failure, but USB can be moved to another computer. I feel Blu-Ray ISO/Music is backed up enough with RAID.

If the amount of vary important data is small, I would go with cloud over a relative's house. Relative's house has more points of failure, my guess will cost more in the end, and require maintenance/thought of a remote box.

 

[sotech]

We use Microservers off-site - the available internet connections here usually aren't sufficient to rsync large amounts of data across the web, so the backup procedure is for someone to pick one up, bring it here, plug it into the LAN, rsync any changes, take it back offsite again. Smaller files get rsync'd over the 'net regularly (documents etc.).

Online backup systems like Backblaze or Crashplan aren't practical here due to the extremely limited upload speeds - when you're talking about media creation (photos/video/etc.) anyhow.

Main server is ZFS, two 6-disk raidz2 pools and the microservers are 4-disk raidz1 pools, all scrubbed weekly.

The national 100mb network can't come fast enough... still 5+ years away here though. Physically moving boxes around is an inelegant but necessary solution for us.

 

[jcl333]

I use SpiderOak(encrypted) and Dropbox and I don't have backups of movies and music. My dad uses Carbonite, which is fine if all your data is on local storage of a single computer, and in my experience is slower than molasses when backing up and restoring. I have read OK things about Crashplan and it is cheap.

Thanks for the tip. <sigh>, another thing to research.... I guess my main concern is uploading the rather large home videos in HD of my kids, and killing my upstream bandwidth in the process or potentially long periods of time. So that means finding a well behaved replication solution that either deals well detecting idle periods, has good time scheduling, or both. And then I have to worry about encryption depending on how much you trust the cloud provider in question. Have you found that the provider is the bottleneck or your own upstream bandwidth?

I suppose it might not be so bad once I got past the initial backlog, and then kept up with it from there. But I can see coming back from a vacation with lots of video and taking weeks to get it all up there.

How many computers are accessing the data? If it is only 1, add a local array and backup to your server. If multiple, I like two arrays in a single box or USB drives for home environments where downtime does not lose money or make clients/bosses angry. A single server does leave you with a single point of failure, but USB can be moved to another computer. I feel Blu-Ray ISO/Music is backed up enough with RAID.

Right now it will be 4, and I will be going to 5 soon. They will be both accessing the data and being backed up themselves. But, this is a home environment, so those computers are my gaming rig, HTPC, wife's computer, etc. There is no boss problem, other than my wife ;-)

I was thinking for speed using a JBOD off of one of the SAS ports, but on the other hand USB does give you the easy ability to plug it into other machines and great portability. I suppose one of those Western Digital or Seagate boxes with a couple 3TB drives in them on USB3 would probably be OK. I could buy two of them, and then leave one at my parents house and swap them every time I visit them, which is maybe once every 4-6 months.

I agree with you completely that RAID along is good enough for music and movies.

If the amount of vary important data is small, I would go with cloud over a relative's house. Relative's house has more points of failure, my guess will cost more in the end, and require maintenance/thought of a remote box.

Hmmm, maybe I could go with a hybrid approach. Use cloud for the small/important data like tax records and such, and use the USB drive solution for pictures and videos. That is, a USB box just sitting at the relatives house, not plugged into anything. I agree that something running at a relatives house could become problematic.

-JCL

 

[jcl333]

We use Microservers off-site - the available internet connections here usually aren't sufficient to rsync large amounts of data across the web, so the backup procedure is for someone to pick one up, bring it here, plug it into the LAN, rsync any changes, take it back offsite again. Smaller files get rsync'd over the 'net regularly (documents etc.).

Online backup systems like Backblaze or Crashplan aren't practical here due to the extremely limited upload speeds - when you're talking about media creation (photos/video/etc.) anyhow.

Main server is ZFS, two 6-disk raidz2 pools and the microservers are 4-disk raidz1 pools, all scrubbed weekly.

The national 100mb network can't come fast enough... still 5+ years away here though. Physically moving boxes around is an inelegant but necessary solution for us.

So, you basically have "portable" servers that you move back and forth? How long does it take for the rsync / how often are you swapping servers? Weekly probably?

-JCL

 

[sotech]

Portable, yes - the HP microservers are tiny little boxes that aren't inconvenient to carry about. There are even a few mini-ITX cases which have handles if you wanted to roll your own. At around $200 per server + drives - and that's including ECC RAM and CPU - they're a super cost effective way of having regularly scrubbed, power efficient offsite backups. We used to have boxes of offline HDDs offsite but the issue was that data corruption would occur and we would only find out when the drives were plugged back in on-site, which isn't particularly reassuring for a backup... so now we know thanks to the weekly scrubs.

Weekly is the theory... fortnightly or monthly more likely, with the exception of a big job that is mission-critical and has to be in three places. Everything is duplicated on-site but I don't really count that as a proper backup as theft/fire/etc. will take out both of the copies here. If something big comes through one of the offsite boxes gets brought over pretty well straight away, raw data loaded on, off it goes again.

As for the time - both arrays saturate gigabit so I'm assuming it'll be ~100MB/s, 6GB/min or so - but I've not ever actually measured it after the initial testing of the Microservers... there's just a script that gets fired off and we leave it to run overnight, by morning it's always done. When we were testing LAN throughput originally we found the onboard NIC wasn't as good at sustaining gigabit speeds compared to a PCI-E Intel one so we routinely put one of those in them now.