Need Advice for local and cloud backup

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

tnkrer

New Member
Sep 20, 2016
28
1
3
54
I have a napp-in-one system ( napp-in-one on dell T20 - my planned setup ) running nicely for almost a year. I set out with the idea that I will add backup after I get the server setup. It's been a year and I need to set up my backup. I read through threads and documents, but I cannot visualize what I need to do to get backups.

My requirements to start with are pretty simple
1. I want to backup/copy certain folders with encryption to another disc. The size of this data will be under 500G. (personal photos and videos and important documents). The backup should be incremental. (ZFS snapshots seems like a great idea, but can I snapshot a folder inside a pool? If not, I may have to break my setup into pools that need to be backed up
2. How can I use the encrypted backup to access files somewhere else. In my past backup strategy, I was using a simple folder sync application that was copying folders from one location to my backup location and it was simple to access my backed up files
3. Here is my setup (without the pfsense and the 1 TB disk is not on the HBA) - In this setup how do I use that 1 TB disk for backup? Can I create another omniOS vm that uses that 1 tb disk and then replicate the snapshot from my napp-in-one to the backup VM? (I am sure my terminology is muddled, since that is my level of understanding at the moment)



Any pointers to guides/how to of the backup set up will really be appreciated, even if they are somewhat different.

Once I figure out local backup, I intend to backup the same to some cloud provider yet to be determined. The purpose of encryption is so that I can move the data as it is to cloud. I don't really need encryption for my local backup

Thanks
 
Last edited:

TangoWhiskey9

Active Member
Jun 28, 2013
402
59
28
Snapshots are great. Just backup everything instead of thinking specific folders.

If something bad happens, you're going to kick yourself if you don't.
 

tnkrer

New Member
Sep 20, 2016
28
1
3
54
Thanks .. I will focus on snapshots as my potential solution, That will mean breaking my pool into two pools, one that I do not backup .. (TV recordings) and one that gets backed up.
So now need to figure out how to make two pools out of one
Here is one zfs snapshot backup plan - That seems good like a good idea and should work in my system -
Automatic ZFS Snapshots and Backups

Also ZFS adminstration document

Lot of reading and trial and error before it becomes real!
 
Last edited:

TangoWhiskey9

Active Member
Jun 28, 2013
402
59
28
Easy to Google that but it's good to work through to learn zfs. Once you're done it'll work great
 

Kybber

Active Member
May 27, 2016
138
43
28
48
Just to be clear: There is no need to divide your pool into two pools, which would require you to destroy and rebuild. What you want is to create more file systems.
 

gea

Well-Known Member
Dec 31, 2010
3,141
1,184
113
DE
You can divide your problem into three demands

- Sync some files onto a backup disk/pool/folder
If the backup target is ZFS, you can additionally use snaps there for versioning

You can use rsync for that if you do not need permissions (rsync does not retain Windows alike nfs4 permissions). You can also use the napp-it datamover (menu filesystems) that allows to setup complex rulesets for copy and sync between local folders and that preserves permissions.

You can use the napp-it "other job" functionality to sync files from your pool to a backup disk/pool/folder automatically.

- Encrypt the data for a cloud backup
This would require to create encrypted file(s) that you can backup to a cloud or any other unsecure destination. You can use tar to create one encrypted file from your data on the backup device. A better method would be creating a ZFS pool on lofi devices. This is a ZFS pool based on encrypted files. Main advantages is that you are able to create ex a raid-z on files. A damaged or missing file would not corrupt such a pool, important if you save the files to unsecure USB devices or disks. With a lofi pool you can simply sync your files with the lofi pool, export the pool and backup the files. The restore, import the lofi pool from the files with snaps, and all the ZFS features like compress, dedup, checksums and redundancy. Lofi Pools are like any other ZFS pool from access. napp-it Pro supports lofi pools from the GUI (menu pools), see Introducing Sparse Encrypted ZFS Pools

- Upload the encrypted backup data/files or other unencrypted files that you want to access directly from the cloud. Use a sync method to sync files from a folder(s) to the cloud. if you want to use a cloudsoftware that is not available on Solaris, use an LX/Linux zone on OmniOS that have direct access to ZFS filesystems.
 
Last edited:

tnkrer

New Member
Sep 20, 2016
28
1
3
54
Thanks Gea .. I am going to use one of the two strategies you have suggested. Use datamover or napp-it other job functionality. (Still need to figure out how both work in napp it UI)
Weekend was spent in getting ny 1TB disk back. At some point I had used it as ESX data store and so had .locker directory on it which was preventing me from removing it from ESX. Now that the scratchConfig (.locker directory) is moved to the SSD, I am ready to move this 1 TB disk to the Dell Perc card, so I can use pass through to let nappit see that disk.

One question though, Once I make that 1 TB disk available to napp it, it will not be seen by any other VM since it is a pass through, right? So only way to access the backup would be through napp it VM instance.. Is that correct?
 
Last edited:

nle

Member
Oct 24, 2012
204
11
18
Yes, that is what pass-through does, lets you dedicate and connect physical HW to a VM.
 

tnkrer

New Member
Sep 20, 2016
28
1
3
54
Thanks nle! Now my 1TB disk is available to napp-it for backup/copy
My napp-it is at 16.02 and it seems datamover was added in 17.04 .. so first step would be to upgrade napp-it .. So that is what I will do this weekend. ETA: Update was way too easy! Had to do a two step upgrade .. but now I am at 17.06 and have datamover
ETA2: It looks like datamover is a pro feature .. Is that correct?

Before (or instead of) cloud, I am thinking that copying this backup to a 1TB portable disk on a weekly basis and keeping that disk in a fireproof safe is sufficient backup for my purposes.

So also want to figure out if I can access the copy of the zfs backup (on the portable disk) on a windows/mac/linux laptop ETA: looks like accessing ZFS on windows directly is not possible. So a solaris/linux VM on windows will allow me to quickly access backup disk if my NAS is not available for longer duration for some reason
 
Last edited:

gea

Well-Known Member
Dec 31, 2010
3,141
1,184
113
DE
A simple rsync call via "other job" should be sufficient for you to copy or sync a folder to a remote or local backup location. Datamover adds complex rules, preserves acl and allows to use snaps as source (copy open files). Another option would be ZFS highspeed replication to keep two filesystems in sync. For a simple filesync to a backup disk use rsync. If you want a backup disk that is accessable directly from Windows, use external USB/NTFS backup disks on Windows with robocopy (free Windows tool) to sync folders from the NAS to the backup disk.

If you want access a ZFS pool "on the fly" you can boot OpenIndiana live from a cd or usb stick, import the pool and access data via the included Mate desktop. A VM would require some sort of Raw Disk Mapping. But as OmniOS/OI/Solaris is installed/ up and ready to use as NAS withing 5 minutes I would not care.
 

tnkrer

New Member
Sep 20, 2016
28
1
3
54
Thanks @gea. I think my backup strategy is now close to being clear in my mind :) And also partially implemented in my napp-it

Step 1: Create ZFS filesystems for data that needs to be backed up - Done
Step 2: Create auto snaps for these file systems using nappit->Jobs->snap->create autosnap job .. accepted defaults, so at 11 pm every night, a snap will be created
Also turned on jobs->autoservice->enable auto 15 min. - Done
Step 3 - create a script with zfs send (or use replication module of napp it) to replicate this filesystem to the backup pool. This will also be a job that will run every night 30 min after the snapshot job - To be done/researched
Step 4 - Create a portable disk with encrypted drive using veracrypt and manually copy these filesystems using a tool like robocopy or foldersync on a monthly basis. - 1 TB disk being encrypted with veracrypt .. That takes a looong time :(

Thanks for all the help! I also use my forum posts as a way to make things clearer to myself .. Hopefully, they will help another newbie at some point ..