Encryption on OpenIndiana and ZFS V. 28

gea

Well-Known Member
Dec 31, 2010
2,485
837
113
DE
Encryption on OpenIndiana

Not integrated in ZFS like with Solaris 11 and ZFS V.31+
but on an underlaying disk level and therefor working with ZFS V.28

The thought is:
- create files on a ZFS dataset (ex 1 GB with the option to backup them to any Filesystem/ Cloud provider)
- build block-devices from the files with lofiadm. (lofiadm supports encryption, must enter a pw here)
- build a regular ZFS pool from these files (use ex ZFS Z2 to recover from backup files with errors)

ex:
1. create a 1G file in /tank/secrets (a ZFS dataset)
cd /tank/secrets
mkfile 1g file1


2. create encrypted blockdevices from these file(s) -> creates a device /dev/lofi/1
lofiadm -c aes-256-cbc -a /tank/secrets/file1
Enter passphrase: ..

-repeat for all disks if you want to build a pool from more disks to have redundancy
(important if you want to backup these files on a non-ZFS file system)


3. Create a regular (ex. basic) ZFS pool from this or these (encrypted) device(s)
zpool create secretpool /dev/lofi/1


The newly created pool works like any ZFS pool.
To take offline you must export the pool and remove the devices

zpool export secretpool
lofiadm -d /tank/secrets/file1

To take online you must build devices from the files again using the same PW and import the pool
lofiadm -c aes-256-cbc -a /tank/secrets/file1
Enter passphrase: ..

If you use the wrong PW, all seems ok but there are no files...


Now you can import your pool from these devices
zpool import -d /dev/lofi shows all available pools

To import the pool, you must use:
zpool import -d /dev/lofi/ secretpool

Only disadvantage may be some lower performance (goes through ZFS twice + encryption).
But its very elegant, easy to implement and it is based simply only on one or more encrypted files.
If you want to backup them, you can just copy them. With small files its not a problem, even on FAT disks
with a max file limit of 2 GB. If you have build redundant ZFS pools from several files (ex Raid-Z1/2/3) its even not
a problem if (1/2/3) files get damaged for whatever reason.

I will add this in napp-it


http://constantin.glez.de/blog/2012/02/introducing-sparse-encrypted-zfs-pools
http://www.cuddletech.com/blog/pivot/entry.php?id=1029
https://blogs.oracle.com/yakshaving/entry/encrypted_fs_on_solaris_10
http://www.idevelopment.info/data/Oracle/DBA_tips/Automatic_Storage_Management/ASM_21.shtml
 

gea

Well-Known Member
Dec 31, 2010
2,485
837
113
DE
It depends only on mkfile and lofiadm.
Both are default tools on every Solaris based system.
 

nterupt

New Member
Nov 26, 2012
2
0
0
I was curious of what you think best practices would be for using this method to basically encrypt the entire ZFS pool.

I didn't think there was a simple way to expand the size of this encrypted zpool without adding "vdevs" or creating a new zpool and copying data over. If I'm incorrect please let me know of a better way to handle this.

It seems that if you created a single file that was as large as the pool you would have two problems:
1. You'd be near the capacity of the pool which from what I've read leads to drastically reduced performance (<= 80% full recommended)
2. It makes it hard to expand the size of your encrypted zpool

For ex:
Unencrypted zpool is a raidz2 vdev with 6 x 3TB disks
You create a single file and encrypt with lofiadm that is roughly 10TB
You increase capacity of the unencrypted zpool by gradually replacing all the 3TB disks with 4TB disks and expanding the pool
I've increased the base zpool capacity by 4TB but I can't possibly create a new zpool and move data over.

The other way to do this would be to create 6 files that are 12TB (12TB/6 drives) vdevs. This way when you add higher capacity drives you can simply create new (larger vdevs) and replace the smaller files.

For ex:
6x2TB encrypted files
Replace drives with 4TB disks and expand underlying zpool (new capacity is ~16TB)
Create a new 2.6TB file (16/6) and replace the smaller files one at a time until you can increase the encrypted zpool capacity.

This still seems to create some problems:
1. You'd be near capacity of the unencrypted pool (> 80% utilization)
2. If you didn't want to incur a performance impact on the encrypted zpool you'd need to stay at < 80% utilization there

That effectively means that to encrypt an entire zpool of Size X, and to maintain "adequate performance", you'd need to keep storage utilization at < 64% (.8*.8) of the base zpool capacity.

For a system a 6x3TB RAIDZ2 zpool would be limited to: (~12TB base storage)*.64 = 7.68TB or (7.68/18) < 43% of raw disk space available for file storage.

I am really looking at OpenIndiana, but one advantage FreeBSD ZFS implementations seem to have is that you can use geli to encrypt a partition on each disk and then hand that to ZFS. It makes it so you still get your encryption but you don't nearly have the same number of issues with growing the encrypted zpool. Am I overcomplicating this or is this just the reality of encrypted ZFS on v28 on Illumos?
 

gea

Well-Known Member
Dec 31, 2010
2,485
837
113
DE
each approach has advantages and disadvantages.

- The OI way with pools based on lofiadm encrypted files on ZFS is best for smaller pools where you want to backup encrypted data with full Raid-Z(1-3) ZFS security to unsecure storage like USB or other non-ZFS storage (cloud etc).

- using enrypted disks and partitions with Geli is more flexible but you loose the backup feature.

-encryption based on ZFS itself is the ideal option but you loose the backup option as well and it is currently available on Oracle Solaris only.
 

nterupt

New Member
Nov 26, 2012
2
0
0
Thanks for the response. I definitely agree that encrypted ZFS ala ZFS v30+ is ideal. If only Oracle would help the ZFS community out on that front...

I just wanted to make sure I understand the limitations and advantages/disadvantages properly since I'm still putting together and testing a new system and definitely qualify as a newbie. I was sort of hoping there was a magical mystery option no. 3 I just wasn't aware of :)
 

cactus

Moderator
Jan 25, 2011
827
76
28
CA
I liked this when I read Constantin's approach(ZFS FS out of encrypted files), but very nice to have it built into Napp-It. Edit: Didn't see the OP date, I'm a little late to the party.

One qualm is the "lofiadm -c aes-256-cbc -a /tank/secrets/file1" command. Once this is done with the wrong password and you write new data, the original data is corrupted.(Let me know if I am wrong) Is there any way around this? I could see setting up a mirror and starting it it degraded to verify the contents of the file system or just having two copies of the encrypted files, but then you are running a parity RAID on top of parity RAID and there is a lot of duplicate data. I also realize back-ups are the obvious solution to this, but not all users are this diligent.(Timidly raises hand). Edit: I also see that this is just one of the limitations of the encrypted files solution. Going to encrypted disks will help with the duplicate data problem.
 
Last edited:

gea

Well-Known Member
Dec 31, 2010
2,485
837
113
DE
One qualm is the "lofiadm -c aes-256-cbc -a /tank/secrets/file1" command. Once this is done with the wrong password and you write new data, the original data is corrupted.(Let me know if I am wrong) Is there any way around this?
You need a check if the 'unlock' was successfull (ex by check a former created file like test.unlock) otherwise you need to discard the unlock. Do not do manually, use a script. (I do it this way in napp-it).

This is one of the ideas behind encryption. A 'empty' file system and a successfully unlocked filesystem with data seems to be the same otherwise its like a delete or self destroy.
 
Last edited: