ZFS Snapshots Send/Recv & Compression

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
Are you setting any of the compression flags on bzip etc?

Interesting if you can try lz/7zip. That does about 30% on my disk image files compared to bzip at hits highest compression settings. (But pegs the CPU)


Sent from my iPhone using Tapatalk
Just --best for a level9 compression.

I'll see what I can do.
 

wildchild

Active Member
Feb 4, 2014
389
57
28
Should help improve throughput , so ssh can actually use the full bw.

You may have to tune the buffer size a bit.
I prefer this over (extra) compression as it doesnt use any cpu extra, thus leaves more for normal operations
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
Well this is almost done at well over 1HR now and it has not compressed it even 10% using --best/9 for bzip so that's a complete waste of time for any level of bzip2... for my data so far at-least.

Hoping @wildchild that will speed things up a bit :)


Just wrapped as I'm typing this... 38.1GB on ZFS and 36.7GB after bzip2/9/best... def not worth it for 1hR+ cpu at nearly 80%.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
This is 100% local from 1 pool to another on the same system... raidz2 of 6 drives to mirror 2 drives.
Absolutely 0 networking involved in this.

@wildchild -- It says I'm missing "gnumake" on solaris, I haven't been able to find that package anywhere... suggestions? Perl, etc, no errors/problems.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
You're right, I'm running OmniOS / Napp-It All In One as a VM.