Ok, found some more time ...
Everything running now (need to file a bug / submit a patch and write-down some build dependencies).
First impression is nice, dedup does a good job with low memory and moderate cpu requirements.
But this seems to be a beast on it's own, here is some output from vdostats that gives an impression.
Lot's of things, for sure interesting to tune this for speed / a certain workload...
Code:
# vdostats --verbose
/dev/mapper/vdoTest :
version : 29
release version : 133524
data blocks used : 179185
overhead blocks used : 1053656
logical blocks used : 577649
physical blocks : 2621440
logical blocks : 25600000
1K-blocks : 10485760
1K-blocks used : 4931364
1K-blocks available : 5554396
used percent : 47
saving percent : 68
block map cache size : 134217728
write policy : async
block size : 4096
completed recovery count : 0
read-only recovery count : 0
operating mode : normal
recovery progress (%) : N/A
compressed fragments written : 181839
compressed blocks written : 64436
compressed fragments in packer : 119
slab count : 3
slabs opened : 3
slabs reopened : 171294
journal disk full count : 0
journal commits requested count : 0
journal entries batching : 0
journal entries started : 37302894
journal entries writing : 0
journal entries written : 37302894
journal entries committed : 37302894
journal blocks batching : 0
journal blocks started : 205420
journal blocks writing : 0
journal blocks written : 205420
journal blocks committed : 205420
slab journal disk full count : 0
slab journal flush count : 2524
slab journal blocked count : 0
slab journal blocks written : 10015
slab journal tail busy count : 0
slab summary blocks written : 9226
reference blocks written : 5141
block map dirty pages : 1352
block map clean pages : 2486
block map free pages : 28930
block map failed pages : 0
block map incoming pages : 0
block map outgoing pages : 0
block map cache pressure : 0
block map read count : 20453407
block map write count : 18649438
block map failed reads : 0
block map failed writes : 0
block map reclaimed : 44
block map read outgoing : 156
block map found in cache : 38196135
block map discard required : 0
block map wait for page : 902872
block map fetch required : 3838
block map pages loaded : 3838
block map pages saved : 11972
block map flush count : 10457
dedupe advice valid : 3459176
dedupe advice stale : 125815
concurrent data matches : 11925
concurrent hash collisions : 0
invalid advice PBN count : 0
no space error count : 516393
read only error count : 0
instance : 0
512 byte emulation : off
current VDO IO requests in progress : 119
maximum VDO IO requests in progress : 2000
dedupe advice timeouts : 0
flush out : 0
write amplification ratio : 0.0
bios in read : 1801096
bios in write : 114645046
bios in discard : 105533171
bios in flush : 4812
bios in fua : 24112
bios in partial read : 0
bios in partial write : 4812
bios in partial discard : 0
bios in partial flush : 4812
bios in partial fua : 0
bios out read : 5298666
bios out write : 3369138
bios out discard : 0
bios out flush : 0
bios out fua : 0
bios meta read : 27954
bios meta write : 310127
bios meta discard : 0
bios meta flush : 50125
bios meta fua : 5556
bios journal read : 11381
bios journal write : 205420
bios journal discard : 0
bios journal flush : 19722
bios journal fua : 5555
bios page cache read : 14295
bios page cache write : 15206
bios page cache discard : 0
bios page cache flush : 13691
bios page cache fua : 0
bios out completed read : 5298666
bios out completed write : 3369138
bios out completed discard : 0
bios out completed flush : 0
bios out completed fua : 0
bios meta completed read : 3899
bios meta completed write : 310127
bios meta completed discard : 0
bios meta completed flush : 26070
bios meta completed fua : 5556
bios journal completed read : 0
bios journal completed write : 205420
bios journal completed discard : 0
bios journal completed flush : 8341
bios journal completed fua : 5555
bios page cache completed read : 3838
bios page cache completed write : 15206
bios page cache completed discard : 0
bios page cache completed flush : 3234
bios page cache completed fua : 0
bios acknowledged read : 1801096
bios acknowledged write : 114645046
bios acknowledged discard : 105533171
bios acknowledged flush : 4812
bios acknowledged fua : 24112
bios acknowledged partial read : 1775137
bios acknowledged partial write : 4812
bios acknowledged partial discard : 0
bios acknowledged partial flush : 4812
bios acknowledged partial fua : 0
bios in progress read : 0
bios in progress write : 0
bios in progress discard : 0
bios in progress flush : 0
bios in progress fua : 0
read cache accesses : 0
read cache hits : 0
read cache data hits : 0
KVDO module bytes used : 424561536
KVDO module peak bytes used : 424565168
KVDO module bios used : 37286
KVDO module peak bio count : 37574
entries indexed : 3425305
posts found : 3584935
posts not found : 3425305
queries found : 53
queries not found : 797
updates found : 307630
updates not found : 0
current dedupe queries : 0
maximum dedupe queries : 1997
Using -o discard on the mount is really important, otherwise no space will be reclaimed on file deletion.
Wonder how this will impact running VM's on a blockbased lun backed with dm-vdo ...
For sure will do some more research soon / try to build a dkms package.