VDO - disk compression and dedup for linux

Discussion in 'Linux Admins, Storage and Virtualization' started by gigatexal, Apr 18, 2018.

  1. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,440
    Likes Received:
    418
  2. dandanio

    dandanio Member

    Joined:
    Oct 10, 2017
    Messages:
    30
    Likes Received:
    8
    I was hoping for a comparison with ZFS, since the comparison with a straight up LVM is just less than favorable...
     
    #2
  3. Evan

    Evan Well-Known Member

    Joined:
    Jan 6, 2016
    Messages:
    2,227
    Likes Received:
    305
    Wow that’s what I would love to have in a well supported way, Like the datadomain type amazing compression but not as expensive.

    A month or 2 before it’s in centos 7.5 (fingers crossed)

    Would be great to use for backup servers.
     
    #3
    gigatexal likes this.
  4. Evan

    Evan Well-Known Member

    Joined:
    Jan 6, 2016
    Messages:
    2,227
    Likes Received:
    305
    @gigatexal you know I tried to find and your yesterday or today to deploy a 7.5 system for testing to see how it works... work gets in the way :(
    Maybe next week, for sure with kids at home no chance on the weekend unless I give up sleep and I anyway didn’t download the iso etc yet.
     
    #4
  5. mrkrad

    mrkrad Well-Known Member

    Joined:
    Oct 13, 2012
    Messages:
    1,226
    Likes Received:
    48
    I'd love to see some sql server and mysql benchmarks with this! mysql compression is horrible with oltp
     
    #5
  6. Evan

    Evan Well-Known Member

    Joined:
    Jan 6, 2016
    Messages:
    2,227
    Likes Received:
    305
    I am not a MySQL guy at all but if I can get somebody to do a benchmark for you I will.
     
    #6
  7. _alex

    _alex Active Member

    Joined:
    Jan 28, 2016
    Messages:
    848
    Likes Received:
    88
    Currently building this on debian testing against 4.16 from experimental to give it a try :D

    Modules build and load fine after passing the right path for the headers (copy/paste from git-repo readme doesn't work).
    Use:
    Code:
    make -C /usr/src/linux-headers-`uname -r` M=`pwd`
    Then copy vdo/kvdo.ko and uds/uds.ko to /lib/modules/`uname -r`/, run depmod and modprobe kvdo modprobe uds

    check with:
    lsmod |grep kvdo
    lsmod |grep uds

    and:
    modinfo kvdo
    modinfo uds

    Unfortunately, userspace vdo seems to have more problems / still hanging, guess include-paths are more rhel / centos - style and i'm running out of time for nerding around atm :(

    I could run some benchmarks when done, have different options to pass into the VM for the hdd's / ssd's, or even a partition from 900P via nvmeof.

    But imho this is not exactly made to run databases on it ...

    Alex
     
    #7
  8. _alex

    _alex Active Member

    Joined:
    Jan 28, 2016
    Messages:
    848
    Likes Received:
    88
    Ok, found some more time ...
    Everything running now (need to file a bug / submit a patch and write-down some build dependencies).

    First impression is nice, dedup does a good job with low memory and moderate cpu requirements.
    But this seems to be a beast on it's own, here is some output from vdostats that gives an impression.

    Lot's of things, for sure interesting to tune this for speed / a certain workload...

    Code:
    # vdostats --verbose
    /dev/mapper/vdoTest :
      version                             : 29
      release version                     : 133524
      data blocks used                    : 179185
      overhead blocks used                : 1053656
      logical blocks used                 : 577649
      physical blocks                     : 2621440
      logical blocks                      : 25600000
      1K-blocks                           : 10485760
      1K-blocks used                      : 4931364
      1K-blocks available                 : 5554396
      used percent                        : 47
      saving percent                      : 68
      block map cache size                : 134217728
      write policy                        : async
      block size                          : 4096
      completed recovery count            : 0
      read-only recovery count            : 0
      operating mode                      : normal
      recovery progress (%)               : N/A
      compressed fragments written        : 181839
      compressed blocks written           : 64436
      compressed fragments in packer      : 119
      slab count                          : 3
      slabs opened                        : 3
      slabs reopened                      : 171294
      journal disk full count             : 0
      journal commits requested count     : 0
      journal entries batching            : 0
      journal entries started             : 37302894
      journal entries writing             : 0
      journal entries written             : 37302894
      journal entries committed           : 37302894
      journal blocks batching             : 0
      journal blocks started              : 205420
      journal blocks writing              : 0
      journal blocks written              : 205420
      journal blocks committed            : 205420
      slab journal disk full count        : 0
      slab journal flush count            : 2524
      slab journal blocked count          : 0
      slab journal blocks written         : 10015
      slab journal tail busy count        : 0
      slab summary blocks written         : 9226
      reference blocks written            : 5141
      block map dirty pages               : 1352
      block map clean pages               : 2486
      block map free pages                : 28930
      block map failed pages              : 0
      block map incoming pages            : 0
      block map outgoing pages            : 0
      block map cache pressure            : 0
      block map read count                : 20453407
      block map write count               : 18649438
      block map failed reads              : 0
      block map failed writes             : 0
      block map reclaimed                 : 44
      block map read outgoing             : 156
      block map found in cache            : 38196135
      block map discard required          : 0
      block map wait for page             : 902872
      block map fetch required            : 3838
      block map pages loaded              : 3838
      block map pages saved               : 11972
      block map flush count               : 10457
      dedupe advice valid                 : 3459176
      dedupe advice stale                 : 125815
      concurrent data matches             : 11925
      concurrent hash collisions          : 0
      invalid advice PBN count            : 0
      no space error count                : 516393
      read only error count               : 0
      instance                            : 0
      512 byte emulation                  : off
      current VDO IO requests in progress : 119
      maximum VDO IO requests in progress : 2000
      dedupe advice timeouts              : 0
      flush out                           : 0
      write amplification ratio           : 0.0
      bios in read                        : 1801096
      bios in write                       : 114645046
      bios in discard                     : 105533171
      bios in flush                       : 4812
      bios in fua                         : 24112
      bios in partial read                : 0
      bios in partial write               : 4812
      bios in partial discard             : 0
      bios in partial flush               : 4812
      bios in partial fua                 : 0
      bios out read                       : 5298666
      bios out write                      : 3369138
      bios out discard                    : 0
      bios out flush                      : 0
      bios out fua                        : 0
      bios meta read                      : 27954
      bios meta write                     : 310127
      bios meta discard                   : 0
      bios meta flush                     : 50125
      bios meta fua                       : 5556
      bios journal read                   : 11381
      bios journal write                  : 205420
      bios journal discard                : 0
      bios journal flush                  : 19722
      bios journal fua                    : 5555
      bios page cache read                : 14295
      bios page cache write               : 15206
      bios page cache discard             : 0
      bios page cache flush               : 13691
      bios page cache fua                 : 0
      bios out completed read             : 5298666
      bios out completed write            : 3369138
      bios out completed discard          : 0
      bios out completed flush            : 0
      bios out completed fua              : 0
      bios meta completed read            : 3899
      bios meta completed write           : 310127
      bios meta completed discard         : 0
      bios meta completed flush           : 26070
      bios meta completed fua             : 5556
      bios journal completed read         : 0
      bios journal completed write        : 205420
      bios journal completed discard      : 0
      bios journal completed flush        : 8341
      bios journal completed fua          : 5555
      bios page cache completed read      : 3838
      bios page cache completed write     : 15206
      bios page cache completed discard   : 0
      bios page cache completed flush     : 3234
      bios page cache completed fua       : 0
      bios acknowledged read              : 1801096
      bios acknowledged write             : 114645046
      bios acknowledged discard           : 105533171
      bios acknowledged flush             : 4812
      bios acknowledged fua               : 24112
      bios acknowledged partial read      : 1775137
      bios acknowledged partial write     : 4812
      bios acknowledged partial discard   : 0
      bios acknowledged partial flush     : 4812
      bios acknowledged partial fua       : 0
      bios in progress read               : 0
      bios in progress write              : 0
      bios in progress discard            : 0
      bios in progress flush              : 0
      bios in progress fua                : 0
      read cache accesses                 : 0
      read cache hits                     : 0
      read cache data hits                : 0
      KVDO module bytes used              : 424561536
      KVDO module peak bytes used         : 424565168
      KVDO module bios used               : 37286
      KVDO module peak bio count          : 37574
      entries indexed                     : 3425305
      posts found                         : 3584935
      posts not found                     : 3425305
      queries found                       : 53
      queries not found                   : 797
      updates found                       : 307630
      updates not found                   : 0
      current dedupe queries              : 0
      maximum dedupe queries              : 1997
    
    Using -o discard on the mount is really important, otherwise no space will be reclaimed on file deletion.
    Wonder how this will impact running VM's on a blockbased lun backed with dm-vdo ...

    For sure will do some more research soon / try to build a dkms package.
     
    #8
    Evan likes this.
  9. mrkrad

    mrkrad Well-Known Member

    Joined:
    Oct 13, 2012
    Messages:
    1,226
    Likes Received:
    48
    I'd just like to see how performance with semi-oltp workload goes with mysql on vdo partition versus using internal innodb compression (which is 100x slower with any sort of updates against large compressed tables)!

    Is vdo inline-dedupe/compression or more like windows 2012/16 job based dedupe?
     
    #9
  10. _alex

    _alex Active Member

    Joined:
    Jan 28, 2016
    Messages:
    848
    Likes Received:
    88
    Don't know how Windows does dedupe, but it's definitely not job-based.
    In both write-modes ACK is sent before any dedupe happens.

    Chapter 29. VDO Integration - Red Hat Customer Portal

    On some quick+dirty benchmarking i was able to squeeze around 150k / 40k r/w IOPS within a zfs-backed VM out of it.
    This is about inline with what i get on a volume without vdo (writes were slightly higher).


    This turned out to be not exactly true, i observed significant higher CPU usage with VDO and also had to increase the number of some threads to bring it to speed like described here. Also had to enable blk-mq within the guest.

    29.6. Tuning VDO - Red Hat Customer Portal

    CPU usage went up by ca. 40% on a 8 core VM, RAM usage increased by what i expected from reading the docs, what isn't too much.

    Guess you can benefit from dm-vdo if you throw in CPU cycles.
     
    #10
Similar Threads: disk compression
Forum Title Date
Linux Admins, Storage and Virtualization Which disks for new array? Aug 6, 2018
Linux Admins, Storage and Virtualization Formatting disks for ZFS on Linux May 8, 2018
Linux Admins, Storage and Virtualization SAS dual-path disk attaching problem (lsi hba with mpt2sas, lsi sas expanders) Apr 30, 2018
Linux Admins, Storage and Virtualization Getting SMART data from ZFS disks in Proxmox Mar 10, 2018
Linux Admins, Storage and Virtualization Interesting Proxmox Disk Usage Nov 12, 2017

Share This Page