KVM Backup Solutions

Discussion in 'Linux Admins, Storage and Virtualization' started by IamSpartacus, Oct 25, 2018.

  1. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    1,436
    Likes Received:
    281
    Are there any decent backup solutions for VM's (in my hase .img disks) running on KVM?
     
    #1
  2. Stephan

    Stephan IT Professional

    Joined:
    Apr 21, 2017
    Messages:
    76
    Likes Received:
    25
    Last time I looked in January this year, there was nothing appealing. So I scripted away myself. Not for the faint of heart, 900 lines bash script, a handful of helper scripts, maybe 3-4 weeks debugging and exploring, soldering (not kidding).

    Step 1 I basically tar up the host filesystems root+boot but without postgresql files and also not any VM qcow2 images and some other stuff.

    Step 2 Snapshot VMs sequentially and atomically and copy off all virtual disks to backup place (in my case from SSD to a HDD RAID5). Commit snapshot before going to the next VM. Running VMs are undisturbed by the backup and operate without any downtime.

    Step 3 Copy powered off VMs as they are, no snapshot like in step 2 necessary. In step 2 and 3 I will export through libvirt a XML definition file for each VM to aid in restoration should that become necessary.

    Step 4 Backup of local postgresql using pg_basebackup and a checkpoint.

    Step 5 Archive step 1-4 data using borg backup with zstd level 3 compression. Right now I have 1.83 TB of raw data compressed and deduplicated down to 174 GB. Borg keeps 25 archives right now with 7 daily, 4 weekly, 12 monthly and 5 yearly max. Integrity check of the archive is run every 30 days.

    Step 6 Mail me any warnings or worse if they happened, including the current runs full log. Package maintainers on Arch Linux have a habit of breaking random packages.

    Step 7 I have bareos tape backup (a bacula clone) running daily with quarterly fulls going to a LTO6 and incrementals going to a LTO5 drive. Since the drives are in my office I modded them using 2.5mm stereo plugs to connect the on-switch of the device potential free with a USB relais box. Reason: Noise. You can't power them on using a mains switch, those drives WANT their button pushed to actually turn on... bummer. So before every tape run a script will execute that turns on the correct drive, waits for it to boot up, loads a tape if present. After each run the tape is ejected and after some cooling off period the drive is turned off again.

    Pretty much a boatload of trouble to get everything going. I think something like Xen Orchestra might be better than this kind of KVM bare metal work. Professionally I would just buy Veeam and VMware to do all this, but I wanted to see how far open source does reach in 2018. Quite far, but you need to be a Linux hacker to pull it off. Haven't published anything due to lack of time and out of fear of needing to support this afterwards, sorry.
     
    #2
  3. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,760
    Likes Received:
    780
    Proxmox VE makes basic backups easy.

    Add a NFS share, map it to Proxmox, schedule VM backups and retention policies.

    Proxmox can also do ZFS snapshot with send/ receive.

    I've turned a low power Proxmox VE host into a NFS server.
     
    #3
    gigatexal likes this.
  4. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    1,436
    Likes Received:
    281
    Are you saying that you can backup VMs from Proxmox VE that aren't actually running on said Proxmox host by simply mapping them via an NFS share?
     
    #4
  5. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,760
    Likes Received:
    780
    I think you can if they're on another Proxmox host in the cluster! I see your point, if you're running RHEL it doesn't help. It's a good reason to think about Proxmox.
     
    #5
  6. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    1,436
    Likes Received:
    281
    I haven't looked at Proxmox in a few years. Is it possible to pass a USB flash drive through to a VM so that it's available pre-OS boot in Proxmox like you can in VMware?
     
    #6
Similar Threads: Backup Solutions
Forum Title Date
Linux Admins, Storage and Virtualization Need a little hand holding on Proxmox VE backup strategies. Aug 11, 2018
Linux Admins, Storage and Virtualization How to backup ZFS setup properly ? Sep 14, 2017
Linux Admins, Storage and Virtualization Backup to NAS and to Cloud Mar 31, 2017
Linux Admins, Storage and Virtualization unlimited cloud backup for large data sets for linux? Aug 8, 2016
Linux Admins, Storage and Virtualization Ceph right for my needs? (Keeping a in sync backup on remote location...) (pve-zync vs ceph?) Jul 23, 2016

Share This Page