Need a little hand holding on Proxmox VE backup strategies.

Discussion in 'Linux Admins, Storage and Virtualization' started by vl1969, Aug 11, 2018.

  1. vl1969

    vl1969 Active Member

    Joined:
    Feb 5, 2014
    Messages:
    478
    Likes Received:
    48
    Hello everyone.
    here is my dilemma,
    I have a "Single" Proxmox server setup, home use. all is current and updated.
    I am using ZFS across the board. all pools are ZFS mirror

    the layout is as follows:
    Code:
    NAME                                 STATE     READ WRITE CKSUM
        pvstore    (2x1TB HDD)               ONLINE       0     0     0
          mirror-0                           ONLINE       0     0     0
            ata-ST1000DM003-1CH162_S1DDPX2G  ONLINE       0     0     0
            ata-TOSHIBA_HDWD110_58KAV7PFS    ONLINE       0     0     0
    
    
    
        NAME                                                  STATE     READ WRITE CKSUM
        rpool    (2x120GB SSD)                                ONLINE       0     0     0
          mirror-0                                            ONLINE       0     0     0
            ata-INTEL_SSDSC2BW120H6_CVTR52060450120AGN-part2  ONLINE       0     0     0
            ata-SanDisk_SDSSDA120G_161443400890-part2         ONLINE       0     0     0
    
    
    
        NAME                                            STATE     READ WRITE CKSUM
        tank0      (3 sets of 2x2TB HDD)                ONLINE       0     0     0
          mirror-0                                      ONLINE       0     0     0
            ata-ST32000542AS_5XW2H5SP                   ONLINE       0     0     0
            ata-ST2000NM0055-1V4104_ZC21117H            ONLINE       0     0     0
          mirror-1                                      ONLINE       0     0     0
            ata-ST2000NM0055-1V4104_ZC211C1R            ONLINE       0     0     0
            ata-ST2000DL003-9VT166_5YD578A7             ONLINE       0     0     0
          mirror-2                                      ONLINE       0     0     0
            ata-ST2000DL003-9VT166_5YD3XYXZ             ONLINE       0     0     0
            ata-Hitachi_HDS722020ALA330_JK1130YAKYEX0T  ONLINE       0     0     0
    
    
    
        NAME                                  STATE     READ WRITE CKSUM
        tank1    (2 sets of 2x3TB HDD)        ONLINE       0     0     0
          mirror-0                            ONLINE       0     0     0
            ata-ST3000DM001-1ER166_ZA501WMX   ONLINE       0     0     0
            ata-TOSHIBA_HDWD130_X6IEDPUAS     ONLINE       0     0     0
          mirror-1                            ONLINE       0     0     0
            ata-TOSHIBA_HDWD130_Y7211HPAS     ONLINE       0     0     0
            ata-TOSHIBA_DT01ACA300_364X6TBKS  ONLINE       0     0     0
    
    
    
        NAME                                 STATE     READ WRITE CKSUM
        tank2    (1 sets of 2x4TB HDD)       ONLINE       0     0     0
          mirror-0                           ONLINE       0     0     0
            ata-ST4000DM004-2CV104_ZFN0TZLW  ONLINE       0     0     0
            ata-ST4000DM004-2CV104_ZFN1AADG  ONLINE       0     0     0
    
    

    now, "rpool" is system nothing new here.
    the "pvstore" is simply an additional storage for the system as in additional local store space. since my system drives are small I needed extra space.

    "tank0" is the main user storage pool. that is where all my media and other files are.

    I have folders "downloads" and "embylib" there.
    all my containers are bind-mount to respective folders based on use.


    "tank1" right now is a kind of online backup pool. I have an rsync job in my file server container syncing embylib to it.

    "tank2" is empty as it is a brand new pool on the machine.


    !!!! WHAT I WANT / NEED !!!!!

    I need to understand how to backup and subsequently restore my system in order of importance.

    #1 - backup my Proxmox setup properly. I know I can always reinstall the whole thing from scratch but setting all up back as it was may be challenging. how do I have at least a template of what I
    have handy?

    #2 - backup all my VM/LXC ? I run the backup from WebUI but than what?

    #2.1 - proper way to backup all my data?
    I have a 5TB external drive. it is currently NTFS formated.
    what would be the best way to drop all my data on it? even if it means I have to re-format it to Linux partition.

    please tell me the best tools, scripts and procedures to use.

    Sincerely Vlad.
     
    #1
  2. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,695
    Likes Received:
    756
    I think the easiest for VMs is NFS or using pve zfs tool.

    For the machine itself, that's a really good question.
     
    #2
  3. niekbergboer

    niekbergboer Member

    Joined:
    Jun 21, 2016
    Messages:
    81
    Likes Received:
    30
    I use http://www.bolthole.com/solaris/zrep for my off-site backups. Works fine, over SSH, and does proper ZFS snapshots and incremental transfers.

    For VMs on zvols, you could in fact do a straight zrep of the zvols.

    What I don't have a good solution for is full Proxmox VM backups; Proxmox insists on always making a full disk-image backup (compressed), while those backups could be a lot smaller if Proxmox could directly use ZFS snapshots (only storing blocks that changed from the last backups).

    If you use Ceph RBD for your VMs, it seems that it is theoretically possible to do snapshot based backups there, too. I wish Proxmox would support that.

    At any rate; my network pipe supports my daily backup load, so I use Proxmox built-in VM backups to write the backups to a ZFS dataset, and I off-site that using zrep. It's not optimal, but it works out of the box, and laziness is an oft-underappreciated trait ;)
     
    #3
  4. vl1969

    vl1969 Active Member

    Joined:
    Feb 5, 2014
    Messages:
    478
    Likes Received:
    48
    thank you guys, I will see if any of this will work.
    right now I have a big issue with my server and not sure how I can fix it yet.
    but as soon as I get it back up I will need to get the data out.
     
    #4
  5. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,695
    Likes Received:
    756
  6. vl1969

    vl1969 Active Member

    Joined:
    Feb 5, 2014
    Messages:
    478
    Likes Received:
    48
    very interesting reading. will surely try.

    right now it looks like I will have to reinstall the whole thing anew. I can not even boot the server up.
     
    #6
Similar Threads: Need little
Forum Title Date
Linux Admins, Storage and Virtualization New Proxmox server - need recommendation on setting up network Aug 9, 2018
Linux Admins, Storage and Virtualization Home Server Setup and Management help needed. May 3, 2018
Linux Admins, Storage and Virtualization Do I need a subscription to use ILO (or IPMI) Apr 29, 2018
Linux Admins, Storage and Virtualization Need some more Linux/NextCloud help :( May 10, 2017
Linux Admins, Storage and Virtualization nested virtualization (how, and what do I need?) Feb 1, 2017

Share This Page