Is VSAN or something else right for my homelab?

Discussion in 'VMware, VirtualBox, Citrix' started by Kev, Nov 18, 2017.

  1. Kev

    Kev Active Member

    Joined:
    Feb 16, 2015
    Messages:
    321
    Likes Received:
    46
    First off, my goal is to provide shared storage between my ESXi hosts and provide a level of redundancy so I can power off any of my hosts to do maintenance.

    Also to preface things, I'm new to VCSA and i'm still evaluating things. I like what I see but there is still tonnes to learn before I can live migrate VMs from one host to another. CPU arch compatibility is one thing that bit me.

    Setup:

    A1SRi-2758F , 32GB DDR, Samsung 845DC EVO MZ-7GE960 as local datastore right now, ConnectX-4 LX
    - Cyberpower Agent for UPS (needs USB connection)
    - DaloRadius for guest Wifi
    - OpenVPN Access Server
    - pFSense 2.4

    As you can see, this machine is like the core services which I will reboot not very often

    X10SRH-CLN4F, E5-2618L v4 @ 2.20GHz, 64GB DDR4, ConnectX-4 LX, 2 x Samsung_SSD_960_EVO 512GB, 2 x Samsung_SSD_850_PRO_512GB (local VMFS)
    - Windows 10 (sparsely used workstation)
    - 2 x Windows 10 running sighthound camera and XProtect
    - Unifi controller
    - Ubuntu (low usage, web server, etc)
    - FreeNAS Corral (PCIe HBA passthrough connected to Lenovo SA120 DAS, 7 x 4TB Raidz2)
    - Xpenology (Using local VMFS as storage for 250GB of all flash, "Download" share with multiple Dockers running. Radarr, Sonarr, Organizr, Homebridge, Plex, Etc)
    - RancherOS and STH Monero docker (Use excess processing)

    This is my main processing machine.

    ASRock D1541D4U-2O8R, XeonD-1541, 48Gb DDR4, Built in SFP+, no local storage
    - RancherOS and STH Monero docker (Use excess processing)

    MacMini6,2: Intel(R) Core(TM) i7-3615QM CPU @ 2.30GHz, 16GB
    - MacOS for development
    - RancherOS and STH Monero docker (Use excess processing)

    Now, I want to revamp my storage for all my virtual machines. I don't need great performance other than booting up and running decently. I have 2 Sandisk 3D NAND SSD 1TB extra that I could use for this project. I was thinking to do a 2 host VSAN (with some observer VM running) and could use my 2 850 Pros as the cache drive and the 1TB as the main storage running on the Atom 2578 and E5-2618L host. I am aiming for mirrored storage for VMs and resiliency to either machines going down for maintenance.

    The XeonD-1541 would be a machine that would use the VSAN storage and I can VMotion between the 2618L and XeonD machine.

    If I ever needed to take down the 2618L for an extended period of time, I can take down the FreeNAS VM and reconfigure it for the ASRock D1541D4U-2O8R with HBA passthrough to my DAS as well.

    Has anyone gone down this path and will I be able to acheive resiliency and availability with this plan? What is the total storage I would get from the VSAN? 1TB of all flash storage?

    Thanks!
     
    #1
    Leonardo Rassi and svtkobra7 like this.
  2. Peanuthead

    Peanuthead Active Member

    Joined:
    Jun 12, 2015
    Messages:
    757
    Likes Received:
    115
    I would not try to over engineer it. Keep it simple to start with. If you have time to set up a shared storage device then go with freeNas or something. If you want hit the ground running and not really mess with any of that then I’m a huge proponent of grabbing a Synology or qnap. I think if you put your 2x 1TB SSD’s in a mirror you should have more than enough to do it you need to do. Also be very honest to yourself and how much room you think you will need now and down the road.
     
    #2
  3. Kev

    Kev Active Member

    Joined:
    Feb 16, 2015
    Messages:
    321
    Likes Received:
    46
    You see, the problem I am trying to solve is having a dedicated freenas VM running on the A1SRi-2758F because if I wanted to take it down for maintenance, the entire homelab datastore goes down with it. VSan sounded like it was built in and offered redundancy.

    To your other point, I should be OK with 1TB storage for the VMs.
     
    #3
  4. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,583
    Likes Received:
    542
    first of all you will need at least four disks (2x (cache+storage)) and 2 dedicated HBAs for vsan ...
     
    #4
    Dean likes this.
  5. Peanuthead

    Peanuthead Active Member

    Joined:
    Jun 12, 2015
    Messages:
    757
    Likes Received:
    115
    Rand is correct. Also the 2758 does not have vtd so passing through a hba for zfs is not going to happen
     
    #5
  6. Kev

    Kev Active Member

    Joined:
    Feb 16, 2015
    Messages:
    321
    Likes Received:
    46
    For VSAN, I thought all you needed was 4 drives, All SSD. I can use 512GB samsung 850 pro as cache and 1TB Sandisk as storage. I don't need more than 1TB of redundant VM storage.
     
    #6
  7. Peanuthead

    Peanuthead Active Member

    Joined:
    Jun 12, 2015
    Messages:
    757
    Likes Received:
    115
    Correct. 1 cache and 1 storage per host with 1 hba per host
     
    #7
  8. Dean

    Dean Member

    Joined:
    Jun 18, 2015
    Messages:
    97
    Likes Received:
    8
    Rule number 1. Ensure your hardware is on the vsan HCL List. If not = epic fail. Vsan will not function.

    I have initiated robo with a witness. As mentioned, ensure your hardware is on the HCL list.



    Sent from my Moto Z (2) using Tapatalk
     
    #8
  9. Kev

    Kev Active Member

    Joined:
    Feb 16, 2015
    Messages:
    321
    Likes Received:
    46
    #9
  10. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,583
    Likes Received:
    542
    Well thats the recommendation to get proper functionality. It does work without but complains a lot (ignorable) and may cause bad performance/weird issues

    William uses a ESX in ESX install where he passes through the SATA ports as 'HBA' and his m.2 as cache drive.
    Sata pass through is a hack (but works) if you don't need them at all and boot from USB (twice in his case as he seems to use the same usb drive for primary and secondary boot)

    VT-D issue will prevent that for you unfortunately.
     
    #10
    Last edited: Nov 19, 2017
  11. Pradawny

    Pradawny Member

    Joined:
    Oct 16, 2016
    Messages:
    41
    Likes Received:
    27
    Don't forget about the VSAN licence, which might be difficult to obtain.
    Are you subscribed to VMUG Advantage?
     
    #11
  12. Kev

    Kev Active Member

    Joined:
    Feb 16, 2015
    Messages:
    321
    Likes Received:
    46
    Alright. I think between all the reading and you guys/gals over here, I’m just going to install freenas 11 and out the two 1tb SSDs mirrored. Should I just make a vmdk file or do raw device pass through? Will I be able to clone the VM and take the drives over to my other servers and do ahci pass through and freenas will just see the previously rdm drives?
     
    #12
  13. Dean

    Dean Member

    Joined:
    Jun 18, 2015
    Messages:
    97
    Likes Received:
    8
    Ok...maybe a little hasty on my part.

    Sent from my Moto Z (2) using Tapatalk
     
    #13
Similar Threads: VSAN something
Forum Title Date
VMware, VirtualBox, Citrix Do you have to use a cache tier with vSAN? Yesterday at 4:54 PM
VMware, VirtualBox, Citrix VSAN 2 Node Cluster Aug 8, 2019
VMware, VirtualBox, Citrix In vSAN all-flash, can I stripe two PCIe caching devices together? May 16, 2019
VMware, VirtualBox, Citrix 6.7U1 Vsan doesn't seem to work with Connectx-3 Nov 7, 2018
VMware, VirtualBox, Citrix vSan does not see disks in Gui Oct 14, 2018

Share This Page