Re-thinking my ESXi homelab storage setup...but to what?

Discussion in 'NAS Systems and Networked Home and SMB Software' started by Ch33rios, Oct 16, 2018.

  1. Ch33rios

    Ch33rios Member

    Joined:
    Nov 29, 2016
    Messages:
    100
    Likes Received:
    6
    Much like when I was a kid and had random urges to switch up my room, I've suddenly had the desire to re-evaluate my storage setup on my homelab server and I'm wondering what I should really do based upon my use. I'll first start with the setup:

    E3-1230v5
    32GB DDR4 RAM
    LSi 9211-8i in IT Mode w/ SAS expander card
    10 x Intel S3500 SSDs (800 GB)
    2 x 480 GB Toshiba consumer-grade SSDs (ESXi datastore)
    4 x 2TB WD-Reds

    VM's I'm running:

    BTRFS NAS Server (LSi card is passed through with the aforementioned Intel/WD drives attached with each currently in separate RAID 10 btrfs pools)
    |_Runs Plex Media Server as well and acts as a NFS/SAMBA file server for the network and other devices

    Photon OS Container Server
    |_Runs various containers with persistent storage of containers on an NFS mount from the NAS server

    Kubernetes Cluster (3 x VMs)
    |_No persistent storage right now but may want to play more with persistent apps mounted via NFS

    Unifi Controller + Unifi Video Server

    Windows 2016 Core

    ------------------------------

    Its worked just fine for over a year but I'm just wondering if its an 'optimal' sort of setup. I've been interested in playing with things like glusterfs and while I could create a virtual cluster and perhaps attach individual drives to the VMs via RDM, I'm not sure if that would be the right choice as a 'production' homelab setup. Although with a proper backup setup perhaps it wouldn't matter?

    In any case, I'm interested in hearing what others might do in my place. Thanks for any input!
     
    #1
  2. spali

    spali New Member

    Joined:
    Nov 4, 2018
    Messages:
    14
    Likes Received:
    1
    I had long time a similar setup, but with 3 hosts running ESXi, but storage/share was virtualized FreeNAS with ZFS, which replicated from one server to the other for backup purposes.
    In general it worked fine, but had a lot the chicken and egg problems with the storage. So I moved to Hyper-V instead of ESXi, to have the storage/share not virtualized. At least for me this made it a lot easier to manage, because all VM's can be sure the storage is available
    (The storage VM could still be starting or even not working at the time of other VM's booted).
    That allows to let the VM's use the share and be always backed up by a single backup job on the physical server. Especially to use persistent storage for your containers, a setup like this would be handy.
    Relying in VM's on the storage which is a VM on the same level was just to complicated and error prone for me.
    I think this would applies also for a single server setup.
    So I would setup a windows server with Hyper-V and File-Server role and put the rest in VM's on it.
     
    #2
  3. acquacow

    acquacow Active Member

    Joined:
    Feb 15, 2017
    Messages:
    245
    Likes Received:
    93
    I'm in the process of going diskless for my next ESXi box. Going to just have a small ITX console-sized box with USB boot. All storage will be provided via iSCSI and NFS from my FreeNAS box.
     
    #3
  4. spali

    spali New Member

    Joined:
    Nov 4, 2018
    Messages:
    14
    Likes Received:
    1
    That's even better as long you have a good storage nerwork in between.
     
    #4
  5. acquacow

    acquacow Active Member

    Joined:
    Feb 15, 2017
    Messages:
    245
    Likes Received:
    93
    Yup, I'm all 10gige with both spinning disk and flash pools.

    By the end of this weekend, I'll also have an all-flash gluster setup too.
     
    #5

Share This Page