Is VSAN or something else right for my homelab?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Kev

Active Member
Feb 16, 2015
461
111
43
41
First off, my goal is to provide shared storage between my ESXi hosts and provide a level of redundancy so I can power off any of my hosts to do maintenance.

Also to preface things, I'm new to VCSA and i'm still evaluating things. I like what I see but there is still tonnes to learn before I can live migrate VMs from one host to another. CPU arch compatibility is one thing that bit me.

Setup:

A1SRi-2758F , 32GB DDR, Samsung 845DC EVO MZ-7GE960 as local datastore right now, ConnectX-4 LX
- Cyberpower Agent for UPS (needs USB connection)
- DaloRadius for guest Wifi
- OpenVPN Access Server
- pFSense 2.4

As you can see, this machine is like the core services which I will reboot not very often

X10SRH-CLN4F, E5-2618L v4 @ 2.20GHz, 64GB DDR4, ConnectX-4 LX, 2 x Samsung_SSD_960_EVO 512GB, 2 x Samsung_SSD_850_PRO_512GB (local VMFS)
- Windows 10 (sparsely used workstation)
- 2 x Windows 10 running sighthound camera and XProtect
- Unifi controller
- Ubuntu (low usage, web server, etc)
- FreeNAS Corral (PCIe HBA passthrough connected to Lenovo SA120 DAS, 7 x 4TB Raidz2)
- Xpenology (Using local VMFS as storage for 250GB of all flash, "Download" share with multiple Dockers running. Radarr, Sonarr, Organizr, Homebridge, Plex, Etc)
- RancherOS and STH Monero docker (Use excess processing)

This is my main processing machine.

ASRock D1541D4U-2O8R, XeonD-1541, 48Gb DDR4, Built in SFP+, no local storage
- RancherOS and STH Monero docker (Use excess processing)

MacMini6,2: Intel(R) Core(TM) i7-3615QM CPU @ 2.30GHz, 16GB
- MacOS for development
- RancherOS and STH Monero docker (Use excess processing)

Now, I want to revamp my storage for all my virtual machines. I don't need great performance other than booting up and running decently. I have 2 Sandisk 3D NAND SSD 1TB extra that I could use for this project. I was thinking to do a 2 host VSAN (with some observer VM running) and could use my 2 850 Pros as the cache drive and the 1TB as the main storage running on the Atom 2578 and E5-2618L host. I am aiming for mirrored storage for VMs and resiliency to either machines going down for maintenance.

The XeonD-1541 would be a machine that would use the VSAN storage and I can VMotion between the 2618L and XeonD machine.

If I ever needed to take down the 2618L for an extended period of time, I can take down the FreeNAS VM and reconfigure it for the ASRock D1541D4U-2O8R with HBA passthrough to my DAS as well.

Has anyone gone down this path and will I be able to acheive resiliency and availability with this plan? What is the total storage I would get from the VSAN? 1TB of all flash storage?

Thanks!
 

Peanuthead

Active Member
Jun 12, 2015
839
177
43
44
I would not try to over engineer it. Keep it simple to start with. If you have time to set up a shared storage device then go with freeNas or something. If you want hit the ground running and not really mess with any of that then I’m a huge proponent of grabbing a Synology or qnap. I think if you put your 2x 1TB SSD’s in a mirror you should have more than enough to do it you need to do. Also be very honest to yourself and how much room you think you will need now and down the road.
 

Kev

Active Member
Feb 16, 2015
461
111
43
41
You see, the problem I am trying to solve is having a dedicated freenas VM running on the A1SRi-2758F because if I wanted to take it down for maintenance, the entire homelab datastore goes down with it. VSan sounded like it was built in and offered redundancy.

To your other point, I should be OK with 1TB storage for the VMs.
 

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
first of all you will need at least four disks (2x (cache+storage)) and 2 dedicated HBAs for vsan ...
 
  • Like
Reactions: Dean

Peanuthead

Active Member
Jun 12, 2015
839
177
43
44
Rand is correct. Also the 2758 does not have vtd so passing through a hba for zfs is not going to happen
 

Kev

Active Member
Feb 16, 2015
461
111
43
41
For VSAN, I thought all you needed was 4 drives, All SSD. I can use 512GB samsung 850 pro as cache and 1TB Sandisk as storage. I don't need more than 1TB of redundant VM storage.
 

Dean

Member
Jun 18, 2015
116
10
18
48
Rule number 1. Ensure your hardware is on the vsan HCL List. If not = epic fail. Vsan will not function.

I have initiated robo with a witness. As mentioned, ensure your hardware is on the HCL list.



Sent from my Moto Z (2) using Tapatalk
 

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
Rule number 1. Ensure your hardware is on the vsan HCL List. If not = epic fail. Vsan will not function.
Well thats the recommendation to get proper functionality. It does work without but complains a lot (ignorable) and may cause bad performance/weird issues

William uses a ESX in ESX install where he passes through the SATA ports as 'HBA' and his m.2 as cache drive.
Sata pass through is a hack (but works) if you don't need them at all and boot from USB (twice in his case as he seems to use the same usb drive for primary and secondary boot)

VT-D issue will prevent that for you unfortunately.
 
Last edited:

Pradawny

Member
Oct 16, 2016
42
27
18
first of all you will need at least four disks (2x (cache+storage)) and 2 dedicated HBAs for vsan ...
Don't forget about the VSAN licence, which might be difficult to obtain.
Are you subscribed to VMUG Advantage?
 

Kev

Active Member
Feb 16, 2015
461
111
43
41
Alright. I think between all the reading and you guys/gals over here, I’m just going to install freenas 11 and out the two 1tb SSDs mirrored. Should I just make a vmdk file or do raw device pass through? Will I be able to clone the VM and take the drives over to my other servers and do ahci pass through and freenas will just see the previously rdm drives?
 

Dean

Member
Jun 18, 2015
116
10
18
48
Well thats the recommendation to get proper functionality. It does work without but complains a lot (ignorable) and may cause bad performance/weird issues

William uses a ESX in ESX install where he passes through the SATA ports as 'HBA' and his m.2 as cache drive.
Sata pass through is a hack (but works) if you don't need them at all and boot from USB (twice in his case as he seems to use the same usb drive for primary and secondary boot)

VT-D issue will prevent that for you unfortunately.
Ok...maybe a little hasty on my part.

Sent from my Moto Z (2) using Tapatalk