FreeNAS to host/run VM's

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

thecoffeeguy

Member
Mar 10, 2016
119
1
18
48
hey folks.
I setup a FreeNAS box in my homelab to hold/run VM's that are part of my ESXi cluster.
This is purely a development/testing type of environment and a playground.

What started out as a quick and dirty setup, has slowly started to evolve to really looking at my setup and figuring out how to maybe get some better performance out of it as well as use best practices.

My setup:
(2) ESXi 7.0 nodes
(1) VMCenter 7.0
(1) FreeNAS box, running 11.2U7 --
Specs: Dual Core Intel CPU, 16gb memory, (2) 2TB Red Drives in a pool as mirror
NFS mount to ESXi to hold VM's centrally
(1) One NIC for mgt
(1) One NIC was configured for iSCSI, but not sure if i need that
(2) Two unused NICS that i've thought about 'bonding' for more 'pipe'

The VM's are all Linux variations of some sort.
not a lot of databases, but that may change moving forward.

The setup works, but I have maybe noticed a bit of a performance hit when moving around in my VM's. I do not think its bad, but I notice it and it bugs me.

I dont have a ton of FreeNAS experience other thatn what I have setup.
I open to suggestions and recommendations

I am also not bound to FreeNAS, so if there is a better solution to use, I am happy to listen. This is bit of a new area for me.

Much obliged friends!

TCG
 

nev_neo

Active Member
Jul 31, 2013
157
44
28
I've tried that once just for testing with BHYVE, it sucked so bad.
Currently i'm only using Freenas for strick storage duties. Its running as a VM on esxi, with the SATA controller passed through.
I think its a better option to use a proper hypervisor for VM duties and let Freenas handle the storage.
I havent tried other options since i'm pretty happy with freenas, but others have had better luck with other platforms like unraid.
 

CreoleLakerFan

Active Member
Oct 29, 2013
485
180
43
hey folks.
I setup a FreeNAS box in my homelab to hold/run VM's that are part of my ESXi cluster.
This is purely a development/testing type of environment and a playground.

What started out as a quick and dirty setup, has slowly started to evolve to really looking at my setup and figuring out how to maybe get some better performance out of it as well as use best practices.

My setup:
(2) ESXi 7.0 nodes
(1) VMCenter 7.0
(1) FreeNAS box, running 11.2U7 --
Specs: Dual Core Intel CPU, 16gb memory, (2) 2TB Red Drives in a pool as mirror
NFS mount to ESXi to hold VM's centrally
(1) One NIC for mgt
(1) One NIC was configured for iSCSI, but not sure if i need that
(2) Two unused NICS that i've thought about 'bonding' for more 'pipe'

The VM's are all Linux variations of some sort.
not a lot of databases, but that may change moving forward.

The setup works, but I have maybe noticed a bit of a performance hit when moving around in my VM's. I do not think its bad, but I notice it and it bugs me.

I dont have a ton of FreeNAS experience other thatn what I have setup.
I open to suggestions and recommendations

I am also not bound to FreeNAS, so if there is a better solution to use, I am happy to listen. This is bit of a new area for me.

Much obliged friends!

TCG
You are I/O bound - the drives are the biggest bottleneck, followed by the single NIC you are using for iscsi, assuming it's 1GB. You can use the two unused NICs for iSCSI MPIO, which will speed up your network access, but randon IOPS and sequential read/writes are still going to suck with those 2TB Reds.

Cheapest way to speed things up is to add a quality pair of SSDs (Intel s3700 are a decent start) to use as a SLOG device to speed up sync writes on your HDDs, and then use iSCSI multi-path IO (mpio) optimally on a separate VLAN to get more network bandwith from your ESX hosts to your FreeNAS VM.