TrueNAS iSCSI sharing of NVME storage to ESXi hosts

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

hydra1981

Member
Jul 19, 2015
53
39
18
43
Hi,

At the moment, I have 3-4 physical machines running ESXi and each one of them has some local storage without any redundancy. I am getting a bit worried that if something goes wrong, I will lose the VMs. I am taking backups using Nakivo but obviously these are not real time. How can I go about this? I have a TrueNAS system that is sharing disk space either as windows shares, iSCSI or NFS so I was wondering if it would make more sense to build an NVME array (say 3+1 cards) and share the storage. One thing that hasn't sunk in yet is, can iSCSI storage be shared between different hosts if the VM storage space ? Any pitfalls in this?

Thank you!

Nick
 
  • Like
Reactions: Samir

realtomatoes

Active Member
Oct 3, 2016
252
33
28
44
i use veeam for VM backup and file-based backup.

i got an external storage where those backups are dumped and another job that uploads files older than7 days to an offsite storage.

----------
i just realized you were looking for towards a shared storage solution rather a backup solution to for IF/When you need backups.

with regads to the shared storage path, compare iSCSI and NFS performance differences. since truenas runs ZFS sync writes are performance killers. so you'll want to spec your truenas box proper to fit the performance you are going for.
 
Last edited:
  • Like
Reactions: Samir

gea

Well-Known Member
Dec 31, 2010
3,333
1,296
113
DE
Sync write is mandatory for VM storage with guest filesystems, does not matter if you use iSCSI or NFS.
With same sync/writeback setting, both are similar fast.

Only on a backup system or a filer, you do not need sync as ZFS cares about itself due Copy on Write.
 
  • Like
Reactions: Samir

gb00s

Well-Known Member
Jul 25, 2018
1,253
667
113
Poland
Have you considered to move away from ESXi and go for Proxmox, building a Proxmox cluster and a HA NVMe storage cluster (DRBD or GlusterFS) for vm's inside the Proxmox cluster? No shared storage via NFS/Samba/iSCSI or whatever needed. As I have no serious experience with ESXi, not sure you can do the same efficient setup with it.
 
Last edited:
  • Like
Reactions: Samir

Tech Junky

Active Member
Oct 26, 2023
625
215
43
I would setup an independent box and use U drives if you're looking for speed and capacity.

I picked up a U.3 15.36TB for just over $1000 and it hits 6.5GB/s for speed. There aren't any M2 options beyond 8TB and those run $800. You can get smaller Gen3 U drives though for $400 for 8TB. Just have to get a bit creative on how to link them to the system. I went with an M2 adapter + cable using Oculink though to make sure I didn't bottleneck the drive. The adapter / cable add about $45 per drive but, there are some direct PCB options that can slot 4 drives for $80 if you have an open slot that bifurcates.
 
  • Like
Reactions: Samir

zachj

Active Member
Apr 17, 2019
188
126
43
Vsphere supports both iscsi and nfs natively. It works and is trivial to setup.

as mentioned I would enable sync on the truenas shares for any data you care about. In my lab I have a separate truenas share with sync disabled for things like iso files that I don’t care about losing
 
  • Like
Reactions: Samir