ESXi all-NVMe storage array requirements

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
E

ectoplasmosis

Active Member
Jul 28, 2021
117
54
28
I'm looking to move a Proxmox VE host to ESXi, to experiment with Nvidia vGPU (RTX A5000 shared between three guests).

The Proxmox host currently has an all-NVMe ext4 striped storage array as 'hot' storage for media ingest, before the content gets archived to a magnetic ZFS pool. The NVMe array consists of 4x Samsung PM9A3 Gen4 drives in software RAID0, connected via a 16-lane PCIe Gen4 bifurcation riser.

Does ESXi support something similar, without using a hardware NVMe RAID controller?
 
zer0sum

zer0sum

Well-Known Member
Mar 8, 2013
881
494
63
As far as I'm aware there is no native software raid capability at al in ESXi :(

If it's just a home lab you could do something silly like pass the disks through to a freenas VM, and then present that storage back to ESXi using iSCSI or NFS :p
 
  • Like
Reactions: ectoplasmosis
E

ectoplasmosis

Active Member
Jul 28, 2021
117
54
28
zer0sum said:
As far as I'm aware there is no native software raid capability at al in ESXi :(

If it's just a home lab you could do something silly like pass the disks through to a freenas VM, and then present that storage back to ESXi using iSCSI or NFS :p
Click to expand...
Ah, that’s a shame!

Are there any other hypervisors that have a free/community tier, support software NVMe RAID and Nvidia vGPU?
 
G

gregsachs

Active Member
Aug 14, 2018
590
204
43
i386 said:
As a role it's not free (part of windows server 20*) and I'm not sure if hyper-v server (the "free" gui free hypervisor) supports storage spaces...
Click to expand...
100% sure that h-v 2016 will do storage spaces on a single host.
 
markarr

markarr

Active Member
Oct 31, 2013
421
122
43
nvidia vGPU has drivers out there for xen, kvm, hyper-v, and esxi. Only used esxi so cant speak to usability of the others.
 
  • Like
Reactions: ectoplasmosis
N

nk215

Active Member
Oct 6, 2015
414
147
43
50
ectoplasmosis said:
I'm looking to move a Proxmox VE host to ESXi, to experiment with Nvidia vGPU (RTX A5000 shared between three guests).

The Proxmox host currently has an all-NVMe ext4 striped storage array as 'hot' storage for media ingest, before the content gets archived to a magnetic ZFS pool. The NVMe array consists of 4x Samsung PM9A3 Gen4 drives in software RAID0, connected via a 16-lane PCIe Gen4 bifurcation riser.

Does ESXi support something similar, without using a hardware NVMe RAID controller?
Click to expand...
Make sure you look at the license cost for vGPU from Nvidia. The last card that has free vGPU is the kepler card (Grid K2/K1).
 
  • Like
Reactions: ectoplasmosis
E

ectoplasmosis

Active Member
Jul 28, 2021
117
54
28
nk215 said:
Make sure you look at the license cost for vGPU from Nvidia. The last card that has free vGPU is the kepler card (Grid K2/K1).
Click to expand...
I’m happy to pay for vGPU licenses, just need to find the right hypervisor to standardise on...
 
D

Dean

Member
Jun 18, 2015
116
11
18
49
Agreed - no nvme raid driver, etc.

BUT, VMware just released details for esxi 7.0 cU3 - supposed to support nvme/tcp protocol. I'd check into it. Release is late October I believe.
 
  • Like
Reactions: ectoplasmosis
J

jerryxlol

Member
Nov 27, 2016
48
5
8
32
Uff i believe you have travelled to the road that is full of licensing :)

VGPU on esxi? - I dont know if its possible to share one gpu more nodes. I am planing horizon on two nodes and both HAS TO HAVE nv a40.
Sharing one gpu over more vms is possible with predefined GPU power and memory. NVIDIA vGPU for vSPhere/ESXi is a premium VMware-licensed feature | NVIDIA Nvidia license ... (recently it has not been and now ? pay for every feature ... )

Esxi native NVME raid? Esxi as is does not provide complex storage layer. So for single host not possible. Is it possible to do FTT when you have it licensed. (it does copy of your data to the more hosts / datastores) (https://www.vmware.com/products/vsphere/fault-tolerance.html)

Multiple hosts Vmware VSAN. (i strongly recommend to study what are the main cons of vsan solution and its pricey)


It depends on the usage ... If it is for production then anything below is higly risky:

NVME passtrough to the VM - > provision NFS or ISCSI back to the esxi. VM NEEDS TO HAVE RESERVED CPU AND MEMORY (pcie passtrough req)
I have tried freenas and openmediavault, I use commonly centos. .. runs okay, but as i mentioned NO PRODUCTION. Limited snapshot capability, etc.
 
You must log in or register to reply here.
Top Bottom