PSA: Vmware ESXi 7.0 does not like consumer NVMe drives

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

barzin98

New Member
Oct 31, 2022
1
0
1
hi
This topic that was raised was very useful and great and helped me a lot to choose Samsung 970 PRO NVMe M2 ; and please don't close this thread so that more people can share their experiences, thanks
 

Jimster480

New Member
Mar 12, 2020
27
8
3
Just an update; this still works fine on the newest ESXI 7.
Does anyone know if it works on ESXI 8? I'm a bit afraid to upgrade to 8.
 

Brunolee

New Member
Apr 16, 2023
1
0
1
I just found out that ESXi 7.0 is really particular about the brand of NVMe drive you try to use.
And it was an issue on earlier versions but it seems to be even stricter now

My Intel 660P works perfectly, but my Adata XPG SX8200 and HP EX920 do not work at all :(
I believe most Intel and Samsung drives like 970's should work fine at least :D

YMMW with other brands on the official HCL - Cisco, Dell, HPE, HGST, Hitachi, Huawei, Intel, Lenovo, Micron, Oracle, Samsung, ScaleFlux, SKHynix, WD
If anyone has other brands working please comment and we can start to create a list.

In previous versions of ESXi you could run a couple of commands to load older nvme drivers like this:


But on ESXi 7.0 that somehow kills all of your network card drivers and your box is now a shiny paperweight :(
Anyone know of any further hacks/tweaks to get it work?
感谢这个方法,非常有用!
我的 Homelab 使用 R730XD,使用 internal SD 作为 ESXi 6.7 boot,PERC H730需要 passthrough 给 Truenas使用,因此无法使用SAS磁盘作为 ESXi Data Store,使用您的方法,成功将 PCIE nvme磁盘驱动成功! thanks again.
 

Attachments

Jimster480

New Member
Mar 12, 2020
27
8
3
Well, nowhere in the thread did it specify the Phison controller so...

Perhaps the thread title should be changed.

Have you tried this?

Honestly I came up with this last night based on the screenshot you posted. Vs my own ESXI instances using SMI controllers.
I know that Samsung controllers still work, you confirming that Phison controllers do work (someone else mentioned other drives) like @StevenDTX who said that Sandisk SN570 drives worked (WD Controller).
Although what is weird is that @dante4 mentioned that his Kingston drive worked; yet it has the same SMI controller as my Mushkin Pilot-E which I already know NEEDS the old driver to work.
Along with @cli-networks who mentioned that SN550 drives don't work.

So it seems that Samsung controllers all work; Sandisk/WD Controllers PARTIALLY work and with your post certain Phison controllers work.
The part I find the most interesting is that 2263EN works based on @dante4 but 2262EN doesn't work (from my own experiences). They are all from the "same generation" and 2262EN is actually the "flagship" controller where 2263EN is the cut down 4-channel one.

The title of this post shouldn't be changed, because upon further investigation; it doesn't seem to matter your drive brand unless maybe its Samsung because otherwise it is hit and miss.

Maybe it has to do with DDR3 on the controller vs DDR4 since the Kingston drive has DDR4 and the Mushkin has DDR3.
Soon I will test ESXI with a PNY XLR8 and report back.
 
  • Like
Reactions: BennyT

Railgun

Active Member
Jul 28, 2018
150
57
28
The title of this post shouldn't be changed, because upon further investigation; it doesn't seem to matter your drive brand unless maybe its Samsung because otherwise it is hit and miss.
That’s the change I was referring to. “Some” consumer drives…
 

Jimster480

New Member
Mar 12, 2020
27
8
3
That’s the change I was referring to. “Some” consumer drives…
Yeah it is definitely some consumer drives. I haven't figured out what actually makes it work or not but I thought I had figured it out last night. Either way I'm glad that the driver still works today and my servers are running fine.
 

galvesribeiro

New Member
Feb 9, 2024
25
7
3
ESXI 8.0 Update 2 - The fling still works. It installs just fine even if it was built for 7.0.

However, in my case, none of the Crucial T705 PCIe Gen5 drives are detected. I seem them in the adapter tab on the Storage section but the disk doesn't appear available to be added to a datastore. The reason is because this new Gen5 drive has a new Phison E26 controller which the driver doesn't know :(
 

galvesribeiro

New Member
Feb 9, 2024
25
7
3
wait a minute, people are still willingly using VMWare????
Yes. Unfortunately, the alternatives, are not as good. Very limited/complicated/non-scalable GPU/PCIe passthru, virtual NICs that can barely support 10G, and literally no good option to replace vSAN. Not to mention the shitload of management features from vCenter.

Until we have good replacements we are hostages. People suggest Nutanix, XCP-ng, Proxmox. Good initiatives but in reality none of them have 1/10th the features ESXI has, unfortunately. Even core things like those I mentioned either doesn't exist or their alternatives are clumsy/hacky and/or limited.

Anyway, hope some day this scenario change. Until there, company will stay on VMWare and at home, VMug still gives me everything for cheap subscription per year.
 

zer0sum

Well-Known Member
Mar 8, 2013
876
490
63
Yes. Unfortunately, the alternatives, are not as good. Very limited/complicated/non-scalable GPU/PCIe passthru, virtual NICs that can barely support 10G, and literally no good option to replace vSAN. Not to mention the shitload of management features from vCenter.

Until we have good replacements we are hostages. People suggest Nutanix, XCP-ng, Proxmox. Good initiatives but in reality none of them have 1/10th the features ESXI has, unfortunately. Even core things like those I mentioned either doesn't exist or their alternatives are clumsy/hacky and/or limited.

Anyway, hope some day this scenario change. Until there, company will stay on VMWare and at home, VMug still gives me everything for cheap subscription per year.
Hmm, in my experience Proxmox is killing it in all of those areas, and is far better than VMware
  • GPU passthrough works great
  • PCIe passthrough is even better and is trivial to get working
  • Virtual NIC's could be faster, but ~10Gbps is fine by me
  • SRIOV is easy to get working if you want faster "virtual" NIC's
I';d argue that it is the opposite and Proxmox has 10 times more features than ESXi :p

What do you need vSAN for?

Clustering? Vmotion? Mgmt? Storage? SDN?
Proxmox is awesome here

Shared Storage?
Ceph and GlusterFS are rock solid for me, and if you want a hacky way to do it, just name your storage the same on each node
 

galvesribeiro

New Member
Feb 9, 2024
25
7
3
Hmm, in my experience Proxmox is killing it in all of those areas, and is far better than VMware
  • GPU passthrough works great
  • PCIe passthrough is even better and is trivial to get working
  • Virtual NIC's could be faster, but ~10Gbps is fine by me
  • SRIOV is easy to get working if you want faster "virtual" NIC's
I';d argue that it is the opposite and Proxmox has 10 times more features than ESXi :p

What do you need vSAN for?

Clustering? Vmotion? Mgmt? Storage? SDN?
Proxmox is awesome here

Shared Storage?
Ceph and GlusterFS are rock solid for me, and if you want a hacky way to do it, just name your storage the same on each node
As I said, they are cool initiatives. But they are far from something that companies would rely on. Not to mention the shady techniques from Proxmox to "require a subscription to update apt repos".

  • Virtual NIC's could be faster, but ~10Gbps is fine by me
Yeah, maybe your use case it is "fine", but it doesn't make it better than what is available on ESXi unfortunately.

I';d argue that it is the opposite and Proxmox has 10 times more features than ESXi :p
Sorry but I'm not in the fanboy/enthusiast zone. I'm grounded on facts. The features available on vCenter to management, workload distribution, clustering, etc, there is nothing like that in the marketing. That is why ESXi is (now unfortunately) the most used hypervisor outside public cloud. There is arguments against that. It is just sad that we had this bad acquisition which f'ed up everything.

What do you need vSAN for?
I hope that was a rhetorical question. There are many usages of vSAN outside just storing VM disks.

Shared Storage?
Ceph and GlusterFS are rock solid for me, and if you want a hacky way to do it, just name your storage the same on each node
GlusterFS is dead. Even TrueNAS deprecated its support. Ceph... Well, the cost of REALLY operating Ceph in production for a company with reasonable size, makes VMWare subscription appear cheap. Not saying it is bad, it is really good and I follow it since the initial versions with the design sessions etc but, it still miles away from vSAN and its distributed storage both in terms of reliability, performance, security.


Again, not calling anything "awesome" and don't want to fanboy anything. Just saying that it is unfortunate that we still need VMWare as there are no matches unfortunately Today. And the bad thing is that Broadcom knows it, and is exploiting the situation with those absurd prices. Again, not talking about homelabs. Talking about real business relying on core infrastructure.
 
  • Like
Reactions: mrpasc and TRACKER