ESXi & nvdimms

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
Has anyone had any success in using ESXi with nvdimms?

Theoretically they are supposed to be supported out of the box but mine do not get detected properly...

Or rather they do get detected with some minor issues, but ESXi does not identify them as persistent memory so they are not usable...

Code:
[root@devesx3ssl:~] dmesg |grep -i nvdimm
TSC: 920250 cpu0:1)BootConfig: 819: injectNVDIMMState = 0 (0)
TSC: 927002 cpu0:1)BootConfig: 828: allowDsmForNVDIMMHealth = TRUE
2019-07-14T18:11:48.916Z cpu29:2097773)NVD: 379: NVDIMM 0x20: NVDIMM Representation string 802c-0f-1649-14f2ac45
2019-07-14T18:12:01.057Z cpu27:2097877)Loading module intelnvdimm ...
2019-07-14T18:12:01.058Z cpu27:2097877)Elf: 2101: module intelnvdimm has license VMware
2019-07-14T18:12:01.058Z cpu27:2097877)Device: 192: Registered driver 'IntelNVDimm' from 35
2019-07-14T18:12:01.058Z cpu27:2097877)Mod: 4962: Initialization of intelnvdimm succeeded with module ID 35.
2019-07-14T18:12:01.058Z cpu27:2097877)intelnvdimm loaded successfully.
2019-07-14T18:12:01.059Z cpu15:2097856)NVD: 306: NVDIMM 0x20: _NCH method failed, status Not supported
2019-07-14T18:12:01.059Z cpu15:2097856)NVD: 1348: NVDIMM 0x20: GetHealthInfo failed, status Not implemented
2019-07-14T18:12:01.059Z cpu15:2097856)NVD: 1628: NVDIMM 0x20: Failed to get LSA data, status Not implemented
2019-07-14T18:12:01.059Z cpu15:2097856)NVD: 718: NVDIMM 0x20: GetCELSize failed, status Not implemented
2019-07-14T18:12:01.059Z cpu15:2097856)IntelNVDimm: IntelNVDimmAttachDev:80: Attached to NVDIMM device 0x20
2019-07-14T18:12:01.059Z cpu15:2097856)Device: 327: Found driver IntelNVDimm for device 0x112843050f812b8e
vmware -v
VMware ESXi 6.7.0 build-13981272
 

Evan

Well-Known Member
Jan 6, 2016
3,346
598
113
Should work (at least on HPE systems I have tested)
Are you trying to use it as a file store for VM’s or are you trying to pass it through directly to a PMem aware VM ?

The answer to both is, yes - HPE DL380 Gen10 with 6248, used whatever was the latest firmware a couple of months ago and same for ESX, just just got a bit of fun to see if it worked before the hardware was used for something else, didn’t really have a use case. Was a 6+6 config x 2 sockets.
 

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
This is Micron on Supermicro, single nvdimm for testing.
Linux/FreeBSD work fine ( or at least better), but Esxi does not properly identifies them so I can't do anything, neither datastore nor passing through. The latter would be primary use case atm
 

Evan

Well-Known Member
Jan 6, 2016
3,346
598
113
Sorry not much help as it just worked when I did it but very different config than what you had running, the system is running Linux now and can’t test ESX again sorry.
 

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
Well that's the issue, its supposed to just work - no debugging options at all on esxi for that.
Is not a huge deal since it runs fine on linux/Freebsd and hopefully OmniOs, just would have been nice not to have to run a dedicated storage box:)
 

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
Just stumbled over a Supermicro FAQ entry (6 weeks old); have not tried it, but maybe it helps somebody

1609025437305.png
 
  • Like
Reactions: gb00s

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
So official statement from Supermicro - there is no system validated (or expected working) for ESXi and NVDimm-Ns.

Inofficial:
The idea that a system that supports NVDimm-N and ESXi might be expected to actually support NVDimm with ESXi is purely speculative and will not be supported (nor is it working).



And yes, there is a long story behind this; quite ridiculous & upsetting o/c.
 

mpepper

New Member
Nov 22, 2020
14
11
3
Found a snippet from a Dell support page that suggest ESXi only supports NVDIMM-N on Intel Xeon Gold (51xx, 61xx) and Platinum (81xx) series. I cannot get ESXi to recognize any NVDIMM-N (both Micron and SKHynix) on a RomeD8-2T w/ Milan 7313P. Windows, Linux, and FreeBSD detect and assign namespace just fine. ESXi does not cooperate though and since no NDCTL is available there is no config option.