EDIT: final update- supermicro support confirmed they are able to reproduce this issue, but will not be fixing it, as the p3700 is "EOL" (it is, but i disagree, to see full info see my last post on page 2) - tks for all the great replies and help from STH!
So ive come across a werid issue in my new test system (just a test setup, esxi 6.5u2 , host only, no vCenter):
I have a 2.5" p3700 nvme disk attached to my supermicro x10 based system (its direct attached via a sm riser card= AOC-2UR8N4-i2XT , which has 4x nvme ports , which goes to a sm nvme backplane, i dont think this HW is relevant though).
I had been running the system great for about 2 weeks, i had a datastore created direct on the nvme disk with several VMs running from it (during most of this time i also had a raid0 data store also created, but unused, from a LSI 2308 raid card , at some point i passed this entire card directly through to a VM, so it no longer appears to esxi). - i had rebooted the esxi host maybe 3 times, never had any issues.
Yesterday my power flickered and when it came back my nvme data store was gone, and under storage->adaptors the nvme "hba" wouldnt come up. I tried removing and reinserting the nvme disk, trying a different nvme bay, rebooting. nothing.
However, this entire time i could see the disk via gui, manage->HW->PCI devices (see image) and also via ssh LSPCI:
0000:08:00.0 Mass storage controller: Intel Corporation DC P3700 SSD [2.5" SFF]
assuming the flash was fried or something, i boot the sys into a ubuntu live CD, and under the disks utility, there is the p3700 disk, with its proper 1.6tb VMFS partition intact.
I then update/patch from 6.5U2 (may2018) to 6.5u2 (latest patches ~ nov1 2018). Reboot, still no nvme.
I attach a random sata disk (to MB sata port), format it as a datastore (VMFS), then reboot, and BOOM the nvme datastore is back! if i remove the sata disk, reboot (so that the nvme is the only attached disk appear to esxi), the NVMe again wont appear!
So it seems that as long as i have some kind of other disk attached (and visible to esxi), my nvme appears properly.
Any ideas what this is about?
this is how lspci + HW tab looks now (with nvme working, bc i have a random sata disk attached with a datastore on it)
0000:07:00.0 Non-Volatile memory controller: Intel Corporation DC P3700 SSD [2.5" SFF] [vmhba3]
vs how lspci + HW tab looks wo the sata disk attached (and thus nvme wont appear) - (note i added the 08:00 to the image, as i have this screen up on a offline laptop, but cant get the image over to this pc to upload)
0000:08:00.0 Mass storage controller: Intel Corporation DC P3700 SSD [2.5" SFF]
thanks!
So ive come across a werid issue in my new test system (just a test setup, esxi 6.5u2 , host only, no vCenter):
I have a 2.5" p3700 nvme disk attached to my supermicro x10 based system (its direct attached via a sm riser card= AOC-2UR8N4-i2XT , which has 4x nvme ports , which goes to a sm nvme backplane, i dont think this HW is relevant though).
I had been running the system great for about 2 weeks, i had a datastore created direct on the nvme disk with several VMs running from it (during most of this time i also had a raid0 data store also created, but unused, from a LSI 2308 raid card , at some point i passed this entire card directly through to a VM, so it no longer appears to esxi). - i had rebooted the esxi host maybe 3 times, never had any issues.
Yesterday my power flickered and when it came back my nvme data store was gone, and under storage->adaptors the nvme "hba" wouldnt come up. I tried removing and reinserting the nvme disk, trying a different nvme bay, rebooting. nothing.
However, this entire time i could see the disk via gui, manage->HW->PCI devices (see image) and also via ssh LSPCI:
0000:08:00.0 Mass storage controller: Intel Corporation DC P3700 SSD [2.5" SFF]
assuming the flash was fried or something, i boot the sys into a ubuntu live CD, and under the disks utility, there is the p3700 disk, with its proper 1.6tb VMFS partition intact.
I then update/patch from 6.5U2 (may2018) to 6.5u2 (latest patches ~ nov1 2018). Reboot, still no nvme.
I attach a random sata disk (to MB sata port), format it as a datastore (VMFS), then reboot, and BOOM the nvme datastore is back! if i remove the sata disk, reboot (so that the nvme is the only attached disk appear to esxi), the NVMe again wont appear!
So it seems that as long as i have some kind of other disk attached (and visible to esxi), my nvme appears properly.
Any ideas what this is about?
this is how lspci + HW tab looks now (with nvme working, bc i have a random sata disk attached with a datastore on it)
0000:07:00.0 Non-Volatile memory controller: Intel Corporation DC P3700 SSD [2.5" SFF] [vmhba3]
vs how lspci + HW tab looks wo the sata disk attached (and thus nvme wont appear) - (note i added the 08:00 to the image, as i have this screen up on a offline laptop, but cant get the image over to this pc to upload)
0000:08:00.0 Mass storage controller: Intel Corporation DC P3700 SSD [2.5" SFF]
thanks!
Last edited: