NVME Napp-It / OmniOS Pass through in ESXI -- date?

[T_Minus]

@gea you mentioned this would be resolved soon -- any road map, or estimated time to when we can pass-through Intel NVME to Napp-IT VM in ESXI?

 

[gea]

There is a new OmniOS bloody 151019 update available

This one includes fixes for NVMe and newest drivers for Intel X550 and 710 (40Gbe)
I am on holiday so I cannot check but maybe this solves the passthrough problem
Install 151019 and do a pkg update

From the anouncement:

new in this update:

* X550 support is now in synch with upstream (last update had a pre-upstream version).

* Bash to 4.3p46

* i40e is now part of "entire" for people installing on boxes Intel XLnnn hardware.

* Curl to 7.50.1

* library/nghttp2 is now in omnios-userland

* libidn is now 1.33

* modcaml update for man pages.

* "if" and "else" statements now in DTrace

* NVME bugfixes (illumos 7275)

* Intel Broadwell "rdc" and "adx" support.

* pvscsi bugfixes (illumos 7218 and 7219)

* ZFS allocation improvements (illumos 7090)

* Sync mDNS with mDNSResponder-625.41.2 (illumos 7139)


I will be updating the LX version later this week, and will announce so on the mailing list.

 

[T_Minus]

Any of the upgrades in Napp-IT fix this? Or is it a separate OS update only?

 

[gea]

napp-it is independent from the underlying OS.
Driver or OS features depend on OpenIndiana, OmniOS or Solaris. This is different to appliances like FreeNAS or NexentaStor where OS and Management is tight together and where you cannot select management and OS release or even a whole OS distribution independently.

For OmniOS you must update to newest 151019 bloody or wait for next stable (around oktober 2016) for newset drivers like NVMe fixes. You can also use Solaris with its own NVMe driver.

Drivers for new X550 (10G) or XL710 (40G) are now in current 151018

btw.
The current bloody is a snapshot of current state
There is ongoing work on the NVMe driver as there seems to be a remaining problem with massive NVMe setup, Bug #7351: NVMe driver sporadically lost track of completed I/O request, which leads to zpool hanging and machine panic. - illumos gate - illumos.org

For details, join maillists illumos-discuss and omnios-discuss

 

[gea]

OmniTi has updated 151019 bloody with newer NVMe 1.1 driver.
This Bloody is propably base of next stable 151020 expected in a few days.

[OmniOS-discuss] Likely last bloody update before r151020

 

[daleg]

FYI, we plan to pull the NVME 1.1 support (with associated blkdev fixes) and e1000g driver support for Kaby Lake chips back to 014 as well.

 

[gea]

There will be an improvement on the Ilumos NVMe driver by Nexenta to fix the ESXi pass-through problem.
Listbox • Email Marketing and Discussion Groups

 

[T_Minus]

When will this trickle down to us?

 

[gea]

Current state is review at Illumos
As the modification is minor, I suppose that it will be available soon at Illumos.

OmniOS bloody will be updated quite often.
It it will be available in OmniOS prior summer with next 151022 lts -
one may need to ask at omnios-discuss.

btw.
It may be good to ask at omnios-dicuss for more persons as integration in OmniOS may be related to public interest.

 

[daleg]

I've been watching this thread, so I know it's desired. I will backport this fix to 014, 018, and 020. We're working on a ton of prep work for 022 at the moment around Python and getting things sorted for being able to support a better installer which supports tsoome's recent work with porting in FreeBSD's loader (goodbye GRUB!), so that plus some other projects means I might not be able to get to this until next week. I'm also evaluating the APIX fix that came with it for possible inclusion in the backport, as well, plus a OpenSSL optimization for 64bit processes which link against libcypto and use ECDH(E)

 

[T_Minus]

I've been watching this thread, so I know it's desired. I will backport this fix to 014, 018, and 020. We're working on a ton of prep work for 022 at the moment around Python and getting things sorted for being able to support a better installer which supports tsoome's recent work with porting in FreeBSD's loader (goodbye GRUB!), so that plus some other projects means I might not be able to get to this until next week. I'm also evaluating the APIX fix that came with it for possible inclusion in the backport, as well, plus a OpenSSL optimization for 64bit processes which link against libcypto and use ECDH(E)

Awesome!

 

[wildchild]

@daleg.. do you know if nexenta's vaai extention are also on the roadmap anytime soon ?
As far as i know nexenta has released them sometime again to the illumos community,just not sure anyone picked it up

 

[daleg]

@daleg.. do you know if nexenta's vaai extention are also on the roadmap anytime soon ?
As far as i know nexenta has released them sometime again to the illumos community,just not sure anyone picked it up

Not our road map, as neither Dan nor myself have the time at the moment nor the VMware knowledge/bits to test it with. I'm not sure what you mean by Nexenta releasing them to the illumos community, because they haven't submitted a patch to add VAAI in illumos-gate to developers@. If what you mean by that is that it is in the illumos-nexenta repo, then if someone were willing to either poke Nextenta directly and get them to submit it upstream, or do the porting and testing work themselves and create a patch for illumos-gate, that is also an option (and probably the most likely way this would appear outside of Nexenta)

 

[gea]

There is a newer NVMe driver version Jan 17. in newest OmniOS Bloody
where NVMe pass-through seems to work.

 

[gea]

Info from OmniTi (OmniOS)

If you are user of LTS (r151014) or Stable (r151020), please "pkg update" and be ready for a new BE and a reboot.

Included in this update are:

- bge fixes on HP Gen9 systems
- MSI-X for NVMe is disable on VMware (where MSI-X is a problem)
- mmap() now properly modifies a file's timestamps in ZFS.
- Default NFS server threads are now increased.
- i40e now uses multiple receive and transmit rings
- One mpt_sas panic bugfix

The install media has been updated too, as the bge fix will help HP Gen9 users.
Install media pointers can be found here:

Installation

Happy updating!
Dan

 

[yu130960]

Done a pkg update. Thanks for the heads up.

 

[sth]

I've previously had good success passing through 2 PCI based P3700 to Napp-it and Omni-OS but am struggling with two 2.5" form factor P3700s connected to a bifurcated Xeon-d 1541 & Supermicro 4 * U2 AOC-SLG3-4e4r. I can pass them through ESXi 6.7 u1 to a Windows Server 2016 without issue but passing them to OmniOS causes them to be detected as unknown types. Given the popularity of these drives I'm figuring this is a user error or gap in my knowledge. My S3700s show fine but the P3700 passed through here is the drive 7 / unknown. Any suggestions as to driver updates etc would be appreciated.

AVAILABLE DISK SELECTIONS:
       0. c2t0d0 <VMware-Virtual disk-2.0-40.00GB>
          /pci@0,0/pci15ad,1976@10/sd@0,0
       1. c61t0d0 <HGST-HUH728080ALN600-A4GNT7J0-7.28TB>
          /pci@0,0/pci15ad,7a0@16/pci15d9,86d@0/disk@0,0
       2. c61t1d0 <HGST-HUH728080ALN600-A4GNT7J0-7.28TB>
          /pci@0,0/pci15ad,7a0@16/pci15d9,86d@0/disk@1,0
       3. c61t2d0 <HGST-HUH728080ALN600-A4GNT7J0-7.28TB>
          /pci@0,0/pci15ad,7a0@16/pci15d9,86d@0/disk@2,0
       4. c61t3d0 <INTEL-SSDSC2BA400G4-G2010160-18.63GB>
          /pci@0,0/pci15ad,7a0@16/pci15d9,86d@0/disk@3,0
       5. c61t4d0 <INTEL-SSDSC2BA800G3-5DV12270-745.21GB>
          /pci@0,0/pci15ad,7a0@16/pci15d9,86d@0/disk@4,0
       6. c61t5d0 <INTEL-SSDSC2BA800G3-5DV12270-745.21GB>
          /pci@0,0/pci15ad,7a0@16/pci15d9,86d@0/disk@5,0
       7. c63t1d0 <drive type unknown>
          /pci@0,0/pci15ad,7a0@16,1/pci8086,3703@0/blkdev@1,0

 

[gea]

Please keep us updated if you find a solution.

At least with an P3600 that i have (pci-e version) I do not have problems.
The <drive type unknown> in the format command itself is uncritical as long as there are no stability problems or other errors.