NVME Napp-It / OmniOS Pass through in ESXI -- date?

Discussion in 'Solaris, Nexenta, OpenIndiana, and napp-it' started by T_Minus, Aug 24, 2016.

  1. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,651
    Likes Received:
    1,388
    @gea you mentioned this would be resolved soon -- any road map, or estimated time to when we can pass-through Intel NVME to Napp-IT VM in ESXI?
     
    #1
  2. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    There is a new OmniOS bloody 151019 update available

    This one includes fixes for NVMe and newest drivers for Intel X550 and 710 (40Gbe)
    I am on holiday so I cannot check but maybe this solves the passthrough problem
    Install 151019 and do a pkg update

    From the anouncement:

    new in this update:

    * X550 support is now in synch with upstream (last update had a pre-upstream version).

    * Bash to 4.3p46

    * i40e is now part of "entire" for people installing on boxes Intel XLnnn hardware.

    * Curl to 7.50.1

    * library/nghttp2 is now in omnios-userland

    * libidn is now 1.33

    * modcaml update for man pages.

    * "if" and "else" statements now in DTrace

    * NVME bugfixes (illumos 7275)

    * Intel Broadwell "rdc" and "adx" support.

    * pvscsi bugfixes (illumos 7218 and 7219)

    * ZFS allocation improvements (illumos 7090)

    * Sync mDNS with mDNSResponder-625.41.2 (illumos 7139)


    I will be updating the LX version later this week, and will announce so on the mailing list.
     
    #2
    T_Minus likes this.
  3. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,651
    Likes Received:
    1,388
    Any of the upgrades in Napp-IT fix this? Or is it a separate OS update only?
     
    #3
  4. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    napp-it is independent from the underlying OS.
    Driver or OS features depend on OpenIndiana, OmniOS or Solaris. This is different to appliances like FreeNAS or NexentaStor where OS and Management is tight together and where you cannot select management and OS release or even a whole OS distribution independently.

    For OmniOS you must update to newest 151019 bloody or wait for next stable (around oktober 2016) for newset drivers like NVMe fixes. You can also use Solaris with its own NVMe driver.

    Drivers for new X550 (10G) or XL710 (40G) are now in current 151018

    btw.
    The current bloody is a snapshot of current state
    There is ongoing work on the NVMe driver as there seems to be a remaining problem with massive NVMe setup, Bug #7351: NVMe driver sporadically lost track of completed I/O request, which leads to zpool hanging and machine panic. - illumos gate - illumos.org

    For details, join maillists illumos-discuss and omnios-discuss
     
    #4
    Last edited: Sep 3, 2016
  5. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    #5
    T_Minus likes this.
  6. daleg

    daleg New Member

    Joined:
    Oct 21, 2016
    Messages:
    10
    Likes Received:
    9
    FYI, we plan to pull the NVME 1.1 support (with associated blkdev fixes) and e1000g driver support for Kaby Lake chips back to 014 as well.
     
    #6
    gea likes this.
  7. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    #7
  8. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,651
    Likes Received:
    1,388
    When will this trickle down to us?
     
    #8
  9. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    Current state is review at Illumos
    As the modification is minor, I suppose that it will be available soon at Illumos.

    OmniOS bloody will be updated quite often.
    It it will be available in OmniOS prior summer with next 151022 lts -
    one may need to ask at omnios-discuss.

    btw.
    It may be good to ask at omnios-dicuss for more persons as integration in OmniOS may be related to public interest.
     
    #9
  10. daleg

    daleg New Member

    Joined:
    Oct 21, 2016
    Messages:
    10
    Likes Received:
    9
    I've been watching this thread, so I know it's desired. I will backport this fix to 014, 018, and 020. We're working on a ton of prep work for 022 at the moment around Python and getting things sorted for being able to support a better installer which supports tsoome's recent work with porting in FreeBSD's loader (goodbye GRUB!), so that plus some other projects means I might not be able to get to this until next week. I'm also evaluating the APIX fix that came with it for possible inclusion in the backport, as well, plus a OpenSSL optimization for 64bit processes which link against libcypto and use ECDH(E)
     
    #10
    T_Minus and gea like this.
  11. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,651
    Likes Received:
    1,388
    Awesome!
    :)
     
    #11
  12. wildchild

    wildchild Active Member

    Joined:
    Feb 4, 2014
    Messages:
    394
    Likes Received:
    56
    @daleg.. do you know if nexenta's vaai extention are also on the roadmap anytime soon ?
    As far as i know nexenta has released them sometime again to the illumos community,just not sure anyone picked it up
     
    #12
  13. daleg

    daleg New Member

    Joined:
    Oct 21, 2016
    Messages:
    10
    Likes Received:
    9
    Not our road map, as neither Dan nor myself have the time at the moment nor the VMware knowledge/bits to test it with. I'm not sure what you mean by Nexenta releasing them to the illumos community, because they haven't submitted a patch to add VAAI in illumos-gate to developers@. If what you mean by that is that it is in the illumos-nexenta repo, then if someone were willing to either poke Nextenta directly and get them to submit it upstream, or do the porting and testing work themselves and create a patch for illumos-gate, that is also an option (and probably the most likely way this would appear outside of Nexenta)
     
    #13
  14. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    There is a newer NVMe driver version Jan 17. in newest OmniOS Bloody
    where NVMe pass-through seems to work.
     
    #14
  15. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    Info from OmniTi (OmniOS)

    If you are user of LTS (r151014) or Stable (r151020), please "pkg update" and be ready for a new BE and a reboot.

    Included in this update are:

    - bge fixes on HP Gen9 systems
    - MSI-X for NVMe is disable on VMware (where MSI-X is a problem)
    - mmap() now properly modifies a file's timestamps in ZFS.
    - Default NFS server threads are now increased.
    - i40e now uses multiple receive and transmit rings
    - One mpt_sas panic bugfix

    The install media has been updated too, as the bge fix will help HP Gen9 users.
    Install media pointers can be found here:

    Installation

    Happy updating!
    Dan
     
    #15
    Last edited: Mar 1, 2017
    yu130960 likes this.
  16. yu130960

    yu130960 Member

    Joined:
    Sep 4, 2013
    Messages:
    122
    Likes Received:
    9
    Done a pkg update. Thanks for the heads up.
     
    #16
  17. sth

    sth Active Member

    Joined:
    Oct 29, 2015
    Messages:
    234
    Likes Received:
    35
    I've previously had good success passing through 2 PCI based P3700 to Napp-it and Omni-OS but am struggling with two 2.5" form factor P3700s connected to a bifurcated Xeon-d 1541 & Supermicro 4 * U2 AOC-SLG3-4e4r. I can pass them through ESXi 6.7 u1 to a Windows Server 2016 without issue but passing them to OmniOS causes them to be detected as unknown types. Given the popularity of these drives I'm figuring this is a user error or gap in my knowledge. My S3700s show fine but the P3700 passed through here is the drive 7 / unknown. Any suggestions as to driver updates etc would be appreciated.

    Code:
    AVAILABLE DISK SELECTIONS:
           0. c2t0d0 <VMware-Virtual disk-2.0-40.00GB>
              /pci@0,0/pci15ad,1976@10/sd@0,0
           1. c61t0d0 <HGST-HUH728080ALN600-A4GNT7J0-7.28TB>
              /pci@0,0/pci15ad,7a0@16/pci15d9,86d@0/disk@0,0
           2. c61t1d0 <HGST-HUH728080ALN600-A4GNT7J0-7.28TB>
              /pci@0,0/pci15ad,7a0@16/pci15d9,86d@0/disk@1,0
           3. c61t2d0 <HGST-HUH728080ALN600-A4GNT7J0-7.28TB>
              /pci@0,0/pci15ad,7a0@16/pci15d9,86d@0/disk@2,0
           4. c61t3d0 <INTEL-SSDSC2BA400G4-G2010160-18.63GB>
              /pci@0,0/pci15ad,7a0@16/pci15d9,86d@0/disk@3,0
           5. c61t4d0 <INTEL-SSDSC2BA800G3-5DV12270-745.21GB>
              /pci@0,0/pci15ad,7a0@16/pci15d9,86d@0/disk@4,0
           6. c61t5d0 <INTEL-SSDSC2BA800G3-5DV12270-745.21GB>
              /pci@0,0/pci15ad,7a0@16/pci15d9,86d@0/disk@5,0
           7. c63t1d0 <drive type unknown>
              /pci@0,0/pci15ad,7a0@16,1/pci8086,3703@0/blkdev@1,0
    
     
    #17
  18. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    2,098
    Likes Received:
    674
    Please keep us updated if you find a solution.

    At least with an P3600 that i have (pci-e version) I do not have problems.
    The <drive type unknown> in the format command itself is uncritical as long as there are no stability problems or other errors.
     
    #18
Similar Threads: NVME Napp-It
Forum Title Date
Solaris, Nexenta, OpenIndiana, and napp-it Samsung 960 NVME on napp-it for cache? Apr 20, 2017
Solaris, Nexenta, OpenIndiana, and napp-it Napp-it SuperStorage Server 6028R-E1CR12L NVMe SSD issue Jul 15, 2016
Solaris, Nexenta, OpenIndiana, and napp-it AMD Epyc NVME support Apr 15, 2019
Solaris, Nexenta, OpenIndiana, and napp-it Trouble installing OmniOS on X11SSH-CTF with NVME M.2 PCIe Drive Aug 27, 2018
Solaris, Nexenta, OpenIndiana, and napp-it Partition same NVME for both SLOG and L2ARC May 6, 2018

Share This Page