Mellanox ConnectX-2 and ESXi 6.5

Discussion in 'VMware, VirtualBox, Citrix' started by freebsdrules, Oct 13, 2017.

  1. freebsdrules

    freebsdrules Member

    Joined:
    Aug 16, 2017
    Messages:
    174
    Likes Received:
    16
    Hi all-

    I'm new to ESXi and I've got a Mellanox ConnectX2 dual port 10gb card I'm trying to get work with version 6.5. I don't see it under the list of available Network Physical NICs and when I go to manage->hardware, I see it but it's greyed out. I tried manually installing the 2.4 driver but still no luck. Is there a driver that works/is supported for this card in 6.5?

    Thanks!
     
    #1
  2. marcoi

    marcoi Well-Known Member

    Joined:
    Apr 6, 2013
    Messages:
    1,321
    Likes Received:
    204
    My card worked out of the box (but it a different card from yours) , it just showed up under physical nics without any additional drivers on esxi 6.5

    upload_2017-10-13_11-1-1.png

    Have you tried the card in a different server/host to see if it working? Or try with a different OS? IE rule out if the card is bad.
     
    #2
    freebsdrules likes this.
  3. freebsdrules

    freebsdrules Member

    Joined:
    Aug 16, 2017
    Messages:
    174
    Likes Received:
    16
    Yes, I put it in my Windows 10 box and it showed up right alongside the other card I have in there. Here's a screenshot of it in the hardware management of esxi but greyed out.

    [​IMG]
     
    #3
  4. marcoi

    marcoi Well-Known Member

    Joined:
    Apr 6, 2013
    Messages:
    1,321
    Likes Received:
    204
    This is what I see under physical nics, it using driver nmlx4_en.
    upload_2017-10-13_11-21-3.png

    Under pci devices i see it like this
    upload_2017-10-13_11-21-59.png

    Can you try a new install of ESXI 6.5 latest release? you can always install to new usb drive, etc to preserve you first install if you need it.
     
    #4
  5. freebsdrules

    freebsdrules Member

    Joined:
    Aug 16, 2017
    Messages:
    174
    Likes Received:
    16
    Hmm, maybe that's what I'll do. I think I'm on the latest release but will double check. Thanks again!
     
    #5
  6. freebsdrules

    freebsdrules Member

    Joined:
    Aug 16, 2017
    Messages:
    174
    Likes Received:
    16
    Just downloaded the newest image and installed. Still same deal. Anybody have any ideas?
     
    #6
  7. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,426
    Likes Received:
    486
    #7
  8. freebsdrules

    freebsdrules Member

    Joined:
    Aug 16, 2017
    Messages:
    174
    Likes Received:
    16
    Thank you, will have to give it a try!
     
    #8
  9. freebsdrules

    freebsdrules Member

    Joined:
    Aug 16, 2017
    Messages:
    174
    Likes Received:
    16
    #9
  10. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,426
    Likes Received:
    486
    What's the exact model and firmware you have?
     
    #10
  11. freebsdrules

    freebsdrules Member

    Joined:
    Aug 16, 2017
    Messages:
    174
    Likes Received:
    16
    Wow, that was a rabbit hole that I didn't think I'd make it back from. Couldn't figure out firmware from esxi, wasn't even acknowledging the card. Thought it might be a bad card but decided to put it back in my windows box and found out it was on an old firmware version (2.7). So, I burned 2.9 and put it back in the esxi server. Boom, recognized on a clean install without doing anything!

    Thanks again for all the help folks.
     
    #11
    Rain likes this.
  12. marcoi

    marcoi Well-Known Member

    Joined:
    Apr 6, 2013
    Messages:
    1,321
    Likes Received:
    204
    glad you got it resolved!
     
    #12
  13. marcoi

    marcoi Well-Known Member

    Joined:
    Apr 6, 2013
    Messages:
    1,321
    Likes Received:
    204
    Hi David,

    Can you elaborate on what you cant install? Do you mean the card physically into your server? Or do you mean once installed it isnt showing up in OS?

    If OS issue, did you check the cards firmware and try to update? If it isnt even recognized by system, then the card might be damaged.

    hope that helps.
     
    #13
  14. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,426
    Likes Received:
    486
    is it configured to ethernet or ib?
    It might be on IB after firmware upgrade and if you want to use it as ETH you might need to set up properly

    # mlxconfig -d /dev/mst/mt4099_pciconf0 s LINK_TYPE_P1=<1|2|3> for P1, port type: IB = 1, ETH = 2, Auto =3

    # mlxconfig -d /dev/mst/mt4099_pciconf0 s LINK_TYPE_P2=<1|2|3> for P2, port type: IB = 1, ETH = 2, Auto =3

    replace model accordingly
     
    #14
  15. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,426
    Likes Received:
    486
    You know its very difficult to help you if you don't provide feedback on the previous suggestions...

    Can you provide the output of mlxconfig?
    Have you tried forcing the port to up on the switch? What switch are you using?
     
    #15
  16. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,426
    Likes Received:
    486
    So the same card wofks fine in Windows but not Esx?

    And both times the ports are set to eth?
     
    #16
Similar Threads: Mellanox ConnectX-2
Forum Title Date
VMware, VirtualBox, Citrix Mellanox ConnectX-2 and ESXi 6.0 - Barely Working - Terrible Performance Nov 7, 2016
VMware, VirtualBox, Citrix ESXi + XPEnology + Mellanox performance? Dec 11, 2018
VMware, VirtualBox, Citrix So I have this Mellanox ConnectX-3 and ESXi 6.7U1 and I want to tune .. Dec 6, 2018
VMware, VirtualBox, Citrix Mellanox unhappy w/ vtD passthru in vSphere Nov 24, 2016
VMware, VirtualBox, Citrix Proxmox issues with SR-IOV and Mellanox Connectx-3 Dec 20, 2015

Share This Page