Mellanox Connectx-3 not being detected

Discussion in 'Networking' started by Kei-0070, May 3, 2019.

  1. Kei-0070

    Kei-0070 Member

    Joined:
    May 3, 2019
    Messages:
    32
    Likes Received:
    12
    Recently bought a Dell branded mellanox connectx-3 (CX322A PN: 19RNV) off ebay to put into my server but so far with my testing, I've not been able to get it to show up.

    Since my server is presently in bits awaiting a rebuild, I tried it in my old X79 based machine. I took it's intel X710 card out and put the mellanox in it's place. It all boots up ok, although I noticed there was no option rom loaded. In windows 10, there is no sign of it or any unknown devices. I tried the mellanox tools and mst status reported there were no mst devices. I tried using centOS 7 and lspci didn't show any mellanox devices either.

    Since I couldn't get anywhere with the X79 system, I moved it into my amd x399 system where again I took my other intel X710 card out and put the mellanox in. This time, it failed to post freezing at debug code 94, pci bus enumeration. I took the card back out and covered the SM bus pins and popped it back in and it still freezes at code 94.

    I could try it in the old server board (amd 790FX) or the new one once I get a cooler bracket (intel Z390). Beyond that, I could take it to work and try it in the hp z800 workstation I have. I've done some reading and could try forcing the pcie slots to gen 2 although I don't see why gen 3 should be an issue. I also need to see about enabling "Above 4G decoding" in the bios if the option exists.

    Beyond that, is there anything I have missed or should try?
     
    #1
    Last edited: May 3, 2019
  2. Terry Wallace

    Terry Wallace PsyOps SysOp

    Joined:
    Aug 13, 2018
    Messages:
    84
    Likes Received:
    35
    Honestly at this point, having used a a ton of Connect-X3 hardware, I'd return to ebay seller. I think you've got a bad card. If you can't even detect it with lspci something is seriously wrong with it.
     
    #2
    arglebargle and Kei-0070 like this.
  3. Kei-0070

    Kei-0070 Member

    Joined:
    May 3, 2019
    Messages:
    32
    Likes Received:
    12
    That was the vibe I was getting from the symptoms.

    I've tried a few more things are still no luck.

    PCIe gen 2 forced on the X79 system did nothing and there is no above 4G decoding option.

    IOMMU & above 4G decoding did nothing on the X399 system, however moving the card to a different slot got it to boot. Once into windows, it was the same result as my other system, no sign of the card.
     
    #3
  4. arglebargle

    arglebargle H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈

    Joined:
    Jul 15, 2018
    Messages:
    634
    Likes Received:
    207
    Same experience here. I've used around a dozen CX2's and 3's in machines and I've encountered a couple that could only be detected in flash recovery mode. They'd flash fine but I never once got them to detect after booting an OS without the jumper closed. I returned both for a refund, no issues with that.
     
    #4
  5. Kei-0070

    Kei-0070 Member

    Joined:
    May 3, 2019
    Messages:
    32
    Likes Received:
    12
    Bridged the flash present pins and stuffed it in my Z800 in work. It doesn't even show up as a pci memory controller and MST still shows "No MST devices found" which suggests it is a very dead card. Seller has agreed to refund it.
     
    #5
Similar Threads: Mellanox Connectx-3
Forum Title Date
Networking Mellanox ConnectX-3 ejectable Thursday at 10:15 PM
Networking Mellanox ConnectX-3 Low File Transfer Speed Sep 9, 2019
Networking Mellanox ConnectX-3 VPI Twinax compatibility with Cisco Nexus 3064 Aug 14, 2019
Networking Mellanox ConnectX-3 EN "Wrong VPD format" issue Jun 20, 2019
Networking Mellanox ConnectX-3 EN duplicate permanent MAC address issue Jun 12, 2019

Share This Page