Mellanox ConnectX-2 EN and Windows 10?

Discussion in 'Networking' started by HWGeek, Sep 30, 2015.

  1. TeleFragger

    TeleFragger Member

    Joined:
    Oct 26, 2016
    Messages:
    139
    Likes Received:
    17
    Yes.... will reply soon. Let me get to work first.....
     
    #41
  2. TeleFragger

    TeleFragger Member

    Joined:
    Oct 26, 2016
    Messages:
    139
    Likes Received:
    17

    so do this... I have a ConnectX-1 card but hopefully it will help as drivers are drivers.
    go to mellanox and DL the 5.50 drivers from them.
    go into device manager on win10 and go under system devices and look for the mellanox card.
    go in and click drivers
    update, let me specify, have drivers, browse...
    point to c:\program files\mellanox\hw and use that driver for it
    once done, you should be able to go back in and get the mellanox add on tabs...
    that is what I had to do and I had to take it a step further.
    for me I had to choose ETH instead of IB ( was checked to use default).
    Then I had to go up under network adapters and do the same for driver but point to mellanox\eth for its drivers.

    so I think Microsoft has default drivers that were provided by mellanox but it doesn't give you the added features and doing this should.

    hope that helps...
     
    #42
  3. shrike0064

    shrike0064 New Member

    Joined:
    Jan 14, 2019
    Messages:
    12
    Likes Received:
    3
    Sir,
    Thank you for your reply as this is driving me up the wall.
    Unfortunately nothing worked. I downloaded the 5.50 driver.
    I found the ETH that you mentioned, but it is already selected and greyed out so I cannot change it to auto.
    When I redirect the driver it comes up with best driver currently installed.
    ARRRRG!
     
    #43
    Last edited: Jan 18, 2019
  4. TeleFragger

    TeleFragger Member

    Joined:
    Oct 26, 2016
    Messages:
    139
    Likes Received:
    17
    no I had that too... you have to do it a different way...

    you have to start with the item under System Devices....


    upload_2019-1-18_21-0-30.png

    upload_2019-1-18_21-0-47.png


    upload_2019-1-18_21-1-2.png

    then click browse....
    you must use the one not in ETH... you must use this one..
    upload_2019-1-18_21-1-39.png

    for me I have this one...
    upload_2019-1-18_21-2-49.png



    and I use it... it looks like the others that were in the list....

    then I close out all the windows and go back in … now I have the new tab...
    upload_2019-1-18_21-3-19.png



    then I choose ETH...

    then I repeat same steps for items under network cards but then I choose ETH folder....

    then I get all the new tabs... only had a few now I got 2 rows...
    upload_2019-1-18_21-4-6.png
     
    #44
    nikalai likes this.
  5. shrike0064

    shrike0064 New Member

    Joined:
    Jan 14, 2019
    Messages:
    12
    Likes Received:
    3
    Yes sir. Did that.
    I have those tabs.
    Those options remain greyed out.
    Has the IP: 169.254.44.65/16
    However my network is 192.168.1. _ _ _
    That's what I don't understand.
    If I set a static I get: 169.254.44.65/16, 192.168.1.55/24
    But still will not connect.
    Gateway: 192.168.1.1
    DNS: 192.168.1.1
     
    #45
    Last edited: Jan 18, 2019
    TeleFragger likes this.
  6. TeleFragger

    TeleFragger Member

    Joined:
    Oct 26, 2016
    Messages:
    139
    Likes Received:
    17
    Weird.. post screenshot?
     
    #46
    shrike0064 likes this.
  7. shrike0064

    shrike0064 New Member

    Joined:
    Jan 14, 2019
    Messages:
    12
    Likes Received:
    3
    Yes sir. Did all of that.
    Greyed out.
    It is assigned IP
     
    #47
  8. shrike0064

    shrike0064 New Member

    Joined:
    Jan 14, 2019
    Messages:
    12
    Likes Received:
    3
    Okay, will have to install some software. Will do in the AM.
     
    #48
  9. TeleFragger

    TeleFragger Member

    Joined:
    Oct 26, 2016
    Messages:
    139
    Likes Received:
    17
    Windows snipping tool? Lol... then can paste straight into this box..
     
    #49
  10. shrike0064

    shrike0064 New Member

    Joined:
    Jan 14, 2019
    Messages:
    12
    Likes Received:
    3
    Apologies about the delay as I had to go into work. Over time even though not getting paid due to shutdown.
    Thanks for the snippet tool. Forgot about that. What other screen shots? Capture1.JPG Capture2.JPG
     
    #50
  11. TeleFragger

    TeleFragger Member

    Joined:
    Oct 26, 2016
    Messages:
    139
    Likes Received:
    17
    Thats weird... firmware update? I know i had a wacked firmware and only saw 1 port... flashed to newer then saw other...
     
    #51
  12. shrike0064

    shrike0064 New Member

    Joined:
    Jan 14, 2019
    Messages:
    12
    Likes Received:
    3
    Tried. Kept saying failed on all 4 cards.
    Now wondering if it's that my PCI-e3 x4 slot somehow isn't compatible. But then why is the card seen and drivers loaded?
    Computers, such fun.

    I found a couple 10gb connectx-3 single ports on eBay for $28 each.
    Bought 2. Hoping these will be supported.
    Found somewhere that the connectx-2 are no longer supported on Windows 10 Client. They may or may not work.
     
    #52
    TeleFragger likes this.
  13. TeleFragger

    TeleFragger Member

    Joined:
    Oct 26, 2016
    Messages:
    139
    Likes Received:
    17

    Well ive got connectx-1 cx4 style ports working in esxi..freenas 11 and win10
     
    #53
    shrike0064 likes this.
  14. shrike0064

    shrike0064 New Member

    Joined:
    Jan 14, 2019
    Messages:
    12
    Likes Received:
    3
    Well, crap.
    I'm going to try these and see what happens. I know all 16 4k IP cameras will be fine on the gigabit LAN but it's the principle of the thing.
    I bought a mini ATX with two slots just for the fiber network I have.
     
    #54
    TeleFragger likes this.
  15. Maxim

    Maxim New Member

    Joined:
    May 17, 2019
    Messages:
    1
    Likes Received:
    0
    Hi guys!

    I will tell my small story. I have just registered account here to help community.

    So I have 2 end points with Mellanox Connect X2 cards.
    One is Fedora 29 and one Windows Server 2008 R2 (I guess everybody knows that it is same thing as Windows 7 x64 but Server).
    I have downloaded latest drivers available for that platform for X2.
    It is older version that drivers for Windows 10 (which itself is terrible piece of shit).

    When I tested link with iperf3 I have experienced speed issues.
    Trandfer speed was limited as described in this thread.

    So my network configuration is the following: Mellanox X2 <-> MikroTik 10 Gbps switch (level 3) <-> Mellanox X2
    I have MikroTik and Mellanox cables but all that does not matter (both works fine).

    In worst case iperf 3 said me 3-4 Gbps in best case up to 7.2 Gbps (depending CPU load - from 100% all cores to idle).
    But even if CPU is idle it is still not expected 10Gbps. So I worried about that and came to this thread (and multiple others).

    I started experiments and finally got full speed. So what was needed to achieve that in my case?

    1) On Windows machine - in Mellanox settings set 9000 in Jumbo Packet field.
    2) Send buffers make like Receive buffers - 4096 (it is maximum).
    3) Interrupt moderation set to Disabled.

    4) On Linux machine set MTU to 9000. There is a lot of docs about that.

    5) And now final and crucial thing - check your router if it is not direct connection.
    In my case it was needed to change MTU/L2MTU from 1500 to 9000 for each SFP+ interface in Mikrotik Router OS settings.
    You will need to find appropriate settings in your router.

    After that I got 9.3-9.5 Gbps in iperf3!

    Have a nice day guys!
     
    #55
  16. tommybackeast

    tommybackeast Active Member

    Joined:
    Jun 10, 2018
    Messages:
    148
    Likes Received:
    36
    Re-visiting a 3yr post - seem Mellanox changed their pages and many old URLs no longer work, any chance you have the current URL for ConnectX-2 Win7 drivers? thanks
     
    #56
  17. saivert

    saivert Member

    Joined:
    Nov 2, 2015
    Messages:
    118
    Likes Received:
    12
    #57
  18. tommybackeast

    tommybackeast Active Member

    Joined:
    Jun 10, 2018
    Messages:
    148
    Likes Received:
    36
    I learned afterwards that while ConnectX-2 was not mentioned, that driver still works for it.

    Simply was confusing language from mellanox
     
    #58
    nikalai likes this.
  19. Parham

    Parham New Member

    Joined:
    Jan 8, 2020
    Messages:
    1
    Likes Received:
    2
    ---------------------------------------------------------------------------------------------
    Hey,
    I just wanted to thank you for your great post and solution.
    I had the same issue with MNPA19-XTR connected to Intel X520-2 (2 port SFP+).

    I had the MNPA19-XTR installed on my desktop connected to one of the ports of the Intel X520-2 on a server machine.

    After a couple of jperf tests I was not getting any speed greater than 7Gbps. Just to confirm your solution I went to my MB BIOS (ASUS MAXIMUS_XI_HERO) and changed the speed of PCIE from X2 to X4 and voila i tested and got the full 10Gbps speeds.

    For people with the same issue make sure you follow the steps below:
    1-If you are using any PCIE anything other than the first PCIE on you MB make sure you have the full speed enabled.
    In my case my GPU was located in PCIE16_1 and the NIC was in PCIE16_3.

    upload_2020-1-9_10-46-59.png
    Here is how to enable it in you ASUS bios:

    20200109_083102 (Large).jpg


    2-I installed latest driver 5.50.14688 and it gave me full speed:
    upload_2020-1-9_10-49-10.png

    3-Under performance make sure to select single port traffic or single stream traffic then hit Run Tuning and restart computer.

    4-Under advanced:
    Interrupt Moderation---->Disable
    Jumbo Packet---->9014 (this could be different but i matched it with the highest value on my server Intel X520-2 NIC)

    5-Receive Buffers and Send Buffer---->4096 or to its maximum value.

    7-Make sure to assign static IP on each NIC.

    8-Add the IP address and name of the target machine to the Host file on windows. Let me know if you have questions on how to do this (this will force windows to use the 10gb and not the 1gb LAN speed).

    6-Use iperf or jperf or RAM disk to test the speeds. Below are my results from Jperf:

    bin/iperf.exe -s -P 0 -i 1 -p 5001 -f g
    ------------------------------------------------------------
    Server listening on TCP port 5001
    TCP window size: 0.00 GByte (default)
    ------------------------------------------------------------
    OpenSCManager failed - Access is denied. (0x5)
    [392] local 192.168.50.3 port 5001 connected with 192.168.50.1 port 53662
    [ ID] Interval Transfer Bandwidth
    [392] 0.0- 1.0 sec 1.07 GBytes 9.21 Gbits/sec
    [392] 1.0- 2.0 sec 1.13 GBytes 9.71 Gbits/sec
    [392] 2.0- 3.0 sec 1.15 GBytes 9.86 Gbits/sec
    [392] 3.0- 4.0 sec 1.15 GBytes 9.85 Gbits/sec
    [392] 4.0- 5.0 sec 1.15 GBytes 9.86 Gbits/sec
    [392] 5.0- 6.0 sec 1.15 GBytes 9.87 Gbits/sec
    [392] 6.0- 7.0 sec 1.15 GBytes 9.88 Gbits/sec
    [392] 7.0- 8.0 sec 1.15 GBytes 9.89 Gbits/sec
    [392] 8.0- 9.0 sec 1.15 GBytes 9.86 Gbits/sec
    [392] 9.0-10.0 sec 1.13 GBytes 9.72 Gbits/sec
    [392] 10.0-11.0 sec 1.14 GBytes 9.75 Gbits/sec
    [392] 11.0-12.0 sec 1.15 GBytes 9.86 Gbits/sec
    [392] 12.0-13.0 sec 1.14 GBytes 9.80 Gbits/sec
    [392] 13.0-14.0 sec 1.14 GBytes 9.76 Gbits/sec
    [392] 14.0-15.0 sec 1.14 GBytes 9.80 Gbits/sec
    [392] 15.0-16.0 sec 1.14 GBytes 9.83 Gbits/sec
    [392] 16.0-17.0 sec 1.15 GBytes 9.89 Gbits/sec
    [392] 17.0-18.0 sec 1.15 GBytes 9.90 Gbits/sec
    [392] 18.0-19.0 sec 1.15 GBytes 9.90 Gbits/sec
    [392] 19.0-20.0 sec 1.14 GBytes 9.75 Gbits/sec
    [ ID] Interval Transfer Bandwidth
    [392] 0.0-20.0 sec 22.8 GBytes 9.80 Gbits/sec

    All the credit goes to mfgmfg for his/her great post and solution!!!!!
    Thanks again for this.
     
    #59
    tommybackeast and nikalai like this.
  20. klui

    klui Member

    Joined:
    Feb 3, 2019
    Messages:
    45
    Likes Received:
    17
    You can get the info you need by going to Mellanox's site and select Support & Education > Ethernet Drivers (or Infiniband/VPI). Then in the navigation pane on the left, click Windows Drivers. In the header portion, click the matrix of "WinOF/WinOF-2 drivers versions vs. supported hardware and firmware." Scroll all the way down until you see CX2 references for "supported" versions. Most likely newer versions will support the part under W10. Just choose WinOF.
     
    #60
    tommybackeast likes this.
Similar Threads: Mellanox ConnectX-2
Forum Title Date
Networking Mellanox Connectx-2 Networkdrive Problems Nov 17, 2019
Networking Mellanox ConnectX-2 VPI Aug 13, 2019
Networking Help Flashing IBM Mellanox ConnectX-2 Jun 2, 2019
Networking Mellanox ConnectX-2 Still Good? May 13, 2019
Networking 10G LACP fails, Ubuntu 18.04, Mellanox ConnectX-2, Quanta LB8/LB4m Mar 14, 2019

Share This Page