Mellanox 40gb only connecting at 10gb

[djdrock]

This is my first time getting into Mellanox, so as a newbie I expected a few issues along the way. The good news is my controllers are working well at 10gbps, but my Windows machines are only connecting at 10gbps (full duplex) in ethernet mode directly connected, no switch.

I am using the Mellanox MCX353A-FCBT ConnectX-3 cards in both systems. Drivers have been updated on both cards to 5.50.14688.0. I am using the MC2206130-003 cable from fs.com. It states that it "supports 10G/40Gigabit Ethernet." It can be found here: Mellanox MC2206130-003 40G QSFP+ DAC Cable

New Hardware/systems:

System 1: Threadripper 1950x, 64gb RAM, x399 mobo, Samsung 970 Evo Plus M.2, Windows 10 Education (basically Enterprise). Mellanox is installed in a PCIex 16 slot.

System 2: Xeon W2102m 16gb RAM, Supermicro X11SRA-F mobo, Samsung 970 Evo Plus M.2. Windows Server 2019. Mellanox is installed in a PCIe x16 slot.

It is quite possible I am missing something obvious, but my normal internet searches have yielded nothing. I read there are some tuning tools for Linux, but can not find anything for Windows based systems.

Using the MST tools, I am able to see that I have a Device ID of 4099 (ConnectX-3) and a PSID of MT_1100120019.

It appears that my current firmware version (2.42.5000) is the latest offered.

Thanks in advance!

 

[djdrock]

I am not sure if this is pertinent, but I ran a command via the mlxtool and it is reporting the following:
Bus:65 Device:0 Function:0

Operational Info for Port# 1:
-----------------------------
State : Active
Physical State : ETH_AN_FSM_AN_GOOD
Speed : 10 Gbps
FEC : No FEC
Loopback Mode : No Loopback
Supported speeds : 0x1000 ( CR-10G )

Bus:65 Device:0 Function:0
TroubleShoot Info Port# 1
--------------------------
Status Opcode : 0
Group Opcode : 0
Message : No issues detected
Bus:65 Device:0 Function:0
Operational Info for Port# 2:
-----------------------------
State : Disabled
Physical State : ETH_AN_FSM_ENABLE
Speed : Unknown
FEC : No FEC
Loopback Mode : No Loopback
Supported speeds : 0x1000 ( CR-10G )

Bus:65 Device:0 Function:0

TroubleShoot Info Port# 2
--------------------------
Status Opcode : 1
Group Opcode : 0
Message : The port is closed by command . Please check that the interface is enabled.

 

[djdrock]

Even though nobody helped, I was able to solve my issue. As it turns out, it was a cable issue. While the MC2206130-003 says it supports 40gb ethernet, it evidently does not, at least with my hardware. Buying the MC2210130-001 cable did the trick. I am now able to connect at 40g. I hope this helps someone in the future.