Mellanox ConnectX3Pro Questions/Issues

[davidi]

Hello everyone I purchased a Mellanox ConnectX3Pro off eBay and I have a few questions that I am hoping someone can help me with. I have been searching but I can't seem to find a clear answer. I installed this card on a Dell r720xd server that is running Proxmox 7.4. Proxmox detected the card just fine. No issue there.

1. Is this card real or a fake? From what I can tell, it's genuine because I was able to use the mxlp program to update the firmware and it updated with zero issues. Please see screenshots below. If it's a fake, I still can return it, so I definitely would like to.

2. I am not getting 10GB out of this card. If I run iperf3, it appears nothing at all gets transmitted. It starts at 2.65GB and then fails. But if I migrate a container from one host to another, that works. So, there is an active link on the card and the other nodes do see it, but I doubt this card is transmitting 10GB even tho the link says it's connected at 10GB Full. My other 2 Proxmox nodes can transmit 10GB in iperf3 when I run it between them, but iperf3 fails when I use this particular node.

3. How do you turn auto negotiation on for this card? I ran mstconfig command, but I do not see an option to turn this on. Is this card not capable of auto negotiation?

Any help would be appreciated. This is my first time using a Mellanox card so please forgive my ignorance on its usage or how to configure it

Here are some screenshots:

 

[davidi]

Anybody?

 

[TonyA]

Anybody?

Have you tried....

sudo ethtool -s enp66s0 autoneg on

 

[davidi]

Have you tried....

sudo ethtool -s enp66s0 autoneg on

Hi there, yes, it was one of the first things I tried and that did not seem to have any effect.

 

[prdtabim]

Looking at the query:
- The card have 2 ports and is a Connectx-3 NON pro. Doesn't have any vpi references in settings and GUIDs are invalid.
- even being a connectx-3 non pro, the configs looks missing some keys from the last versions. I sugest you to reset the configs using mlxconfig

mlxconfig – Changing Device Configuration Tool

docs.nvidia.com

 

[davidi]

Looking at the query:
- The card have 2 ports and is a Connectx-3 NON pro. Doesn't have any vpi references in settings and GUIDs are invalid.
- even being a connectx-3 non pro, the configs looks missing some keys from the last versions. I sugest you to reset the configs using mlxconfig

mlxconfig – Changing Device Configuration Tool

docs.nvidia.com

Hmmm interesting. Based on your reply, I am going to pull the card from this server tomorrow, take the heatsink off and see what it is. Then I am going to put it into a non-production server and use mlxconfig like you suggested. I honestly should have done that from the beginning, but this card went into a production server to replace a 10GB Broadcom NIC that quit working.

I've never used mxlconfig before, will that fix the GUIDs and restore all the features of the card with a full reset?

 

[prdtabim]

Hmmm interesting. Based on your reply, I am going to pull the card from this server tomorrow, take the heatsink off and see what it is. Then I am going to put it into a non-production server and use mlxconfig like you suggested. I honestly should have done that from the beginning, but this card went into a production server to replace a 10GB Broadcom NIC that quit working.

I've never used mxlconfig before, will that fix the GUIDs and restore all the features of the card with a full reset?

I used mlxconfig to reset the configs after an upgrade in an very old card. Half of the settings are missing.

 

[davidi]

I used mlxconfig to reset the configs after an upgrade in an very old card. Half of the settings are missing.

I finally had a chance to pull the card and remove the heatsink. It says it's a ConnectX3Pro. I haven't had a chance to mess with mlxconfig. But, if it doesn't fix the GUID's or the restore the features of the card, what is the point? I'm thinking I may just return this card.