Mellanox ConnectX-2 Firmware

i386

Well-Known Member
Mar 18, 2016
3,241
1,049
113
33
Germany
To enable rdma you must run one of the mellanox cmdlets to change/enable roce v1 (newer winof versions have the default setting roce = disabled and older roce = roce v2, which was introduced with connectx-3 pro).
Can't remeber the command but it's somewhere on their website.

Btw I never got rdma running on windows clients (7, 10) with connectx-2 or connectx-3. Even following some microsoft tutorials which mention the connectx-3.
 

Incroyable HULK

New Member
Mar 31, 2012
9
0
1
Hello, thanks for posting this. I just got two HP Cards delivered and I was hoping to flash the firmware to 2.10.720

When I try to do so I get the following warning. Not sure what I should do:
Code:
-E- PSID mismatch. The PSID on flash (HP_0F60000010) differs from the PSID in the given image (MT_0F60110010).
The PSID is the only difference I can see between the output you posted and my card.

Any help appreciated!
 

MrSquish

New Member
Mar 14, 2017
2
0
1
40
Looks like i've gone and bricked one of my cards now as well. :\

I tried running the flint-d mt26448_pciconf0 -i command to reverse the firmware, but it just tells me this:
-E- Can not open mst26448_pciconf0: No such device MFE_CR_ERROR.

Did you have to do any other steps to recover after you bricked yours?
 

i386

Well-Known Member
Mar 18, 2016
3,241
1,049
113
33
Germany
Sadly , no RDMA for Windows Clients OS.
Actually it works with windows 10, at least "Get-SmbMultichannelConnection" commandlet tells me that the card is rdma capable.
The problem is that the driver disables "Force ndk to work with global pause" as default. Enable it and rdma should wok.
 

f5inet

New Member
Mar 21, 2018
4
3
3
44
Hello, thanks for posting this. I just got two HP Cards delivered and I was hoping to flash the firmware to 2.10.720

When I try to do so I get the following warning. Not sure what I should do:
Code:
-E- PSID mismatch. The PSID on flash (HP_0F60000010) differs from the PSID in the given image (MT_0F60110010).
The PSID is the only difference I can see between the output you posted and my card.

Any help appreciated!
One Year later, but... Help incoming!

Ok, i have the same problem. A dirt cheap ConnectX-2 from ebay, that is retired from a working HPe computer. The PSID was the same as you (HP_0F60000010) and the firmware was 2.9.1200 that is not RDMA capable.

I downloaded the mlx firmware, and change the PSID (MT_0F60110010 => HP_0F60000010)on the INI to generate a working firmware for my cards, and then, burn the firmware. worked like a charm after two reboots (the OS has to recognize the 'new' card). Then, install the WINOF 5.40 drivers, and reports the ConnectX-2 cards with the new (2.10.720) firmware and RDMA Capable.

In my case, as I am working with a Hyper-V internal Switch in a hyperconverged cluster (S2D & HyperV hosts), I must enable 'networkdirect' in the HyperV virtual ethernet device in Device Manager. After that, the cmdlet Get-SmbServerNetworkInterface reports RDMA capable.

PS C:\Users\Administrator.<edited>> Get-SmbServerNetworkInterface

Scope Name Interface Index RSS Capable RDMA Capable Speed IpAddress
---------- --------------- ----------- ------------ ----- ---------
* 4 True True 10 Gbps fe80::6866:5820:52c8:8ef3
* 4 True True 10 Gbps 192.168.5.14

This is my package to upgrade the firmware to the cheap dirt HP Branded Mellanox ConnectX-2 (HP_0F60000010) on ebay.

MEGA

Execute the BAT on the directory (or edit it, to view the commands), and install WINOF 5.40 included in the package (or download from oficial mellanox website, o whatever you need for your OS)
 

nikey22

New Member
Feb 19, 2018
14
0
1
50
Is there any other benefit to going to the 2.10.720 firmware if you are not dealing with RDMA
I have only a peer-to-peer connection with workstation/NAS server.

Thanks.
 

i386

Well-Known Member
Mar 18, 2016
3,241
1,049
113
33
Germany
Like most firmware (or bios) updates for enterprise gear it contains bug fixes for edge cases. If you're happy with your current version and don't need/want rdma, no need to update to v2.10.720.
 

f5inet

New Member
Mar 21, 2018
4
3
3
44
Is there any other benefit to going to the 2.10.720 firmware if you are not dealing with RDMA
I have only a peer-to-peer connection with workstation/NAS server.

Thanks.
not really. RDMA main benefit is inter-server communication speed. RDMA is lower CPU use, also.

I am implementing RDMA because in a hyperconverged cluster, the Servers are being used as S2D AND Hyper-V Hosts, so , every CPU slice that I can get from S2D to Hyper-V VMs is really a plus for me.
 
Last edited:

donald24

New Member
Mar 4, 2018
4
0
1
47
Thank you for this thread, activating RDMA on my 2016-fileserver connecting to my win10-workstation made a serious difference in terms of speed and latency! the bottlenecks jumped straight to the source drives ;)
 

CoadyTech

New Member
Nov 12, 2017
6
1
3
30
United Kingdom
One Year later, but... Help incoming!

Ok, i have the same problem. A dirt cheap ConnectX-2 from ebay, that is retired from a working HPe computer. The PSID was the same as you (HP_0F60000010) and the firmware was 2.9.1200 that is not RDMA capable.

I downloaded the mlx firmware, and change the PSID (MT_0F60110010 => HP_0F60000010)on the INI to generate a working firmware for my cards, and then, burn the firmware. worked like a charm after two reboots (the OS has to recognize the 'new' card). Then, install the WINOF 5.40 drivers, and reports the ConnectX-2 cards with the new (2.10.720) firmware and RDMA Capable.

In my case, as I am working with a Hyper-V internal Switch in a hyperconverged cluster (S2D & HyperV hosts), I must enable 'networkdirect' in the HyperV virtual ethernet device in Device Manager. After that, the cmdlet Get-SmbServerNetworkInterface reports RDMA capable.

PS C:\Users\Administrator.<edited>> Get-SmbServerNetworkInterface

Scope Name Interface Index RSS Capable RDMA Capable Speed IpAddress
---------- --------------- ----------- ------------ ----- ---------
* 4 True True 10 Gbps fe80::6866:5820:52c8:8ef3
* 4 True True 10 Gbps 192.168.5.14

This is my package to upgrade the firmware to the cheap dirt HP Branded Mellanox ConnectX-2 (HP_0F60000010) on ebay.

MEGA

Execute the BAT on the directory (or edit it, to view the commands), and install WINOF 5.40 included in the package (or download from oficial mellanox website, o whatever you need for your OS)
I just tried this on a HP ConnectX-2 card. The update did initially go through fine however it's now causing ESXi 6.7 to crash when Windows initialises the NIC in PCIe passthrough mode. VMware does seem to be able to use the card though natively.

If I manage to fix it I'll report back. For the moment though it looks like the ConnectX-3 is a better bet.
 

DaveP

Member
Jul 30, 2016
50
2
8
41
One Year later, but... Help incoming!

Ok, i have the same problem. A dirt cheap ConnectX-2 from ebay, that is retired from a working HPe computer. The PSID was the same as you (HP_0F60000010) and the firmware was 2.9.1200 that is not RDMA capable.

I downloaded the mlx firmware, and change the PSID (MT_0F60110010 => HP_0F60000010)on the INI to generate a working firmware for my cards, and then, burn the firmware. worked like a charm after two reboots (the OS has to recognize the 'new' card). Then, install the WINOF 5.40 drivers, and reports the ConnectX-2 cards with the new (2.10.720) firmware and RDMA Capable.

In my case, as I am working with a Hyper-V internal Switch in a hyperconverged cluster (S2D & HyperV hosts), I must enable 'networkdirect' in the HyperV virtual ethernet device in Device Manager. After that, the cmdlet Get-SmbServerNetworkInterface reports RDMA capable.

PS C:\Users\Administrator.<edited>> Get-SmbServerNetworkInterface

Scope Name Interface Index RSS Capable RDMA Capable Speed IpAddress
---------- --------------- ----------- ------------ ----- ---------
* 4 True True 10 Gbps fe80::6866:5820:52c8:8ef3
* 4 True True 10 Gbps 192.168.5.14

This is my package to upgrade the firmware to the cheap dirt HP Branded Mellanox ConnectX-2 (HP_0F60000010) on ebay.

MEGA

Execute the BAT on the directory (or edit it, to view the commands), and install WINOF 5.40 included in the package (or download from oficial mellanox website, o whatever you need for your OS)
Getting file not available on Mega, any chance you can re-upload that file somewhere?
 

arglebargle

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈
Jul 15, 2018
657
241
43
Reuploaded: MEGA

By mistake, I deleted it from Mega. Restored.
I've mirrored a copy on Google drive too:
Connect-X2 Firmware - Google Drive

Thanks a ton for re-uploading, all of the other links to the v2.10.720 firmware that I could find are toast. I've upgraded an MHQH29C and a pair of MNPA19-XTRs without issue tonight.

To enable rdma you must run one of the mellanox cmdlets to change/enable roce v1 (newer winof versions have the default setting roce = disabled and older roce = roce v2, which was introduced with connectx-3 pro).
Can't remeber the command but it's somewhere on their website.

Btw I never got rdma running on windows clients (7, 10) with connectx-2 or connectx-3. Even following some microsoft tutorials which mention the connectx-3.
M$ gated RDMA support on client OSes behind a license upgrade to Windows 10 Pro for Workstations, I think it's an extra $125-130 upgrade over a standard Win10 Pro license. If I understand this correctly it unlocks the "optional feature" packages to enable RDMA but I'm not sure what support is actually in the OS beyond SMB Direct. It'd be grand if there were a way to use NFS over RDMA directly on Win10 but I don't see that happening.

Yet Another Network Blog: Infiniband/RDMA on Windows - now on Windows 10 too
 
Last edited:

arglebargle

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈
Jul 15, 2018
657
241
43
I've upgraded a couple of cards on Linux without issue. Just download and install the OFED package from Mellanox, it has everything you'll need (minus the new firmware.)