Mellanox ConnectX-4 Re-flash to 100GbE?

MiniKnight

Well-Known Member
Mar 30, 2012
3,034
940
113
NYC
Does anyone here know if it's possible to flash a Mellanox ConnectX-4 40/56gbe or 50gbe card to 100GbE firmware?

I was looking at the cards, and I don't see anything outside of the same silicon. It makes me think that what Mellanox is really doing is just using the same card, changing firmware, and slapping the sticker that goes with the firmware.

Has anyone done this? Is there a guide?
 

i386

Well-Known Member
Mar 18, 2016
2,997
960
113
33
Germany
Hmm, keep in mind that with cx4 mellanox started to use pcie x16 for 100+ Gbit/s networking.
If your card has just x8 you might see 100gbit after Cross flashing but have only ~80gbit.
 

Rand__

Well-Known Member
Mar 6, 2014
6,087
1,460
113
I have thought the same, if you have a QSFP+ card (which is x16 usually) then it might work. Have not found a cheap enough 50 GB card to test though.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,342
5,471
113
@rand_ and @MiniKnight

If you guys have decent instructions, I might be able to find a CX414A. I think it is PCIe x8 though.
 

Rand__

Well-Known Member
Mar 6, 2014
6,087
1,460
113
The process should be identical to the cx3 crossflash with change psid flag.

Not sure if one would need a modified binary or not for cx4+
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,342
5,471
113
I am not sure I ever did CX3. Happen to have a link?
 

_alex

Active Member
Jan 28, 2016
874
96
28
Bavaria / Germany
if it doesn't work due to flint complaining - it is quite easy to get the mstflint sources, bypass the ckeck that fails, compile and flash whatever you want (at your own risk!)
did this to get fcct/pro fw on the el-cheapo hp cx-3 to see how much vxlan-offload is really worth...
 
  • Like
Reactions: AveryFreeman

Rand__

Well-Known Member
Mar 6, 2014
6,087
1,460
113
MCX414A-BCAT
ConnectX-4 EN network interface card, 40/56GbE dual-port QSFP28, PCIe3.0 x8, tall bracket, ROHS R6

And the 414 is indeed a x8 card so probably would not work fully at 100GBs; but it would be very interesting if you can flash a VPI firmware on it;)
 

_alex

Active Member
Jan 28, 2016
874
96
28
Bavaria / Germany
for sure scary to push the button the first time, and might go south.
but the only way to know what works and what not is to try, and recovery - at least for cx-3 - is not totally impossible.
Not sure if flashing the cards on physical x8 to EDR will ever work, but quite sure x-flashing the cards on x16 between EN / VPI will be an option.
 

Rand__

Well-Known Member
Mar 6, 2014
6,087
1,460
113
The 413 is a "ConnectX-4 Lx EN network interface card, 40GbE single-port QSFP28, PCIe3.0 x8" - so very nice to hear that, thanks:)
 

consp

New Member
Mar 23, 2021
1
0
1
It also works for VPI cards:

I can confirm that flashing a MCX455A-FCA to ECA works so maybe other models will as well. My (broken channel 2 ...) 100g cable shows up as 100g now. Since 25G worked fine I wondered if it was possible to use all four channels on 25G. The test mode supported it at least despite one strand being broken, so now does the firmware report it as well despite my cable not working properly. Therefore I cannot test the throughput properly until I get a new cable though.

Code:
Before:
Part Number:           MCX455A-FCA_Ax
Description:           ConnectX-4 VPI adapter card; FDR IB (56Gb/s) and 40GbE; single-port QSFP28; PCIe3.0 x16; ROHS R6
//
Supported Cable Speed           : 0x3801f1d3 (56G,40G,25G,10G,1G)

After:
Part Number:           MCX455A-ECA_Ax
Description:           ConnectX-4 VPI adapter card; EDR IB (100Gb/s) and 100GbE; single-port QSFP28; PCIe3.0 x16; ROHS R6
//
Supported Cable Speed           : 0xf8f1f0d3 (100G,50G,40G,25G,10G,1G)
edit: 100G works fine.
 
Last edited: