Lenovo Connectx-4 card flash to Mellanox

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
A

AK360

Member
Feb 25, 2021
25
26
13
Hello helpful people,

I have just received 3x Lenovo 01GR250 CX4 2x25Gb cards. Is there an idiots guide out there to flash these to MCX4121A-ACAT firmware which is what they are under the skin :) ??

I searched the forums but couldn't see anything for Connectx 4 other than someone trying to hack a 50Gb to 100Gb.

Any help gratefully received!

Alex
 
I

i386

Well-Known Member
Mar 18, 2016
4,411
1,638
113
35
Germany
I used the same procedure to flash a hpe card (825110-B21) to a vanilla mellanox nic (MCX455A-ECAT) as with the connect-x3 cards.
 
A

AK360

Member
Feb 25, 2021
25
26
13
i386 said:
I used the same procedure to flash a hpe card (825110-B21) to a vanilla mellanox nic (MCX455A-ECAT) as with the connect-x3 cards.
Click to expand...
Thank you It did work with a bit of modifying, for any one wondering:

********** Backup Settings **************
flint -d mt4117_pciconf0 query full > flint_query.txt

flint -d mt4117_pciconf0 hw query > flint_hwinfo.txt

flint -d mt4117_pciconf0 ri orig_firmware.bin

flint -d mt4117_pciconf0 dc orig_firmware.ini

mlxburn -d mt4117_pciconf0 -vpd > orig_vpd.txt

************** Write Mellanox Firmware ***************

flint -d mt4117_pciconf0 -ocr hw set Flash0.WriteProtected=Disabled

flint -nofs --use_image_ps --ignore_dev_data -allow_psid_change -d mt4117_pciconf0 -i fw-ConnectX4Lx-rel-14_30_1004-MCX4121A-ACA_Ax-UEFI-14.23.17-FlexBoot-3.6.301.bin burn

************* After power power cycle program GUID and MAC **************

flint -d mt4117_pciconf2 -guid XXXX -mac XXXX -ocr sg
 
  • Like
Reactions: lightsword, rukiri, mr.brainless and 3 others
K

Karpo

New Member
Mar 4, 2023
14
22
3
Hi,

bought Connectx-4 100G Dell model:
Device Type: ConnectX4
Part Number: 06W1HY_0JJN39_Ax
Description: Mellanox ConnectX-4 Single Port EDR PCIE Adapter LP
PSID: DEL2180110032
PCI Device Name: mt4115_pciconf0

As far as I understand this is same as CX455A-ECAT.

I changed it IB->Ethernet successfully and can get 94Gb/sec in iPerf. Enabled SR-IOV in firmware, updated EXSi drivers and enabled SR-IOV in drivers but card stays in 'Reboot to apply' in ESXi 7.

Guessing that Dell firmware could be the reason so tried above steps to flash it as Mellanox CX455A-ECAT.

Not sure if this response is expected when disabling write protection:

sudo flint -d /dev/mst/mt4115_pciconf0 -ocr hw set Flash0.WriteProtected=Disabled

-W- Firmware flash cache access is enabled. Running in this mode may cause the firmware to hang.
-E- Unknown option "set" for the "Hw" command. you can use query.

Checking write protection, it seems to be still on:

sudo flint -d 03:00.0 -ocr hw query
-W- Firmware flash cache access is enabled. Running in this mode may cause the firmware to hang.
HW Info:
HwDevId 521
HwRevId 0x0
Flash Info:
Type W25QxxBV
TotalSize 0x1000000
Banks 0x1
SectorSize 0x1000
WriteBlockSize 0x10
CmdSet 0x80
QuadEn 1
Flash0.WriteProtected Top,8-SubSectors
JEDEC_ID 0x1840ef

Flashing fails:

sudo flint -nofs --use_image_ps --ignore_dev_data -allow_psid_change -d /dev/mst/mt4115_pciconf0 -i ./fw-ConnectX4-rel-12_28_2006-MCX455A-FCA_Ax-UEFI-14.21.17-FlexBoot-3.6.102.bin burn

Current FW version on flash: 12.28.4512
New FW version: 12.28.2006

Note: The new FW version is older than the current FW version on flash.

Do you want to continue ? (y/n) [n] : y
You are about to replace current PSID on flash - "DEL2180110032" with a different PSID - "MT_2160111021".
Note: It is highly recommended not to change the PSID.

Do you want to continue ? (y/n) [n] : y

Burn process will not be failsafe. No checks will be performed.
ALL flash, including the device data sections will be overwritten.
If this process fails, computer may remain in an inoperable state.

Do you want to continue ? (y/n) [n] : y

-E- Burning FS3 image failed: Cannot burn device data sections, Flash is write protected.


Any ideas on how to disable write protection?
 
K

Karpo

New Member
Mar 4, 2023
14
22
3
Answering myself, got flashing working by installing mstflint (and getting right CX455-ECA firmware, rather than FCA above):

sudo apt install mstflint
sudo mstflint -d 03:00.0 -i fw-ConnectX4-rel-12_28_2006-MCX455A-ECA_Ax-UEFI-14.21.17-FlexBoot-3.6.102.bin -allow_psid_change burn

<reboot>

sudo flint -d /dev/mst/mt4115_pciconf0 query full
Image type: FS3
FW Version: 12.28.2006
FW Release Date: 15.9.2020
Part Number: MCX455A-ECA_Ax
Description: ConnectX-4 VPI adapter card; EDR IB (100Gb/s) and 100GbE; single-port QSFP28; PCIe3.0 x16; ROHS R6
Product Version: 12.28.2006
Rom Info: type=UEFI version=14.21.17 cpu=AMD64,AARCH64
type=PXE version=3.6.102 cpu=AMD64
Description: UID GuidsNumber
Base GUID: 248a070300a8---- 4
Base MAC: 248a07a8---- 4
Image VSD: N/A
Device VSD: N/A
PSID: MT_2180110032
Security Attributes: N/A
Default Update Method: fw_ctrl

However EXSi still keeps SR-IOV as 'reboot needed', maybe Dell T5810 does not support it in bios.
 
  • Like
Reactions: mr.brainless
K

Karpo

New Member
Mar 4, 2023
14
22
3
After flashing with mstflint using Mellanox firmware with below switches, it also changes Part Number:
sudo mstflint -d 03:00.0 -i fw-ConnectX4-rel-12_28_2006-MCX455A-ECA_Ax-UEFI-14.21.17-FlexBoot-3.6.102.bin -allow_psid_change burn
 
F

Freebsd1976

Active Member
Feb 23, 2018
404
73
28
mcdaguo said:
Although you can use flint -d mt4115_pciconf0 q full to see that the Part Number has been changed to MCX455-ECA, but you can still see the Part Number and serial number before the OEM network card in the network card information. How can I modify the PN and SN of the OEM network card information? View attachment 27809
View attachment 27808
Click to expand...
These call vpd, vpd is readonly. it can’t be modify.
and more important, shouldn’t modify them, Even if someone could( almost impossible).

unless someone want to make “genuine” nic , just like many intel x520 on second hand market.
 
Last edited:
M

mach3.2

Active Member
Feb 7, 2022
143
104
43
Karpo said:
Hi,

bought Connectx-4 100G Dell model:
Device Type: ConnectX4
Part Number: 06W1HY_0JJN39_Ax
Description: Mellanox ConnectX-4 Single Port EDR PCIE Adapter LP
PSID: DEL2180110032
PCI Device Name: mt4115_pciconf0

As far as I understand this is same as CX455A-ECAT.

I changed it IB->Ethernet successfully and can get 94Gb/sec in iPerf. Enabled SR-IOV in firmware, updated EXSi drivers and enabled SR-IOV in drivers but card stays in 'Reboot to apply' in ESXi 7.

Guessing that Dell firmware could be the reason so tried above steps to flash it as Mellanox CX455A-ECAT.

Not sure if this response is expected when disabling write protection:

sudo flint -d /dev/mst/mt4115_pciconf0 -ocr hw set Flash0.WriteProtected=Disabled

-W- Firmware flash cache access is enabled. Running in this mode may cause the firmware to hang.
-E- Unknown option "set" for the "Hw" command. you can use query.

Checking write protection, it seems to be still on:

sudo flint -d 03:00.0 -ocr hw query
-W- Firmware flash cache access is enabled. Running in this mode may cause the firmware to hang.
HW Info:
HwDevId 521
HwRevId 0x0
Flash Info:
Type W25QxxBV
TotalSize 0x1000000
Banks 0x1
SectorSize 0x1000
WriteBlockSize 0x10
CmdSet 0x80
QuadEn 1
Flash0.WriteProtected Top,8-SubSectors
JEDEC_ID 0x1840ef

Flashing fails:

sudo flint -nofs --use_image_ps --ignore_dev_data -allow_psid_change -d /dev/mst/mt4115_pciconf0 -i ./fw-ConnectX4-rel-12_28_2006-MCX455A-FCA_Ax-UEFI-14.21.17-FlexBoot-3.6.102.bin burn

Current FW version on flash: 12.28.4512
New FW version: 12.28.2006

Note: The new FW version is older than the current FW version on flash.

Do you want to continue ? (y/n) [n] : y
You are about to replace current PSID on flash - "DEL2180110032" with a different PSID - "MT_2160111021".
Note: It is highly recommended not to change the PSID.

Do you want to continue ? (y/n) [n] : y

Burn process will not be failsafe. No checks will be performed.
ALL flash, including the device data sections will be overwritten.
If this process fails, computer may remain in an inoperable state.

Do you want to continue ? (y/n) [n] : y

-E- Burning FS3 image failed: Cannot burn device data sections, Flash is write protected.


Any ideas on how to disable write protection?
Click to expand...
For those who landed here using Google, you need to use the OEM version of MFT to disable flash write protection. The regular flint won't allow you to disable flash write protection.

You do that by installing MFT with the --oem flag.

See step c under Linux install.

Compilation and Installation

docs.nvidia.com docs.nvidia.com
 
  • Like
Reactions: lightsword, obovata, blunden and 1 other person
O

obovata

New Member
Dec 18, 2023
3
3
3
Thank you everyone for the helpful posts in this thread. I picked up a Dell branded Connectx-4 LX (p/n 0MRT0D) recently and was able to successfully flash to the stock Mellanox firmware (14.32.1010, PSID MT_2420110034).

Unfortunately the Mellanox firmware produces an issue with the link LEDs on the card. They remain illuminated green even when there were no transceivers installed. The activity LEDs still worked correctly. Everything else on the card seemed to work normally but I was left wondering if there were any other not so obvious issues from the firmware change.

I ended up flashing back to the original Dell firmware. Thank you AK360 for including that flint command to back up the original firmware! I can confirm that restoring from it works. The LEDs on my card are now back to operating as they should. I also noticed that I did not have to manually set the GUID and MAC again after flashing despite the Mellanox to Dell PSID change.

Curiously the latest Dell firmware is a bit newer than the latest Mellanox one (14.32.2004 vs. 14.32.1010). Unfortunately the release notes Dell provides leaves the feature summary and fixes sections blank so who knows what difference it makes. I did catch that the UEFI driver was incremented up from 14.25.17 to 14.25.18 as well:

cx4lxfw14322004notes.png

If anyone with a Dell branded Connectx-4 LX that needs to go back to the original Dell firmware and did not make a backup, I found that you can extract out the firmware from the Windows installer on Dell's site. Use 7zip on the exe and you'll find the .bin in the payload folder. There are few but Dell helpfully includes the applicable part numbers in the filename: fw-ConnectX4Lx-rel-14_32_2004-020NJD_0MRT0D_Ax-UEFI-14.25.18-FlexBoot-3.6.502.bin.
 
  • Like
Reactions: rtech and Exhaust8890
J

j.battermann

Member
Aug 22, 2016
86
21
8
44
mach3.2 said:
For those who landed here using Google, you need to use the OEM version of MFT to disable flash write protection. The regular flint won't allow you to disable flash write protection.

You do that by installing MFT with the --oem flag.

See step c under Linux install.

Compilation and Installation

docs.nvidia.com docs.nvidia.com
Click to expand...
Do you happen to know if the windows tools / WinMFT also have an OEM flag?
 
J

j.battermann

Member
Aug 22, 2016
86
21
8
44
j.battermann said:
Do you happen to know if the windows tools / WinMFT also have an OEM flag?
Click to expand...
Answering my own question just a few minutes later: the windows installer does indeed contain a flint_oem.exe but it only installs flint_ext.exe by default, which in turn the flint.bat recognizes / uses primarily:

1715984992503.png

So what you can do is extract the windows installer's .msi, grab the flint_oem.exe, remove/rename the flint_ext.exe in the target installation dir (after you installed WinMFT) and off you go.

There might by an environment variable or something which the .msi picks up to actually make that entire thing easier, but I am not really in the mood to dissect wix files right now.
 
  • Like
Reactions: lightsword, rukiri, Exhaust8890 and 1 other person
J

j.battermann

Member
Aug 22, 2016
86
21
8
44
j.battermann said:
Answering my own question just a few minutes later: the windows installer does indeed contain a flint_oem.exe but it only installs flint_ext.exe by default, which in turn the flint.bat recognizes / uses primarily:

View attachment 36754

So what you can do is extract the windows installer's .msi, grab the flint_oem.exe, remove/rename the flint_ext.exe in the target installation dir (after you installed WinMFT) and off you go.

There might by an environment variable or something which the .msi picks up to actually make that entire thing easier, but I am not really in the mood to dissect wix files right now.
Click to expand...
... and flashing worked, thanks for the instructions!

1715985625786.png
 
  • Like
Reactions: Exhaust8890
H

Hand

New Member
Sep 5, 2024
3
0
1
AK360 said:
Thank you It did work with a bit of modifying, for any one wondering:

********** Backup Settings **************
flint -d mt4117_pciconf0 query full > flint_query.txt

flint -d mt4117_pciconf0 hw query > flint_hwinfo.txt

flint -d mt4117_pciconf0 ri orig_firmware.bin

flint -d mt4117_pciconf0 dc orig_firmware.ini

mlxburn -d mt4117_pciconf0 -vpd > orig_vpd.txt

************** Write Mellanox Firmware ***************

flint -d mt4117_pciconf0 -ocr hw set Flash0.WriteProtected=Disabled

flint -nofs --use_image_ps --ignore_dev_data -allow_psid_change -d mt4117_pciconf0 -i fw-ConnectX4Lx-rel-14_30_1004-MCX4121A-ACA_Ax-UEFI-14.23.17-FlexBoot-3.6.301.bin burn

************* After power power cycle program GUID and MAC **************

flint -d mt4117_pciconf2 -guid XXXX -mac XXXX -ocr sg
Click to expand...
Forgive my ignorance. I don't understand the last part where you have to program the GUID and MAC, do we use the values from pre-flash?
 
H

Hand

New Member
Sep 5, 2024
3
0
1
i386 said:
Yes
Click to expand...
Alright so I used the base guid and base mac from the backup flint query file and rebooted, but when running query again, it shows the base guid and base mac as N/A.

Interestingly enough both GuidsNumber now show up as 4, but previously were 8 before flashing the stock fw.
 
I

ix64

New Member
Sep 25, 2024
1
0
1
I got the exact same card (0MRT0D), and it seems like the base GUID and MAC cannot be set with flint for some reason.

However, I can use the Dell updater to update the card, and it persists the both the Base GUID and MAC

At least with fw version 14.32.2004, ASPM is well and working
 
H

Hand

New Member
Sep 5, 2024
3
0
1
ix64 said:
I got the exact same card (0MRT0D), and it seems like the base GUID and MAC cannot be set with flint for some reason.

However, I can use the Dell updater to update the card, and it persists the both the Base GUID and MAC

At least with fw version 14.32.2004, ASPM is well and working
Click to expand...
Yeah, I ended up going back to dell FW and updated to latest version, then I tried adding the GUID and MAC and now it shows up like so:

Code: 
Image type:            FS3
FW Version:            14.32.2004
FW Release Date:       13.1.2022
Part Number:           020NJD_0MRT0D_Ax
Description:           Mellanox 25GBE 2P ConnectX-4 Lx Adapter
Product Version:       14.32.2004
Rom Info:              type=UEFI version=14.25.18 cpu=AMD64
                       type=PXE version=3.6.502 cpu=AMD64
Description:           UID                GuidsNumber
Base GUID:             98039b030094a338        4
Orig Base GUID:        N/A                     4
Base MAC:              98039b94a338            4
Orig Base MAC:         N/A                     4
Image VSD:             N/A
Device VSD:            N/A
PSID:                  DEL2420110034
Security Attributes:   N/A
Default Update Method: fw_ctrl
This is how it first looked:

Code: 
Image type:            FS3
FW Version:            14.31.2250
FW Release Date:       24.11.2021
Part Number:           020NJD_0MRT0D_Ax
Description:           Mellanox 25GBE 2P ConnectX-4 Lx Adapter
Product Version:       14.31.2250
Rom Info:              type=UEFI version=14.24.15 cpu=AMD64
                       type=PXE version=3.6.404 cpu=AMD64
Description:           UID                GuidsNumber
Base GUID:             98039b030094a338        8
Base MAC:              98039b94a338            8
Image VSD:             N/A
Device VSD:            N/A
PSID:                  DEL2420110034
Security Attributes:   N/A
Default Update Method: fw_ctrl
However, one issue I could never fix was restoring the VPD, it now shows up as no VPD found. The whole thing was kind of a waste of time for me and I wish I had seen obovata's comment before trying lol.
 
You must log in or register to reply here.
Top Bottom