Looking for a Connectx-3 custom firmware package

dazagrt

New Member
Mar 1, 2021
9
1
3
OK, the Dell Isilon part number 105-000-855-00 comes up as Mellanox Dual 10Gb Ethernet CNA
 

Rand__

Well-Known Member
Mar 6, 2014
6,081
1,458
113
Its an EMC card with custom EMC firmware on it.
You might be able to flash it to a regular MLX one if you want that with --allow_psid_change
 

Rand__

Well-Known Member
Mar 6, 2014
6,081
1,458
113
I'm confused :D
Understandable. :p

Its indeed a bit weird that the pci id points to a regular card while the info says its a 10g card.
Probably some OEM/EMC fw "magic"...

Maybe pics indeed would help to see if its more like a 312 or 314
 

i386

Well-Known Member
Mar 18, 2016
2,981
946
113
33
Germany
I think I would risk it and flash one card with the fimware for "MCX312C-XCCT" (Dual 10GbE (High Message Rate))
 

Rand__

Well-Known Member
Mar 6, 2014
6,081
1,458
113
Well it certainly is no 314;)
Its at least worth a try if you have the need for different fw...
 

dazagrt

New Member
Mar 1, 2021
9
1
3
I tried and it borked at the supplied image.
I'm just trying to work out how to backup the current firmware.
 

BeTeP

Well-Known Member
Mar 23, 2019
573
369
63
And use the correct PRO firmware, otherwise you might brick the card.
Unless by "bricking" you meant "render temporarily inoperable" there is 0% chance of that happening. You wouldn't be able to brick (i.e. permanently destroy beyond repair) that card by flashing if you tried. Mellanox has one of the easiest recovery procedures in the industry.
 

klui

Active Member
Feb 3, 2019
545
240
43
Unless by "bricking" you meant "render temporarily inoperable" there is 0% chance of that happening. You wouldn't be able to brick (i.e. permanently destroy beyond repair) that card by flashing if you tried. Mellanox has one of the easiest recovery procedures in the industry.
Besides Livefish and I2C reprogramming, do you have any advice on recovering a CX3 that does not show up in lspci/Windows device manager?
 

BeTeP

Well-Known Member
Mar 23, 2019
573
369
63
If the eeprom content is the only thing preventing the card from properly enumerating (i.e. it's not otherwise damaged) just use the flash recovery (FLASH_PRESENT) jumper - it forces the card to boot from ROM instead.
 

dazagrt

New Member
Mar 1, 2021
9
1
3
So I've got them through now. This quick note is for the next person seeking help.
How it came together was something like this:

Download the tools from Mellanox.
There are two you can use; the MFT suite or MLXup, get them from here:
mlxup - Mellanox Update and Query Utility

All of the firmware downloads are found here:
Firmware Downloads

We need to get some info from the network card. Use anything you like to query the cards, be it mlxup.exe, mlxfwmanager.exe
Simply running mlxup.exe without any other syntax will query all cards. You will get an output like this:

.\mlxup.exe
Querying Mellanox devices firmware ...

Device #1:
----------

Device Type: ConnectX3Pro
Part Number: MCX312C-XCC_Ax
Description: ConnectX-3 Pro EN NIC; 10GbE; dual-port SFP+; PCIe3.0 x8 8GT/s; RoHS R6; hardware revision C

PSID: MT_2340111023
PCI Device Name: mt4103_pci_cr0
Port1 MAC: 98039be75470
Port2 MAC: 98039be75471
Versions: Current Available
FW 2.42.5000 2.42.5000
PXE 3.4.0752 3.4.0752

Status: Up to date

The hardware name is what we can use. PCI Device Name: mt4103_pci_cr0

Now if we were out of date after running mlxup.exe it would ask us if we wanted to update. If we said yes, it will automatically download the file from Mellanox and install it.

Now if we wanted to use our firmware or a different firmware that mlxup.exe wasn't aware of, all that we need to do is add the device and image file we wanted to flash with the -d and -i switches.
Our command would now look something like this.

.\mlxup.exe -d mt4103_pci_cr0 -u -i my-image-file.bin

The (-d device) comes before the (-u update) and then the image file we want to use (-i image-file).

If you are trying to flash a Dell card (like mine) back the standard Mellanox firmware, for example, you will need to add some more parameters to the command to make it flash your image file.

I have found that switching to the program called flint (which is found in MFT Suite of tools) and adding the following to the end of the command will allow it to run the flashing process.
-allow_psid_change burn

Now our whole command looks like this:
.\flint -d mt4103_pci_cr0 -i my-image-file.bin -allow_psid_change burn

once you run the above command you will get something like this:

You are about to replace current PSID on flash - "EMC0000000002" with a different PSID - "MT_2340111023".

Note: It is highly recommended not to change the PSID.

Do you want to continue ? (y/n) [n] : y

You must answer yes to get the flash through!

The flash won't take long. When it is finished do a full shutdown. Then boot up in the normal way.

All done.
Hope that helps.
 
Last edited:

Stephan

Active Member
Apr 21, 2017
405
236
43
Germany
Didn't want to open a separate thread...

Any drawbacks on Sun Oracle 7046442 (ConnectX-3 Rev A4) to run a bastardized IBM1090111019.bin from the Lenovo package i.e. 2.42.5032 together with fw-ConnectX3-rel-2_42_5000-MCX354A-FCB_A2-A5-FlexBoot-3.4.752 configuration?

Differences are very few:

< Name = 00D9551_Ax
> Name = MCX354A-FCB_A2-A5
< Description = Mellanox ConnectX-3 VPI Dual-port QSFP FDR14/40GbE HCA
> Description = ConnectX-3 VPI adapter card; dual-port QSFP; FDR IB (56Gb/s) and 40GigE; PCIe3.0 x8 8GT/s; RoHS R6
< PSID = IBM1090111019
> PSID = MT_1090120019
< hca_header_subsystem_id = 0x0076
> hca_header_subsystem_id = 0x0050

Used fs2_update_ini.py to merge and flint verify says bootable and no problems detected.
 

Stephan

Active Member
Apr 21, 2017
405
236
43
Germany
Leaving this here for posterity for people who got an Oracle 7046442 rev A3/A4. Flashing to Lenovo 2.42.5032 code base worked for me. Very last version to ever come out for ConnectX3 I guess. See commands below (I used Windows and a .cmd wrapper for the Python script) on how to build the firmware. Oracle had no boot ROM but I saved firmware 2.35.6312 and 2.35.6308 from the cards if anyone needs it.

For giggles I ported Mellanox Firmware Tools 4.18.0 ("MFT") to ArchLinux, including the necessary drivers which are now automatically compiled through DKMS for the kernel currently installed. So you can do "mst start" "mst status" "mget_temp -d /dev/mst/mt4099_pci_cr0" etc. and things will just work. I backuped and flashed four cards using this package. PM me if you want any of this.

Code:
  Device Type:      ConnectX3
  Part Number:      7046442_7092757_Ax
  Description:      Oracle Dual port; ConnectX-3 ; Infiniband Adapter FDR/QDR
  PSID:             ORC1090120019
  PCI Device Name:  /dev/mst/mt4099_pci_cr0
  Port1 GUID:       XXXXXXXXXXXXXXXX
  Port2 GUID:       XXXXXXXXXXXXXXXX
  Versions:         Current        Available
     FW             2.35.6308      N/A
Code:
  Device Type:      ConnectX3
  Part Number:      MCX354A-FCB_A2-A5
  Description:      ConnectX-3 VPI adapter card; dual-port QSFP; FDR IB (56Gb/s) and 40GigE; PCIe3.0 x8 8GT/s; RoHS R6
  PSID:             MT_1090120019
  PCI Device Name:  /dev/mst/mt4099_pci_cr0
  Port1 GUID:       XXXXXXXXXXXXXXXX
  Port2 GUID:       XXXXXXXXXXXXXXXX
  Versions:         Current        Available
     FW             2.42.5032      N/A
     PXE            3.4.0754       N/A
     UEFI           14.11.0047     N/A
Code:
# Extract config from Mellanox firmware
WinMFT\flint.exe -i fw-ConnectX3-rel-2_42_5000-MCX354A-FCB_A2-A5-FlexBoot-3.4.752.bin dc fw-ConnectX3-rel-2_42_5000-MCX354A-FCB_A2-A5-FlexBoot-3.4.752.cfg

# Update IBM firmware with Mellanox config
fs2_update_ini.cmd CX3-2_42_5032.bin fw-ConnectX3-rel-2_42_5000-MCX354A-FCB_A2-A5-FlexBoot-3.4.752.cfg

# Query and verify image
WinMFT\flint.exe -i CX3-2_42_5032.bin q full
WinMFT\flint.exe -i CX3-2_42_5032.bin verify

# Burn firmware to an image
mlxburn -fw fw-ConnectX3-rel.mlx -conf MCX312A-XCB_A2-A6.ini -wrimage mlnx_firmware.bin

# Checks
flint -i mlnx_firmware.bin verify
flint -i mlnx_firmware.bin query full
 
  • Like
Reactions: istamov and UhClem