Flashing stock Mellanox firmware to OEM (EMC) ConnectX-3 IB/Ethernet dual-port QSFP adapter

Churchill

Admiral
Jan 6, 2016
788
185
43
I did what @kiteboarder did on 3 cards and worked like a champ through the process.

My only question is how can I truly test the cards to see if they will appear as full Ethernet devices?
 
  • Like
Reactions: kiteboarder

Milos

New Member
Feb 22, 2018
21
1
3
Black hole with internet
These are the notes I took down for myself when I did this a year or so back. YMMV. Please be careful using -allowpsidchange option.

Flashing firmware on HP 649281-B21 to stock Mellanox MCX354A-FCBT (ConnectX-3) guide:

For Windows Server 2016:

Step 1 - download and install:
Mellanox WinMFT_x64.exe (latest version from Mellanox Website under firmware tools)

Step 2 - download firmware for your card from Mellanox Website. Mine was an HP 649281-B21 which means i needed the MCX354A-FCBT firmware. Be sure which card is yours.
Rename downloaded firmware file to fw.bin
copy it to:
C:\Program Files\Mellanox\WinMFT>

Step 3 - open command prompt as administrator
cd to directory:
C:\Program Files\Mellanox\WinMFT>

Run:
0) mst status
1) mlxfwmanager
2) flint -d mt4099_pci_cr0 -i fw.bin -allow_psid_change burn

where "mt4099_pci_cr0" is the name output from commands 0 or 1

It should say "You are about to replace current PSID on flash - "HP_02802xxxxx" with a different PSID... etc.
Do you want to continue, type Y.


Flashing is done , 3 cards all is ok thanks a lot.
 
  • Like
Reactions: kiteboarder

cuco

Member
Feb 13, 2018
51
2
8
35
Hello,
i bought two Fujitsu single port QSFP cards. MT4099 so ConnectX-3 but i cant exactly figure out which hardware version it is, to choose the right firmware.
This is the Link i purchased them from:
Fujitsu ConnectX-3 VPI 1-Port QSFP FDR 56GbE LP - S26361-F4533-L102 | eBay

Another question. I will use the cards in a server 2019 envirenment just to connect my alsways on server with my optional running server via a Brocase 6610. No special configs with VLans or so. Should or do i have to flash them to ethernet mode?
Perhaps anyone can help. Would be very kind.

Thanks a lot!
 

mb300sd

Active Member
Aug 1, 2016
197
74
28
30
Hello,
i bought two Fujitsu single port QSFP cards. MT4099 so ConnectX-3 but i cant exactly figure out which hardware version it is, to choose the right firmware.
This is the Link i purchased them from:
Fujitsu ConnectX-3 VPI 1-Port QSFP FDR 56GbE LP - S26361-F4533-L102 | eBay

Another question. I will use the cards in a server 2019 envirenment just to connect my alsways on server with my optional running server via a Brocase 6610. No special configs with VLans or so. Should or do i have to flash them to ethernet mode?
Perhaps anyone can help. Would be very kind.

Thanks a lot!
Looks like MCX353A-FCBT to me. Change to ethernet mode in device manager (click the network adapter, View->Devices by connection, open the properties for the device 1 up from it)
 

cuco

Member
Feb 13, 2018
51
2
8
35
Ok this seems to have worked. I uses mighty fohdeesha's example.
Card is set to ethernet mode as well. I use the WinOF 5.50.51 drivers. But there is now WakeOnLan option.
When i read right 5.50.51 does not support WOL.
I tried to use the 5.35 with the latest firmware. but this does not seem to work. the system doesnt use the driver.
I am running on Server2019 and their only is a server 2016 5.35 driver. Any idea how to get WOL to work? Do i have to downgrade the firmware?
 

KC8FLB

Member
Aug 12, 2018
56
38
18
These are the notes I took down for myself when I did this a year or so back. YMMV. Please be careful using -allowpsidchange option.

Flashing firmware on HP 649281-B21 to stock Mellanox MCX354A-FCBT (ConnectX-3) guide:

For Windows Server 2016:

Step 1 - download and install:
Mellanox WinMFT_x64.exe (latest version from Mellanox Website under firmware tools)

Step 2 - download firmware for your card from Mellanox Website. Mine was an HP 649281-B21 which means i needed the MCX354A-FCBT firmware. Be sure which card is yours.
Rename downloaded firmware file to fw.bin
copy it to:
C:\Program Files\Mellanox\WinMFT>

Step 3 - open command prompt as administrator
cd to directory:
C:\Program Files\Mellanox\WinMFT>

Run:
0) mst status
1) mlxfwmanager
2) flint -d mt4099_pci_cr0 -i fw.bin -allow_psid_change burn

where "mt4099_pci_cr0" is the name output from commands 0 or 1

It should say "You are about to replace current PSID on flash - "HP_02802xxxxx" with a different PSID... etc.
Do you want to continue, type Y.
Thank you, worked wonderfully on a Win 10 box.
 
  • Like
Reactions: kiteboarder

saverio

New Member
Jun 24, 2020
5
0
1
Hello,
I have flashed a CX354A-QCBT to -FCBT using the guide in this thread. I tried to cross connect the two ports using an FS.com QSFP DAC, but the link is 10Gb, not 40. If I try to set it manually to 40000 with ethtool, it doesn't go up.


Could it be a cable issue?

Code:
Querying Mellanox devices firmware ...

Device #1:
----------

  Device Type:      ConnectX3
  Part Number:      MCX354A-FCB_A2-A5
  Description:      ConnectX-3 VPI adapter card; dual-port QSFP; FDR IB (56Gb/s) and 40GigE; PCIe3.0 x8 8GT/s; RoHS R6
  PSID:             MT_1090120019
  PCI Device Name:  0000:02:00.0
  Port1 MAC:        506b4b7cf4b1
  Port2 MAC:        506b4b7cf4b2
  Versions:         Current        Available
     FW             2.42.5000      2.42.5000
     PXE            3.4.0752       3.4.0752

  Status:           Up to date
Code:
ethtool enp2s0
Settings for enp2s0:
        Supported ports: [ FIBRE ]
        Supported link modes:   1000baseKX/Full
                                10000baseKX4/Full
                                10000baseKR/Full
                                40000baseCR4/Full
                                40000baseSR4/Full
                                56000baseCR4/Full
                                56000baseSR4/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  1000baseKX/Full
                                10000baseKX4/Full
                                10000baseKR/Full
                                40000baseCR4/Full
                                40000baseSR4/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Port: Direct Attach Copper
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000014 (20)
                               link ifdown
        Link detected: yes
 

Freebsd1976

Active Member
Feb 23, 2018
220
33
28
Hello,
I have flashed a CX354A-QCBT to -FCBT using the guide in this thread. I tried to cross connect the two ports using an FS.com QSFP DAC, but the link is 10Gb, not 40. If I try to set it manually to 40000 with ethtool, it doesn't go up.


Could it be a cable issue?

Code:
Querying Mellanox devices firmware ...

Device #1:
----------

  Device Type:      ConnectX3
  Part Number:      MCX354A-FCB_A2-A5
  Description:      ConnectX-3 VPI adapter card; dual-port QSFP; FDR IB (56Gb/s) and 40GigE; PCIe3.0 x8 8GT/s; RoHS R6
  PSID:             MT_1090120019
  PCI Device Name:  0000:02:00.0
  Port1 MAC:        506b4b7cf4b1
  Port2 MAC:        506b4b7cf4b2
  Versions:         Current        Available
     FW             2.42.5000      2.42.5000
     PXE            3.4.0752       3.4.0752

  Status:           Up to date
Code:
ethtool enp2s0
Settings for enp2s0:
        Supported ports: [ FIBRE ]
        Supported link modes:   1000baseKX/Full
                                10000baseKX4/Full
                                10000baseKR/Full
                                40000baseCR4/Full
                                40000baseSR4/Full
                                56000baseCR4/Full
                                56000baseSR4/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  1000baseKX/Full
                                10000baseKX4/Full
                                10000baseKR/Full
                                40000baseCR4/Full
                                40000baseSR4/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Port: Direct Attach Copper
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000014 (20)
                               link ifdown
        Link detected: yes
Maybe or may not be , I have flash 5 qcbt to fcbt .
1 one of them only one port can work on 40/56 ,the second port work on 10g.
2. the other four card work as normal fcbt .
 
Jan 10, 2019
86
17
8
blog.azureinfra.com
and for the Windows Users: (very similar to the opening post:)

#Download Tools: https://www.mellanox.com/downloads/MFT/WinMFT_x64_4_15_0_104.exe
#Download Firmware: http://www.mellanox.com/downloads/f...00-MCX354A-FCB_A2-A5-FlexBoot-3.4.752.bin.zip

#Install the tools, then go to
CD .\Program Files\Mellanox\WinMFT

#Query the device name (use the 2nd name returned looking like: mt4099_pciconf0)
mdevices status

#Backup old firmware
flint -d mt4099_pciconf0 dc orig_fw.ini

#Burn new firmware
flint -d mt4099_pciconf0 -i c:\<folder>\fw-ConnectX3-rel-2_42_5000-MCX354A-FCB_A2-A5-FlexBoot-3.4.752.bin -allow_psid_change burn

#disable boot stuff
mlxconfig -d mt4099_pciconf0 set BOOT_OPTION_ROM_EN_P1=false
mlxconfig -d mt4099_pciconf0 set BOOT_OPTION_ROM_EN_P2=false
mlxconfig -d mt4099_pciconf0 set LEGACY_BOOT_PROTOCOL_P1=0
mlxconfig -d mt4099_pciconf0 set LEGACY_BOOT_PROTOCOL_P2=0


#remove the boot ROM
flint -d mt4099_pciconf0 --allow_rom_change drom

#check current settings
mlxconfig -d mt4099_pciconf0 query


#Reboot Machine
 

dpsi

New Member
Dec 14, 2015
3
0
1
Just wanted to mention that I was able to do this succesfully with a Revision A6 649281-B21 using the A2-A5 firmware.