Flashing the Cisco CX623106A NIC to OEM firmware

thankfly · Nov 13, 2024

It's truly that the card marked AN can flash to AS, and AS cannot flash to AN.
Why? I dont know.

thankfly · Nov 17, 2024

thankfly said:
It's truly that the card marked AN can flash to AS, and AS cannot flash to AN.
Why? I dont know.

I have tried for many times, and it's impossible to flash an AS to AN, and not sure why.

mha · Feb 18, 2025

1)
Is it possible to have the VPI cards run a regular Dx firmware to enable TLS offloading?

I have a CX653105A / 07TKND single-port VPI adapter that can run in Infiniband or Ethernet mode just fine. I know from the posts here that I can flash the Mellanox 200G firmware on it. Would it, however, run the corresponding Dx firmware from, for instance, MCX623105AC-VDAT?

This is not possible. The flasher will complain that the firmware was specifically designed for a Dx card, tried it today.

2)
I am also a bit confused on the ConnectX-6 DX cards. From NVidias home page resources you can see that they mention that the TLS offloading crypto engine is available without differentiating between cards;

NVIDIA ConnectX-6 Dx Network Adapters ("IPsec and TLS in-line crypto and block crypto acceleration")
https://www.nvidia.com/content/dam/...ethernet-adapters/connectX-6-dx-datasheet.pdf ("Inline hardware TLS encryption and decryption AES-GCM 128/256-bit key")

However looking at the following resource:

NVIDIA ConnectX-6 Dx Ethernet Adapter Cards User Manual

It would seem like the following models are designated as "No Crypto";

MCX623105AN-CDAT

MCX623106AN-CDAT

MCX623106AS-CDAT

Will the previously discussed Cisco ConnectX-6 DX card (and probably Dell and others) run the MCX623106AS-CDA firmware to enable TLS offloading?

Any input from you would be appreciated. Bought a bunch of cards with the hope of having TLS offloading only to see that the feature is not available on my adapter

Code:

# ethtool -k ext3 | grep -i tls
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
tls-hw-record: off [fixed]

Code:

# lspci -v -s 08:00.0
08:00.0 Ethernet controller: Mellanox Technologies MT2892 Family [ConnectX-6 Dx]
        Subsystem: Mellanox Technologies MT2892 Family [ConnectX-6 Dx]
        Physical Slot: 1
        Flags: bus master, fast devsel, latency 0, IRQ 16, NUMA node 0, IOMMU group 56
        Memory at 96000000 (64-bit, prefetchable) [size=32M]
        Expansion ROM at 92b00000 [virtual] [disabled] [size=1M]
        Capabilities: [60] Express Endpoint, MSI 00
        Capabilities: [48] Vital Product Data
        Capabilities: [9c] MSI-X: Enable+ Count=64 Masked-
        Capabilities: [c0] Vendor Specific Information: Len=18 <?>
        Capabilities: [40] Power Management version 3
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [1c0] Secondary PCI Express
        Capabilities: [230] Access Control Services
        Capabilities: [320] Lane Margining at the Receiver <?>
        Capabilities: [370] Physical Layer 16.0 GT/s <?>
        Capabilities: [420] Data Link Feature <?>
        Kernel driver in use: mlx5_core
        Kernel modules: mlx5_core

Code:

Image type:            FS4
FW Version:            22.38.1002
FW Release Date:       3.8.2023
Part Number:           0F6FXM_08P2T2_Ax
Description:           Mellanox ConnectX-6 Dx Dual Port 100 GbE QSFP56 Network Adapter
Product Version:       22.38.1002
Rom Info:              type=UEFI version=14.31.20 cpu=AMD64,AARCH64
                       type=PXE version=3.7.201 cpu=AMD64
Description:           UID                GuidsNumber
Base GUID:             58a2e        8
Base MAC:              58a2e            8
Image VSD:             N/A
Device VSD:            N/A
PSID:                  DEL0000000027
Security Attributes:   secure-fw
Default Update Method: fw_ctrl
Life cycle:            GA SECURED
Secure Boot Capable:   Enabled
EFUSE Security Ver:    0
Image Security Ver:    0
Security Ver Program:  Manually ; Disabled

All input would be appreciated!

Tux · Jul 30, 2025

Just to let here know, thanks for your guide @bitbckt - it directed me to the right way... will do small recap of all steps...

I was able to successfully crossflash 4 branded HPE NV60100M 100Gb 2p Strg Offload Adptr (SPS: P46007-001, Model No: CX623106A, PN: R8M41-63001) - upgradeable only through iLO 5+ from PSID HPE0000000062 to PSID MT_0000000436 with original Mellanox FW, that is after crossflash upgradeable from OS (used Rocky Linux 9.5 with mlnx_ofed 24.10-2.1.8.0 driver).

This PN R8M41-63001 corresponds to MCX623106AC-CDAT (HPE & NVIDIA Networking OEM Solutions Reference Guide).

Used process was shorting the J7 pin (e.g. with paperclip) on the card (when system was off), then power on, start MST configuration module and when card was in recovery mode due to shorted pin, it allowed crossflashing to different PSID...

Bash:

# mst start
# flint -d /dev/mst/mt530_pciconf0 --psid MT_0000000436 -i fw-ConnectX6Dx-rel-22_35_4506-MCX623106AC-CDA_Ax-UEFI-14.29.15-FlexBoot-3.6.902.signed.bin -ocr --nofs --allow_psid_change --yes burn

Then turned system off, removed the shorted pin, started system normally and then did second step of the upgrade to confirm, I'm actually able to upgrade FW without recovery mode...

Bash:

# mlxfwmanager -i fw-ConnectX6Dx-rel-22_45_1020-MCX623106AC-CDA_Ax-UEFI-14.38.16-FlexBoot-3.7.500.signed.bin -u

Original FW (initial release version afaik):

Bash:

# ethtool -k ens1f0np0 | grep -i tls
tls-hw-tx-offload: on
tls-hw-rx-offload: off
tls-hw-record: off [fixed]

Bash:

# lspci -v -s 08:00.0
08:00.0 Ethernet controller: Mellanox Technologies MT2892 Family [ConnectX-6 Dx]
    Subsystem: Hewlett Packard Enterprise MT2892 Family [ConnectX-6 Dx]
    Physical Slot: 1
    Flags: bus master, fast devsel, latency 0, IRQ 16, NUMA node 0, IOMMU group 55
    Memory at 94000000 (64-bit, prefetchable) [size=32M]
    Capabilities: [60] Express Endpoint, MSI 00
    Capabilities: [48] Vital Product Data
    Capabilities: [9c] MSI-X: Enable+ Count=64 Masked-
    Capabilities: [c0] Vendor Specific Information: Len=18 <?>
    Capabilities: [40] Power Management version 3
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
    Capabilities: [180] Single Root I/O Virtualization (SR-IOV)
    Capabilities: [1c0] Secondary PCI Express
    Capabilities: [230] Access Control Services
    Capabilities: [320] Lane Margining at the Receiver <?>
    Capabilities: [370] Physical Layer 16.0 GT/s <?>
    Capabilities: [420] Data Link Feature <?>
    Kernel driver in use: mlx5_core
    Kernel modules: mlx5_core

Bash:

# flint -d /dev/mst/mt4125_pciconf0 q
Image type:            FS4
FW Version:            22.31.2006
FW Release Date:       31.8.2021
Product Version:       22.31.2006
Rom Info:              type=UEFI version=14.24.15 cpu=AMD64,AARCH64
                       type=PXE version=3.6.404 cpu=AMD64
Description:           UID                GuidsNumber
Base GUID:             b83fd2xxxxxxxxxx        4
Base MAC:              b83fd2xxxxxx            4
Image VSD:             N/A
Device VSD:            N/A
PSID:                  HPE0000000062
Security Attributes:   secure-fw

Latest version after crossflash to Mellanox FW and PSID:

Bash:

# ethtool -k ens1f0np0 | grep -i tls
tls-hw-tx-offload: on
tls-hw-rx-offload: off
tls-hw-record: off [fixed]

Bash:

# lspci -v -s 4f:00.0
4f:00.0 Ethernet controller: Mellanox Technologies MT2892 Family [ConnectX-6 Dx]
    Subsystem: Mellanox Technologies Device 0041
    Flags: bus master, fast devsel, latency 0, IRQ 36, NUMA node 0
    Memory at 3801fe000000 (64-bit, prefetchable) [size=32M]
    Expansion ROM at 90a00000 [disabled] [size=1M]
    Capabilities: [60] Express Endpoint, MSI 00
    Capabilities: [48] Vital Product Data
    Capabilities: [9c] MSI-X: Enable+ Count=64 Masked-
    Capabilities: [c0] Vendor Specific Information: Len=18 <?>
    Capabilities: [40] Power Management version 3
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
    Capabilities: [1c0] Secondary PCI Express
    Capabilities: [230] Access Control Services
    Capabilities: [320] Lane Margining at the Receiver <?>
    Capabilities: [370] Physical Layer 16.0 GT/s <?>
    Capabilities: [420] Data Link Feature <?>
    Kernel driver in use: mlx5_core
    Kernel modules: mlx5_core

Bash:

# flint -d /dev/mst/mt4125_pciconf0 q
Image type:            FS4
FW Version:            22.45.1020
FW Release Date:       6.5.2025
Product Version:       22.45.1020
Rom Info:              type=UEFI version=14.38.16 cpu=AMD64,AARCH64
                       type=PXE version=3.7.500 cpu=AMD64
Description:           UID                GuidsNumber
Base GUID:             b83fd2xxxxxxxxxx        4
Base MAC:              b83fd2xxxxxx            4
Image VSD:             N/A
Device VSD:            N/A
PSID:                  MT_0000000436
Security Attributes:   secure-fw

devrand · Feb 9, 2026

@bitbckt
Thanks for your guide!

Sucessfully flashed my Lenovo 4XC7A08248 (should be MCX623106AS-CDAT) with PSID LNV0000000029 to MT_0000000437, OEM firmware 22.31.1014 to firmware 22.47.1088

Search

Flashing the Cisco CX623106A NIC to OEM firmware

thankfly

New Member

thankfly

New Member

mha

New Member

Tux

New Member

devrand

New Member