Supermicro AOC-S25G-M2S ConnectX-4 LX SFP28 to Mellanox firmware

[john389]

Hi,

I tried flashing the Supermicro OEM variant of the ConnectX-4 LX dual port SFP28 card to the newest Mellanox MCX4121A-ACAT firmware today. Results were as follows:

• Flashing using `flint` worked, however ...
• Sometimes the driver hangs, complaining about CRC errors and such, then the device is no longer visible and I need to reboot; the system hangs and needs a hard-reset to function again
• Only one network port get's a carrier and active LED lights, when a module/DAC cable is plugged in, the other will always remain at NO-CARRIER

When looking at the firmware configuration of the original Supermicro firmware I saw quite a lot of entries that the default Mellanox firmware did not have. Here are a few:

nv_config.global.pci.settings.fpp_en = 1
management.expander_ini.pca9536_exist = 1
management.expander_ini.smbus_slv_indx = 7
management.expander_ini.pca9536.def_val = 0xf
management.expander_ini.pca9536.config = 0.0
pcie_cfg.pcie_tx_config = 3
pcie_cfg.use_1_8_rail_on_tx = 1
pcie_cfg.ob_bias_gen3 = 15
pcie_cfg.pcie_cml100_reg = 8
pcie_cfg.pcie_leva = 9
pcie_cfg.eqlz_ph3_array_force_single_entry.enable = 1
pcie_cfg.eqlz_ph3_array_force_single_entry.ob_tap0 = 21
pcie_cfg.eqlz_ph3_array_force_single_entry.ob_tap1 = 92
pcie_cfg.eqlz_ph3_array_force_single_entry.ob_tap2 = 7
. . .

Obviously they are from different sections.

When flashing my older ConnectX-3 cards, I could download an older MFT version, a MLX firmware file, combine it with an altered config file and generate a new firmware image, which can be burnt to the card.

For ConnectX-4 cards I have no idea how to accomplish that.

Considering that the device shows up after crossflashing, maybe the settings need to be integrated into the default Mellanox firmware for everything to work correctly?

Could someone help?

Also: Does anyone know where Supermicro has placed their firmware for the card? All links seem to be dead.

 

[klui]

Supermicro-specific SKUs (AOC-....) tend to be custom designs. They typically don't use reference designs so there will be situations where stock firmware won't work properly. Just look at a normal Mellanox ConnectX-4 Lx and compare it to their S25G-m2S. If you're concerned about FW updates, send an email to support.

Supermicro cards are very lenient on the types of transceivers they accept. I have one of their I2S cards and they don't have any vendor locking.

 

[i386]

(I think supermicro uses use the reference design + smbus (or i2c?) stuff to get the nic/adapter information to show up in ipmi)

Also: Does anyone know where Supermicro has placed their firmware for the card? All links seem to be dead.

You could try their wdl server: www.supermicro.com - /wdl/
But I don't know in what section sm would put the driver/firmware...

 

[john389]

Thank you both for the information. It looks like my best bet is to contact Supermicro support for a newer firmware. I couldn't find anything on their wdl server.