Today arrived at the office to test a few things
Do you have Mellanox or OEM cards? OEMs, like HP come with secure-fw, meaning crossflash is nearly impossible without an i2c flash access.
This one is probably pure Mellanox (I bought just 1, my first ConnectX-5 to test it, now it's fully working under Windows10 but it's PCIe 3.0 x16 like cheaper ConnectX4 100GbE) also it' is "too fast" to use it in something else than server (nice one, to fully utilize it if PCIE 4.0 will be working).
But today I flashed also 455A-FCAT to ECAT (40GbE to 100GbE) from DELL and it's also easy only difference is at Windows 10 drivers fail to start it, but you can still flash it easily (Windows command):
flint -d mt4115_pciconf0 -i c:\M\fw-ConnectX4-rel-12_28_2006-MCX455A-ECA_Ax-UEFI-14.21.17-FlexBoot-3.6.102.bin -allow_psid_change burn
and to switch Ethernet permanently:
mlxconfig -d mt4115_pciconf0 set LINK_TYPE_P1=ETH
but I am still worry about flash PCIe 3.0 to 4.0, my logic says something inside should be different to support double the speed. hmm...
CCAT to CCHT no problem but CDAT hmm.... especially when they are changing name from EN to EX
ConnectX-5 card is VPI (both IB and Eth), and it's really far from Ex cards (Eth only).
Fortunately I need Eth only
but still Eth and IB is nothing to PCIe 3.0 and PCIe 4.0 I think - but not sure :/
Please dump the firmware config here ((mst)flint -d <your card> q; (mst)flint -d <your card> dc;
good idea
Does your system support PCIe 4.0 in that particular slot (lspci -s <addr> -vvv)
No - not yet but I will organize it when working
My 2 today test bench are (old workstations):
TR 1950x - I can connect 4 PCIe 3.0 x16 cards
Xeon 1650 v2 - I can connect 2 PCIe 3.0 x16 cards and one x4.
Problem with PCIe 4.0 and 2 ports is, how to saturate it (200GbE) from SSD's - So waiting for TR 6000 or Xeon 12gen
Fastest testbench I can organize for while (until it goes as a workstation) is 12900kf with DDR5 memory, If using PCIe 5.0 slot for Mellanox, still 12 PCIe 4.0 lanes left so about 18 GB/s from NVME is possible, but 2x100GbE is still faster - about 22GB/s. 5950x is too slow in single thread (I believe) and limited bandwidth thru chipset (~8GB/s).
My CX6 cross-flash has been successful, there are also successful reports of CX5 cross-flashes here.
Can you recognize on the pictures if they looks similar or same to EX PCIe 4.0 cards ?
*sorry for the pictures quality, I forgot phone from the car and use old samsung gear wristwatch to take a photos