Bought cheap MCX353A-FCBT ConnectX-3 Primergy Fujitsu branded card from eBay:
MCX353A-FCBT FUJITSU CONNECTX3 VPI 1 PORT QSFP 40GBE ADAPTER CARD W/BOTH BRACKET | eBay
Reported PSID: FJT1060100019
This Fujitsu PSID prevented me from upgrading to the latest firmware version
Read forums here, figured out how to reflash to latest firmware and stock PSID.
When you reflash you should see: You are about to replace current PSID on flash - "FJT1060100019" with a different PSID - "MT_1100120019"
Only issue I see so far is connecting a Linux to Windows machine only gets about 13 Gbps with multiple threads iperf3 (see iperf3 at 40Gbps and above), will need to investigate/tune further. I can get 40 Gbps reliably between these cards, either directly or through a 40 Gbps switch, using multiple iperf processes in parallel. I was not getting the correct speed in the beginning because I was using one of the cards in a PCIe 2 slot instead of 3.
References:
-
家庭向け(仮) 40GbE NIC 導入方法 | 自宅ラックエンジニアの活動ログ
-
https://forums.servethehome.com/ind...ectx-3-en-with-stock-firmware.8689/post-80356
===== LINUX REFLASH =====
Ubuntu install example: apt install mstflint
lspci | grep Mellanox (to get the device ID 06:00.0 below - replace with your own device ID!)
mstflint -d 06:00.0 query full > flint_query.txt
mstflint -d 06:00.0 hw query > flint_hwinfo.txt
mstflint -d 06:00.0 ri orig_firmware.bin
mstflint -d 06:00.0 dc orig_firmware.ini
mstflint -d 06:00.0 rrom orig_rom.bin
mstvpd -m 06:00.0 > orig_vpd.txt
# Download firmware for MCX353A-FCBT from
Firmware for ConnectX®-3 IB/VPI
wget
http://www.mellanox.com/downloads/f...00-MCX353A-FCB_A2-A5-FlexBoot-3.4.752.bin.zip
unzip fw-ConnectX3-rel-2_42_5000-MCX353A-FCB_A2-A5-FlexBoot-3.4.752.bin.zip
mstflint -d 06:00.0 -i fw-ConnectX3-rel-2_42_5000-MCX353A-FCB_A2-A5-FlexBoot-3.4.752.bin -allow_psid_change burn
===== WINDOWS REFLASH =====
Run cmd windows as Administrator!
Install MFT:
Mellanox Firmware Tools (MFT)
mst start
mst status
(gets list of devices change mt4099_pci_cr0 below with your device id)
flint -d mt4099_pci_cr0 -i fw-ConnectX3-rel-2_42_5000-MCX353A-FCB_A2-A5-FlexBoot-3.4.752.bin -allow_psid_change burn