Mellanox ConnectX 3 VPI (MCX354A-FCBT) HP OEM but with Mellanox OEM Firmware - 40 USD each

arglebargle

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈
Jul 15, 2018
653
228
43
  • Like
Reactions: BackupProphet

mpeter

New Member
Feb 15, 2019
2
2
3
Bricked other devices in the past, its worth $10 to me to not have to learn or deal with this specific procedure. Might be simple, might not. Don’t care, still worth $10 to have it flashed by someone else.
 
  • Like
Reactions: Sleyk

herby

Active Member
Aug 18, 2013
177
51
28
Mellanox noob here... Will these work in non hp systems?
Yep, I have a couple in Supermicro boards now that I flashed myself with instructions found on these forums. I did have to do the tape over the B-5/B-6 pins trick on one NIC because it had an SMBus conflict with the motherboard, but that's not much of a hastle.
 

arglebargle

H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈
Jul 15, 2018
653
228
43
Yep, I have a couple in Supermicro boards now that I flashed myself with instructions found on these forums. I did have to do the tape over the B-5/B-6 pins trick on one NIC because it had an SMBus conflict with the motherboard, but that's not much of a hastle.
I had to do this for one machine too, I had no problem in about 10 other systems.
 

jpk

Member
Nov 6, 2015
58
21
8
41
I don't know if it's appropriate to post this here, so I can delete if necessary, but here are the instructions I wrote down for flashing the MCX354A cards I bought. I have used mine in Supermicro, Dell and Oracle servers.

For CentOS 7

download firmware tools from:
http://www.mellanox.com/page/management_tools

which leads you to:
http://www.mellanox.com/downloads/MFT/mft-4.10.0-104-x86_64-rpm.tgz


start the mellanox stuff with mst and look at what cards are installed:
# mst start
# ls -l /dev/mst
mt4099_pciconf0 mt4099_pci_cr0 mt4099_pci_msix0


Next, download firmware from
http://www.mellanox.com/page/firmware_table_ConnectX3IB

Need the firmware for the MCX354A-FCBT which should has the PSID MT_1090120019, and then extract it to get the .bin
http://www.mellanox.com/downloads/firmware/fw-ConnectX3-rel-2_42_5000-MCX354A-FCB_A2-A5-FlexBoot-3.4.752.bin.zip

then run this for each card, changing the /dev/mst/mt4099_pci_crX number from /dev/mst/

# flint -d /dev/mst/mt4099_pci_cr0 -i fw-ConnectX3-rel-2_42_5000-MCX354A-FCB_A2-A5-FlexBoot-3.4.752.bin -allow_psid_change burn

This should output something like this:

Current FW version on flash: 2.10.2280
New FW version: 2.42.5000


You are about to replace current PSID on flash - "HP_0280210019" with a different PSID - "MT_1090120019".
Note: It is highly recommended not to change the PSID.

Do you want to continue ? (y/n) [n] : y
Burning FS2 FW image without signatures - OK
Restoring signature - OK

And now the mlxfwmanager should show something like this:

# mlxfwmanager
Querying Mellanox devices firmware ...

Device #1:
----------

Device Type: ConnectX3
Part Number: 649281-B21_B1
Description: HP IB 4X FDR CX-3 PCI-e G3 Dual Port HCA
PSID: HP_0280210019
PCI Device Name: /dev/mst/mt4099_pci_cr1
Port1 MAC: 24be059e3f31
Port2 MAC: 24be059e3f32
Versions: Current Available
FW 2.10.2280 N/A
PXE 3.3.0650 N/A

Status: No matching image found

Device #2:
----------

Device Type: ConnectX3
Part Number: MCX354A-FCB_A2-A5
Description: ConnectX-3 VPI adapter card; dual-port QSFP; FDR IB (56Gb/s) and 40GigE; PCIe3.0 x8 8GT/s; RoHS R6
PSID: MT_1090120019
PCI Device Name: /dev/mst/mt4099_pci_cr0
Port1 MAC: 24be059fe0c1
Port2 MAC: 24be059fe0c2
Versions: Current Available
FW 2.42.5000 2.42.5000
FW (Running) 2.10.2280 N/A
PXE 3.4.0752 3.4.0752

Status: Up to date


Change the default port type from VPI (auto) to eth (again, changing 0 to 1, etc for multiple cards)

# mlxconfig -d /dev/mst/mt4099_pciconf0 set LINK_TYPE_P1=2
# mlxconfig -d /dev/mst/mt4099_pciconf0 set LINK_TYPE_P2=2
 

WANg

Well-Known Member
Jun 10, 2018
728
371
63
Why would you get those for $40 over just flashing the Mlx firmware yourself on the ones for $30 ?
(BO on HP 544QSFP Infiniband 10/40GB FDR/EN PCIe Dual Port HCA 656089-001 649281-B21 | eBay )
It's only a $7 difference unless you buy in bulk and the seller accepts your offer, it's much less risky letting someone else do the firmware swap (regardless of whether it's easy or not it's still your time here), and also, you have a choice between SFF and normal form factor brackets - I am pretty sure that the Mellanox SFF bracket alone is worth $10 including shipping and handling.
 
  • Like
Reactions: Sleyk

101

New Member
Apr 30, 2018
27
9
3
Seller is clearly adding value, and they have more units than they know what to do with. Great option, and good on them for seizing the opportunity.
 
  • Like
Reactions: Sleyk

Sleyk

Well-Known Member
Mar 25, 2016
995
388
63
Stamford, CT
Yup. For me, I want to learn and experiment on how to do it, so I will be flashing mines personally. This is how I learned to flash all my old LSI cards. For me, it was a fun and cool experience. I picked up 4 x 40Gb HP cards for $15 per card = $60 total from a seller accepting best offers and will flash 'em when they arrive.

But I can see someone just wanting a pre-flashed working card, and that above auction in first post isn't half bad. $5 bucks more per card and its already flashed, plus you get to choose what bracket you need. Not bad at all. Cards have been going for around this price, so no loss to the buyer if they wanted to pick up a set of say, 2 x high profile cards + 2 x low profile cards.

For me, the extra $5 bucks saved per card is money to buy other things I need (cables, etc) but to a noob flasher, or to someone who just can't be bothered with wanting to learn it, the $5 bucks is worth it. Time is money too right? ;)