Confusing Mellanox Mezzanine Cards for C6320

[CA_Tallguy]

I've purchased a couple mezzanine cards off ebay and am a bit confused. The BCAA/3CYRK I believe was originally sold by Dell for the C6220 and I bought 3 of these to try in my C6320 sleds. THEY DO WORK (use 4NWN5 bridge card) and they come up as Connectx-3 PRO and MT27520 family.

Since I wasnt postitve they would work I ALSO bought FCAA/Y5WKX off eBay. This is supposed to be a better card I think, with VPI, and originally sold for C6320. But this card DOES NOT SHOW PRO and is coming up as MT27500 family.

This is completely baffling. Shouldn't the FCAA card with VPI show as PRO as well? From what I can tell in web research, MT27500 family is not a PRO card.

I'm wondreing -- did someone flash my BCAA/3CYRK's with better firmware? Or did I get sold a counterfeit or reflashed FCAA/Y5WKX? Or perhaps all is well and FCAA/Y5WKX is indeed VPI and is NOT a pro card?

In any case WHICH IS THE BETTER CARD? I don't know enough about VPI to know if I need it yet. But PRO sounds better than not pro to me! But what I thought was the better card and I paid more for does NOT show as pro.

The MCX384A-BCAA // Dell part 3CYRK cards come up in VMware as:

Mellanox Technologies MT27520 Family [ConnectX-3 Pro]

MT27520 Family [ConnectX-3 Pro]
ID 0000:02:00.0
Device ID 0x1007
Vendor ID 0x15b3
Function 0x0
Bus 0x2
Vendor Name Mellanox Technologies
Class ID 0x200
Subdevice ID 0x10
Subvendor ID 0x15b3
Slot 0x0

The MCX384A-FCAA // Dell part Y5WKX cards come up in VMware as:

Mellanox Technologies MT27520 Family [ConnectX-3] <<< NOT PRO!!

MT27500 Family [ConnectX-3] <<< DIFFERENT FAMILY AND NOT SHOWING "PRO"
ID 0000:02:00.0
Device ID 0x1003
Vendor ID 0x15b3
Function 0x0
Bus 0x2
Vendor Name Mellanox Technologies
Class ID 0x280
Subdevice ID 0x71
Subvendor ID 0x15b3
Slot 0x0

Side note: you need to use the 4NWN5 bridge/interposer board for these on a C6320 sled! There are many that look similar but they are NOT the same....

Differences in Mezzanine Bridge Cards for PowerEdge C | DELL Technologies

I'm trying to figure out which of the following bridge cards is the right one to use for installing Mellanox part MCX384A-BCAA // Dell part 3CYRK into C6320.... (and if 3CYRK is com...

www.dell.com

 

[CA_Tallguy]

I am continuing my saga with these cards and trying to update firmware. So far cannot find it but have inquired on Dell forums.

Here's a repost from that thread of what I have encountered so far (so this info will be archived for others here as well). To run these updates on linux, you simply download the file and chmod +x then run it as root. It will update or spit out errors if you don't pass their system screening. But I have also learned that you can add an "-- extract" option (double dash) along with a name for a folder to extract the content files. So if I can find the appropriate firmware I can try to use the Mellanox tools to update the cards. Hopefully. There is another flag to add to "disable security" in the update but I am not sure if that will bypass the system check. And in any of these scenarios, it would be very good to be certain that you are attempting to flash the appropriate FW, including that it will work on a mezzanine card (vs regular PCIe).

I believe that I am looking for the firmware BIN files where the payloads inside are meant for or compatible with the 3CYRK and Y5WKX cards.

I have tried a bunch of updates but none of them work, at least as BIN files. Mostly I am getting stopped because my system configuration does not validate (no more specific info is given by the installer)....

Network_Firmware_RN0JN_LN_02.42.50.00.BIN
Network_Firmware_P29N2_LN_02.42.50.00.BIN
Network_Firmware_M8H5D_LN_02.42.50.00.BIN
Network_Firmware_84DJH_LN_02.42.50.00.BIN
Network_Firmware_5RPT5_LN_02.33.50.50.BIN

I always get error "This Update Package is not compatible with your system configuration" except one update I tried for C6220 that complained more specifically that the problem was my system was C6320.

So for all the other update attempts, including those above/below, the problem may not be my C6320 but another configuration item.... even the wrong card. (And actually most of the payloads appear to be for standard PCIe cards, not mezzanine cards, which is potentially a very significant difference) I think 5RPT5 only contained FW for the non-PRO card and I was trying to update PRO so that may have been reason for failure on that one.

But also, looking at the logs for the updates suggests that these packages contain custom firmware for specific Dell product numbers and none have matched 3CYRK. (See list of firmware packages and firmware payloads below.

My plan is to extract the appropriate firmware for the 3CYRK and Y5WKX when I find those packages or packages that are supposedly compatible. AFAIK there is no way for customers to search for the firmware by the NIC product number.

5RPT5: fw-ConnectX3-rel_06256K_02.33.50.50.bin

5RPT5: fw-ConnectX3-rel_08KP6W_0M9NW6_02.33.50.50.bin

5RPT5: fw-ConnectX3-rel_0W0RM9_0Y3KKR_02.33.50.50.bin

5RPT5: fw-ConnectX3-rel_06256K_02.33.50.50.bin

5RPT5: fw-ConnectX3-rel_08KP6W_0M9NW6_02.33.50.50.bin

5RPT5: fw-ConnectX3-rel_0W0RM9_0Y3KKR_02.33.50.50.bin

84DJH : fw-ConnectX3Pro-rel-2_42_5000-019RNV_0YHTD6-UEFI-15.11.45-FlexBoot-3.4.752.bin

84DJH : fw-ConnectX3Pro-rel-2_42_5000-02T0WW-CLP-8006.0.0-UEFI-15.11.45-FlexBoot-3.4.752.bin

84DJH : fw-ConnectX3Pro-rel-2_42_5000-0C8Y42_0R3F0N-UEFI-15.11.45-FlexBoot-3.4.752.bin

84DJH : fw-ConnectX3-rel-2_42_5000-06256K-CLP-8005-UEFI-15.11.45-FlexBoot-3.4.752.bin

84DJH : fw-ConnectX3-rel-2_42_5000-08KP6W_0M9NW6-UEFI-15.11.45-FlexBoot-3.4.752.bin

84DJH : fw-ConnectX3-rel-2_42_5000-0W0RM9_0Y3KKR-UEFI-15.11.45-FlexBoot-3.4.752.bin

M8H5D : fw-ConnectX3Pro-rel-2_42_5000-019RNV_0YHTD6-UEFI-15.11.45-FlexBoot-3.4.752.bin

M8H5D : fw-ConnectX3Pro-rel-2_42_5000-02T0WW-CLP-8006.0.0-UEFI-15.11.45-FlexBoot-3.4.752.bin

M8H5D : fw-ConnectX3Pro-rel-2_42_5000-0C8Y42_0R3F0N-UEFI-15.11.45-FlexBoot-3.4.752.bin

M8H5D : fw-ConnectX3-rel-2_42_5000-06256K-CLP-8005-UEFI-15.11.45-FlexBoot-3.4.752.bin

M8H5D : fw-ConnectX3-rel-2_42_5000-08KP6W_0M9NW6-UEFI-15.11.45-FlexBoot-3.4.752.bin

M8H5D : fw-ConnectX3-rel-2_42_5000-0W0RM9_0Y3KKR-UEFI-15.11.45-FlexBoot-3.4.752.bin

P29N2 : fw-ConnectX3Pro-rel-2_42_5000-019RNV_0YHTD6-UEFI-15.11.45-FlexBoot-3.4.752.bin

P29N2 : fw-ConnectX3Pro-rel-2_42_5000-02T0WW-CLP-8006.0.0-UEFI-15.11.45-FlexBoot-3.4.752.bin

P29N2 : fw-ConnectX3Pro-rel-2_42_5000-0C8Y42_0R3F0N-UEFI-15.11.45-FlexBoot-3.4.752.bin

P29N2 : fw-ConnectX3-rel-2_42_5000-06256K-CLP-8005-UEFI-15.11.45-FlexBoot-3.4.752.bin

P29N2 : fw-ConnectX3-rel-2_42_5000-08KP6W_0M9NW6-UEFI-15.11.45-FlexBoot-3.4.752.bin

P29N2 : fw-ConnectX3-rel-2_42_5000-0W0RM9_0Y3KKR-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3Pro-rel-2_42_5000-019RNV_0YHTD6-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3Pro-rel-2_42_5000-02T0WW-CLP-8006.0.0-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3Pro-rel-2_42_5000-0C8Y42_0R3F0N-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3-rel-2_42_5000-06256K-CLP-8005-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3-rel-2_42_5000-08KP6W_0M9NW6-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3-rel-2_42_5000-0W0RM9_0Y3KKR-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3Pro-rel-2_42_5000-019RNV_0YHTD6-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3Pro-rel-2_42_5000-02T0WW-CLP-8006.0.0-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3Pro-rel-2_42_5000-0C8Y42_0R3F0N-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3-rel-2_42_5000-06256K-CLP-8005-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3-rel-2_42_5000-08KP6W_0M9NW6-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3-rel-2_42_5000-0W0RM9_0Y3KKR-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3Pro-rel-2_42_5000-019RNV_0YHTD6-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3Pro-rel-2_42_5000-02T0WW-CLP-8006.0.0-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3Pro-rel-2_42_5000-0C8Y42_0R3F0N-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3-rel-2_42_5000-06256K-CLP-8005-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3-rel-2_42_5000-08KP6W_0M9NW6-UEFI-15.11.45-FlexBoot-3.4.752.bin

RN0JN : fw-ConnectX3-rel-2_42_5000-0W0RM9_0Y3KKR-UEFI-15.11.45-FlexBoot-3.4.752.bin

 

[CA_Tallguy]

Here's a table I'm starting with information on various Dell cards.

I **THINK** these are all mezzanine cards. I'm having trouble finding firmware for my 3CYRK so maybe I will try to force 1965W update if I can find that firmware as they are as close as you can get to the same thing AFAIK. I wonder what the difference is between the two versions they have for BCAA and FCAA models. Perhaps the added 1 in the PSID means they came after the 0 versions?

Mellanox has a Dell firmware page at this link but it is not comprehensive. They also have an archive of outdated versions. As of this writing, the only firmware they have for the DEL111000 range cards is for the Y5WKX which is the extra "1" PSID version of that card. Too bad they don't have it for the BCAA "1" version too as that's what I need.

I'll try to fill out this table and link firmware as I come across more info.

DELL#	MELLANOX#	FAMILY	Device, Subdev, Class	PSID	FIRMWARE
3CYRK - PRO	MCX384A-BCAA	MT27520 [ConnectX-3 Pro]	0x1007 0x10 0x200	DEL1110001023
1965W - PRO?	MCX384A-BCAA			DEL1110000023
Y5WKX - VPI	MCX384A-FCAA	MT27520 [ConnectX-3]	0x1003 0x71 0x280	DEL1110001019	2.42.5000
K75R1 - VPI	MCX384A-FCAA			DEL1110000019
X8GJ4	MCX343A-XCAA			DEL1120000023
T483W	MCX383A-FCxx			DEL0A30000019