Flashing stock Mellanox firmware to OEM (EMC) ConnectX-3 IB/Ethernet dual-port QSFP adapter

[tommybackeast]

Please excuse some basic questions about to be asked; as "modern" Dell Servers are new for me.

Goal: use this card HP 649281-B21 Infiniband 10/40GB Dual Port Adapter Low Profile Bracket | eBay in a Dell T620 Tower Server; with above card connecting via fiber-optic cable to the 10GB port on the Brocade 6450. Right now, the Dell T620 has been defaulted back to 'factory'.

I am aware firmware for above card must be flashed, but confusion. When the card goes into the Dell T620, I plan to run Dell's version of ESXi 6.5 U2. I also have a Win7 Pro workstation.

I strongly believe in asking for help BEFORE I do something I don't know; rather than asking for help once i screwed things up.

So does the HP 649281-B21 Infiniband 10/40GB Dual Port card go into the Win7 Workstation and run Mellanox firmware flashing **selecting Win7 or ESXI ** (I would guess Win7 since its in a Win7 box; but once the firmware is flashed, it is going into the Dell T620 which then will get ESXi installed.
--or-- do I install the card as is, into the Dell T620, install Dell ESXI 6.5 U2; then flash the firmware using the ESXi version.

Lastly, I hope i typed my question(s) clearly enough and thank you.

 

[arglebargle]

You're overthinking things. The firmware for the card is universal, the OFED download is a suite of drivers and utilities for working with the card with whatever operating system the current machine is running. Think of OFED as a driver + tools package.

Put the card in the Win7 box, download OFED for windows and the firmware package then use MST and flint to flash the card. When you're done move the flashed card to your ESXi box and install the OFED support package so you have drivers. If you read back through the thread I'm pretty sure there are step by step recipes to flash the card. You shouldn't be able to do anything that will actually damage or brick the card with the firmware tools so just read the thread carefully, refer to the manual if needed (it's very complete, they cover everything comprehensively) and you'll be done in no time.

 

[tommybackeast]

Thank you VERY much. Yes, when there's a situation I am not 100% confident in, I do tend to overthink things. lol -

 

[tommybackeast]

Got it working in Windows 7 after changing the Ports to Ethernet. By default it is set to Auto.

PGlover : I also was running Win7 Pro box (to flash the firmware):

but I am failing on converting it from AUTO to ETHERNET (I did reboot and got the same message when trying a 2nd time to issue the below dos command)

C:\>cd program files
C:\Program Files>cd mellanox
C:\Program Files\Mellanox>mst status
MST devices:
------------
mt4099_pci_cr0
mt4099_pciconf0

C:\Program Files\Mellanox>mlxconfig -d mt4099_pci_cr0 set LINK_TYPE_P1=2 LINK_TY
PE_P2=2
-E- Failed to open device: mt4099_pci_cr0. Cannot perform operation, Driver migh
t be down.

C:\Program Files\Mellanox>

How did you get yours to change to ETHERNET?

 

[arglebargle]

Use the second device name "mt4099_pciconf0". I think I had this problem the first time I did this too, it's a little weird in that some things work with both and others don't.

 

[tommybackeast]

Use the second device name "mt4099_pciconf0". I think I had this problem the first time I did this too, it's a little weird in that some things work with both and others don't.

Thank you, I won't be in front of the hardware to try your command for an hour+.

It was suggest to me that I also ask my followup question to you re ESXi. Right now, the card is in a Win7 box to do the flashing. Once that is done; it will be removed and I shall power down the Dell T620 Tower Server to insert the card. Note: I will be using this card in 10GB and connecting it via fibercables to a brocade 6450 10gb port. ESXi is brand new for me (just installed Dell's ESXi 6.5U2 on the Dell T620's internal Thumbdrive port).

Since weather has ruined my evening, I had planned to read the ESXi manual for the first time tonight re hardware installation. My first question would be : insert the card by itself and power Server back on -or- insert card, its HP Mellanox 655874-B21 QSFP/SFP Adapter ; with the fiber optical transceiver, and fiber-optic cable inserted into the switch and THEN turn the Server back on

 

[Hindsight]

You can put and pull the transceiver if it's on. No big deal. Long as server is off when you put the card in.

 

[tommybackeast]

Use the second device name "mt4099_pciconf0". I think I had this problem the first time I did this too, it's a little weird in that some things work with both and others don't.

Did not work


C:\Program Files\Mellanox>mst status
MST devices:
------------
mt4099_pci_cr0
mt4099_pciconf0

C:\Program Files\Mellanox>mlxconfig -d mt4099_pciconf0 set LINK_TYPE_P1=2 LINK_T
YPE_P2=2
-E- Failed to open device: mt4099_pciconf0. Cannot perform operation, Driver mig
ht be down.

C:\Program Files\Mellanox>

 

[tommybackeast]

You can put and pull the transceiver if it's on. No big deal. Long as server is off when you put the card in.

Thank you. ESXI is new for me, modern Dell Servers are new for me, fiber optical transceivers are new for me, etc = hence, many questions and I truly appreciate the outstanding help from members on this board.

 

[arglebargle]

Try "mlxconfig q" with nothing else and see if it returns adapter status.

 

[tommybackeast]

Try "mlxconfig q" with nothing else and see if it returns adapter status.

C:\Program Files\Mellanox>mst status
MST devices:
------------
mt4099_pci_cr0
mt4099_pciconf0

C:\Program Files\Mellanox>
C:\Program Files\Mellanox>mlxconfig q
-E- Failed to open device: mt4099_pciconf0. Cannot perform operation, Driver mig
ht be down.

C:\Program Files\Mellanox>

 

[tommybackeast]

Try "mlxconfig q" with nothing else and see if it returns adapter status.

C:\Program Files\Mellanox>mlxfwmanager --query
Querying Mellanox devices firmware ...

Device #1:
----------

Device Type: ConnectX3
Part Number: MCX354A-FCB_A2-A5
Description: ConnectX-3 VPI adapter card; dual-port QSFP; FDR IB (56Gb/s)
and 40GigE; PCIe3.0 x8 8GT/s; RoHS R6
PSID: MT_1090120019
PCI Device Name: mt4099_pci_cr0
Port1 MAC: --removed
Port2 MAC: --removed
Versions: Current Available
FW 2.42.5000 2.42.5000
PXE N/A 3.4.0752

Status: Up to date


C:\Program Files\Mellanox>

 

[arglebargle]

Try "mstconfig q" maybe? I've never had mlxconfig do that on a CX-3.

 

[tommybackeast]

Will these network adapters card work in a Windows 7 PC? I am having a hard time finding the Ethernet drivers for a Windows 7 OS.

Do you recall your commands to flash this card in the your Win7 Box.

I seem to be having the same problem you did; but it seems you solved your issue, may I ask how? thanks

 

[tommybackeast]

Try "mstconfig q" maybe? I've never had mlxconfig do that on a CX-3.

C:\Program Files\Mellanox>
C:\Program Files\Mellanox>
C:\Program Files\Mellanox>mstconfig q
'mstconfig' is not recognized as an internal or external command,
operable program or batch file.

 

[tommybackeast]

Try "mstconfig q" maybe? I've never had mlxconfig do that on a CX-3.

fohdeesha (in a PM) shared his thoughts that the way the card is now flashed, it will AUTO select its speed, based on what connectors are used. Worthy experiment. so my plan now is to : 1. Shut down win7 box and remove the card. 2) shut down Dell Server (do I need to pull the power plugs?) 3) insert card into Dell Server which has Dell ESXI 6.5U loaded onto internal USB thumbdrive. I also have iDRAC7 enterprise if that matters. 4) insert the GENUINE 655902-001 655874-B21 HP MELLANOX QSFP/SFP+ ADAPTER KIT 886111656585 | eBay

into the card, insert the Brocade optical 10GB TRanceivers into above adapter and into switch; and insert the 5m Fiber cable connecting this card to Brocade 6450 10gb port. 5) Turn on Dell Server power 6) then launch ESXi from Win7 browser and try to figure out how to install the card driver

GENUINE 655902-001 655874-B21 HP MELLANOX QSFP/SFP+ ADAPTER KIT 886111656585 | eBay

 

[arglebargle]

Oh, it'll almost certainly autodetect ethernet -- all of mine do when plugged into ethernet devices -- I just thought you had some reason to set the config parameters manually.

If there's an OFED package available for ESXi (I haven't looked, I don't use it) then you can change card configuration on that machine too.

 

[nezach]

I was configuring my card on linux using latest MFT package and everything was working great until I enabled SR-IOV using mlxconfig utility. Once I enabled that and rebooted I started getting same message:

-E- Failed to open device: mt4099_pciconf0. Cannot perform operation, Driver might be down.

I then disabled SR-IOV using Flexboot config screen and mlxconfig started working fine again.

 

[nthu9280]

@tommybackeast et al
I finally got around to started flashing the cards that have been sitting in a box for a long time. Got the same error in windows. tried admin / elevated powershell and not getting the error and was able to successfully flash first one.

 

[tommybackeast]

@tommybackeast et al
I finally got around to started flashing the cards that have been sitting in a box for a long time. Got the same error in windows. tried admin / elevated powershell and not getting the error and was able to successfully flash first one.

Thanks, I wonder why powershell worked