Strange Mellanox problem. No eth, only IB. Flash don't work.

[DouglasteR]

As the title summarize, i have an Oracle CX354A dual port that for some reason is hard locked in IB.




Even if i flash it (and i've tried dozens of firmwares) with eth image the adapter simply don't work, the interface stay online but braindead:


I've tried 4 sets of QSFP->SFP+->10gb and 5gb UTP adapters to no avail.

These sets are working fine in other mellanoxs.

Can anyone shed some light ? I don't want to believe that this is incapable of ethernet. I have one visually the same working just fine.

Thanks for the help guys.

 

[Blue)(Fusion]

Did you catually set the configuration of the port(s)?

To check the setting on Port 1:

mstconfig -d pci0:1:0:0 q LINK_TYPE_P1

To set Port 1 to Ethernet mode on next boot:

mstconfig -d pci0:1:0:0 s LINK_TYPE_P1=ETH

Note, this is on FreeBSD with mstflint and you'll have to use the appropriate PCI address for your setup.

 

[DouglasteR]

Already tried ! It indeed confirms it is in eth mode, but it stays braindead.

When the card arrived, i put it directly in Pfsense and when i tried to switch from ib to eth the adapter rejected the change.

 

[Sean Ho]

FCBT firmware 2.42, as shown by mstflint q ? PSID set correctly during flash?

Noticing you're using SFP+/RJ45 transceivers; is the device on the other end of the link 10GbaseT, or 2.5/5?

 

[i386]

These sets are working fine in other mellanoxs.

From the same batch/seller or from different ones?
Did you try if it works in infiniband mode? Maybe you got a dead card from the seller...

 

[DouglasteR]

FCBT firmware 2.42, as shown by mstflint q ? PSID set correctly during flash?

Noticing you're using SFP+/RJ45 transceivers; is the device on the other end of the link 10GbaseT, or 2.5/5?

IRRC i overrided the PSID (with the flag). But i´ve saved the original in the .ini.

The other end device is a SFP+ mikrotik switch.

From the same batch/seller or from different ones?
Did you try if it works in infiniband mode? Maybe you got a dead card from the seller...

Hmm, they are from the same seller, but different batches.

I will try to connect it directly to another mellanox tonight and report back.

Thanks guys.

 

[DouglasteR]

Doing a loopback with the QSFP+ cable ! It seems to detect the link !



And in Windows, the adapter now seems operational !?:



Interfaces are configured as 10.0.0.1 and .2 .

Strange, maybe it really didn't like my QSFP->SFP+ adapters

 

[Stephan]

maybe it really didn't like my QSFP->SFP+ adapters

Adapter was MAM1Q00A-QSA or HP 655874-B21? Those should work. In those I have Cisco GLC-T 30-1410-03 V3 to connect slow gear.

 

[DouglasteR]

I'm back guys, sorry for the delay.

Tested multiple Gbic and none worked.

Then i tested with a twinax and it instantly connected at 10gbps. So it CAN'T do anything bellow 10GB/40GB. 1, 2,5 and 5 don't work !

Thanks for the help and patience ! SOLVED !