Mellanox SX6012, HP 544+ QSFP+ and 40GBASE-LR4 modules

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
M

Matta

Member
Oct 16, 2022
36
6
8
Hello,

Recently I bought Mellanox SX6012 and successfully converted it to Ethernet mode.
I have bunch of HP 544+ QSFP+ with FlexLOM to PCIe adapters that I'll be using with DACs and (hopefully) with 40GBASE-LR4 modules - specifically these:
AOI | AQOLLCQ4EDMA0662 | 40G 1310nm Transceiver Module | eBay

I've put one of these card into my windows computer, everything is working fine with QSFP+ DACs.
However, If I connect my computer and SX6012 with two of those modules I get no connection.
Mellanox_AOI.jpg

I'm not sure what seems to be the problem. Cable is 10m LC/LC duplex SMF.
 
M

Matta

Member
Oct 16, 2022
36
6
8
Thanks for the fast reply !

Would be great if you could point me to correct thread (or other source) with some kind of guide to do this kind of thing.
I've done SATA HBA cards flashing but never this.
Thanks once more !
 
M

Matta

Member
Oct 16, 2022
36
6
8
Before I dwell further into flashing, can I try loopback test on Mellanox itself - put two of these module in slots and connect them with cable ? Will that work ?
 
M

Matta

Member
Oct 16, 2022
36
6
8
I just connected and port light won't turn on, same port status as above. If I do loopback with DAC port lights turn on and there is connection. It seems it's not issue with card in computer.
Modules do get recognized in ports, as seen above in picture.
 
K

klui

Well-Known Member
Feb 3, 2019
846
463
63
Right type of cable and polarity looks correct. There are several possibilities:
  • dirty transceivers/ferrules - use click cleaner and for home use, optionally use inspection scope. Just because parts are new doesn't mean they're clean--in fact most new parts are dirty
  • defective cable - use VFL to check, can also use to confirm polarity
  • cable too short and you need an attenuator for LR optics - only power meter with transceiver datasheet will tell if this is the case
Never used this company's transceivers before.

EDIT: Per @Stephan I would also try flashing with stock Mellanox firmware. I've never used these FlexLOM cards before but I recall when my CX3/3-Pros didn't work ethtool showed the transceivers but they would not link up with what acted like modified Mellanox firmware (newer FW version from what Mellanox has on their website and card has PSID starting wht MT_).
 
Last edited:
  • Like
Reactions: Stephan
Stephan

Stephan

Well-Known Member
Apr 21, 2017
945
714
93
Germany
Also don't use Windows, get a recent Linux ISO for techies (Kali LinuxRescue etc.) put on USB drive prepared with Ventoy and check ethtool if transceivers show up at all. Some older transceivers get very hot and need good cooling, Mellanox original firmware doesn't care, maybe HP got too many support calls and decided to put in a whitelist of transceivers they could verify cooling for.
 
  • Like
Reactions: klui
K

klui

Well-Known Member
Feb 3, 2019
846
463
63
Shit I'm getting old. I typed FlexLOM but brain didn't register OP's is HP-branded. My HP-branded CX3 won't link with Arista and Intel transceivers, but as I wrote above even though the transceivers are shown using ethtool. Flashed to stock Mellanox fixed that. I was pulling out my hair blaming my my cable plant for days until I performed a simple loopback test--the disadvantage of having too much hardware and playing musical chairs with them.
 
M

Matta

Member
Oct 16, 2022
36
6
8
NablaSquaredG said:
try

fae cable-stamping-unlock 40g_lr4

on the switch
Click to expand...
Wohooo, progress ! Thank you mr. NablaSquaredG ! :)
Just got successful loopback connection on two switch ports, it seems cable and modules are good.
No connection with card though, going to reflash it to stock Melannox.
 
  • Like
Reactions: NablaSquaredG
M

Matta

Member
Oct 16, 2022
36
6
8
I've successfully flashed the card to Mellanox firmware, set ports to ethernet but still have no connection. :(

Am I still missing something ?
Mellanox_card.jpg

Edit:
With this reflashing now even DAC won't work.
Back to square one.

Edit2:
Reverted to HP firmware, DAC working again, modules not working.
Suggestions ?
Maybe try firmware from other brand (IBM, Dell, ...) ? I'm not sure if it's even possible.
 
Last edited:
M

Matta

Member
Oct 16, 2022
36
6
8
I don't hate it, I just don't have enough hours on it to be comfortable. :D
Anyway, this particular card is in my Windows PC anyway. :)
 
You must log in or register to reply here.
Top Bottom