Mellanox CX556A-EDAM reads garbage from transceivers and does not establish link

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

NablaSquaredG

Layer 1 Magician
Aug 17, 2020
1,382
839
113
Hey,

So a couple of months ago I bought a Mellanox CX556A-EDAM (Yes, EDAM, not EDAT)

However; I'm having some issues with it now. When connecting it to another ConnectX-5 with 100GBase-CWDM4 transceivers, the EDAM claims to establish a link, but the other end does not.

And when I try to read the transceiver with ethtool -m, I get the following output:

Code:
sudo ethtool -m enp78s0f0
        Identifier                                : 0x11 (QSFP28)
        Extended identifier                       : 0x07
        Extended identifier description           : 1.5W max. Power consumption
        Extended identifier description           : No CDR in TX, CDR present in RX
        Extended identifier description           : 5.0W max. Power consumption,  High Power Class (> 3.5 W) not enabled
        Connector                                 : 0x00 (unknown or unspecified)
        Transceiver codes                         : 0x0f 0x00 0x0f 0x00 0x00 0x00 0x55 0x55
        Transceiver type                          : 40G Ethernet: 40G Base-CR4
        Transceiver type                          : 40G Ethernet: 40G Base-SR4
        Transceiver type                          : 40G Ethernet: 40G Base-LR4
        Transceiver type                          : 40G Ethernet: 40G Active Cable (XLPPI)
        Transceiver type                          : FC: Twisted Pair (TP)
        Transceiver type                          : FC: Video Coax (TV)
        Transceiver type                          : FC: Multimode, 50m (M5)
        Transceiver type                          : FC: Single Mode (SM)
        Transceiver type                          : FC: 800 MBytes/sec
        Transceiver type                          : FC: 400 MBytes/sec
        Transceiver type                          : FC: 200 MBytes/sec
        Transceiver type                          : FC: 100 MBytes/sec
        Encoding                                  : 0x00 (unspecified)
        BR, Nominal                               : 0Mbps
        Rate identifier                           : 0x00
        Length (SMF,km)                           : 0km
        Length (OM3 50um)                         : 0m
        Length (OM2 50um)                         : 0m
        Length (OM1 62.5um)                       : 0m
        Length (Copper or Active cable)           : 0m
        Transmitter technology                    : 0x00 (850 nm VCSEL)
        Laser wavelength                          : 1654.600nm
        Laser wavelength tolerance                : 0.000nm
        Vendor name                               : __$/___D________
        Vendor OUI                                : 0f:00:00
        Vendor PN                                 : _UU___________$/
        Vendor rev                                : __
        Vendor SN                                 : _____UU_________
        Date code                                 : __$/___D
...which is obviously nonsense

I have tried the following things:
  • Flash a stock CX556A-EDAT firmware (16.31.1014)
  • Reflash the firmware it came with (16.98.3042)
  • Force the link to PCIe3.0
None of this helped.

Interestingly, flint query shows the following:

Code:
PSID:                  MT_0000000078
Orig PSID:             MT_0000000009
MT_0000000009 is a standard CX556A-EDAT...

Also, the card does not have VPD!

Any ideas?
 

Freebsd1976

Active Member
Feb 23, 2018
395
73
28
Hey,

So a couple of months ago I bought a Mellanox CX556A-EDAM (Yes, EDAM, not EDAT)

However; I'm having some issues with it now. When connecting it to another ConnectX-5 with 100GBase-CWDM4 transceivers, the EDAM claims to establish a link, but the other end does not.

And when I try to read the transceiver with ethtool -m, I get the following output:

Code:
sudo ethtool -m enp78s0f0
        Identifier                                : 0x11 (QSFP28)
        Extended identifier                       : 0x07
        Extended identifier description           : 1.5W max. Power consumption
        Extended identifier description           : No CDR in TX, CDR present in RX
        Extended identifier description           : 5.0W max. Power consumption,  High Power Class (> 3.5 W) not enabled
        Connector                                 : 0x00 (unknown or unspecified)
        Transceiver codes                         : 0x0f 0x00 0x0f 0x00 0x00 0x00 0x55 0x55
        Transceiver type                          : 40G Ethernet: 40G Base-CR4
        Transceiver type                          : 40G Ethernet: 40G Base-SR4
        Transceiver type                          : 40G Ethernet: 40G Base-LR4
        Transceiver type                          : 40G Ethernet: 40G Active Cable (XLPPI)
        Transceiver type                          : FC: Twisted Pair (TP)
        Transceiver type                          : FC: Video Coax (TV)
        Transceiver type                          : FC: Multimode, 50m (M5)
        Transceiver type                          : FC: Single Mode (SM)
        Transceiver type                          : FC: 800 MBytes/sec
        Transceiver type                          : FC: 400 MBytes/sec
        Transceiver type                          : FC: 200 MBytes/sec
        Transceiver type                          : FC: 100 MBytes/sec
        Encoding                                  : 0x00 (unspecified)
        BR, Nominal                               : 0Mbps
        Rate identifier                           : 0x00
        Length (SMF,km)                           : 0km
        Length (OM3 50um)                         : 0m
        Length (OM2 50um)                         : 0m
        Length (OM1 62.5um)                       : 0m
        Length (Copper or Active cable)           : 0m
        Transmitter technology                    : 0x00 (850 nm VCSEL)
        Laser wavelength                          : 1654.600nm
        Laser wavelength tolerance                : 0.000nm
        Vendor name                               : __$/___D________
        Vendor OUI                                : 0f:00:00
        Vendor PN                                 : _UU___________$/
        Vendor rev                                : __
        Vendor SN                                 : _____UU_________
        Date code                                 : __$/___D
...which is obviously nonsense

I have tried the following things:
  • Flash a stock CX556A-EDAT firmware (16.31.1014)
  • Reflash the firmware it came with (16.98.3042)
  • Force the link to PCIe3.0
None of this helped.

Interestingly, flint query shows the following:

Code:
PSID:                  MT_0000000078
Orig PSID:             MT_0000000009
MT_0000000009 is a standard CX556A-EDAT...

Also, the card does not have VPD!

Any ideas?
maybe it is es version?
try dac ?