SOLVED Mellanox SX6036 + LR4 Transceivers does not work

[Blue)(Fusion]

I'd just like to report that the CX4 came in today and it and the SX6036 work fine with the Cisco QSFP-40G-SR-BD transceivers and 100m OM4.

 

[Benno]

I'd just like to report that the CX4 came in today and it and the SX6036 work fine with the Cisco QSFP-40G-SR-BD transceivers and 100m OM4.

What is the power consumption of that Cisco QSFP-40G-SR-BD transceiver in your situation?

 

[Blue)(Fusion]

I don't know actual power draw, but literature says this model uses 3.5 watts. It gets quite hot in my desktop and I added some small heatsinks to the exposed external surfaces of the transceiver and a 40mm fan to the CX4 chipset heatsink to improve cooling and this seems to have helped alot, atleast to my finely tuned fingers.

 

[luckylinux]

Just a word of caution for those, like me, that naively thought that also CWDM4 QSFP28 Transceivers could be used.

Unfortunately not with the Mellanox SX6036 Switch.

I naively thought that CWDM4 being also a Long Range Variant should be compatible with the Switch.

Big Mistake . The Switches will refuse to link to each other (2 x Mellanox SX6036 connected together), either in Ethernet or Infiniband Mode, either in Auto-Negotiation or forced FDR / QDR Speed, etc.

Now I need to buy some LR4 Modules instead .

 

[NablaSquaredG]

Yeah that happens from time to time.

If you have the possibility to code the transceiver, code it to 40G-LR4 with Mellanox OUI, and it will very likely just work.

I had to do that on Mellanox SN3700s too because Cumulus refused to force a 100G transceiver to 40G... But the other side, Arista 7050TX, was happily using the CWDM4 at 40G.

 

[luckylinux]

If you have the possibility to code the transceiver, code it to 40G-LR4 with Mellanox OUI, and it will very likely just work.

How would I even do that ? Where ? Is it a special Firmware Programmer Kit or something ?

While Google AI Search is Garbage 80+% of the Time, so not to be taken as an absolute Source of Truth by any Means, it seemed to suggest some Issues related to "Protocol" and that thee CWDM4 Transceiver has no Fallback Mechanism. It also seems to require RS-FEC for Error correction, which the Mellanox SX6036 lacks.

switch-104 [standalone: master] (config interface ethernet 1/1) # show interfaces ethernet 1/1

Eth1/1:
  Admin state                               : Enabled
  Operational state                         : Down
  Last change in operational status         : 0:07:05 ago (20 oper change)
  Boot delay time                           : 0 sec
  Description                               : N\A
  Mac address                               : 00:02:c9:a8:2e:ac   
  MTU                                       : 1500 bytes (Maximum packet size 1522 bytes)
  Fec                                       : auto
  Flow-control                              : receive off send off
  Actual speed                              : 40 Gbps
  Auto-negotiation                          : Disabled
  Width reduction mode                      : Unknown
  Switchport mode                           : access
  MAC learning mode                         : Enabled
  Last clearing of "show interface" counters: Never
  60 seconds ingress rate                   : 0 bits/sec, 0 bytes/sec, 0 packets/sec
  60 seconds egress rate                    : 0 bits/sec, 0 bytes/sec, 0 packets/sec

  Rx:
    0                     packets
    0                     unicast packets
    0                     multicast packets
    0                     broadcast packets
    0                     bytes
    0                     discard packets
    0                     error packets
    0                     fcs errors
    0                     undersize packets
    0                     oversize packets
    0                     pause packets
    0                     unknown control opcode
    0                     symbol errors

  Tx:
    0                     packets
    0                     unicast packets
    0                     multicast packets
    0                     broadcast packets
    0                     bytes
    0                     discard packets
    0                     error packets
    0                     hoq discard packets

switch-104 [standalone: master] (config interface ethernet 1/1) # show interfaces ethernet 1/1 transceiver
Port 1/1 state
        identifier             : QSFP28
        cable/module type      : Optical module
        ethernet speed and type: 100G CWDM4 MSA, FEC based
        vendor                 : INNOLIGHT
        supported cable length : 2000m SMF
        part number            : TR-FC13T-NFB
        revision               : 1A
        serial number          : INIAF7401353

 

[NablaSquaredG]

How would I even do that ? Where ? Is it a special Firmware Programmer Kit or something ?

One possibility is the stuff from REVELPROG, e.g. Programmer Revelprog-IS + SFP QSFP Adapter

It also seems to require RS-FEC for Error correction, which the Mellanox SX6036 lacks.

It's a bit more complicated than that - but the TL;DR is that the end devices themselves (e.g. switch and NIC) set / negotiate that. FEC can mostly be turned off, and transceivers are not inherently aware whether FEC is being used or not.

related to "Protocol" and that thee CWDM4 Transceiver has no Fallback Mechanism.

Not sure what you mean - most CWDM4 transceivers are dual rate 40G / 100G, and I think I've had that exact model you're quoting above running at 40G.

 

[luckylinux]

One possibility is the stuff from REVELPROG, e.g. Programmer Revelprog-IS + SFP QSFP Adapter

It's a bit more complicated than that - but the TL;DR is that the end devices themselves (e.g. switch and NIC) set / negotiate that. FEC can mostly be turned off, and transceivers are not inherently aware whether FEC is being used or not.

FEC not being a Deal breaker is definitively a positive Sign, thanks .

However 200+ EUR for that Kit is a lot.

Aliexpress has something comparable-ish for 90 EUR all inclusive, not sure how good or bad it is though:

https://www.aliexpress.com/item/1005006659842448.html

https://www.aliexpress.com/item/1005010796102652.html

As well as

Please contact with us for the software ,We will send the software by email!

Not sure what you mean.

I think the AI mentioned that the Clock / Data Rate on the Electrical Side (inside the Switch) was hardcoded to 25gbps / Lane in the case of CWDM4 Transceivers.

Most CWDM4 transceivers are dual rate 40G / 100G, and I think I've had that exact model you're quoting above running at 40G.

It could be, I just fail to find a Datasheet stating that and Google AI once again keeps insisting:

The InnoLight TR-FC13T-NFB is a native 100G (4x25G) transceiver and does not natively support 40G speed (such as 40GBASE-LR4 or 40G QSFP+).

Here is what you need to know about 40G compatibility and your options:

Why It Is Not Directly Compatible

- Rate Mismatch: The internal clock, data rates (25 Gbps per lane vs. 10 Gbps per lane), and firmware are locked to 100G operations
- Wavelength Differences: 100G CWDM4 uses 1271–1331 nm grid wavelengths, whereas typical 40G LR4 optics use a different LWDM grid (1271–1331 nm but with different channel spacing and optical specifications).

Switch Port Compatibility (Backward Compatibility)
While the transceiver cannot change its speed, your network switch port might support 40G:
- Dual-Rate Ports: Many 100G QSFP28 ports on switches (like Cisco, Arista, or Juniper) can be configured to run at 40G speeds
- The Catch: To use a port at 40G, you must replace the 100G transceiver with a dedicated 40G QSFP+ transceiver (e.g., 40G-LR4 or 40G-PLR4).

Now the Wavelength is just BS, as two of the same Transceivers will just talk to each other in the same Manner, so for me the Problem should only be on the Electrical Side (Protocol, Frequency, Parity/Error Correction, etc).

The AI was however quite optimistic about reprogramming some Registers for some weird Reason (which is usually a Sign of Hallucination):

switch-104 [standalone: master] # show interfaces ethernet 1/1 transceiver raw
Port 1/1 raw transceiver data:

I2C Address 0x50, Page 0, 0:255:
  0000  11 07 00 ff 00 ff 01 00 00 55 55 55 00 55 55 00  .........UUU.UU.
  0010  00 00 00 00 00 00 27 2e 00 00 7f 0b 00 00 00 00  ......'.........
  0020  00 00 18 e8 1c 47 1b dd 27 af 4e 6e 4e aa 4e d2  .....G..'.NnN.N.
  0030  4e 4b 29 ae 24 45 37 13 25 2b 00 00 00 00 00 00  NK).$E7.%+......
  0040  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0050  00 00 00 00 00 00 00 aa aa 00 00 00 00 01 00 00  ................
  0060  00 00 ff 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0070  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0080  11 cc 07 80 00 00 00 00 00 00 00 07 ff 02 02 00  ................
  0090  00 00 00 40 49 4e 4e 4f 4c 49 47 48 54 20 20 20  ...@INNOLIGHT   
  00a0  20 20 20 20 00 44 7c 7f 54 52 2d 46 43 31 33 54      .D|.TR-FC13T
  00b0  2d 4e 46 42 20 20 20 20 31 41 65 a4 39 08 46 12  -NFB    1Ae.9.F.
  00c0  06 07 ff fa 49 4e 49 41 46 37 34 30 31 33 35 33  ....INIAF7401353
  00d0  20 20 20 20 31 38 30 32 32 32 20 20 0c 18 67 4e      180222  ..gN
  00e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
I2C Address 0x50, Pages 1, 128:255:
  0080  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0090  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00b0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00c0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
I2C Address 0x50, Pages 2, 128:255:
  0080  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0090  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00b0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00c0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00e0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
I2C Address 0x50, Pages 3, 128:255:
  0080  3a 00 fa 00 35 00 fd 00 00 00 00 00 00 00 00 00  :...5...........
  0090  90 88 71 48 8c a0 75 30 00 00 00 00 00 00 00 00  ..qH..u0........
  00a0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00b0  6e 17 00 b1 57 73 01 62 92 7c 13 88 88 b8 1d 4c  n...Ws.b.|.....L
  00c0  6e 17 02 c4 62 1f 04 62 00 00 00 00 00 00 00 00  n...b..b........
  00d0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  00e0  a7 07 00 00 00 00 00 00 00 00 11 11 00 00 22 22  ..............""
  00f0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00


switch-104 [standalone: master] # show interfaces ethernet 1/1 transceiver
Port 1/1 state
        identifier             : QSFP28
        cable/module type      : Optical module
        ethernet speed and type: 100G CWDM4 MSA, FEC based
        vendor                 : INNOLIGHT
        supported cable length : 2000m SMF
        part number            : TR-FC13T-NFB
        revision               : 1A
        serial number          : INIAF7401353

Though with its Track Record the Changes of this going smoothly are very close to zero .

 

[nasbdh9]

While 99% of the DSP chips on 100G CWDM4 chipset support 40G, encoding support and switch compatibility are required, as is the problem you're currently experiencing.

Multimode BIDI and singlemode LR4 40G transceivers are currently more expensive than 200G or 400G SR4 modules (excluding those nearly unusable secondhand junk), so if you don't have cabling constraints, please buy a 40G SR4 transceiver (MPO-12) or DAC cable.

 

[luckylinux]

While 99% of the DSP chips on 100G CWDM4 chipset support 40G, encoding support and switch compatibility are required, as is the problem you're currently experiencing.

Multimode BIDI and singlemode LR4 40G transceivers are currently more expensive than 200G or 400G SR4 modules (excluding those nearly unusable secondhand junk), so if you don't have cabling constraints, please buy a 40G SR4 transceiver (MPO-12) or DAC cable.

I just pulled a 20 meter SMF (Single Mode Fiber) through my Loft ... 20 Meter DAC isn't an Option.

Between Switch and NIC, yeah, DAC all the Way, though those are THICK Cables.

For long Runs, Fiber is the only Option.

 

[nasbdh9]

I just pulled a 20 meter SMF (Single Mode Fiber) through my Loft ... 20 Meter DAC isn't an Option.

Between Switch and NIC, yeah, DAC all the Way, though those are THICK Cables.

For long Runs, Fiber is the only Option.

Considering replace to MPO cables? 40G transceivers with MPO interfaces are significantly cheaper and readily available inexpensive new stock (multimode SR4 or even singlemode PSM4).

 

[luckylinux]

Considering replace to MPO cables? 40G transceivers with MPO interfaces are significantly cheaper and readily available inexpensive new stock (multimode SR4 or even singlemode PSM4).

Have you looked at how much MPO Breakout Cables cost ? Let alone that I would need to pull like 4x24F or more to achieve the same as I have now ?

I saw some LR4 Lite for like 30 EUR / each on eBay. That's relatively expensive but Way less than those MPO Cables that take a bunch of Fibers.

 

[nasbdh9]

I saw some LR4 Lite for like 30 EUR / each on eBay. That's relatively expensive but Way less than those MPO Cables that take a bunch of Fibers.

The significant price difference of MPO cables across different regions is indeed a problem.

 

[luckylinux]

Any other Idea before I try to purchase some other Transceivers or those Programmers ? Either Solution sucks honestly .

 

[luckylinux]

I finally bit the bullet and bought some Finisar FTL4C1QL1L LR4 Transceiver.

After forcing Port 1 to Ethernet Mode on all 4 Switches, it finally linked, basically it worked out of the Box

Not sure why in Default/Infiniband Mode it would stay down, could it be that it does NOT support Infiniband at all ?

user@desktop:~$ ssh -o KexAlgorithms=+diffie-hellman-group14-sha1 -o HostKeyAlgorithms=+ssh-rsa admin@192.168.5.101

Mellanox MLNX-OS Switch Management

(admin@192.168.5.101) Password:
Last login: Sun Aug 30 14:26:18 2026 from 192.168.3.72

Mellanox Switch

switch-101 [standalone: master] > enable
switch-101 [standalone: master] # configure terminal
switch-101 [standalone: master] (config) # port 1/1 type ethernet force
switch-101 [standalone: master] (config) # show port type
Ethernet:   1/1
InfiniBand: 1/2 1/3 1/4 1/5 1/6 1/7 1/8 1/9 1/10 1/11 1/12 1/13 1/14 1/15 1/16 1/17 1/18 1/19 1/20 1/21 1/22 1/23 1/24 1/25 1/26 1/27 1/28 1/29 1/30 1/31 1/32 1/33 1/34 1/35 1/36
switch-101 [standalone: master] (config) # interface ethernet 1/1
switch-101 [standalone: master] (config interface ethernet 1/1) # show interfaces ethernet 1/1

Eth1/1:
  Admin state                               : Enabled
  Operational state                         : Up
  Last change in operational status         : 0:02:35 ago (9 oper change)
  Boot delay time                           : 0 sec
  Description                               : N\A
  Mac address                               : 00:02:c9:5d:3d:ec  
  MTU                                       : 1500 bytes (Maximum packet size 1522 bytes)
  Fec                                       : auto
  Flow-control                              : receive off send off
  Actual speed                              : 40 Gbps
  Auto-negotiation                          : Enabled
  Width reduction mode                      : Unknown
  Switchport mode                           : access
  MAC learning mode                         : Enabled
  Last clearing of "show interface" counters: Never
  60 seconds ingress rate                   : 0 bits/sec, 0 bytes/sec, 0 packets/sec
  60 seconds egress rate                    : 264 bits/sec, 33 bytes/sec, 1 packets/sec

  Rx:
    3                     packets
    0                     unicast packets
    3                     multicast packets
    0                     broadcast packets
    192                   bytes
    0                     discard packets
    0                     error packets
    0                     fcs errors
    0                     undersize packets
    0                     oversize packets
    0                     pause packets
    0                     unknown control opcode
    0                     symbol errors

  Tx:
    79                    packets
    0                     unicast packets
    79                    multicast packets
    0                     broadcast packets
    5056                  bytes
    0                     discard packets
    0                     error packets
    0                     hoq discard packets

switch-101 [standalone: master] (config interface ethernet 1/1) # show interfaces ethernet 1/1 transceiver
Port 1/1 state
        identifier             : QSFP+
        cable/module type      : Optical cable/ module - LR4
        ethernet speed and type: 40GBASE - LR4
        vendor                 : Finisar
        supported cable length : 2000m SMF
        part number            : FTL4C1QL1L
        revision               : A0
        serial number          : UXR01ET

switch-101 [standalone: master] (config interface ethernet 1/1) # exit
switch-101 [standalone: master] (config) # configuration write

EDIT 1: try to force Port to Infiniband, not sure if this provides better Diagnostic

switch-101 [standalone: master] (config) # port 1/1 type infiniband force
switch-101 [standalone: master] (config) # interface ib 1/1
switch-101 [standalone: master] (config interface ib 1/1) # show interfaces ib 1/1
IB1/1 state:
        Logical port state          : Down
        Physical port state         : Polling
        Current line rate           : -
        Supported speeds            : sdr, ddr, qdr, fdr10, fdr
        Speed                       : -
        Supported widths            : 1X, 4X
        Width                       : -
        Max supported MTUs          : 4096
        MTU                         : 0
        VL capabilities             : VL0 - VL7
        Operational VLs             : -
        Description                 :
        IB Subnet                   : infiniband-default
        Phy-profile                 : high-speed-ber
        Width reduction mode        : Disabled

        RX bytes                    : 0
        RX packets                  : 0
        RX errors                   : 0
        Symbol errors               : 0
        VL15 dropped packets        : 0

        TX bytes                    : 0
        TX packets                  : 0
        TX wait                     : 0
        TX discarded packets        : 0



switch-101 [standalone: master] (config interface ib 1/1) # show interfaces ib 1/1 transceiver
IB1/1 state:
        identifier                  : QSFP+
        cable/module type           : Optical cable/ module - LR4
        infiniband speeds           : unknown
        vendor                      : Finisar
        supported cable length      : 2000m SMF
        part number                 : FTL4C1QL1L
        revision                    : A0
        serial number               : UXR01ET

 

[Ahrimanes]

If someone else is looking for information, I want to add my experience with 2 SX6036 switches and 100m MM OM4 with LC connectors.

Cisco QSFP-40G-SR-BD - Switch identifies the modules, but could never get it to power the laser diodes. Spent hours and hours with modified firmware and cable-stamping etc.

FS QSFP-LX4-40G - Worked after cable-stamping command. Got link with acceptable RX power values.