Kaiam XQX4002 100G CWDM4 @ ConnectX 4. Anyone experience?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

servethebasement

New Member
Oct 10, 2023
3
0
1
Hello,
I have seen the incredibly cheap Intel 100G transceivers linked here (somewhere).
Only problem is that shipping to europe is 30€ + taxes so the $5 seems less nice after that.

I instead bought a set of Kaiam XQX4002 from France. (lot 2x KAIAM XQX4002 100G-CWDM4 QSFP28 100 Gb 1310nm 2km Transceiver | eBay)
35€ in total for 2 incl. shipping which is acceptable.

Only problem is that I cannot get a link.
Cabling is 30m OS2 + 2m OS2 after Keystone module.
Here is ethtool output from my server:
Code:
root@px1:~# ethtool -m enp129s0np0
        Identifier                                : 0x11 (QSFP28)
        Extended identifier                       : 0xcc
        Extended identifier description           : 3.5W max. Power consumption
        Extended identifier description           : CDR present in TX, CDR present in RX
        Extended identifier description           : High Power Class (> 3.5 W) not enabled
        Power set                                 : Off
        Power override                            : On
        Connector                                 : 0x07 (LC)
        Transceiver codes                         : 0x80 0x00 0x00 0x00 0x00 0x00 0x00 0x00
        Transceiver type                          : 100G Ethernet: 100G CWDM4 MSA with FEC
        Encoding                                  : 0x07 ((256B/257B (transcoded FEC-enabled data))
        BR, Nominal                               : 25500Mbps
        Rate identifier                           : 0x02
        Length (SMF,km)                           : 2km
        Length (OM3 50um)                         : 0m
        Length (OM2 50um)                         : 0m
        Length (OM1 62.5um)                       : 0m
        Length (Copper or Active cable)           : 0m
        Transmitter technology                    : 0x40 (1310 nm DFB)
        Laser wavelength                          : 1310.000nm
        Laser wavelength tolerance                : 65.000nm
        Vendor name                               : KAIAM CORP
        Vendor OUI                                : 14:ed:e4
        Vendor PN                                 : XQX4000
        Vendor rev                                : 1A
        Vendor SN                                 : AR742R0D4A
        Date code                                 : 180330
        Revision Compliance                       : SFF-8636 Rev 2.5/2.6/2.7
        Module temperature                        : 36.39 degrees C / 97.50 degrees F
        Module voltage                            : 3.2881 V
        Alarm/warning flags implemented           : Yes
        Laser tx bias current (Channel 1)         : 40.044 mA
        Laser tx bias current (Channel 2)         : 40.574 mA
        Laser tx bias current (Channel 3)         : 40.836 mA
        Laser tx bias current (Channel 4)         : 38.702 mA
        Transmit avg optical power (Channel 1)    : 1.2870 mW / 1.10 dBm
        Transmit avg optical power (Channel 2)    : 1.0267 mW / 0.11 dBm
        Transmit avg optical power (Channel 3)    : 1.2072 mW / 0.82 dBm
        Transmit avg optical power (Channel 4)    : 1.3600 mW / 1.34 dBm
        Rcvr signal avg optical power(Channel 1)  : 0.9715 mW / -0.13 dBm
        Rcvr signal avg optical power(Channel 2)  : 0.3771 mW / -4.24 dBm
        Rcvr signal avg optical power(Channel 3)  : 0.7329 mW / -1.35 dBm
        Rcvr signal avg optical power(Channel 4)  : 0.8895 mW / -0.51 dBm
        Laser bias current high alarm   (Chan 1)  : Off
        Laser bias current low alarm    (Chan 1)  : Off
        Laser bias current high warning (Chan 1)  : Off
        Laser bias current low warning  (Chan 1)  : Off
        Laser bias current high alarm   (Chan 2)  : Off
        Laser bias current low alarm    (Chan 2)  : Off
        Laser bias current high warning (Chan 2)  : Off
        Laser bias current low warning  (Chan 2)  : Off
        Laser bias current high alarm   (Chan 3)  : Off
        Laser bias current low alarm    (Chan 3)  : Off
        Laser bias current high warning (Chan 3)  : Off
        Laser bias current low warning  (Chan 3)  : Off
        Laser bias current high alarm   (Chan 4)  : Off
        Laser bias current low alarm    (Chan 4)  : Off
        Laser bias current high warning (Chan 4)  : Off
        Laser bias current low warning  (Chan 4)  : Off
        Module temperature high alarm             : Off
        Module temperature low alarm              : Off
        Module temperature high warning           : Off
        Module temperature low warning            : Off
        Module voltage high alarm                 : Off
        Module voltage low alarm                  : Off
        Module voltage high warning               : Off
        Module voltage low warning                : Off
        Laser tx power high alarm   (Channel 1)   : Off
        Laser tx power low alarm    (Channel 1)   : On
        Laser tx power high warning (Channel 1)   : Off
        Laser tx power low warning  (Channel 1)   : On
        Laser tx power high alarm   (Channel 2)   : Off
        Laser tx power low alarm    (Channel 2)   : On
        Laser tx power high warning (Channel 2)   : Off
        Laser tx power low warning  (Channel 2)   : On
        Laser tx power high alarm   (Channel 3)   : Off
        Laser tx power low alarm    (Channel 3)   : On
        Laser tx power high warning (Channel 3)   : Off
        Laser tx power low warning  (Channel 3)   : On
        Laser tx power high alarm   (Channel 4)   : Off
        Laser tx power low alarm    (Channel 4)   : On
        Laser tx power high warning (Channel 4)   : Off
        Laser tx power low warning  (Channel 4)   : On
        Laser rx power high alarm   (Channel 1)   : Off
        Laser rx power low alarm    (Channel 1)   : Off
        Laser rx power high warning (Channel 1)   : Off
        Laser rx power low warning  (Channel 1)   : Off
        Laser rx power high alarm   (Channel 2)   : Off
        Laser rx power low alarm    (Channel 2)   : Off
        Laser rx power high warning (Channel 2)   : Off
        Laser rx power low warning  (Channel 2)   : Off
        Laser rx power high alarm   (Channel 3)   : Off
        Laser rx power low alarm    (Channel 3)   : Off
        Laser rx power high warning (Channel 3)   : Off
        Laser rx power low warning  (Channel 3)   : Off
        Laser rx power high alarm   (Channel 4)   : Off
        Laser rx power low alarm    (Channel 4)   : Off
        Laser rx power high warning (Channel 4)   : Off
        Laser rx power low warning  (Channel 4)   : Off
        Laser bias current high alarm threshold   : 100.000 mA
        Laser bias current low alarm threshold    : 5.000 mA
        Laser bias current high warning threshold : 90.000 mA
        Laser bias current low warning threshold  : 10.000 mA
        Laser output power high alarm threshold   : 3.1623 mW / 5.00 dBm
        Laser output power low alarm threshold    : 0.2239 mW / -6.50 dBm
        Laser output power high warning threshold : 1.7783 mW / 2.50 dBm
        Laser output power low warning threshold  : 0.2818 mW / -5.50 dBm
        Module temperature high alarm threshold   : 73.00 degrees C / 163.40 degrees F
        Module temperature low alarm threshold    : 0.00 degrees C / 32.00 degrees F
        Module temperature high warning threshold : 68.00 degrees C / 154.40 degrees F
        Module temperature low warning threshold  : 5.00 degrees C / 41.00 degrees F
        Module voltage high alarm threshold       : 3.4600 V
        Module voltage low alarm threshold        : 3.1300 V
        Module voltage high warning threshold     : 3.4100 V
        Module voltage low warning threshold      : 3.1800 V
        Laser rx power high alarm threshold       : 3.1623 mW / 5.00 dBm
        Laser rx power low alarm threshold        : 0.0708 mW / -11.50 dBm
        Laser rx power high warning threshold     : 1.7783 mW / 2.50 dBm
        Laser rx power low warning threshold      : 0.0891 mW / -10.50 dBm


root@px1:~# ethtool enp129s0np0
Settings for enp129s0np0:
        Supported ports: [ FIBRE ]
        Supported link modes:   1000baseKX/Full
                                10000baseKR/Full
                                40000baseKR4/Full
                                40000baseCR4/Full
                                40000baseSR4/Full
                                40000baseLR4/Full
                                25000baseCR/Full
                                25000baseKR/Full
                                25000baseSR/Full
                                50000baseCR2/Full
                                50000baseKR2/Full
                                100000baseKR4/Full
                                100000baseSR4/Full
                                100000baseCR4/Full
                                100000baseLR4_ER4/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: Yes
        Supported FEC modes: None        RS      BASER
        Advertised link modes:  1000baseKX/Full
                                10000baseKR/Full
                                40000baseKR4/Full
                                40000baseCR4/Full
                                40000baseSR4/Full
                                40000baseLR4/Full
                                25000baseCR/Full
                                25000baseKR/Full
                                25000baseSR/Full
                                50000baseCR2/Full
                                50000baseKR2/Full
                                100000baseKR4/Full
                                100000baseSR4/Full
                                100000baseCR4/Full
                                100000baseLR4_ER4/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  Not reported
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Auto-negotiation: on
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Supports Wake-on: d
        Wake-on: d
        Link detected: no


root@px1:~# mlxlink -d 81:00.0

Operational Info
----------------
State                              : Polling
Physical state                     : ETH_AN_FSM_AN_GOOD_CHECK
Speed                              : N/A
Width                              : N/A
FEC                                : N/A
Loopback Mode                      : No Loopback
Auto Negotiation                   : ON

Supported Info
--------------
Enabled Link Speed                 : 0xf8f1f0d3 (100G,50G,40G,25G,10G,1G)
Supported Cable Speed              : 0x00800000 (100G)

Troubleshooting Info
--------------------
Status Opcode                      : 0
Group Opcode                       : N/A
Recommendation                     : No issue was observed

Tool Information
----------------
Firmware Version                   : 12.28.2006
MFT Version                        : mft 4.25.0-62


root@px1:~# mlxconfig -d 81:00.0 q

Device #1:
----------

Device type:    ConnectX4       
Name:           MCX455A-ECA_Ax  
Description:    ConnectX-4 VPI adapter card; EDR IB (100Gb/s) and 100GbE; single-port QSFP28; PCIe3.0 x16; ROHS R6
Device:         81:00.0         

Configurations:                                      Next Boot
        MEMIC_BAR_SIZE                              0               
        MEMIC_SIZE_LIMIT                            _256KB(1)       
        FLEX_PARSER_PROFILE_ENABLE                  0               
        FLEX_IPV4_OVER_VXLAN_PORT                   0               
        ROCE_NEXT_PROTOCOL                          254             
        NON_PREFETCHABLE_PF_BAR                     False(0)        
        VF_VPD_ENABLE                               False(0)        
        STRICT_VF_MSIX_NUM                          False(0)        
        VF_NODNIC_ENABLE                            False(0)        
        NUM_PF_MSIX_VALID                           True(1)         
        NUM_OF_VFS                                  8               
        NUM_OF_PF                                   1               
        FPP_EN                                      True(1)         
        SRIOV_EN                                    True(1)         
        PF_LOG_BAR_SIZE                             5               
        VF_LOG_BAR_SIZE                             1               
        NUM_PF_MSIX                                 63              
        NUM_VF_MSIX                                 11              
        INT_LOG_MAX_PAYLOAD_SIZE                    AUTOMATIC(0)    
        PCIE_CREDIT_TOKEN_TIMEOUT                   0               
        PARTIAL_RESET_EN                            False(0)        
        SW_RECOVERY_ON_ERRORS                       False(0)        
        RESET_WITH_HOST_ON_ERRORS                   False(0)        
        PCI_DOWNSTREAM_PORT_OWNER                   Array[0..15]    
        CQE_COMPRESSION                             BALANCED(0)     
        IP_OVER_VXLAN_EN                            False(0)        
        MKEY_BY_NAME                                False(0)        
        UCTX_EN                                     True(1)         
        PCI_ATOMIC_MODE                             PCI_ATOMIC_DISABLED_EXT_ATOMIC_ENABLED(0)
        TUNNEL_ECN_COPY_DISABLE                     False(0)        
        LRO_LOG_TIMEOUT0                            6               
        LRO_LOG_TIMEOUT1                            7               
        LRO_LOG_TIMEOUT2                            8               
        LRO_LOG_TIMEOUT3                            13              
        TX_SCHEDULER_BURST                          0               
        LOG_DCR_HASH_TABLE_SIZE                     14              
        MAX_PACKET_LIFETIME                         0               
        DCR_LIFO_SIZE                               16384           
        LINK_TYPE_P1                                ETH(2)          
        ROCE_CC_PRIO_MASK_P1                        255             
        CLAMP_TGT_RATE_AFTER_TIME_INC_P1            True(1)         
        CLAMP_TGT_RATE_P1                           False(0)        
        RPG_TIME_RESET_P1                           300             
        RPG_BYTE_RESET_P1                           32767           
        RPG_THRESHOLD_P1                            1               
        RPG_MAX_RATE_P1                             0               
        RPG_AI_RATE_P1                              5               
        RPG_HAI_RATE_P1                             50              
        RPG_GD_P1                                   11              
        RPG_MIN_DEC_FAC_P1                          50              
        RPG_MIN_RATE_P1                             1               
        RATE_TO_SET_ON_FIRST_CNP_P1                 0               
        DCE_TCP_G_P1                                1019            
        DCE_TCP_RTT_P1                              1               
        RATE_REDUCE_MONITOR_PERIOD_P1               4               
        INITIAL_ALPHA_VALUE_P1                      1023            
        MIN_TIME_BETWEEN_CNPS_P1                    0               
        CNP_802P_PRIO_P1                            6               
        CNP_DSCP_P1                                 48              
        LLDP_NB_DCBX_P1                             False(0)        
        LLDP_NB_RX_MODE_P1                          OFF(0)          
        LLDP_NB_TX_MODE_P1                          OFF(0)          
        ROCE_RTT_RESP_DSCP_P1                       0               
        ROCE_RTT_RESP_DSCP_MODE_P1                  DEVICE_DEFAULT(0)
        DCBX_IEEE_P1                                True(1)         
        DCBX_CEE_P1                                 True(1)         
        DCBX_WILLING_P1                             True(1)         
        KEEP_ETH_LINK_UP_P1                         True(1)         
        KEEP_IB_LINK_UP_P1                          False(0)        
        KEEP_LINK_UP_ON_BOOT_P1                     False(0)        
        KEEP_LINK_UP_ON_STANDBY_P1                  False(0)        
        DO_NOT_CLEAR_PORT_STATS_P1                  False(0)        
        AUTO_POWER_SAVE_LINK_DOWN_P1                True(1)         
        NUM_OF_VL_P1                                _4_VLs(3)       
        NUM_OF_TC_P1                                _8_TCs(0)       
        NUM_OF_PFC_P1                               8               
        VL15_BUFFER_SIZE_P1                         0               
        DUP_MAC_ACTION_P1                           LAST_CFG(0)     
        SRIOV_IB_ROUTING_MODE_P1                    LID(1)          
        IB_ROUTING_MODE_P1                          LID(1)          
        PCI_WR_ORDERING                             per_mkey(0)     
        MULTI_PORT_VHCA_EN                          False(0)        
        PORT_OWNER                                  True(1)         
        ALLOW_RD_COUNTERS                           True(1)         
        RENEG_ON_CHANGE                             True(1)         
        TRACER_ENABLE                               True(1)         
        IP_VER                                      IPv4(0)         
        BOOT_UNDI_NETWORK_WAIT                      0               
        UEFI_HII_EN                                 False(0)        
        BOOT_DBG_LOG                                False(0)        
        UEFI_LOGS                                   DISABLED(0)     
        BOOT_VLAN                                   1               
        LEGACY_BOOT_PROTOCOL                        NONE(0)         
        BOOT_INTERRUPT_DIS                          False(0)        
        BOOT_LACP_DIS                               False(0)        
        BOOT_VLAN_EN                                False(0)        
        BOOT_PKEY                                   0               
        P2P_ORDERING_MODE                           DEVICE_DEFAULT(0)
        DYNAMIC_VF_MSIX_TABLE                       False(0)        
        ADVANCED_PCI_SETTINGS                       False(0)        
        SAFE_MODE_THRESHOLD                         10              
        SAFE_MODE_ENABLE                            True(1)

Seems to be recognized okay.
Here is the closest I could find on windows:
Code:
PS > mlxcables.bat -d mt4115_pciconf0_cable_0 -DDM
Cable DDM:
----------
Temperature    : 49C
Voltage        : 3.2745V
Channel 1:
        RX Power : 0.9332dBm
        TX Power : 1.3399dBm
        TX Bias  : 38.1800mA
Channel 2:
        RX Power : -0.2091dBm
        TX Power : -15.9517dBm
        TX Bias  : 60.5400mA
Channel 3:
        RX Power : 1.1214dBm
        TX Power : 0.7998dBm
        TX Bias  : 45.3660mA
Channel 4:
        RX Power : 1.5152dBm
        TX Power : 0.9205dBm
        TX Bias  : 45.6360mA
----- Flags -----
Temperature:
        Alarm  high  : 0
        Warning high : 0
        Warning low  : 0
        Alarm  low   : 0
Voltage:
        Alarm  high  : 0
        Warning high : 0
        Warning low  : 0
        Alarm  low   : 0
Channel 1:
        RX Power alarm  high : 0
        RX Power warning high: 0
        RX Power warning low : 1
        RX Power alarm  low  : 0

        TX Power alarm  high : 0
        TX Power warning high: 1
        TX Power warning low : 0
        TX Power alarm  low  : 0

        TX Bias alarm  high  : 0
        TX Bias warning high : 0
        TX Bias warning low  : 0
        TX Bias alarm  low   : 0

Channel 2:
        RX Power alarm  high : 0
        RX Power warning high: 0
        RX Power warning low : 0
        RX Power alarm  low  : 0

        TX Power alarm  high : 0
        TX Power warning high: 0
        TX Power warning low : 1
        TX Power alarm  low  : 1

        TX Bias alarm  high  : 0
        TX Bias warning high : 0
        TX Bias warning low  : 0
        TX Bias alarm  low   : 0

Channel 3:
        RX Power alarm  high : 0
        RX Power warning high: 0
        RX Power warning low : 0
        RX Power alarm  low  : 0

        TX Power alarm  high : 0
        TX Power warning high: 0
        TX Power warning low : 0
        TX Power alarm  low  : 0

        TX Bias alarm  high  : 0
        TX Bias warning high : 0
        TX Bias warning low  : 0
        TX Bias alarm  low   : 0

Channel 4:
        RX Power alarm  high : 0
        RX Power warning high: 0
        RX Power warning low : 0
        RX Power alarm  low  : 0

        TX Power alarm  high : 0
        TX Power warning high: 0
        TX Power warning low : 0
        TX Power alarm  low  : 0

        TX Bias alarm  high  : 0
        TX Bias warning high : 0
        TX Bias warning low  : 0
        TX Bias alarm  low   : 0

----- Thresholds -----
        Temperature high alarm threshold   : 73C
        Temperature high warning threshold : 68C
        Temperature low  warning threshold : 5C
        Temperature low  alarm threshold   : 0C

        Voltage high alarm threshold   : 3.4600V
        Voltage high warning threshold: 3.4100V
        Voltage low  warning threshold: 3.1800V
        Voltage low  alarm   threshold: 3.1300V

        RX Power high alarm threshold : 5.0000dBm
        RX Power high warn threshold  : 2.5001dBm
        RX Power low  warn threshold  : -10.5012dBm
        RX Power low  alarm threshold : -11.4997dBm

        TX Power high alarm threshold : 5.0000dBm
        TX Power high warn threshold  : 2.5001dBm
        TX Power low  warn threshold  : -5.5006dBm
        TX Power low  alarm threshold : -6.4995dBm

        TX Bias high alarm threshold  : 100.0000mA
        TX Bias high warn threshold   : 100.0000mA
        TX Bias low  warn threshold   : 0.0000mA
        TX Bias low  alarm threshold  : 0.0000mA
 

TRACKER

Active Member
Jan 14, 2019
251
108
43
I have XQX4302 from France and there were two things i had to do before i got link - enable FEC91 and set manually 100Gbps speed (that is on Mkrotik CRS504 switch).
 

servethebasement

New Member
Oct 10, 2023
3
0
1
I have a direct connection from my promox server to my windows PC. No switch in between.
Ethtool and mlxlink only show RS-FEC (528,514).
I tried enabling with ethtool and mlxlink without success.
Also forced 100G with ethtool without success.
 

servethebasement

New Member
Oct 10, 2023
3
0
1
I managed to get some Intel 100G transcrivers on loan from work.
Plugged them in and have a stable link.
So it's not the cable.
I've also cleaned the transceivers with the cheapest LC cleaning pen I could buy.

Do you happen to have any kind of manual/datasheet for the Kaiam transceivers?
 

TRACKER

Active Member
Jan 14, 2019
251
108
43
Unfortunately i did not find any technical details for these KAIAM transceivers on internet. Perhaps i was not looking on the right place(s) :)