Hi,
I've followed the instructions and successfully flashed some:
ConnectX-3 VPI adapter card; dual-port QSFP; QDR IB (40Gb/s) and 10GigE; PCIe3.0x8 8GT/s; RoHS R6
to generic firmware.
And for most of it, the cards seem to work fine.
I can get around >30Gbit/s with a single
iperf3
between 2 servers with these cards, which seems great.
But once I actually put them to real work, they'll cap at 17Gbit/s:
View attachment 22863
At this point, iperf3 will also return poor speeds (sub 10Mbit), so it's clear that it has reached it's capped.
Tranceivers are Arista:
Code:
Identifier : 0x0d (QSFP+)
Extended identifier : 0xc0
Extended identifier description : 3.5W max. Power consumption
Extended identifier description : No CDR in TX, No CDR in RX
Extended identifier description : High Power Class (> 3.5 W) not enabled
Connector : 0x07 (LC)
Transceiver codes : 0x02 0x00 0x00 0x00 0x10 0x10 0x01 0xd5
Transceiver type : 40G Ethernet: 40G Base-LR4
Transceiver type : FC: long distance (L)
Transceiver type : FC: Longwave laser (LL)
Transceiver type : FC: Single Mode (SM)
Transceiver type : FC: 1200 MBytes/sec
Transceiver type : FC: 800 MBytes/sec
Transceiver type : FC: 400 MBytes/sec
Transceiver type : FC: 200 MBytes/sec
Transceiver type : FC: 100 MBytes/sec
Encoding : 0x03 (NRZ)
BR, Nominal : 10300Mbps
Rate identifier : 0x00
Length (SMF,km) : 10km
Length (OM3 50um) : 0m
Length (OM2 50um) : 0m
Length (OM1 62.5um) : 0m
Length (Copper or Active cable) : 0m
Transmitter technology : 0x40 (1310 nm DFB)
Laser wavelength : 1310.000nm
Laser wavelength tolerance : 10.000nm
Vendor name : Arista Networks
Vendor OUI : 64:9d:99
Vendor PN : QSFP-LR4-40G
Vendor rev : 1A
Vendor SN : F2020105366
Revision Compliance : SFF-8636 Rev 2.5/2.6/2.7
Module temperature : 39.28 degrees C / 102.70 degrees F
Module voltage : 3.1662 V
Alarm/warning flags implemented : Yes
Laser tx bias current (Channel 1) : 38.464 mA
Laser tx bias current (Channel 2) : 40.896 mA
Laser tx bias current (Channel 3) : 37.952 mA
Laser tx bias current (Channel 4) : 36.864 mA
Transmit avg optical power (Channel 1) : 0.8780 mW / -0.57 dBm
Transmit avg optical power (Channel 2) : 0.8186 mW / -0.87 dBm
Transmit avg optical power (Channel 3) : 1.0237 mW / 0.10 dBm
Transmit avg optical power (Channel 4) : 0.8855 mW / -0.53 dBm
Rcvr signal avg optical power(Channel 1) : 0.6321 mW / -1.99 dBm
Rcvr signal avg optical power(Channel 2) : 1.0105 mW / 0.05 dBm
Rcvr signal avg optical power(Channel 3) : 0.9729 mW / -0.12 dBm
Rcvr signal avg optical power(Channel 4) : 0.9047 mW / -0.43 dBm
And they seem to be in fine shape.
Has anyone encountered a similar issue? I'm puzzled of what it could be.