QSFP+ to QSFP+ link/connection (4x10G vs 40Gfull)

[Spyder Web]

What's the functional difference?

The reason I ask is because I have an Arista 7050QX-32-S (24x QSFP+ 4x10G-capable-ports and 8 QSFP+ 40G-only-ports) and recently I picked up some Arista 40G-UNIV transceivers expecting, perhaps niavely, for them to functionally be an SMF/MMF compatable version of the 40G-SRBD, but they will only link/connect at 4x10G and seemingly cannot be forced to 40Gfull like my other QSFP+ transceivers (SR4, SRBD) and DACs. As such I have been unable to get them to link at 40G with any of my Mellanox ConnectX-3 or ConnectX-4 adapters and my Intel XL-710's flat out reject them.

 

[Spyder Web]

Attached is an example of what I'm seeing. The best/only information I've been able to find on the 40G-UNIV is the Arista FAQ and this blog (How to Run 40GbE Over Duplex LC MMF Cabling) where they describe both the QSFP SRBD and UNIV:
"The working principle of Cisco QSFP BiDi transceiver is that it uses has two 20Gbps channels, each transmitted and received simultaneously over two wavelengths on a single MMF strand"
"[The UNIV] working principle is similar with the Juniper LX4 mentioned above, having 4 channels of 10G multiplexed inside the module to transmit and receive an aggregate 40Gbps signal over 2 strands of fiber".

 

[Spyder Web]

The basis of using two optical transceivers end-to-end is that they must follow the same standard. Obvious 40GBASE-SRBD and 40GBASE-UNIV are two different technical standards.

In simple, 40G-SRBD and 40G-UNIV shouldn't work together because they work with different numbers of wavelengths. SRBD uses only two wavelengths in a strand but UNIV uses four different wavelengths in a strand. Even if they can both use MMF duplex LC cable, they can't work togheter.

Thanks for the response, but I think you miss understood my question/issue. I'm not trying to mix SRBD and UNIV transceivers. An SRBD to SRBD will link at 40Gfull in either type of the Arista's QSFP+ ports; however, the UNIV to UNIV will only link at 4x10gfull in a 4x10 capable QSFP+ port and will not function at all in 40G only QSFP+ port.

 

[Stril]

Hi!

I am using Arista 40Gbase-UNIV in 7050QX-switches, but they run fine on 40GbE:

Et32 connected in Po1 full 40G 40GBASE-UNIV

--> Connected Arista 7050QX-32 to Arista 7050QX-32 on EOS 4.16.7M

Stril

 

[i386]

Did you try

shutdown
no shutdown

in the specific interface after changing the speed?

 

[Spyder Web]

I am using Arista 40Gbase-UNIV in 7050QX-switches, but they run fine on 40GbE:

That's awesome and what I was hoping/expecting when I ordered them. Have you tested these with any host adapters?

Did you try

shutdown
no shutdown

in the specific interface after changing the speed?

I did not, but I'm 99.9% sure the problem persisted through an EOS upgrade/reboot (4.17.6F->4.20.2.1F). I'm assuming/hoping the dangers of eBay struck and I got a bum/dirty set of UNIV's; though I ordered/tested 2 separate sets of both the UNIV transceivers and LC duplex patch cables. I have since returned the UNIV's, though with renewed hope from the responses I believe I'll give another couple sets a shot.

 

[Stril]

Hi!

I did only use Arista-UNIV as interconnect between two Arista switches

...but of course, I saw this problem with some DACs.

 

[Spyder Web]

I'm terribly sorry for my negelcting. Since Strill said the 40G-UNIV can be used in the 40G only port so it isn't the issue about the 4x10G or 40G only the port. If you still want to make use of those Arista 40G-UNIV transceivers, I think you may need to buy Mellanox 40G-UNIV transceivers for the Mellanox 40G adapters and Intel 40G-UNIV transceivers for the Intel XL-710. It seems more like a problem of transceiver compatibility on those host equipment. But I would rather put them aside and use inexpenisve DACs with differently branded ends.

No worries; I'm sure I could have phrased my questions/issues better at the beginning. Unfortunately these workstations are about a 10M run from the switch in another room; so at the upper thresholds of supported DAC range. I'd normally go with an AOC, but fortunately these 40G-UNIV transceivers are going for really cheap on ebay. I got them for $35/ea shipped and with the LC duplex MMF it ended up being cheaper than either DAC/AOC offerings from fs.com.

Arista -- QSFP-40G-UNIV -- 40G QSFP+ Universal Transceiver | eBay
Arista QSFP-40G-UNIV 40G QSFP+ Universal Transceiver | eBay

Great news, the latest 40G-UNIV transceivers actually work at 40G-Full in both the arista switch and the Mellanox cards.

 

[mattaw]

Can you confirm that the QSFP-40G-UNIV worked in a connectx-3 card? I am having the devil of a time getting some Fiberstore QSFP BiDi's to work. The card firmware notes that the Cisco BD is good for 40GB, but the power draw is 3W and the card specs state only 1.5W is available.

Tearing my hair out trying to figure out what to purchase to try next. Do let me know if you can!

 

[Stril]

Hi!

I can only confirm for ConnectX-4

 

[mattaw]

Thanks anyway! I am debating whether to purchase a couple of LR4's, or a UNIV and test with them as my el-cheapo FS DB's don't work (could be a bad one, after all ebay). Oh well, keeping the 40G dream at home alive

 

[upgrayedd]

ARISE THREAD!

these cx-3 vpi cards (649281-B21 HP fw or crossflash) do not appear to support 40gbase-lr4. don't know the underlying cause yet (module power?). i cannot see that the .ini files in the firmware have an option to enable or disable that link mode as with the 56gbit modes. i have tried both a contemporary HP JG661A and arista UNIV. i have a cx-4 on the slow boat and expect that will have LR4 link mode.


# ethtool enp130s0d1
Settings for enp130s0d1:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseKX/Full
10000baseKX4/Full
10000baseKR/Full
40000baseCR4/Full
40000baseSR4/Full
56000baseCR4/Full
56000baseSR4/Full

 

[Stril]

Hi!

I had to update the firmware to be able to use all of my QSFP+-Modules

 

[upgrayedd]

Hi!

I had to update the firmware to be able to use all of my QSFP+-Modules

what are your card models, fw versions and module models?

i.e. are you using a cx-3 non-pro and either an arista UNIV or LR4 type optic?

 

[upgrayedd]

connectx-4 arrived on the slow boat. it advertises 40GBASE-LR4 link mode as research suggested it would. also even though the HW is more like the dual port 40G parts (CX454A) including x8 pci-e phy, it shows 100gbit link modes:

    Supported link modes:   1000baseKX/Full
                            10000baseKR/Full
                            40000baseKR4/Full
                            40000baseCR4/Full
                            40000baseSR4/Full
                            40000baseLR4/Full
                            56000baseKR4/Full
                            25000baseCR/Full
                            25000baseKR/Full
                            25000baseSR/Full
                            50000baseCR2/Full
                            50000baseKR2/Full
                            100000baseKR4/Full
                            100000baseSR4/Full
                            100000baseCR4/Full
                            100000baseLR4_ER4/Full

​

i'm guessing this is because is flashed as a MT_2190110032 / CX456A-ECAT.

anyhow, the Arista UNIV optics work which is handy because they are CWDM and so work over cheap OM3 MMF or SMF. so now i can get to my fileserver at 40gbit ^_^