QSFP28 100G to differents speeds like 40Gb

nephri

Active Member
Sep 23, 2015
543
107
43
44
Paris, France
Hi,
i just saw this link Run QSFP28 100G Port in 40Gbps, 4x 10GbE, and 4x 25GbE Modes | FS Community

I knew you could achive 4x25Gb or 4x10Gb but i was sure we can't use QSFP+ tranceiver for having 40Gb but te FS thread seems to say we can...

If i read it correctly, we can do :
- Use a QSFP28 transceiver (or DAC) for 100G connectivity (you use 4x lanes at 25G)
- Use a QSFP+ transceiver (or DAC) for 40G connectivity (you use 4x lanes at 10G)
- Use a QSFP+ transceiver (or DAC) for 4x 10G connectivity with a breakout cable to 4x SFP+ at the other side
- Use a QSFP28 transceiver (or DAC) for 4x 25G connectivity with a breakout cable to 4x SFP25 at the other side

Do you have experienced a 40G connectivity on a 100G port ? on which brands ?
I have 4 machines using 40G NIC and i would be able to upgrade my switch keeping theses NICs for a while...

PS: i'm studying new Mikrotik switchs, someone have 40G experiences on with Mikrotik switches ?

Best regards,
Sébastien.
 

i386

Well-Known Member
Mar 18, 2016
3,403
1,141
113
33
Germany
I have 4 machines using 40G NIC and i would be able to upgrade my switch keeping theses NICs for a while...
I have the "opposite" problem: I have multiple 100GBE nics but can't/won't update to a 100GBE switch in the near future* and have to stick with my 40GBE switch :D
The 100GBE nics are mellanox cx-4 vpi cards (2 are hpe branded) and they work fine with all qsfp+ and qsfp14 cable connected to an arista 40GBE switch. The only problem I had was with a 100GBE AOC cable that was not properly recognized on the arista and worked only with 10GBE.

* the "good" 100GBE switches went up from ~1.1k $ to 10k+ $ during the pandemic/shortages and I don't have such a budget for my homelab
 

Rand__

Well-Known Member
Mar 6, 2014
6,333
1,599
113
IIrc Mellanox, Dell and Celestica 100G switches connected fine at 40G to each other and CX3+'s. They also do 100G o/c with the correct fec mode set.
I currently run MLX 100G to brocade 7450's at 40G

The above is with DAC's.

Have not tried 40G transceivers, but 100G is pickier for me (due to high power requirements on mine, MLX to MLX works fine, but Brocade won't. Dont have the Dell/Celestica switches any more to test).

But else your list looks valid:)
 

nephri

Active Member
Sep 23, 2015
543
107
43
44
Paris, France
Thanks for theses answers. It's a good news i will study my options.


I would find a combo of switches for handle RJ45 and fibers at high speeds.

I found Mikrotik have good products like :
- CRS312-4C+8XG-RM for RJ45 MikroTik
- CRS504-4XQ-IN for 100G MikroTik

But theses would be ideal if:
- CRS312-4C+8XG-RM had a 100G port for uplink instead of 4x SFP+
- CRS504-4XQ-IN had 8x 100 G instead of 4x

I know we can have a Celestica DX010 (32x 100G) relatively cheap but it's too power hungry even when empy and 32x ports was too much for me.

For my home setup, i would love a unique switch with 6+ QSFP+ and 8+ RJ45-10G

If you know some ref to know, don't hesitate to share.
 

A-W-A

New Member
May 14, 2022
27
9
3
Do you need something more capable and with more 100G ports than the FS S8550-6Q2c.

Andrew
 

blinkenlights

Active Member
May 24, 2019
161
76
28
It's interesting that someone (FS) took the time to write an article about QSFP28 interoperability. My homelab switch is an ICX7450-24 (2x 40GigE, 1x 4x10GigE modules) with the SFP+/QSFP+ modules fully populated. Sometimes I've found 25GigE (SFP28) and 100GigE (QSFP28) cables new in box for less than their SFP+/QSFP+ counterparts.

For what it's worth, this is what I have seen:
* Chelsio T62100-CR -> 3.0M QSFP28 Passive DAC (Dell/EMC) -> ICX7450 negotiates at 40GigE as expected
* Chelsio T6225-CR -> 0.5M SFP28 Active Optical (custom) -> ICX7450 negotiates at 10GigE as expected
* Chelsio T62100-CR -> 3.0M QSFP28 Active Optical (NVIDIA/Mellanox) -> ICX7450 does not negotiate, can't force speed either
* Chelsio T62100-CR -> 3.0M QSFP28 Active Optical (Juniper) -> ICX7450 negotiates at 40GigE as expected, but only once (required reboot if unplugged)

If you are using longish Passive DACs with 40GigE infrastructure, it might be worth trying the beefy QSFP28 cables :cool:
 

bsiu922

New Member
Sep 1, 2022
1
0
1
Quick question. any one try to use the S8550-6Q2C to replace Mellanox SX6005 or SX 6012?