Cable confusion - How can I recognize which will do 56Gb?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Sniper_X

Member
Mar 11, 2021
115
16
18
I have seen so many QSFP+ cables and own several copper (passive) and Optical (Active) cables.
I can get 40Gb Ethernet all day long.

However, how do I recognize if a cable can go 56Gb/s?

I know that eBay sellers mostly post proper specs on their auctions, but sometimes they don't.

I'm trying to minimize how much more i have to spend here, so is there some way to verify which will/should do 56Gb/s?
 

NablaSquaredG

Layer 1 Magician
Aug 17, 2020
1,319
800
113
See my testing results here:

TL;DR:
All 100G Ethernet (Mellanox or Generic) or Mellanox FDR cables can do 56G Ethernet (but only do 40G Infiniband)
Only genuine Mellanox FDR cables can do 56G Infiniband and 56G Ethernet.
Mellanox EDR Ethernet cables DO NOT support 56G Infiniband (but 56G Ethernet works). Mellanox EDR Infiniband cables support BOTH 56G Infiniband and 56G Ethernet

Mellanox FDR cables have part number MC22071XX-XXX:
56G FDR IB part numbers are MC2207130-XXX -> Short length (up to 2m), thinner, MC2201728-XXX -> Longer (up to 3m), thicker and MC2207126-004 (4m), very thick.

MC22061XX are QDR cables afaik, so will not support 56G Ethernet or Infiniband.

MCP170L-FXX are variants of the MC220171XX-XXX (LSZH jacket instead of standard jacket)

There are also FDR AOCs, I don't know
 
  • Like
Reactions: klui

Sniper_X

Member
Mar 11, 2021
115
16
18
Thanks for this information!

I have this cable:
Siemon QSFP28-04 QSFP+ 4M QSFP28 100G Passive Copper Cable

Part#: QSFP28-04
UPC: 700416985549

But I'm not getting 56Gb/s links using ConnectX-3 cards, Ethernet mode and a fully licensed SX6036 with ports set to 56Gbit.

Was this wrong information?
s-l1600[1].jpgs-l1600[1].jpg
 

NablaSquaredG

Layer 1 Magician
Aug 17, 2020
1,319
800
113
You'll need to give us some more information in order to debug the issue, e.g. ibstat output, ibstatus output, mlxfwmanager output, cable / transceiver diagnostics, etc...
 

Sniper_X

Member
Mar 11, 2021
115
16
18
Here is the info for the active fiber cable I have that claims to be capable up to 100Gb:

Transceiver Information (reported by SX6036 - fully licensed)
========================
Identifier: QSFP+

Cable/Module type: Optical cable/ module
Eth speed (type): 40G Active Cable (XLPPI)
Vendor: ONET
Cable length: 12 m
Part number: 1AT-1Q2M12XX-03A
Revision: 04

Serial number: 1QA-0437690

This ONET cable will do 40Gbit, but not 56Gbit ethernet when connceted between two SX6036 switches (fully Licensed)

According to the post linked above, 100G cables should do 56Gbit since "Mellanox is not picky about Ethernet speeds" no?
 

Sniper_X

Member
Mar 11, 2021
115
16
18
Apparently this cable won't do 56Gbit Ethernet either.
I suppose I have cables that aren't that useful beyond 40Gbit.
1617562669523.png
 

NablaSquaredG

Layer 1 Magician
Aug 17, 2020
1,319
800
113
Do you have a ConnectX card for further diagnosis?

Which port speed profile is selected?
Can you please show a full screenshot of the port with the cable in question?
Which MLNX-OS version?
 

NablaSquaredG

Layer 1 Magician
Aug 17, 2020
1,319
800
113
Eh... yeah, the cables you have bought (QSFP28-04) are NOT QSFP28 cables, they are QSFP+ 40G cables.

Very unfortunate product naming.

See the manufacturer's datasheet:

Cable Assembly, High Speed Interconnect, 40Gb/s, Passive Direct Attach Cable (Copper), QSFP+ to QSFP+, 28 AWG, 4 Meter, PVC, Black

So - No misinformation from my side as you claimed :)

P.S. Your alleged 100G AOC cable is also a QSFP+ 40G cable as the label on the transceiver says.
 

Sniper_X

Member
Mar 11, 2021
115
16
18
Do you have a ConnectX card for further diagnosis?

Which port speed profile is selected?
Can you please show a full screenshot of the port with the cable in question?
Which MLNX-OS version?
The testing I’m performing is done between two SX6036 switches.

This is the most straightforward way to test cable capabilities specifically.

The cables are only linking up at 40Gbit.
All my cables behave the same.

I’ll have to get others that are true Mellanox products.
 

Sniper_X

Member
Mar 11, 2021
115
16
18
So - No misinformation from my side as you claimed :)

I never claimed you gave misinformation.

I was asking since it was your post that outlined clearly what is and isn't possible based on actual testing you had done.
After reading your posts, I assumed you were correct and that my cables were mislabeled and/or were defective or not working right.

It turns out that was the issue and the auction description was incorrect.

I have requested a return due to this.

P.S. Your alleged 100G AOC cable is also a QSFP+ 40G cable as the label on the transceiver says.
Fine. I get it.

The packaging and the auction description said otherwise though.

I will deal with this with the seller, and have requested a return of this item as well.

Many times an advertised speed of a cable is what the manufacturer is willing to guarantee, but the cable will go faster in reality.
Especially when the devices connected are all from the same manufacturer and that manufacturer has a proprietary faster speed.

This is seen quite often on networking products.
(e.g. proprietary WiFi speeds & protocols, also you can get often 10Gbe out of CAT5e cables)

It was reasonable to assume that with end-to-end Mellanox hardware, using an active, optical 40Gbit cable could very likely bump to 56Gbit.

Especially since your testing found that Mellanox is far less "picky" about cables when it came to Ethernet speeds.


By the way...

It's odd that you chose to assume I was accusing you of spreading misinformation, when it was more likely I was asking if you agreed that maybe my cables were defective or not working right.

Finally, there was no need for the snarky "...as the label on the transceiver says." retort - suggesting I'm being unobservant or something.
 

JTF195

New Member
Nov 15, 2017
13
7
3
33
Eh... yeah, the cables you have bought (QSFP28-04) are NOT QSFP28 cables, they are QSFP+ 40G cables.

Very unfortunate product naming.

See the manufacturer's datasheet:

Cable Assembly, High Speed Interconnect, 40Gb/s, Passive Direct Attach Cable (Copper), QSFP+ to QSFP+, 28 AWG, 4 Meter, PVC, Black

So, Siemon used the "QSFP28-4" product code because it's 28AWG, 4 meters.

That's kinda dumb...
 

applepi

Member
Jun 15, 2013
86
67
18
Been banging my head on this all day to realize my QSFP28-### cable is only QSFP+ ughhhh.