Purchased QSFP+ cards : Need cable help

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

0xbit

New Member
Aug 7, 2017
23
4
3
Good Day Servethehome forum members,
I've just purchased two MCX314A-BCCT Dual 40/56GbE cards and now need cables for them.
I already have, in my cart, two Mellanox QSFP+ to SFP+ adapters for backwards compatibility to SFP+ cards I have.

Now, I am on the hunt for QSFP+ cables.. I've seen multiple types and most are listed as infiniband. Is there a difference between the infiniband QSFP+ cables and ethernet? Or is a QSFP+ cable w/ line rated speed a QSFP+ w/ line rated speed? I only need a 3m run and was wondering if the following would suffice for 56GbE speeds on my Nic :


Can experts provide thoughts? I'll be connecting two computers directly via these two cards.
:) Thanks so much you guys.
 

i386

Well-Known Member
Mar 18, 2016
4,221
1,540
113
34
Germany
Infiniband FDR qsfp cables are actually QSFP14 cables (4 lanes a 14gbit/s) and can be used for infiniband up to FDR (SDR,DDR,QDR are also supported) and as external SAS3 cables (the hgst 60 bay jbod for example needs qsfp14 cables).

40gbe qsfp cables are QSFP+ cables (4 lanes a 10gbit/s).

If you want to use only 40gbe (or qdr infiniband) then you can buy the "cheaper" qsfp+ cables.
 
  • Like
Reactions: 0xbit

0xbit

New Member
Aug 7, 2017
23
4
3
Infiniband FDR qsfp cables are actually QSFP14 cables (4 lanes a 14gbit/s) and can be used for infiniband up to FDR (SDR,DDR,QDR are also supported) and as external SAS3 cables (the hgst 60 bay jbod for example needs qsfp14 cables).

40gbe qsfp cables are QSFP+ cables (4 lanes a 10gbit/s).

If you want to use only 40gbe (or qdr infiniband) then you can buy the "cheaper" qsfp+ cables.
Excellent information i386 !
So, generically speaking it's simply :
QSFP+ w/ SDR,DDR,QDR, FDR speccing
which can be used for infiniband/Ethernet.
I just learned something today..
upload_2017-11-5_12-23-22.png
So, just to confirm, I am looking at
QSFP+ cables rated at either 40GbE/56GbE for my Mellanox Nics which support 40GbE/56GbE.
As such, I am looking at 40GbE (QDR) QSFP+ or 56GbE (FDR) QSFP+.
So, this :
New Mellanox HP 498386-B23 3M 4x DDR/QDR QSFP IB Fiber Optic Cable | eBay
would get me 40GbE speeds (QDR)

and this :
00ND493 IBM MELLANOX TECHNOLOGIES INFINIBAND QSFP 56GB/S PASSIVE COPPER CABLE | eBay
would get me 56GbE speeds (FDR)

Both will work on my Mellanox QSFP+ ethernet nic?

Final questions, are there any gotchas with FDR(56GbE) that I should be aware of vs 40GbE? The mellanox cards are x8 PCIe 3.0 and dual port. However, Pcie 3.0 x8 is bandwidth limited to 63.04 Gbps. So, It doesn't seem I'll be able to get 56Gbps speeds x2 anyway. That being said, will i get reduced latency of transmit given that the lines run at 14Gbps vs 10Gpbs? Really apologize for the amateur questions if they seem as such.

If all i need is 3m, is there a benefit to going fiber optical vs Copper w.r.t to latency? Bit errors?
Thank you so very much for the all of the help. Hoping to finalize my purchases by this evening so I have yummy stuff to play with soon ^_^.
 

i386

Well-Known Member
Mar 18, 2016
4,221
1,540
113
34
Germany
QSFP+ w/ SDR,DDR,QDR, FDR speccing
QSFP+ w/ SDR,DDR,QDR*

FDR is Fourteen Data Rate Infiniband (14gbit/s) and requires qsfp14 cables, qsfp+ cables are not enough.

56 GBE is Mellanox proprietary implementation of ethernet, it's not a standard and not compatible to other proprietary 56 gbit/s ethernet implementations (eg Cisco uplink modules with 56gbit/s)

If you need just 3m use DAC cables (Direct Attach Copper cables), they are usually cheaper than 2 optical transceivers + fibers.
 
  • Like
Reactions: nthu9280 and 0xbit

0xbit

New Member
Aug 7, 2017
23
4
3
QSFP+ w/ SDR,DDR,QDR*

FDR is Fourteen Data Rate Infiniband (14gbit/s) and requires qsfp14 cables, qsfp+ cables are not enough.

56 GBE is Mellanox proprietary implementation of ethernet, it's not a standard and not compatible to other proprietary 56 gbit/s ethernet implementations (eg Cisco uplink modules with 56gbit/s)

If you need just 3m use DAC cables (Direct Attach Copper cables), they are usually cheaper than 2 optical transceivers + fibers.
Thank you so much i386 ! You helped immensely and in a super timely fashion. I'm off to ebay to make some purchases :). Super excited !
 

0xbit

New Member
Aug 7, 2017
23
4
3
My Qsfp14 cables came in for my fun little Mellanox 40gigEth Nics and the cables work like a charm !
Mellanox QSFP Passive Copper FDR14 Infiniband Cable running in 40GigEth mode.
Thank you so much i386 !
 

xephael

New Member
Jun 4, 2019
7
0
1
QSFP+ w/ SDR,DDR,QDR*

FDR is Fourteen Data Rate Infiniband (14gbit/s) and requires qsfp14 cables, qsfp+ cables are not enough.

56 GBE is Mellanox proprietary implementation of ethernet, it's not a standard and not compatible to other proprietary 56 gbit/s ethernet implementations (eg Cisco uplink modules with 56gbit/s)

If you need just 3m use DAC cables (Direct Attach Copper cables), they are usually cheaper than 2 optical transceivers + fibers.

I have a Mellanox branded cable that mentions 56GbE QDR/FDR10 but I'm not seeing 56GbE in mxltool when looking at port rates. Does that mean Mellanox's store is wrong about it?
 

i386

Well-Known Member
Mar 18, 2016
4,221
1,540
113
34
Germany
The link you posted is for a qdr 40gbit/s cable.

The text is generic and says UP TO 14gbit/s per lane. The "specifications" for this cable are listed under condtion.
 

azev

Well-Known Member
Jan 18, 2013
768
251
63
I am currently still battling some compatibility issue with the same Mellanox cards and Cisco Nexus 3064 switches.
I've purchase 2 cisco compatible qsfp+ cable (non cisco branded) hoping to get 40Gb ethernet link on my nexus switches, and both does not work.
Not sure what switches you are planning to connect these cards to but it may have some impact to the type of cables that you need.