40gbe DAC/fiber cables

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Jeff Robertson

Active Member
Oct 18, 2016
429
115
43
Chico, CA
I picked up a couple of HP branded Mellanox Connectx-3 40gbe cards that I need to connect together with either a 5M DAC or fiber cable. Any thoughts as to whether either of these would work?

Dell DAC-QSFP-40G-5M Copper Direct Attach Cable DP/N J90VN | eBay

NEW Mellanox MFS4R12CB-005 Fiber Optic Cable with QSFP connectors | eBay

For reference these are the cards that I need to directly link:

649281-B21 656089-001 661685-001 HP IB FDR/EN 10/40GB 2P 544QSFP ADAPTER | eBay

The end goal is to simply link both cards @ 40gbe.

Thanks!
 
  • Like
Reactions: T_Minus

Marsh

Moderator
May 12, 2013
2,643
1,496
113
Both cables should work.
For 5M run, I prefer fiber optic cable , plus cost is cheaper.
 

whitey

Moderator
Jun 30, 2014
2,766
868
113
41
Depends what he is hooking to/use case. If it's all IB then by all means either of those will work, if you're like me who has an existing 10G switch (Juniper EX3300) and QDR IB switch (Mellanox IS5022) and want to protect your 10G investment these cards will do 10/40G eth and 56G IB. With the cards being VPI series you can simply use one of these (THANKS @Patrick) to convert QSFP port to SFP+ and connect to 10G switch...wait for 40G ethernet switches to come down over the next year or two and past/future-proof yourself. That's what imma doin! These adapters will work w/ SFP+ DAC's and lamps/OM3/4 fiber, tested both to be working like a charm although I personally am migrating to fiber due to tidiness/OCD-ness even for my 10G connects :-D

NEW HP Mellanox QSFP/SFP+ Adapter Kit 655874-B21 MAM1Q00A-QSA | eBay

EDIT: I DO however like those fiber QSFP cables for my IB work, TIRED of dealing w/ unruly IB copper cables...may have to order three of those. Never seen them that cost effective and while I saw some HACKED/CHOPPED off version on ebay for cheap (I think mention of those here as well) I am no fiber splicing monkey :-D
 

Mark Wilde

New Member
Aug 21, 2016
18
5
3
37
Yes, you should use the fiber optic cable with QSFP+ connectors. Besides the reason that they are cheaper than the other, fiber optic cable usually have better performance than the copper optics.
 

Marsh

Moderator
May 12, 2013
2,643
1,496
113
@Jeff Robertson

Please , let us know what is the accepted BO price.
From another ebay seller, I was able to buy 4 x $15 (Best offer price ) per cable.
I am looking at the fiber cable that you linked , they are nice cable and I don't mind have few more.
 

whitey

Moderator
Jun 30, 2014
2,766
868
113
41
@Jeff Robertson

Please , let us know what is the accepted BO price.
From another ebay seller, I was able to buy 4 x $15 (Best offer price ) per cable.
I am looking at the fiber cable that you linked , they are nice cable and I don't mind have few more.
sh|t, I was gonna hold off a few weeks but I'd better jump before @Marsh buys them all haha j/k

Anyone know if those are 5meter...is that what the PN indicates?

Interested as well in someone probing them on BO pricing and reporting back :-D
 

whitey

Moderator
Jun 30, 2014
2,766
868
113
41
yes, the -005 suffix on the Mellanox MFS4R12CB-005 indicates a 5m cable. I'm pretty sure they are QDR (32Gb/s) capable.
Nice, QDR should be 40Gb/s capable though right or is there some bit encoding overhead I am not accounting for? No biggie either way good to know though cause in my head I think DDR 20G, QDR 40G, FDR 56G, EDR 100G, etc. :-D
 

aero

Active Member
Apr 27, 2016
346
86
28
54
QDR is 32gbps after taking the 8b/10b encoding into account. That's 8 bits of data for every 10 bits transmitted. Same with DDR, it's 16gbps.

With anything FDR and newer it's 64b/66b encoding, so much more efficient.

I've always disliked how infiniband tends to advertise the raw phy speeds. It's disingenuous.

Edit: by comparison, 10GbE is 10.3125gbps, using 64b/66b encoding, yielding a full usable 10gbps.
 
Last edited:

britinpdx

Active Member
Feb 8, 2013
367
184
43
Portland OR
Nice, QDR should be 40Gb/s capable though right or is there some bit encoding overhead I am not accounting for?
Indeed, QDR is 40Gb/s .. I should have stated that shows as a 32Gbs "link speed" in Windows ..



This is a Connectx-3 card (FDR capable) linked up to a Connectx-2 card (QDR capable). Link speed is certainly limited by the CX-2 card. Until I swap out the CX-2 I won't know if the MFS4R12CB-005 supports FDR or not.
 

whitey

Moderator
Jun 30, 2014
2,766
868
113
41
Well just to report back, I got the big 'decline-o' on offer of $20 for each of those cables (qty 3) so we at least have a starting point. Someone float him an offer of $25 each to feel him out, maybe $30 or so is all he will take ATM.

EDIT: Pitched in another offer @ $30 each that was accepted for qty3, oh and I had ebay bucks $24.46 avail so it was abt the same price as my $20 each offer plus $5...woohoo
 
Last edited:

i386

Well-Known Member
Mar 18, 2016
4,220
1,540
113
34
Germany
Not all qsfp cables Support 56gbit, I tried this Last Weekend with a brocade cable ans 2 connectx 3 vpi cards.
Reading through Wikipedia and mellanox Forums I think the Link Speeds are Programmed in the qsfp Connectors.
 

whitey

Moderator
Jun 30, 2014
2,766
868
113
41
Got my QSFP+ fiber cables in today, damn they are NICE lookin'/feelin' compared to the monstrosity copper QSFP+ cables.
 
Last edited:

whitey

Moderator
Jun 30, 2014
2,766
868
113
41
$30 each (got 3), didn't wanna piss them off and try $25 a piece since they rejected my $20 each offer. Seems reasonable for a high quality cable.
 

talmania

New Member
Jan 4, 2013
10
1
3
Thanks to Jeff and the others in this thread I've jumped on 2 of the HP's as well and the QSFP+ to SFP+ adapters Whitey linked. 60 shipped for each card and 14 shipped for the adapters. Bit more than the connectx-2 but I love the future proofing!!

Now to figure out which switch to get!
 

talmania

New Member
Jan 4, 2013
10
1
3
Yeah I know I've been leaning that way and think I'll end up with a Cisco for learning and use the uplinks to the Ubiquiti.