Mellanox 40GbE
- Thread starter T_Minus
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Thanks! I'll have to keep trying there, lol
MySQL Connect failed, server:localhost
URL Alias definitions are not found!
Sweet! The 'chart' I think is what will come in handy.
If only I could afford ConnectX5 100Gbe
****DROOOOL****
LOL
I'm working on a NAS cluster with these cards, very impressive to see 100GbitIf only I could afford ConnectX5 100Gbe
(not at home eh )If you need a cheap 40gb mellanox card, check ebay for rebranded cards like this one: 649281-B21 656089-001 661685-001 HP IB FDR/EN 10/40GB 2P 544QSFP ADAPTER | eBay
I'm sure this is covered elsewhere, but how does that set-up work?
I thought the QSFP to SFP+ breakouts were just for a QSFP switch port --> SFP+ NICs.
I believe like this: Using a 40GbE (QSFP+) NIC with a 10GbE Switch (SFP+)
@Patrick have you used the QSA module? Many people do not recommend to use the adapters owing to the quality issue or the high insertion loss.
I have but usually with 3m DACs so I have not seen any issues with about 20 in use over the past few months.
Hey Guys I was led here by Patricks quest for 40GBE. I myself have just bought two nice Arista 7050QX-32's and picked up some 3M Dacs. Now I am buying NICS.
I run EXSI and KVM so I thought I may be able to make use of the PRO models of ConnectX-3
After HOURS of looking at cards, websites and stuff seems my options are:
Pro does hardware offload - MCX314A-BCCT Dual 40/56GbE
Normal does some stuff - MCX314A-BCBT Dual 40/56GbE QSFP
With Infiniband - I don't need this:
Normal - MCX354A-FCBT Dual FDR 56Gb/s or 40/56GbE
PRO - MCX354A-FCCT Dual VPI FDR/40/56GbE
So I can see prices anywhere from $50 - $300 for these cards.
The pro cards seem to be in the $1000+ Retail range, and more heavy duty in terms of options.
I can get either the Generic MELLANOX cards in Mellanox ConnectX-3 EN or for the same price I can get the Mellanox ConnectX-3 Pro but they are OEM brand IBM/LENOVO .
Wondering if there would be any issue running OEM cards without reflashing them in EXSI 6.5 or Ubuntu 14.05 - 16.04 and Windows etc.
Do all these OEM cards work fine in Supermicro and Dell servers? Seems like a much better deal, if I have to flash them I could, done it to LSI cards before but just wondering if I even have to bother.
Thanks!
Once I get the deal I will send you the link also
I run EXSI and KVM so I thought I may be able to make use of the PRO models of ConnectX-3
After HOURS of looking at cards, websites and stuff seems my options are:
Pro does hardware offload - MCX314A-BCCT Dual 40/56GbE
Normal does some stuff - MCX314A-BCBT Dual 40/56GbE QSFP
With Infiniband - I don't need this:
Normal - MCX354A-FCBT Dual FDR 56Gb/s or 40/56GbE
PRO - MCX354A-FCCT Dual VPI FDR/40/56GbE
So I can see prices anywhere from $50 - $300 for these cards.
The pro cards seem to be in the $1000+ Retail range, and more heavy duty in terms of options.
I can get either the Generic MELLANOX cards in Mellanox ConnectX-3 EN or for the same price I can get the Mellanox ConnectX-3 Pro but they are OEM brand IBM/LENOVO .
Wondering if there would be any issue running OEM cards without reflashing them in EXSI 6.5 or Ubuntu 14.05 - 16.04 and Windows etc.
Do all these OEM cards work fine in Supermicro and Dell servers? Seems like a much better deal, if I have to flash them I could, done it to LSI cards before but just wondering if I even have to bother.
Thanks!
Once I get the deal I will send you the link also
usually there is no issue in reflashing those, done a couple of HPs myself for use in SM boards.
PROs have a better RocE support iirc and therefore might be more desirable. Normal ones can be had as cheaply as $50 per on a deal; have not seen PRO's so low.
VPI's can run in Infiniband or Ethernet mode so I'd not write them off.
PROs have a better RocE support iirc and therefore might be more desirable. Normal ones can be had as cheaply as $50 per on a deal; have not seen PRO's so low.
VPI's can run in Infiniband or Ethernet mode so I'd not write them off.
Got a super basic question... Purchased 3 HP 544qsfp 649281-b21; Will be doing networking without switch for now.
I need only 10G as of now (SFP+) but having a cable that supports 40G is also great. Question; if i buy QSFP+ Direct Attached Cable and not SFP+ will I still be able to use 10G ETHERNET between these cards?
I need only 10G as of now (SFP+) but having a cable that supports 40G is also great. Question; if i buy QSFP+ Direct Attached Cable and not SFP+ will I still be able to use 10G ETHERNET between these cards?
A qsfp or Quad-sfp is a 40 gig cable. Physically distinct from a sfp cable which run 10 or 1 gig
A qsfp card to card connection will be 40 gig Ethernet , skipping the discussion of infiniband mode.
You can buy, and certain network cards support, a breakout cable that’s a single qsfp on one end and 4 sfp on the other. That breaks a 40 gig into 4 separate 10 gig connections. I think I would avoid that for what you describe. I have a few of those cards and support for them is well documented on the forums.
A qsfp card to card connection will be 40 gig Ethernet , skipping the discussion of infiniband mode.
You can buy, and certain network cards support, a breakout cable that’s a single qsfp on one end and 4 sfp on the other. That breaks a 40 gig into 4 separate 10 gig connections. I think I would avoid that for what you describe. I have a few of those cards and support for them is well documented on the forums.