Mellanox Connectx-2 VPI Dual Port Adapter - $49.99 OBO

[T_Minus]

Just to confirm, these are VPI but can only do 10GbE, and not 40GbE?

AUCTION STATES:

Mellanox ConnectX-2 VPI Dual-Port 40GB Adapter Card HCA-30024 700Ex2-Q
40GB/S QSFP PCIE2.0 X8 5.0GT/S

Where are you getting that info?

 

[TeeJayHoward]

AUCTION STATES:

Mellanox ConnectX-2 VPI Dual-Port 40GB Adapter Card HCA-30024 700Ex2-Q
40GB/S QSFP PCIE2.0 X8 5.0GT/S

Where are you getting that info?

I was just reading it online myself. Can't remember the site, but I think it was in the user manual for these cards. It's 10 gig ethernet, or 40 gig Infiniband. No idea about IPoIB - Could be 40 gig there.

 

[T_Minus]

Ironically on: Mellanox Products: ConnectX-2 VPI Dual-Port Adapter InfiniBand and Ethernet Connectivity

It says:

• One adapter for InfiniBand, 10 Gigabit
  Ethernet and Data Center Bridging fabrics
• Virtual Protocol Interconnect
• 1us MPI ping latency over InfiniBand
• One 10, 20, or 40Gb/s InfiniBand
• One 10 Gigabit Ethernet SFP+ port
• PCI Express 2.0 (up to 5GT/s)
• CPU offload of transport operations
• End-to-end QoS and congestion control
• Hardware-based I/O virtualization
• TCP/UDP/IP stateless offload
• Fibre Channel encapsulation (FCoIB or FCoE)
• RoHS-R6

Why would auction state SFP and Mellanox site SFP+ ??? Guess we need to check card versions when we get them.

Also, to run IB at 40Gb/s what other transceiver/fiber is needed? I don't have interest in running it at 10GbE at this time, what about IPoIB?

 

[TeeJayHoward]

link

Also, to run IB at 40Gb/s what other transceiver/fiber is needed? I don't have interest in running it at 10GbE at this time, what about IPoIB?

1. 3.4.1 Optical Modules for QSFP
   
   These cards support optical modules for active fiber optical cables. The part ordering number for this module is MFM4R12C-QDR.

Why would auction state SFP and Mellanox site SFP+ ???

1. 3.4.2 Modules for QSFP to SFP+
   
   These cards support modules for SFP+ cables. The Mellanox Quad to Single Small Form Factor Pluggable adapter (QSFP to SFP+ adapter or Mellanox QSA) is built in QSFP form factor with a receptacle for an SFP+ cable connector. The part ordering number for this module is MAM1Q00A-QSA.
   

 

[TeeJayHoward]

Doing some more research into the qsfp vs qsfp+ form factors. The labels on those cut $3 cables showed a couple of mellanox switch models. The user manuals for both switches show just qsfp cables. Nothing in there at all about qsfp+.

I took a chance. I'm hoping they're MPO. I'm hoping they support INF-8438 (which is, from what I can tell, the qsfp spec?). The seller refused an offer of 40@1.25 per. He accepted my offer of 40@2 per.

 

[PnoT]

Anyone have a link to the short brackets as the vendor has not replied back to my question about whether they're short/tall on the cards.

 

[PithyChats]

Doing some more research into the qsfp vs qsfp+ form factors. The labels on those cut $3 cables showed a couple of mellanox switch models. The user manuals for both switches show just qsfp cables. Nothing in there at all about qsfp+.

I took a chance. I'm hoping they're MPO. I'm hoping they support INF-8438 (which is, from what I can tell, the qsfp spec?). The seller refused an offer of 40@1.25 per. He accepted my offer of 40@2 per.

Please report back, I might pick up a few myself. What cable will you be using? The auction states that cables were cut, so you'll need some new cables.

 

[MiniKnight]

Mellanox SSF 8436 QSFP MC2207310 Infiniband Optical Transceiver Emcore | eBay

Can you buy some of those QSFP+ transceivers and an optical cable, and connect two of these Connect-X2 cards together, or is there a difference between QSFP and QSFP+? At $3 per end, those would be some super cheap cables!

Come to think of it, what kind of connector's on the optical side? MPO? LC-LC? MTP?

I just bought 8 for $2.50 ea. If we figure out how to use them, great. If we don't then I'm out $20. Hoping they work in ConnectX-3

 

[buryat]

Anyone have a link to the short brackets as the vendor has not replied back to my question about whether they're short/tall on the cards.

The seller says:

• Uses Standard Profile.

I'd like to trade short for full-height brackets, I have 7 short, if someone wants to trade PM me.

 

[T_Minus]

EDIT: NM, I have enough large chassis

 

[PnoT]

The seller says:

• Uses Standard Profile.

I'd like to trade short for full-height brackets, I have 7 short, if someone wants to trade PM me.

I might let me see what I have.

 

[TeeJayHoward]

I just bought 8 for $2.50 ea. If we figure out how to use them, great. If we don't then I'm out $20. Hoping they work in ConnectX-3

That was basically my thought. My concern is that Mellanox wouldn't have put an MTP/MPO connector on there... That it's an optical cable that's not designed to be disconnected.

If nothing else, they'll serve to keep the dust out of the QSFP slots!

 

[Spartus]

That was basically my thought. My concern is that Mellanox wouldn't have put an MTP/MPO connector on there... That it's an optical cable that's not designed to be disconnected.

If nothing else, they'll serve to keep the dust out of the QSFP slots!

Please report back on this

 

[TeeJayHoward]

Please report back on this

I definitely will. With pictures. They should arrive some time this week. I think Thursday's the day eBay's telling me, but they're only a few hundred miles away, so they may get here quicker.

 

[digivampire]

I've just checked the Mellanox website on cabling regarding the transceivers (Cables - Mellanox Technologies). It seems like they should be supported and here is my reasoning:

Ebay seller lists the tag on the cabling as Mellanox MC2207310-015 part number. This is indeed FDR (http://www.mellanox.com/related-docs/prod_cables/DS_FDR_56Gbs_Active_Optical.pdf), but it is quoted as 2 equivalent part numbers, MC2207310-015 or MC2207312-003. From the cabling page you can see that the listed part number has two notes 1 and 4. From the note 4 quote:

Supported with the following SW/FW revisions or later: ConnectX-3 FW:2.11.0860, ConnectX-2 FW:2.9.1000, InfiniScale-4 FW: 7.4.2310, SwitchX FW:9.1.6670, 9.1.6494, 9.2.0000, SwitchX OS:3.2.0503-15, 3.3.3000, Tools:MFT 3.1.0-1

Since ConnectX-2 IS supported and it is NOT capable of FDR only QDR, so this should work! Now the question remains what the optical connectors are and if they are replaceable, we should see that shortly as I also ordered about a dozen of these. The listed incompatibility TeeJayHoward found I believe is purely when it comes to negotiation, i.e. both sides have to agree on either FDR or QDR transmit rates/protocols as posted by the standard.

 

[TeeJayHoward]

Since ConnectX-2 IS supported and it is NOT capable of FDR only QDR, so this should work! Now the question remains what the optical connectors are and if they are replaceable, we should see that shortly as I also ordered about a dozen of these. The listed incompatibility TeeJayHoward found I believe is purely when it comes to negotiation, i.e. both sides have to agree on either FDR or QDR transmit rates/protocols as posted by the standard.

I saw this as well, which is why I even attempted it. Additionally, in the Connect-X2 manual, the SFF-8436 standard was mentioned.

 

[dec]

The infiniband 1.2.1 standard, which is what these cards are built to, does not support 4x interface width at QDR rates using fiber. IB-4x-DDR-SX is defined to do 4x DDR at 850nm with an MPO fiber connector.

Mellanox may have a non-standard cable which supports higher rates over optical cables, however it's likely to be a fixed cable.

I would suggest you're probably safer with QSFP-QSFP copper cables to get the full 4x QDR rates that these cards (appear to be) capable of.

 

[PithyChats]

Here is the low profile version for the same price (albeit technically MHQH29C-XTR, but some may prefer this).

Mellanox Dualport Infiniband Adapter Card 40GBPS PCI E Low Profile MHQH29C XTR | eBay

 

[RandyC]

I picked up 8 of these @ $40 each last week. They just got delivered yesterday.

I pm'ed Chuckleb through ebay (by coincidence.) about low profile brackets. He had some spares and was nice enough to send me a few. I don't know if he has any more.

I also got 5 of these copper cables to see if they work well:
New QLogic Compliant 1M 4X QSFP IB Copper Cable – CBL1 0600130 | eBay

I have this Sun IB switch arriving in a few days (I got it through a different seller @$325):
Sun Data Center Infiniband Switch X2821A Z 602 4758 | eBay

Also, if anyone needs low profile cards:
MHQH29B XTR Mellanox Infiniband 4X QDR QSFP Connectx 2 VPI Dual Port 40GB s HCA | eBay

If anyone has good success with other cables let me know, as I need more, and don't know if copper or optical is preferred.

 

[TeeJayHoward]

If anyone has good success with other cables let me know, as I need more, and don't know if copper or optical is preferred.

I spent a lot of time talking with Rob over at Mellanox's store. Here's what I learned about Infiniband cables:

• Infiniband DOES care which cables are used.
• Passive cables are thicker, cheaper, and work great at short distances. There's no laser inside - It's just straight copper.
• Active cables are thinner, use more power, and work great at medium distances. There's a laser inside.
• Transceivers are used for long distances. There's obviously a laser inside.
  • Transceivers are broken down by protocol used - 40GbE, IB, or VPI (Both)
  • You cannot mix transceivers (Example: 40GbE on one end, IB on the other)
  • Mellanox cards work with transceivers from almost any brand. They're not like Brocade or HP, where you have use Brocade/HP branded transceivers.
• If you want to convert between IB and Ethernet, you need to use a switch capable of said conversion.
  • Said switches need to be licensed. Even if they're capable, they won't do it without a license.