Mellanox Connectx-2 VPI Dual Port Adapter - $49.99 OBO

ultradense

Member
Feb 2, 2015
61
11
8
40
Thanks! Great info.

Does this also mean you can mix and match brands of (passive) cables/nics/switches without any problems? If so, why are all vendors claming you should match by brand compatibility?

To bad I just ordered a bunch of cables a few hours ago...
 

PnoT

Active Member
Mar 1, 2015
625
146
43
Texas
This is all going in the same 25U rack for iSCSI in ESXi, so 3 meter copper cable should be fine.

I just hope the Sun X2821A-Z Switch isn't a brick attached to a jet engine. (I really hope it has a subnet manager, but no biggie if not.)

Does anyone have some good tutorials on setting up IB and iSCSI / ZFS on Ubuntu or CentOS to be used as storage for VMWare?
I'm still thinking about buying one of those switches and I've heard it's pretty loud but would love to hear some feedback on firmware, flashing, and maybe ideas on how to quiet that thing down. I'm going to pull the trigger on one.
 

TeeJayHoward

Active Member
Feb 12, 2013
376
111
43
Does this seller normally have issues shipping? I'm anxious to receive my cards. I paid on the 28th, had a shipping label created on the 30th, and since then... Nothing.
 

PnoT

Active Member
Mar 1, 2015
625
146
43
Texas
Does this seller normally have issues shipping? I'm anxious to receive my cards. I paid on the 28th, had a shipping label created on the 30th, and since then... Nothing.
Ebay confirmed my item was shipped but no tracking information is available and it's been 2 days. A USPS track indicates the following:

March 30, 2015 , 11:20 am
Shipping Label Created
BROOKLYN, NY 11220
 

TeeJayHoward

Active Member
Feb 12, 2013
376
111
43
Ebay confirmed my item was shipped but no tracking information is available and it's been 2 days. A USPS track indicates the following:

March 30, 2015 , 11:20 am
Shipping Label Created
BROOKLYN, NY 11220
Same here.
 

TeeJayHoward

Active Member
Feb 12, 2013
376
111
43
Please report back, I might pick up a few myself. What cable will you be using? The auction states that cables were cut, so you'll need some new cables.
Please report back on this
I just bought 8 for $2.50 ea. If we figure out how to use them, great. If we don't then I'm out $20. Hoping they work in ConnectX-3
They're definitely NOT MPO. If you could get ahold of the cables Mellanox uses, I'm sure it would work, but the connector's not like anything I've seen before. Pics:


 
  • Like
Reactions: NME and T_Minus

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,342
1,797
113
CA
I just got my copper cables today for doing: HOST <--COPPER--> HOST
We'll see how they work when I wrap up these ESXI builds, got the final parts coming next week.

A lot more than I expected at the start :D
 

tubeamps

Active Member
Dec 1, 2014
102
25
28
They're definitely NOT MPO. If you could get ahold of the cables Mellanox uses, I'm sure it would work, but the connector's not like anything I've seen before. Pics:


Can you post a picture of the end of the cable and other side?

These adaptors and the ConnectX-2 both appear to be SFF-8436 compliant. In this picture that looks similar to a MPT/MPO-12 connector. I'm not that familiar with MPO, but it almost looks like an MPO cable without the outermost housing. Maybe this was done to make things fit?

I should receive my adaptors today. I'm happy to at least see QSFP to SFP+ transceivers for fairly cheap. If nothing else these can serve as decent 10gb ethernet adaptors until until I can work something else out. I would really like some inexpensive 40gb 50-100ft fiber runs though.
 
  • Like
Reactions: TeeJayHoward

TeeJayHoward

Active Member
Feb 12, 2013
376
111
43
Can you post a picture of the end of the cable and other side?
Sure. I don't want to muck up this thread any more than I already have, though, so let's move the conversation to my QSFP thread here. I'm uploading the requested photos now.

These adaptors and the ConnectX-2 both appear to be SFF-8436 compliant. In this picture that looks similar to a MPT/MPO-12 connector. I'm not that familiar with MPO, but it almost looks like an MPO cable without the outermost housing. Maybe this was done to make things fit?
It DOES look very similar. I wonder...
 

sasha

New Member
Dec 9, 2013
11
1
3
Reading the thread and getting lost a little bit.

Le me ask a few specific questions:

if I buy 2 x of these and put them into 2 servers running esx

  • I should be able to connect them directly without switch for IPoIB
    • So essentially I'd have two separate links @ QDR between two servers (running subnet manager on esx)
  • What cable/modules do I need to achieve full QDR rate between them, MAM1Q00A-QSA and something else?
  • If I scale back my ambition, and decide to go with just 10G ethernet for now..what cable/modules?
thank you in advance for clarifying it for me
 

Chuckleb

Moderator
Mar 5, 2013
1,017
331
83
Minnesota
If you are looking for ease of use, go with the 10GbE option and a QSFP cable between them. That works very easily, no special configurations needed.

Moving to in difficulty, to go IPoIB, you will need to run the aforementioned subnet manager. For IPoIB or any IB, you will want to use the QSFP cables, don't convert using the QSA adapters.

Also don't bother trying dual link IB, adds complexity and you can't saturate 40Gbps over the PCIe anyway. Single link is fine.

Basically, card and single QSFP cable between will give you lots of options.
 
  • Like
Reactions: epicurean

TeeJayHoward

Active Member
Feb 12, 2013
376
111
43
Step 4: MLNX_VPI_WinOF-4_80_All_win2012R2_x64 ( This is the version I used , there is a newer 4_90 version now ),
During Installation , Installer should upgrade card with the latest firmware, if it recognized the card.
This didn't work for me. The 4.80 version only has 2.9.1200. The 4.90 version doesn't support the ConnectX2 cards at all. Here's how I got one of my cards to 2.10.720:
1) Collect the necessary files: WinMFT, fw-ConnectX2-rel-2_9_1000-MHQH29B_A3-A5.bin, MHQH29B_A3-A5.ini, 2.10.720.mlx (thanks dba!). Put them all in the same directory. Execute any commands from that directory as an Administrator.
2) Install WinMFT.
3) Change the PSID (and downgrade the firmware to 2.9.1000).
flint -allow_psid_change -d mt26428_pci_cr0 -i fw-ConnectX2-rel-2_9_1000-MHQH29B_A3-A5.bin burn
4) Reboot
5) Upgrade the firmware to 2.10.720.
mlxburn -dev mt26428_pci_cr0 -fw 2.10.720.mlx -conf MHQH29B_A3-A5.ini
I've since simplified the process by using mlxburn to combine the .mlx and the .ini into a .bin file which we can use to change the PSID and upgrade the firmware in one step. Here's a 2.10.720 image for the MHQH29B-XTR cards. I would really, REALLY appreciate it if someone would upload it to their server and let us grab it from there - You can't trust my stuff to be up and running when you need it. :D
1) Install WinMFT.
2) Open up a command prompt as an administrator, and CD to the directory you copied the 2.10.720 image to. Run the following command:
flint -allow_psid_change -d mt26428_pci_cr0 -i MHQH29B-2.10.720.bin burn
3) Reboot
1) Download the ESXi MFTs for your version (5.5 link | 6.0 link - Both from here)
2) Upload the mft .vib file and the firmware .bin to your host using whatever method you want. (I'm partial to dumping it in a datastore, then moving it to /var/tmp.)
3) SSH into your ESXi host and install the mft:
esxcli software vib install -v /var/tmp/mft-3.8.0.56-10EM-600.0.0.2295424.x86_64.vib
4) Reboot
5) Update the firmware:
/opt/mellanox/bin/flint -allow_psid_change -d mt26428_pciconf0 -i /var/tmp/MHQH29B-2.10.720.bin burn
6) Reboot again
Code:
yum install -y pciutils usbutils wget rpm-build
cd /var/tmp
wget http://www.mellanox.com/downloads/MFT/mft-3.8.0-56.tgz
wget http://www.landynamics.org/pool/Applications/Drivers/MHQH29B-XTR/MHQH29B-2.10.720.bin
tar zxvf mft-3.8.0-56.tgz
cd mft-3.8.0-56
./install.sh
mst start
cd /var/tmp
flint -allow_psid_change -d /dev/mst/mt26428_pci_cr1 -i /var/tmp/MHQH29B-2.10.720.bin burn
 
Last edited:

epicurean

Active Member
Sep 29, 2014
743
63
28
I've used generic cables w/o any problems, but you can almost always find cheap copper QSFP cables on ebay.
Hi Chuck,
Could point me to sources that sell these generic cables and connectors that will allow me to work with these adapters without much grief?
 

cw823

Active Member
Jan 14, 2014
363
145
43
Hi Chuck,
Could point me to sources that sell these generic cables and connectors that will allow me to work with these adapters without much grief?
^^^^^^^ what he said.

But pretend I'm an idiot and make the explanation as simple as possible. :)