Is this the same deal back on?
Mellanox MCX311A-XCAT 10 Gigabit Ethernet PCIe Adapter Card ConnectX-3 MCX311A | eBay
Mellanox MCX311A-XCAT 10 Gigabit Ethernet PCIe Adapter Card ConnectX-3 MCX311A | eBay
Mellanox MCX311A-XCAT 10 Gigabit Ethernet PCIe Adapter Card ConnectX-3 CX311A $12 obo + $15 Shipping
- Thread starter zeynel
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
I think for stable/reliable Performance the whole network should be lossless.
This means also the Switch should be configured for 'Global Pause' or 'PFC' on the Ports used for RoCE - what can be quite painful and a limit on usable Switches.
Never setup RoCE because of this but would be interested in some reports if it's worth the efforts involved.
I got a few requests for printed brackets. Will do, but haven't designed them yet, because I haven't received the cards yet. If you are in a rush I would suggest ordering from Mellanox. I'll report back once I have designed and printed mine.
Cheers, all.
Cheers, all.
I am starting to play with the cards as well and so far I am finding that the firmware is slightly outdated.
If you are interested in updating the firmware, you can download the MFT package from Mellanox using this link:
Mellanox Firmware Tools
MFT User Manual
In the manual they have a section that talks about being able to check the firmware directly against their website and perform the update.
However what I've discovered is that the firmware that appears to be available from Mellanox website directly is actually newer than what the tool is able to detect.
Here is the link to the latest firmware that I was able to find for these cards:
FW-ConnectX3-rel-2_42_5000-MCX311A-XCA_Ax-FlexBoot-3.4.752.bin.zip
I also think its possible to enable RDMA with these cards using the Quanta LB6M 10GB switch because it supports DCB and PFC which should enable the lossless network. Now I just need to learn how to setup VLANs, enable Trunk Ports and I should be a happy camper
If you are interested in updating the firmware, you can download the MFT package from Mellanox using this link:
Mellanox Firmware Tools
MFT User Manual
In the manual they have a section that talks about being able to check the firmware directly against their website and perform the update.
However what I've discovered is that the firmware that appears to be available from Mellanox website directly is actually newer than what the tool is able to detect.
Here is the link to the latest firmware that I was able to find for these cards:
FW-ConnectX3-rel-2_42_5000-MCX311A-XCA_Ax-FlexBoot-3.4.752.bin.zip
I also think its possible to enable RDMA with these cards using the Quanta LB6M 10GB switch because it supports DCB and PFC which should enable the lossless network. Now I just need to learn how to setup VLANs, enable Trunk Ports and I should be a happy camper
Fudge
Double Fudge
So, now I have super pricey Pro cards that I didn't even need .. FML
Any other benefit of these pro cards? do they do more hardware level processing of network traffic data? Time to fire up ebay, offload these cards I have and wait for a mcx311a-xcat-10-gbit sale .
Also, I never found that Connect-x2 ETH HP firmware website link even though everybody mentions it? Can you provide linkage please?
@0xbit,
Here is a link to the simple comparison that I found on the Mellanox website:
Adapter comparison
The biggest things that I see are related to RoCE v2 support (Routing) and hardware offloading for Overlay Networks (VXLAN/NVGRE) which are important for SDN (Software Defined Networking).
So it all depends on how crazy you want to go with your setup. If you are playing with things like AzureStack, then this is probably important and in fact you probably want to go to ConnectX-4 or 5 cards. However if you are doing a smaller home setup, then these cards should be plenty.
For me, I am going to figure out how to get RDMA working and in order to get around the RoCE v1 limitation of routing, I am going to change my network config to probably leverage the /16 address space because I am planning to build out a SDN based lab using SCVMM with Network Controllers, etc....
Once I get this sorted, I really want to get the ConnectX-5 cards because of the eSwitch capability and the fact that you can link up to 4 servers together without a switch by looping the dual ports together. Here is a LINK
I did install the WinOF package but the firmware was still older than what was available.
I ended up having to download the firmware directly from their site and using flint tool to flash it to the latest version.
Went ahead and ordered the brackets from Mellanox Store.
Mellanox MTM007160 FRU Tall Perforated bracket for 1-port SFP+ adapter with gasket. Fits MCX312B / MCX311A
Mellanox MTM007160 FRU Tall Perforated bracket for 1-port SFP+ adapter with gasket. Fits MCX312B / MCX311A
Yeah if you get cheap shipping then the store is probably worth it. I just got my cards so I'll do up a bracket design this weekend. Took a while because I had to get it brokered across the boarder into canada etc.
hey, how many of you guys need these brackets and how many? shipping from Mellanox might be expensive for small orders, but if I order in bulk, the high shipping costs is reduced per bracket and then I can probably re-send them back out 1st class post for cheap, likely $3 or so within USA. So, for small quantities, this might make sense to do.... i'm happy to donate my time as long as my costs are covered. the postman picks up packages from my place every day anyway and I've got some used envelopes I can reuse. let me know if that's something you guys want to consider and we can then figure out if there's enough people and the details.
Just FYI,
Mellanox told me that the perforated ones would take 4-6 weeks to ship.
They we're okay with shipping a replacement bracket MTM005078 which was " not perforated, but should fit the MCX312B", those being 16 in stock.
I preferred to wait.
Mellanox told me that the perforated ones would take 4-6 weeks to ship.
They we're okay with shipping a replacement bracket MTM005078 which was " not perforated, but should fit the MCX312B", those being 16 in stock.
I preferred to wait.
Thanks for the offer - I'm seeing USD80 for the parts and USD70 for shipping 6 of them to Australia. Crazy.
What do you reckon it would end up being if I brokered through you? Obviously the parts are the same.
well, it looks like Mellanox just jacked up the price. it was previously $4 per bracket; shipping was a bit high. they probably saw a sudden demand for it and changed the price...