Mellanox MCX311A-XCAT 10 Gigabit Ethernet PCIe Adapter Card ConnectX-3 CX311A $12 obo + $15 Shipping

Discussion in 'Great Deals' started by zeynel, Nov 27, 2017.

  1. BLinux

    BLinux cat lover server enthusiast

    Joined:
    Jul 7, 2016
    Messages:
    2,359
    Likes Received:
    823
    well, i received my order of 10 today. they are tiny, and indeed half height brackets. they were just wrapped in bubble wrap; would really have preferred them to individually placed in anti-static bags, but oh well... will try to test them out over the weekend of next week.
     
    #41
    zeynel likes this.
  2. iLya

    iLya Member

    Joined:
    Jun 22, 2016
    Messages:
    48
    Likes Received:
    5
    The latest sale where it states 50 items per lot is actually a mistake. I just got notified by the seller asking if I wanted a refund or the individual cards. That being said, if these are truly the ConnectX-3 cards and are fully functional, then even at $23 a card with the shipping is still a great price.
     
    #42
  3. hakabe

    hakabe Member

    Joined:
    Jul 6, 2016
    Messages:
    112
    Likes Received:
    2
    Hopefully for $23 you'll get it shipped properly (anti-static bags etc). People should give feedback on that even when the price was low as $12. I would prefer packed inside a newspaper than inside a bubble wrap.
     
    #43
    MikeH likes this.
  4. iLya

    iLya Member

    Joined:
    Jun 22, 2016
    Messages:
    48
    Likes Received:
    5
    @hakabe, have you tested your cards yet? with the normal price of ~$80-$100 a card on ebay for something comparable, I am wondering if this is just an awesome deal or if someone made a different mistake or sold a bunch of untested cards.
     
    #44
  5. hakabe

    hakabe Member

    Joined:
    Jul 6, 2016
    Messages:
    112
    Likes Received:
    2
    No, my package is on its way. Probably it takes a week to get into the EU. On eBay there's Dual port 10/40Gbps Mellanox NICs for ~$60.

    I am keen to see the results from the users who has already received the NICs.
     
    #45
  6. britinpdx

    britinpdx Active Member

    Joined:
    Feb 8, 2013
    Messages:
    348
    Likes Received:
    151
    I received my cards yesterday and did some quick testing this morning on one of the cards.

    As previously noted, these are half height cards with low profile brackets. Compared to the previous gen ConnectX2 card (MNPA19) the MCX311A card is approx 3/4 length, and has a x4 pci-e connector compared to the x8 pci-e connector on the MNPA19. The 2 bracket mounting holes appear to be identical. The "status LEDs" are different between the MNPA19 and MCX311A .

    I swapped out the tall bracket from a MNPA19-XTR card that I have in order to test the MCX311A card in my S2012R2 4U test box (which I have previously had all sorts of Mellanox cards in) and it was recognized right away.

    I grabbed the latest drivers/firmware/MFT/documentation from the Mellanox site.Previous generations of Mellanox VPI/Eth cards required FW to be downloaded and burnt but in reading the docs for these cards they apparently can be flashed "online". The FW on the card was older so I flashed using the "online" ability ..

    Code:
    PS C:\downloads\mellanox cx311a s2012r2> mst status
    MST devices:
    ------------
      mt4099_pci_cr0
      mt4099_pciconf0
    PS C:\downloads\mellanox cx311a s2012r2> mlxfwmanager -d mt4099_pci_cr0 --query
    Querying Mellanox devices firmware ...
    
    Device #1:
    ----------
    
      Device Type:      ConnectX3
      Part Number:      MCX311A-XCA_A1
      Description:      ConnectX-3 EN network interface card; 10GigE; single-port SFP+; PCIe3.0 x4 8GT/s; RoHS R6
      PSID:             MT_1170110023
      PCI Device Name:  mt4099_pci_cr0
      Port1 MAC:        0002c9239a10
      Port2 MAC:        0002c9239a11
      Versions:         Current        Available
         FW             2.10.4290      N/A
         PXE            3.4.0000       N/A
    
      Status:           No matching image found
    
    PS C:\downloads\mellanox cx311a s2012r2> mlxfwmanager --online -u -d mt4099_pci_cr0
    Querying Mellanox devices firmware ...
    
    Device #1:
    ----------
    
      Device Type:      ConnectX3
      Part Number:      MCX311A-XCA_A1
      Description:      ConnectX-3 EN network interface card; 10GigE; single-port SFP+; PCIe3.0 x4 8GT/s; RoHS R6
      PSID:             MT_1170110023
      PCI Device Name:  mt4099_pci_cr0
      Port1 MAC:        0002c9239a10
      Port2 MAC:        0002c9239a11
      Versions:         Current        Available
         FW             2.10.4290      2.40.7000
         PXE            3.4.0000       3.4.0746
    
      Status:           Update required
    
    Release notes for the available Firmware:
    -----------------------------------------
    
      For more details, please refer to the following FW release notes:
        1- ConnectX3 (2.40.7000):    http://www.mellanox.com/pdf/firmware/ConnectX3-FW-2_40_7000-release_notes.pdf
        2- ConnectX3Pro (2.40.7000): http://www.mellanox.com/pdf/firmware/ConnectX3Pro-FW-2_40_7000-release_notes.pdf
        3- Connect-IB (10.16.1020):  http://www.mellanox.com/pdf/firmware/ConnectIB-FW-10_16_1020-release_notes.pdf
        4- ConnectX4 (12.20.1010):   http://www.mellanox.com/pdf/firmware/ConnectX4-FW-12_20_1010-release_notes.pdf
        5- ConnectX4Lx (14.20.1010): http://www.mellanox.com/pdf/firmware/ConnectX4Lx-FW-14_20_1010-release_notes.pdf
        6- ConnectX5 (16.20.1010):   http://www.mellanox.com/pdf/firmware/ConnectX5-FW-16_20_1010-release_notes.pdf
    
    ---------
    Found 1 device(s) requiring firmware update...
    
    Perform FW update? [y/N]: y
    
    Please wait while downloading MFA(s) 100%
    Device #1: Updating FW ...
    Done
    
    Restart needed for updates to take effect.
    PS C:\downloads\mellanox cx311a s2012r2>
    
    The tall bracket for the MCX311A appears to be available at the Mellanox Store ..

    So all in all I rate this under the "awesome deal" category !
     
    #46
    epicurean, hakabe and roswellian like this.
  7. iLya

    iLya Member

    Joined:
    Jun 22, 2016
    Messages:
    48
    Likes Received:
    5
    Great news @britinpdx, I purchased 2 of the cards under the first deal and then regretted that I didn't get more even if I couldn't use them all, I could have at least have some spares. When I realized that the same seller was selling more cards, I got more of those cards through a offer so I got them a bit cheaper than $23 and now I am happy that I did.
    With these cards supporting RDMA, they should do great for using them for SDS(Software Defined Storage) testing which requires really low latency network configuration.
     
    #47
  8. britinpdx

    britinpdx Active Member

    Joined:
    Feb 8, 2013
    Messages:
    348
    Likes Received:
    151
    Verified with the Finisar FTLX8571D3BCL.
     
    #48
  9. Richard Wad

    Richard Wad New Member

    Joined:
    Feb 7, 2017
    Messages:
    28
    Likes Received:
    5
    Got mine in. All but one were made in China. One was made in Israel. Looks identical to the Chinese cards.
    All came wrapped nicely in pink antistatic bubble wrap. This wrap will protect cards from static buildup but not static discharge, just have to ground yourself before handling.
     
    #49
  10. hakabe

    hakabe Member

    Joined:
    Jul 6, 2016
    Messages:
    112
    Likes Received:
    2
    One foot to the heater radiator and you're all set :D
     
    #50
  11. 0xbit

    0xbit New Member

    Joined:
    Aug 7, 2017
    Messages:
    23
    Likes Received:
    4
    Only Connectx-3 Pro cards and Connectx-4 and above cards support RDMA aka ROCE for these Eth cards.
     
    #51
  12. iLya

    iLya Member

    Joined:
    Jun 22, 2016
    Messages:
    48
    Likes Received:
    5
    #52
  13. 0xbit

    0xbit New Member

    Joined:
    Aug 7, 2017
    Messages:
    23
    Likes Received:
    4
    I'm right. The Brochure caused me to think the same thing thus why I own some pro cards and non pro cards. Here's the details you're looking for : http://www.mellanox.com/page/products_dyn?product_family=79 .. Scroll down to hardware support at the bottom.
    discussion :
    Difference between PRO VPI and PRO EN adapter (... | Mellanox Interconnect Community

    I too wish that features were detailed on a concise splash page for equal comparison. Don't even get me started on roce v1 vs. v2. So, there it is.. Roce is only available on connectx-3Pro and Connectx-4 and up. There is a software emulation for Roce for cards that don't support it but you're ofc looking at added latency as it wont be done in hardware on the Connectx card? Here's linkage to the software implementation :
    Software-based RoCE, A New Way to Experience RDMA - RoCE Initiative
     
    #53
  14. iLya

    iLya Member

    Joined:
    Jun 22, 2016
    Messages:
    48
    Likes Received:
    5
    @0xbit, I guess I am still hopeful and here is why:
    1. User ophirmaor in the community thread is explicitly stating that RDMA is supported on both types (VPI and EN) and that Pro supports RoCE v2 but non-Pro supports v1
    2. Reading the WinOF (not WinOF-2) user manual which can be found here: LINK
      is stating that RoCE is supported on ConnectX-3 and ConnectX-3 Pro with firmware version 2.30.3000 or higher (page 44 under System Requirements)

    What I never even though about is the SoftROCE which sounds kind of interesting and I am wondering if it is a cheap solution for the home lab configurations as long as your hardware is beefy enough. Unfortunately it is only supported for Linux OSs which doesn't help with Hyper-V scenarios.

    Regarding lack of RoCE v2 support, the only time I see this really causing me an issue is if I decide to do nested virtualization and expose storage directly to some of those nested hypervisors which would most likely require traffic routing and this RoCE routing is only supported in v2.

    I really need to plug these cards in and play with them a bit and see if I can gain any benefit over my normal non RoCE storage access that I have going on at the moment.
     
    #54
    0xbit likes this.
  15. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,683
    Likes Received:
    412
    RDMA over converged ethernet works with mellanox adapters since the cx-2 ethernet adapters (the "cheap as dirt" cards :D).
    For cx-2 you will need a certain firmware version that's only available through hp, cx-3 and newer adapters should support it with any firmware version.
    For rdma on windows you need to enable "NDK Global Pause" in thre driver settings.
     
    #55
    0xbit likes this.
  16. 0xbit

    0xbit New Member

    Joined:
    Aug 7, 2017
    Messages:
    23
    Likes Received:
    4
    Fudge :(

    Double Fudge :(

    So, now I have super pricey Pro cards that I didn't even need .. FML
    Any other benefit of these pro cards? do they do more hardware level processing of network traffic data? Time to fire up ebay, offload these cards I have and wait for a mcx311a-xcat-10-gbit sale .

    Also, I never found that Connect-x2 ETH HP firmware website link even though everybody mentions it? Can you provide linkage please?
     
    #56
    Last edited: Dec 3, 2017
  17. hakabe

    hakabe Member

    Joined:
    Jul 6, 2016
    Messages:
    112
    Likes Received:
    2
  18. Spartus

    Spartus Member

    Joined:
    Mar 28, 2012
    Messages:
    139
    Likes Received:
    19
    So, seeing as how I will be 3d printing myself a dozen full height brackets for these cards, if anyone else needs some send me a PM.
     
    #58
    Antonio likes this.
  19. BLinux

    BLinux cat lover server enthusiast

    Joined:
    Jul 7, 2016
    Messages:
    2,359
    Likes Received:
    823
    finally had some time to test these out last night. some basic 2-way testing with iperf3 revealed two things to me:

    1) if you setup the IP wrong and end up iperf3 connecting to yourself, it'll show you 20Gbps bandwidth!!!
    2) if you didn't know your motherboard has 1 slot that is PCI-E 1.0, this card will only operate around 5Gbps in that slot!

    after all that silliness, all 10 of my order worked perfectly fine with FS SFP+ modules and Cisco DACs. these don't seem to be too picky about SFP+ or cables.
     
    #59
  20. epicurean

    epicurean Member

    Joined:
    Sep 29, 2014
    Messages:
    543
    Likes Received:
    20
    Awesome! Could I trouble u to print say 4 for me?
     
    #60
Similar Threads: Mellanox MCX311A-XCAT
Forum Title Date
Great Deals Mellanox MCX311A-XCAT 10 Gigabit Ethernet Card w/Transceiver Low Profile - $65 Mar 4, 2017
Great Deals Mellanox ConnectX-3 on ebay look strange Oct 21, 2019
Great Deals Mellanox ConnectX-3 Pro (HPE) for $49.99 Oct 6, 2019
Great Deals Mellanox QSFP to SFP+ Adapter $10 Oct 1, 2019
Great Deals Mellanox 354A Pro $40 Jul 17, 2019

Share This Page