Inexpensive 40G NIC for VMWare home lab?

Discussion in 'Networking' started by Ron Kelley, Jan 13, 2020.

  1. Ron Kelley

    Ron Kelley New Member

    Joined:
    Jan 13, 2020
    Messages:
    12
    Likes Received:
    2
    Greetings all,

    Looking for pair of inexpensive 40GbE NICs for my VMWare home lab (2x servers running 6.5u3). The servers currently have dual-port 10G NICs, and I would like to bump them to 40G. I already have 40G DACs from a previous setup. This will be pure Ethernet - no InfiniBand.

    I saw this from a recent thread:

    649281-B21 656089-001 661685-001 HPE IB FDR/EN 10/40GB 2P 544QSFP ADAPTER | eBay

    With a conversion guide here:
    https://forums.servethehome.com/ind...net-dual-port-qsfp-adapter.20525/#post-198015


    Currently, these cards are $29/ea on eBay, but I don't know anything about them (VMware compatibility, software/firmware versions, etc). Can someone please let me know if they will work w/VMWare once flashed?

    Thank you.
     
    #1
    SRussell likes this.
  2. sash

    sash Member

    Joined:
    Nov 22, 2019
    Messages:
    30
    Likes Received:
    4
    They work just fine. I just installed one into one of my hosts. I'm running latest release of vCenter and ESXi though. Support was out of the box Screenshot 2020-01-13 at 23.07.30.png
     
    #2
  3. Ron Kelley

    Ron Kelley New Member

    Joined:
    Jan 13, 2020
    Messages:
    12
    Likes Received:
    2
    Thanks for the reply! I see the seller is out of town until Jan 20, but I found a similar card here using Rev: A5

    649281-B21 656089-001 661685-001 HPE IB FDR/EN 10/40GB 2P 544QSFP ADAPTER | eBay

    Do you know if this is the same card as in the other eBay link? They appear to be the same to me but I would like a second pair of eyes.
     
    #3
  4. sash

    sash Member

    Joined:
    Nov 22, 2019
    Messages:
    30
    Likes Received:
    4
    I bought from this seller as well a couple of days back precisely for the same reason)
    Mine is going overseas. So it will be a couple of weeks before I get it.
     
    #4
  5. sash

    sash Member

    Joined:
    Nov 22, 2019
    Messages:
    30
    Likes Received:
    4
    I've checked my current card from the seller that is away at the moment. It is revision A2.
    Screenshot 2020-01-14 at 14.57.21.png
     
    #5
  6. Ron Kelley

    Ron Kelley New Member

    Joined:
    Jan 13, 2020
    Messages:
    12
    Likes Received:
    2
    Thanks. I ordered 2 cards from the second seller for $25/ea last night ($50 total - free shipping inside the US). According to the pics on eBay, these should be revision A5 cards.

    The cards should arrive by Friday/Saturday this week, and I will update this post when I have a chance to apply the Mellanox firmware.

    Thanks for the great feedback so far!
     
    #6
  7. Ron Kelley

    Ron Kelley New Member

    Joined:
    Jan 13, 2020
    Messages:
    12
    Likes Received:
    2
    Follow-up on this thread. As per the previous post, I ordered 2x dual-port 40G NICs from eBay on Tuesday, and the cards arrived yesterday. Both in factory wrapped anti-static bags with rev A5 stickers. I installed one in my Debian server, followed the conversion guide mentioned in the first post, and had the new firmware installed/running in about 15mins! The second card only took about 10mins to configure.

    To verify network connectivity, I used a 100G DAC cable and connected the ports in a back-to-back config (Port-1 to Port-2). iPerf showed 4986MBytes/sec. Excellent!!!

    screenshot_07.png

    Finally, I popped these into my ESXi 6.5U3 servers, configured the network, attached some VMs to the NICs, and ran some speed tests. Again, these cards pushed over 4GB/sec between VMs (across two servers).

    Many thanks to this forum for the great instructions and suggestions. Also, kudos to the eBay seller - very prompt updates and fast shipping.
     
    #7
    ste.mo likes this.
Similar Threads: Inexpensive VMWare
Forum Title Date
Networking Quietest Inexpensive QSFP+ (or units that allow DIY tasks to become quiet) Oct 27, 2018
Networking Inexpensive SFP+ for Server 2016 Dec 3, 2016
Networking Can anyone recommend a good/inexpensive 48 port GbE switch? Jun 11, 2015
Networking Inexpensive 8 port Gigabit Managed switch Jul 1, 2014
Networking VMWare NSX - I've some basic design questions Dec 17, 2019

Share This Page