10GBit card upgrade

Discussion in 'Networking' started by Octopuss, Jun 30, 2019.

  1. Octopuss

    Octopuss Member

    Joined:
    Jun 30, 2019
    Messages:
    61
    Likes Received:
    7
    I want to upgrade my home server with 10GBit speed, and have a dilemma what card to choose.

    The options are Chelsio T520-SO-CR (if I can find one of the last revisions) or Intel X550-T1/T2 (I only need one port, but two never hurt if the price is the same or similar).

    I'd like to hear about any other viable alternatives. The only hard requirements are:
    1) Reasonably low power consumption, because I don't want the server to get too hot, because with 3x HDD and a SAS HBA is already is pretty warm these days.
    2) It should be SFP+.

    Today I learned about Mellanox cards. Quite a lot of stuff to digest for non-networking type of person.
    They seem to be dirt cheap, possibly because some of them are old producs I guess?
    Can anyone tell me anything about those? Only the X-2 card seems to support SFP+, but maybe the higher speed standards are backwards compatible? Even the ConnectX-4 Lx cost next to nothing compared to the Intel/Chelsio ones.
     
    #1
  2. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,682
    Likes Received:
    411
    The connect-x 2 are cheap because they are eol since 2015/16? and come from decommissioned servers.
    Connect-x 3 are cheap because they are almost eol (december 2019 if I remeber correctly) (and have already been removed from the active products page by mellanox).
    I like mellanox cards becuase they work so far with all transceivers and cables I could find.
    There are cards with sfp+ (connect-x 3) and sfp28 (connect-x 4 and newer). Yo can also get nics with qsfp and use adapters for sfp
    See this front page post by @Patrick : https://www.servethehome.com/using-a-40gbe-qsfp-nic-with-a-10gbe-switch-sfp/
     
    #2
  3. Octopuss

    Octopuss Member

    Joined:
    Jun 30, 2019
    Messages:
    61
    Likes Received:
    7
    So I can't just plug SFP+ transceiver into card that has (q)sfp28 slot?
     
    #3
  4. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,682
    Likes Received:
    411
    With sfp28 you can. For qsfp you will need an adapter if you want to use sfp
     
    #4
  5. Octopuss

    Octopuss Member

    Joined:
    Jun 30, 2019
    Messages:
    61
    Likes Received:
    7
    That sounds good.
    Guess I could keep an eye out for ConnectX-4. I guess 3 would be enough, but non-EOL hardware will probably get longer driver support.
     
    #5
  6. Octopuss

    Octopuss Member

    Joined:
    Jun 30, 2019
    Messages:
    61
    Likes Received:
    7
    What about card revisions? Does it matter? What kind of differences can there be?
     
    #6
  7. Fritz

    Fritz Well-Known Member

    Joined:
    Apr 6, 2015
    Messages:
    1,885
    Likes Received:
    385
    #7
  8. Octopuss

    Octopuss Member

    Joined:
    Jun 30, 2019
    Messages:
    61
    Likes Received:
    7
    #8
  9. hceuterpe

    hceuterpe New Member

    Joined:
    May 22, 2019
    Messages:
    4
    Likes Received:
    0
    How about Connect X-3 Pro? I bought three of these dual cards in January and they work great. Unlike the EN, The Pro has no sign of EOL, and I highly suspect it will be around for a while since it's now the only SFP+ 10 GigE card now they make that Mellanox still considers mainstream supported.
     
    #9
  10. klui

    klui Member

    Joined:
    Feb 3, 2019
    Messages:
    38
    Likes Received:
    16
    "EN" only refer to their ethernet-only based ASICs and they have CX4, 5, and 6 ENs. All CX3s, including 3-Pros will be EoS (sale) the end of this year. 5 years afterwards, EoS (service) begins and that date marks when they stop improving or maintaining the product.

    http://www.mellanox.com/page/eol
     
    #10
  11. hceuterpe

    hceuterpe New Member

    Joined:
    May 22, 2019
    Messages:
    4
    Likes Received:
    0
    Wrong. The EN based on their literature is effectively the prior generation vs. their Pro. Their release dates are several years apart, as well. Only the OCP form factor cards have been stated, and that won't suit that many folks here. The cards I got aren't even on the Mellanox spreadsheet for EOL products.
     
    #11
  12. klui

    klui Member

    Joined:
    Feb 3, 2019
    Messages:
    38
    Likes Received:
    16
    That's news to me
    http://www.mellanox.com/products/ethernet-adapter-cards/
    Why doesn't http://www.mellanox.com/page/products_dyn?product_family=204&mtag=connectx_4_en_card show any CX4, 5, or 6 Pro products? That's because they don't make any with that designation.
     
    #12
  13. Octopuss

    Octopuss Member

    Joined:
    Jun 30, 2019
    Messages:
    61
    Likes Received:
    7
    The only 3 card they list on their website is ConnectX-3 Pro EN. So, heh? I don't know.
     
    #13
  14. TeleFragger

    TeleFragger Member

    Joined:
    Oct 26, 2016
    Messages:
    134
    Likes Received:
    16

    I went 10gb but I went with a super older Mellanox card... I went with ConnectX-1 cards, only because I picked up a few cards and a ton of cables from craigslist CHEAP... $55...
    it is a special CX4 connector... I got a 6 port switch for $50 and I even bought a ConnectX-2 CX4 port style for $18.... so I am $125ish in on 6 10gb ports in my lab.

    I do wish I could tie it into my 1gb network, but I cant or still learning how too....

    I thought of going sfp+ and even any other card but to me 10gb is 10gb and my solution was for cost. These cards do get warm though....

    I'm happy with my setup as I am able to copy files from my win10 box (nvme) to freenas (SSD Pool) @ 1.93GB/s...

    my machines are
    1. Server 2019 - file share
    2. Windows 10 - plex
    3. ESXI 6.7u2 - vm lab
    4. ESXI 6.7u2 - vm lab
    5. Freenas - messin around with storage... iSCSI
    6. Windows 10 - gaming rig
     
    #14
  15. epicurean

    epicurean Member

    Joined:
    Sep 29, 2014
    Messages:
    543
    Likes Received:
    20
    #15
  16. Fritz

    Fritz Well-Known Member

    Joined:
    Apr 6, 2015
    Messages:
    1,885
    Likes Received:
    385
    Don't know. Maybe someone else will chime in
     
    #16
  17. Octopuss

    Octopuss Member

    Joined:
    Jun 30, 2019
    Messages:
    61
    Likes Received:
    7
    It's me again.
    What would you say is a good price for Mellanox ConnectX-4 Lx EN (whichever variant). I found one on Ebay for $150 (plus shipping to Europe which sucks, but...), but I have no idea if that's good or average.
     
    #17
Similar Threads: 10GBit card
Forum Title Date
Networking Do Mellanox x2/x3 10gbit sfp cards work with Sophos? Mar 21, 2016
Networking 10gbit routing Jul 19, 2019
Networking What is a normal amount of Retr in a 10gbit iperf3 test? Nov 27, 2018
Networking 10gbit interface compatibility (Intel X553 <-> Mellanox ConnectX-2) Aug 12, 2018
Networking Can't get 10Gbit speeds on Intel X550-T1 Jul 3, 2018

Share This Page