10GBe vs. 25GBe - Cost Analysis and Part Numbers

Discussion in 'Networking' started by Edd189, Oct 31, 2019.

  1. Edd189

    Edd189 New Member

    Joined:
    Oct 18, 2019
    Messages:
    19
    Likes Received:
    2
    I am upgrading my backbone in my home and am trying to decide between 10GBe and 25GBe. Right now I only need two high speed connections, but will plan for more. Along the way, I have researched part numbers and prices and pulled everything into a spreadsheet. I figured I would share here as someone else may find this very helpful.

    To the community, does anyone have a recommendation for a better 25GBe switch than the one I have listed? Also, if you see anything incorrect or incompatible, please let me know!
     

    Attached Files:

    #1
  2. altmind

    altmind Member

    Joined:
    Sep 23, 2018
    Messages:
    67
    Likes Received:
    17
    Will you compare 25g prices with 40gig?

    I find multigig tech(new hardware) highly expensive, sometimes more expensive than next closest speed. And for case of 25g sfp, even the sfp connector is different, so theres little to reuse.
     
    #2
  3. Edd189

    Edd189 New Member

    Joined:
    Oct 18, 2019
    Messages:
    19
    Likes Received:
    2
    I have scoured the internet for information on 40Gbe and the conclusion I reached was that Ethernet wasn’t possible, only Infiniband. I wasn’t interested in that option. But please correct me if I’m wrong!
     
    #3
  4. Edd189

    Edd189 New Member

    Joined:
    Oct 18, 2019
    Messages:
    19
    Likes Received:
    2
    The IS5022 switch looks perfect when paired with a ConnectX-3 Pro, but no Ethernet. I suppose it would be OK for direct connections though.
     
    #4
  5. altmind

    altmind Member

    Joined:
    Sep 23, 2018
    Messages:
    67
    Likes Received:
    17
    What you mean 40g ethernet is impossible? There are 40gbe cards, there are 40gb sfp, there are 40gbe. They usually use MTP-12, so its basically 10x4 bonding, no gains in latency, only in bandwidth.
     
    #5
  6. Edd189

    Edd189 New Member

    Joined:
    Oct 18, 2019
    Messages:
    19
    Likes Received:
    2
    Impossible in my price range, I think. Do you know a 40Gbe switch that works with Ethernet?
     
    #6
  7. altmind

    altmind Member

    Joined:
    Sep 23, 2018
    Messages:
    67
    Likes Received:
    17
    Cisco N3K-C3132Q-40GE $650 used, or nexus 3064x if you are good with 4 qsfp and 48 sfp+.

    used nexus 3k is in general very price competitive.
     
    #7
  8. Rand__

    Rand__ Well-Known Member

    Joined:
    Mar 6, 2014
    Messages:
    3,592
    Likes Received:
    544
    And of course all the converted Mellanox/EMC ones (search Sx6036, sx6018, sx6012) here on StH;)
     
    #8
  9. RageBone

    RageBone Active Member

    Joined:
    Jul 11, 2017
    Messages:
    267
    Likes Received:
    72
    those, though i haven't found any the last few times i looked.

    But since op currently only needs two connections, pear to pear directly t is still an option and would cost you the 3 cards, one dualport, and the cables.

    On the other hand with 10GbE, CX2 / CX3 sfp+ cards seem to be ok, though i went the cx2/3 QSFP 40GbE rout, without a switch since i will only have less then 6 connections ever* and i can do that with spare adapters for less power and what my 18 port switch cost me.

    SFP+ has the advantage in switch availability and cost i think. .
     
    #9
  10. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,396
    Likes Received:
    1,125

    a ton of us are running 40gbE ethernet here on this forum, the ICX6610 for example is $200 and has 2x 40gbE ports (and a bunch of 10gb and 1gb): https://forums.servethehome.com/ind...s-cheap-powerful-10gbe-40gbe-switching.21107/

    there's other switches like the arista 7050S-64 with 4x 40gbE (and a bunch of 10gb) ports for around 500, or if you want just a ton of 40gbE density there's stuff like the arista 7050qx-32s for around $900 with 32 40gbE ports. I would stay far away from the old cisco nexus stuff, it's generally twice the price of the aristas and 3x as loud

    mellanox connectx 40gbE NICs are $40
     
    #10
    JustinClift, SRussell and RageBone like this.
  11. funkywizard

    funkywizard mmm.... bandwidth.

    Joined:
    Jan 15, 2017
    Messages:
    607
    Likes Received:
    253
    I'm happy to tell you that you're misinformed. 40gbe is widely available and relatively inexpensive.

    ConnectX-3 cards can do either IB or ethernet mode

    Arista makes 7050QX32 switches (32x40g) that are routinely $500 or less on ebay

    There may also be cheaper / better options.

    As well, the 40g is typically backwards compatible with 10g. 24 of the 32 ports on the 7050QX32 for example, can be set to 4x10g mode
     
    #11
  12. Edd189

    Edd189 New Member

    Joined:
    Oct 18, 2019
    Messages:
    19
    Likes Received:
    2
    Wow! Thanks! I didn’t know I could get a legit switch for that low cost.
     
    #12
    fohdeesha likes this.
  13. Edd189

    Edd189 New Member

    Joined:
    Oct 18, 2019
    Messages:
    19
    Likes Received:
    2
    Here is what I have for 40GBe fiber parts and costs.

    I also added an option to buy the NIC cards at 40GBe so I could direct attach my server/workstation to play with, but run the full backbone at 10GBe. That's where I'm leaning now.
     

    Attached Files:

    #13
Similar Threads: 10GBe 25GBe
Forum Title Date
Networking Help with home 10GbE network (10Gbase-T and SFP+) Nov 19, 2019
Networking The usual noob question(s) about cheap 10gbe again Nov 17, 2019
Networking Broadcom BCM57416 10Gbe vs Intel X550 10Gbe adapters? Sep 29, 2019
Networking 10GbE network speed slow on Netgear XS716-T Sep 21, 2019
Networking Cisco Meraki MS22, good 10Gbe switch? Aug 19, 2019

Share This Page