Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Discussion in 'Networking' started by fohdeesha, Jul 12, 2018.

  1. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,388
    Likes Received:
    1,117
    updated the icx7xxx firmware on the guide from 8080ca to the latest 8080d. Interestingly this release included new bootloaders for both the icx7150 and icx7250, but no mention of bootloader changes in the changelog.

    just bugfixes: Imgur
     
    #1241
  2. Roelf Zomerman

    Joined:
    Jan 10, 2019
    Messages:
    34
    Likes Received:
    4
    Does anyone have some NIC tips, for the ones that work with Server 2019 and this switch and that are not too expensive?..
    I have the HP NC523SFP but there are no native 2016/2019 drivers.. and using the 2012R2 drivers result in a bluescreen as soon as I change settings (like MTU)... My connection is using SFP Direct cables..
     
    #1242
  3. ljvb

    ljvb Member

    Joined:
    Nov 8, 2015
    Messages:
    76
    Likes Received:
    9
    Try the 571, it is newer, Solar Flair based 10G card still being sold by HP. I just picked one up from ebay for 35$ (I got the FLR version since I have HP servers, but they have a regular non FLR pci version). I'll know more when I get it, should arrive tomorrow, but it is still actively supported by HP, so drivers should be available.
     
    #1243
  4. ljvb

    ljvb Member

    Joined:
    Nov 8, 2015
    Messages:
    76
    Likes Received:
    9
    Heh.. uhh.. no.. That is why it is turned off and sitting in the corner of my Garage while I try to figure out how to sell a 300 pound brick.. My wife did not like the $300 a month power biill... Tried local pickup a few times on Craigslist and Ebay but no luck. At this point, and the fact it is like 5'F today and my house is freezing.. I'm debating on cranking that sucker up inside the house.. it generates enormous amounts of heat :)
     
    #1244
  5. mrgstiffler

    mrgstiffler New Member

    Joined:
    Jan 20, 2019
    Messages:
    6
    Likes Received:
    4
    Mellanox cards are cheap and work. I have a single port CX-2 in my Server 2019 machine and a dual port in my Unraid.
     
    #1245
  6. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,388
    Likes Received:
    1,117
    Mellanox Connectx-3's are down to 20 dollars for a single port 10gbe version, work in literally everything
     
    #1246
    tommybackeast, tjk and maes like this.
  7. 747builder

    747builder Member

    Joined:
    Dec 17, 2017
    Messages:
    92
    Likes Received:
    38
    any reason people arent using the chelsio's? they seem supported on about everything also including FreeBSD/FreeNAS
     
    #1247
  8. kapone

    kapone Active Member

    Joined:
    May 23, 2015
    Messages:
    616
    Likes Received:
    247
    umm...yeah...
     
    #1248
  9. 747builder

    747builder Member

    Joined:
    Dec 17, 2017
    Messages:
    92
    Likes Received:
    38
    care to elaborate?
     
    #1249
  10. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,388
    Likes Received:
    1,117
    mellanox has been perfectly supported in freebsd/freenas for a very long time. the chelsio cards you will find for 20 or 30 dollars (the price of a much newer connect x3) are going to be ancient. Mellanox also has super open firmware tools that allow you to do a lot of stuff
     
    #1250
  11. Roelf Zomerman

    Joined:
    Jan 10, 2019
    Messages:
    34
    Likes Received:
    4
    yeah thnx!.. will also checkout the MCX354A-QCBT version.. and hook it up to the VDX or ICX7450...
     
    #1251
  12. 747builder

    747builder Member

    Joined:
    Dec 17, 2017
    Messages:
    92
    Likes Received:
    38
    will have to check them out. any preferred models in that pricerange that is dual sfp+ ? i have a few servers that take low profile cards only and i have a mix of FreeBSD/FreeNAS and Vmware in the rack so a model that can go in any would be awesome. hardware tends to be a mixture of Dell and Supermicro..
     
    #1252
  13. frameshift18

    frameshift18 New Member

    Joined:
    Jan 9, 2019
    Messages:
    4
    Likes Received:
    0
    Anywhere specific you are seeing that? I see $28 for low pro and $38 for full height on ebay currently
     
    #1253
  14. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,388
    Likes Received:
    1,117
    looks like the listing I had saved at $19 sold out. the brackets are also a couple dollars if you want to just swap them out
     
    #1254
  15. ic7Yoox0EedieHe

    ic7Yoox0EedieHe New Member

    Joined:
    Oct 5, 2016
    Messages:
    5
    Likes Received:
    4
    #1255
    tommybackeast likes this.
  16. juey

    juey Member

    Joined:
    Oct 1, 2018
    Messages:
    39
    Likes Received:
    10
    Jumbo Frames have very few advantages, Speed Increase is bellow 10-15 % and they make a heck of trouble with much devices anyway.
     
    #1256
  17. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,388
    Likes Received:
    1,117
    yeah I honestly had no clue people were still using jumbo frames. I think it's been a long time since I've seen CPU/hardware old enough that increasing MTU size gave me more than 4 or 5% increase at the expensive of 50% more headache. Even my 40gbE storage is at 1500mtu

    I'll add them to the OP if you can confirm they don't fragment/work properly linked at 10gbE (from your post in that thread it seems they were linked at 1gbps when you ran the test)
     
    #1257
    BackupProphet likes this.
  18. Roelf Zomerman

    Joined:
    Jan 10, 2019
    Messages:
    34
    Likes Received:
    4
    The funny thing I noticed is that my iPerf max'd out at 5Gbps, but copying a Windows file between two servers actually managed to get the adapter to 9.7Gbps.. not sure how/why that is possible.. but just going by the numbers of iPerf seems not to be everything..
     
    #1258
  19. DanielWood

    DanielWood Member

    Joined:
    Sep 14, 2018
    Messages:
    41
    Likes Received:
    12

    I’m guessing SMB multichannel was at play, so you were no longer maxing out a single CPU thread.
     
    #1259
  20. gslavov

    gslavov New Member

    Joined:
    Jun 7, 2017
    Messages:
    17
    Likes Received:
    4
    Depends on what parameters you run iPerf with. It is notoriously bad on Windows with defaults. Try it with at least 4 threads and change the window size to 1M and see what you get.

     
    #1260
Similar Threads: Brocade Series
Forum Title Date
Networking [SOLVED] Help Needed - Brocade ICX 6450 + Ruckus R720 Nov 25, 2019
Networking Brocade ICX 6610 - what does dhcp-client enable do? Nov 19, 2019
Networking MikroTik CSS326-24G-2S+RM vs Brocade ICX6450-24 Nov 1, 2019
Networking Brocade ICX6450-24P vs Aruba S2500-24P for Homelab Sep 22, 2019
Networking Brocade VDX 6720 - what do I need to know? Sep 16, 2019

Share This Page