Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Discussion in 'Networking' started by fohdeesha, Jul 12, 2018.

  1. Zervun

    Zervun Member

    Joined:
    Feb 2, 2019
    Messages:
    44
    Likes Received:
    6
    Any of the rear 40g ports (there are 4 of them per switch 2x breakout, 2x stack) can be used for stacking. However the 2x breakout ports cannot be used for 40g, only the 40g only ones. (Fohdeesha just answered my question on this yesterday)

    So what I am going to do with mine is stack across the breakout ports, and use the 4 total ports across my 2x 6610's for 40g.

    The max ports per 6610 you can get straight 40g is 2x per switch (if you disable stacking and/or use stacking on the breakout ports).

    As for POE - I hadn't seen much difference in price on the 6610's POE vs. non-POE so you might as well get a POE one. By default POE is disabled and you can enable it per port when you need it. I believe the switches are built with the same number fans/power supplies.

    Make sure you get them with 2x power supplies and 2x fans as they are expensive/hard to come by (especially the fans). Also keep in mind the exhaust direction - the E models are rear exhaust direction

    If you are looking for something not as loud the 6450 is the way to go but you have less ports/no 40g no dual power supplies. I would not get the 6610s if they are in the same room as you are.
     
    #1341
  2. Kewjoe

    Kewjoe New Member

    Joined:
    Feb 3, 2019
    Messages:
    6
    Likes Received:
    2
    Got my ICX6450-24P yesterday. Followed the guide. Everything is working great. Thanks to @fohdeesha for the awesome thread and all the help!. I have A LOT of reading to do :)
     
    #1342
    Last edited: Feb 21, 2019
    fohdeesha likes this.
  3. Freebsd1976

    Freebsd1976 Member

    Joined:
    Feb 23, 2018
    Messages:
    100
    Likes Received:
    8
    Would you mind post item link?
     
    #1343
  4. Kewjoe

    Kewjoe New Member

    Joined:
    Feb 3, 2019
    Messages:
    6
    Likes Received:
    2
    #1344
    Last edited: Feb 21, 2019
  5. Freebsd1976

    Freebsd1976 Member

    Joined:
    Feb 23, 2018
    Messages:
    100
    Likes Received:
    8
    #1345
  6. Blue)(Fusion

    Blue)(Fusion Member

    Joined:
    Mar 1, 2017
    Messages:
    86
    Likes Received:
    14
    I can't believe I didn't see this thread much much sooner.

    I currently have a silly Netgear XSM7224S for 24x 10G and another Netgear GSM7328PS for 24x PoE 1G in my rack. This could knock out both into one awesome switch.

    I definitely would need the 2x 40G ports to breakout to 10G links. Do I need the premium version for this option? The premium is of course quite a bit more expensive...

    Also, does anybody have a list of what SFP+ modules are compatible with this? I have a ton of Netgear and off-brand SFP+ modules and Cisco DACs.
     
    #1346
  7. dlaffin1012

    dlaffin1012 New Member

    Joined:
    Feb 21, 2019
    Messages:
    12
    Likes Received:
    0
    Ok that is good to hear that I can still have 4 usable 40g ports. Sinc ea lot of you here have a lot of good knowledge, I suppose I can ask some advice on my setup.

    Currently- I have a dedicated computer running pfsense as the firewall and router. That runs into the LB4M switch and everything from there is connected to the switch. I have a HP Proliant ML350P G8 running a few VM's one being my NAS. that NAS is connected via SFP+. I also have a separate computer running as my backup for the NAS also running SFP+.

    Currently Looking to Setup- I would like to get two 6610's and stack them. Then connect the NAS and Backup System via 40G but also have LACP on those 40G so I can have redundancy. By this I mean having one cable in the first switch and one in the second. This would require me to have dual QSFP cards in each system or one dual port card in each system. I also wanted to do the same to the pfsense system with the 10G ports, so once I start doing more VLANs there wouldn't be a bottleneck and I would also have redundancy if a switch goes down. I also wanted to have dual 10G cables for my other VM's for redundancy as well.

    My main questions for my new setup- I keep reading that I should do two dual cards rather than dual port cards because if one card goes down the other is there vs if the dual port card goes down, then I have no connection. the issues I run in to is that both the pfsense and backup systems have motherboards with only one PCIE slot because of their form factor. So I would have to replace the entirety of both systems with systems with more PCIE slots. Starts to get pricey. Is it really worth having dual cards vs dual port cards. Is the switch more likely to die before a card or is it the other way around.

    My other question being do I turn off routing on pfsense and allow the switch to do it, one so I can dedicate pfsense as a firewall and also because of VLAN bottlenecking? Although I assume 10G would stop it from bottlenecking.

    I was thinking of condensing all my systems into the HP server but
    1- it is frowned upon to have the backup and NAS in the same system.
    2- it is frowned upon to virtualize pfsense into a server.
     
    #1347
  8. Kewjoe

    Kewjoe New Member

    Joined:
    Feb 3, 2019
    Messages:
    6
    Likes Received:
    2
    Coming from someone who found this thread a month ago. I highly suggest you read at least the first post in this thread. I believe both of your questions are answered (at least in part). But there is a lot of great info throughout the thread (search by keyword "DAC" and you'll find several conversations about it for instance).
     
    #1348
    fohdeesha likes this.
  9. Kewjoe

    Kewjoe New Member

    Joined:
    Feb 3, 2019
    Messages:
    6
    Likes Received:
    2
    There are far more knowledgeable people that can answer your questions. But I plan to do the inter-vlan routing on my ICX6450 and leave the firewall duties on my (virtualized :)) pfsense. It's a beefy L3 switch, might as well take advantage of getting full line speeds.
     
    #1349
  10. kapone

    kapone Well-Known Member

    Joined:
    May 23, 2015
    Messages:
    655
    Likes Received:
    268
    If this is for "home" not a "homelab", you're overthinking this. If it is in fact a lab, fire away.
     
    #1350
  11. trippinnik

    trippinnik New Member

    Joined:
    Oct 13, 2018
    Messages:
    11
    Likes Received:
    0
    Just got one of these bad boy 6610s. Looks like it already has the licenses installed so I want to back them up before reseting. What do I need to do that? Apologies if this is answered above I tried searching for the answer and didn't come up with results.
     
    #1351
  12. Zervun

    Zervun Member

    Joined:
    Feb 2, 2019
    Messages:
    44
    Likes Received:
    6
    A bit overkill for a home lab. Since you don't need many 40g ports I'd just simplify it on one switch and keep the other switch as backup saving power/noise. Other stuff is going to die probably before a network card. Can always swap one in easily unless this is mission critical business stuff or you just want to play around with LACP etc. Depending on the motherboard the PCIE speeds might be reduced with multiple cards but probably won't matter. Some of the Mellenox cards run at 8x (will run at 4x fine and doubtful you will hit any cap on that.
     
    #1352
    Last edited: Feb 23, 2019
  13. Zervun

    Zervun Member

    Joined:
    Feb 2, 2019
    Messages:
    44
    Likes Received:
    6
    For a fully licensed switch there will be 4 licenses under "show license"
     
    #1353
  14. trippinnik

    trippinnik New Member

    Joined:
    Oct 13, 2018
    Messages:
    11
    Likes Received:
    0
    I read if I erase the config I loose the licenses that are installed. do i just need the License ID to reinstall? I have two licenses:
    ICX6610-10G-LIC-POD
    ICX6610-PREM-LIC-SW
     
    #1354
  15. ViciousXUSMC

    ViciousXUSMC Active Member

    Joined:
    Nov 27, 2016
    Messages:
    160
    Likes Received:
    63
    Been having fun with the HPE Aruba S2500 switches that were so cheap.
    Now kind of thinking about getting one of these.

    Normally I associate Brocade with Fiber Channel, and I love the black color vs the dull grey of the Aruba stuff :)

    Question however (and I did read about 25 pages of this thread)
    The OP made no mention of a requirement for licensing, but several posts in the thread make mention of needing licensing to unlock features (4 different licenses if I am correct?)

    So perhaps these are not the cheap switches they appear to be after licensing cost, that is what I need to verify before I look to buy.

    Could somebody please break down the licensing for me a bit as far as what each license does and about how much it costs?
    Once you do obtain one, I assume it's perpetual and not something you need an annual agreement for?

    Thanks!
     
    #1355
  16. jarekd

    jarekd New Member

    Joined:
    Apr 21, 2018
    Messages:
    7
    Likes Received:
    0
    Ruckus is still selling licenses for ICX 6xxx series (e.g. ICX6610).

    Asked their ordering@ and all licenses are still orderable until November, 2023.
    New hardware cannot be ordered any more.
     
    #1356
  17. Zervun

    Zervun Member

    Joined:
    Feb 2, 2019
    Messages:
    44
    Likes Received:
    6
    See post #6
     
    #1357
  18. Ouraing

    Ouraing New Member

    Joined:
    Dec 31, 2018
    Messages:
    25
    Likes Received:
    21
    Factory resetting does remove the licenses, so you want to make sure you have a backup of the license info to reinstall it.

    You have Ports on Demand licenses and you have Feature licenses, the details vary slightly between the 6610 and 6450.

    For a 6610 (Ruckus ICX 6610 Switch Data Sheet | Technical Documents | Ruckus Wireless Support):
    Without a license ports 1/3/1 to 1/3/8 (the front ports) are 1Gb only, a single PoD license enabled 4 ports at a time so to be fully licensed you need 2 PoD licenses to get 8 10Gb ports. The rear ports are fully enabled without a license. Fully licensed you can have up to 16 10Gb ports and 2 40Gb ports.

    Basic layer 3 (static routes, ACLs) is included in the base license. Most dynamic routing features for IPV4 and IPV6 (OSPF, VRRP, PBR, RIP) require a premium. Note this includes IPV6 versions of these protocols as well. If you want BGP, GRE, IPV6 Tunneling or VRP you need an advanced license (which includes the premium features).

    For a 6450 (Ruckus ICX 6430 and 6450 Switch Data Sheet | Technical Documents | Ruckus Wireless Support):
    Without a license 2 of the 4 front SFP+ ports are limited to 1Gb only. A PoD license enables 2 ports at a time, so to be fully licensed you only need a single PoD license.

    Basic layer 3 (static routes, ACLs) for IPV4 and IPV6 are included in the base license. More advanced IPV4 features like OSPF, VRRP, and GRE require the premium license to unlock, no IPV6 dynamic features. No BGP or IPV6 tunneling support on this model either.
     
    #1358
  19. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,482
    Likes Received:
    1,249
    as stated 9.2E-5 times, licenses are free for sth members
     
    #1359
    ewer0012 likes this.
  20. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,482
    Likes Received:
    1,249
    you can't back them up or export them (at least not without a JTAG unit). If you just run "erase startup-config" from the enable level that will wipe the config and keep the licenses
     
    #1360
Similar Threads: Brocade Series
Forum Title Date
Networking Mikrotik compatible with Brocade 57-0000075-01 transceivers? Dec 17, 2019
Networking [SOLVED] Help Needed - Brocade ICX 6450 + Ruckus R720 Nov 25, 2019
Networking Brocade ICX 6610 - what does dhcp-client enable do? Nov 19, 2019
Networking MikroTik CSS326-24G-2S+RM vs Brocade ICX6450-24 Nov 1, 2019
Networking Brocade ICX6450-24P vs Aruba S2500-24P for Homelab Sep 22, 2019

Share This Page