Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Discussion in 'Networking' started by fohdeesha, Jul 12, 2018.

  1. rootwyrm

    rootwyrm New Member

    Joined:
    Mar 25, 2017
    Messages:
    8
    Likes Received:
    1
    So, I have a conundrum... stuck looking at the 7450 and similar because I rather want BGP (I'd prefer ISIS but Brocade doesn't even do that on the VDX.)

    But the noise level is nowhere near acceptable for something that has to be 10 feet from my head, and can't go in the rack. The 6450 with fan modification is right about the limit of what I would consider acceptable (with some additional baffle.) Anyone had any luck getting the BGP capable models down to about that noise level? Failing that, any suggestions for alternatives when I don't mind spending more if I have to?
     
    #2621
    dashpuppy likes this.
  2. dashpuppy

    dashpuppy Member

    Joined:
    Dec 16, 2018
    Messages:
    31
    Likes Received:
    4
    Why not change the fans ? Or put resistors on the stock fans to slow them down ?
     
    #2622
  3. rootwyrm

    rootwyrm New Member

    Joined:
    Mar 25, 2017
    Messages:
    8
    Likes Received:
    1
    Because there isn't thermal margin on the 7450 and the like to just swap the fans, and the primary issue is the PSU fans. Someone is welcome to do that on their ICX or VDX, so long as they post the literal melted PCBs and PSUs that result. There is no thermal margin on stuff like the 6610 and 7450 and above.

    The only way you're doing it is one of two ways:
    1. Chuck the 1U form factor in the bin (which is fine since this is on a workbench, but now has you up against the firmware expected RPM issue)
    2. Software mod a lower model to do BGP (which is not a small task since RFC2835/5925 needs kernel support)
     
    #2623
  4. dashpuppy

    dashpuppy Member

    Joined:
    Dec 16, 2018
    Messages:
    31
    Likes Received:
    4
    Pretty sure 100% NOTHING is going to melt if you change the fans and lower their speed. The switch doesn't pull KW's of power.
     
    #2624
  5. vangoose

    vangoose Member

    Joined:
    May 21, 2019
    Messages:
    80
    Likes Received:
    25
    I got a 6650 because I wanted bgp as well as 10Gb ports counts.

    I think I gave up on it, will use 7250 stack for 10Gb ports, and use SRX as bgp router for kubernetes.
     
    #2625
  6. juey

    juey Member

    Joined:
    Oct 1, 2018
    Messages:
    50
    Likes Received:
    14
    Go for a nasty Linux Box with FRRouting.
     
    #2626
  7. juey

    juey Member

    Joined:
    Oct 1, 2018
    Messages:
    50
    Likes Received:
    14
    The 6610 ASIC will melt...ive watched transceiver temperatures go up to 55-60 degree celsius and cpu up to 65-70, with stock cooling.
     
    #2627
  8. Jorge Perez

    Jorge Perez New Member

    Joined:
    Dec 8, 2019
    Messages:
    2
    Likes Received:
    0
    So is there something similar to these switches in the routing department?
     
    #2628
  9. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,489
    Likes Received:
    1,254
    they are routers, how many routes do you need to take? or do you mean firewall? (NAT etc)
     
    #2629
  10. Jorge Perez

    Jorge Perez New Member

    Joined:
    Dec 8, 2019
    Messages:
    2
    Likes Received:
    0
    Yeah, I meant a gateway to handle NAT.
     
    #2630
  11. frogtech

    frogtech Well-Known Member

    Joined:
    Jan 4, 2016
    Messages:
    1,261
    Likes Received:
    172
    @fohdeesha

    while my switch is booting, this message comes up:

    Starting kernel ...
    bcmiproc-gmac 18023000.ethernet: can't find MAC address

    Is that normal? The switch boots otherwise.

    Also, when I run the command to license the 8x10g ports, I get a message re: an evaluation of 45 days. Does the latest firmware still utilize honor based licensing and will the ports be 10 gig after the 45 days? This is used in a lab so it's not in production or used to make money.
     
    #2631
    Last edited: Dec 9, 2019
  12. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,489
    Likes Received:
    1,254


    its normal and yes theyre honor based you'll be fine
     
    #2632
  13. Wolfstar

    Wolfstar Member

    Joined:
    Nov 28, 2015
    Messages:
    80
    Likes Received:
    46
    And I'm 100% sure you're wrong. I've actually seen photos my guys took of Brocade FCX648S-HPOE switches after they caught fire, and saw one that had burned out myself. The 6610s and the FCX line use extremely similar hardware. I can't say I've seen the same of the 6610s, but that's because we don't use them at work - we jumped straight to ICX7450s.

    Now, having seen burned out Brocade switches before, you might wonder why I'm still a fan of these things. The reason is simple - They STILL WORKED. Yes, I had a total of four incidents, one of which involved two switches in the same stack, that resulted in literally scorched and charred circuit boards, but the switches were still operating. In these cases, they were PoE boards, but even so.

    They've had their problems over the years, but being built fragile is not really one of them.
     
    #2633
    rootwyrm, itronin and fohdeesha like this.
  14. frogtech

    frogtech Well-Known Member

    Joined:
    Jan 4, 2016
    Messages:
    1,261
    Likes Received:
    172
    Any idea what a typical idle temp is for the ICX7250? The air coming from the rear of the switch is cool-ish, not very warm, but the heatsink for the main ASIC is rather warm, not hot but not comfortable either if you set your hand on it for a couple of minutes. I am just wondering I guess what the operating temperature specification range is for it and if it's going to be naturally higher than your traditional desktop or server CPU.
     
    #2634
  15. eduncan911

    eduncan911 Member

    Joined:
    Jul 27, 2015
    Messages:
    91
    Likes Received:
    33
    I've tried searching this thread, and browsing a dozen or so pages...

    Can someone tell me if "PoE" is PoE+? E.g. 48V not for 24V like for Ubiquiti's newer AC Pro and AP AC HD?

    These sound awesome... I just picked up my Mikrotik 24 port PoE+ + 4x 10Gbps cages, and was looking at their newer 10 Gbps 10GbaseT switch for $500.

    But after finding these, I'm debating selling the NIB Mikrotik for a couple of these.

    My only issue is my desktop, and two laptops, are already have 10GBaseT RJ45 10Gbps - so I need an inexpensive RJ45 10Gbps solution at my switch with VLANs.
     
    #2635
  16. Aestr

    Aestr Well-Known Member

    Joined:
    Oct 22, 2014
    Messages:
    869
    Likes Received:
    274
    The PoE variants of both the 6450 and 6610 supply 802.3at Power over Ethernet Plus (PoE+). You're better off looking up data sheets for this type of info.

    I am currently using Mikrotik S+RJ10 modules both a 6450 and 6610 without issues to convert SFP+ to 10Gbase-T. They are pretty cheap, and as long as you don't need too many you'll come out ahead.
     
    #2636
    eduncan911 likes this.
  17. CED6688

    CED6688 New Member

    Joined:
    Dec 4, 2019
    Messages:
    15
    Likes Received:
    8
    I wanted BGP to pair my physical network with a Kubernetes cluster running Calico networking (no IPIP, VXLAN, or NAT required). On my current 6610, not only did I get this working in about 15 minutes, but I was able to redistribute the routes to other parts of my network via OSPF. Basically all of your cluster IPs are part of your normal infrastructure, allowing you to set policy both within the cluster and at the network layer. It's pretty cool.

    When I drop this to a 6450, I've decided to just do router-on-a-stick with a Mikrotik RB4011, which on its single SFP+ port can route 10gb/s on that one port (I've tested it) and supports BGP. I'll then have it redistribute the routes via OSPF to the rest of the network, which the 6450 will manage. All of my nodes have a separate interface to my storage VLAN, so that won't hit the Mikrotik, so the Mikrotik will really only be routing traffic that enters/leaves the cluster...and 10gb/s should be plenty enough for this for me.

    That said, BGP support in the 7250 or 6450 would have eliminated a device and some complexity. The 6610 is incredible and is less than 40db from the next room, but it draws too much attention from the wife when she enters that room (~55db), so this is my current plan... though there is a part of me thinking about trying to swap in a rev b power supply. It would be good if someone can post noise measurement of a 6610 operating with one of those... I feel like I'm just at the borderline of acceptability. :)

    Clayton
     
    #2637
  18. vangoose

    vangoose Member

    Joined:
    May 21, 2019
    Messages:
    80
    Likes Received:
    25
    This is mine at idle.

    The stack unit 1 chassis info:

    Power supply 1 (AC - Regular) present, status ok
    Power supply 2 not present

    Fan ok, speed (auto): [[1]]<->2

    Fan controlled temperature: 75.6 deg-C

    Fan speed switching temperature thresholds:
    Speed 1: NM<----->98 deg-C
    Speed 2: 67<----->105 deg-C (shutdown)

    Fan 1 Air Flow Direction: Front to Back
    Slot 1 Current Temperature: 75.6 deg-C (Sensor 1)
    Slot 2 Current Temperature: NA
    Warning level.......: 100.0 deg-C
    Shutdown level......: 105.0 deg-C
     
    #2638
    frogtech likes this.
  19. vangoose

    vangoose Member

    Joined:
    May 21, 2019
    Messages:
    80
    Likes Received:
    25
    Exact same reason I got a 6650. My 6650 comes with redundant rev B PSU, they are loud even in a separate room.

    I gave up now until I can put a AC unit int the room and close the door.

    I'll configure my SRX300 or F5 to be BGP router for kubernetes.
     
    #2639
  20. juey

    juey Member

    Joined:
    Oct 1, 2018
    Messages:
    50
    Likes Received:
    14
    As said, it supports both standards. Ive been running a Unifi AP-HD attached to a 6450-24P, so it works with Unifi Gear.
     
    #2640
    eduncan911 likes this.
Similar Threads: Brocade Series
Forum Title Date
Networking Mikrotik compatible with Brocade 57-0000075-01 transceivers? Dec 17, 2019
Networking [SOLVED] Help Needed - Brocade ICX 6450 + Ruckus R720 Nov 25, 2019
Networking Brocade ICX 6610 - what does dhcp-client enable do? Nov 19, 2019
Networking MikroTik CSS326-24G-2S+RM vs Brocade ICX6450-24 Nov 1, 2019
Networking Brocade ICX6450-24P vs Aruba S2500-24P for Homelab Sep 22, 2019

Share This Page