Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Discussion in 'Networking' started by fohdeesha, Jul 12, 2018.

  1. kapone

    kapone Well-Known Member

    Joined:
    May 23, 2015
    Messages:
    655
    Likes Received:
    268
    Correct. That in between is much newer, and these switches are already EOL, which is why you can buy a 0.5tbps switching bandwidth switch for ~$200 (vs $10K or more)
     
    #2661
  2. vangoose

    vangoose Member

    Joined:
    May 21, 2019
    Messages:
    77
    Likes Received:
    25
    No, it's same as 6610. I opened the cover, looked at it and gave up.

    I've changed my plan for now. I have 2 7250 now, may get another 7250 and use it as core. another 2 7250 will run in layer 2 since I want pfc. BGP will be handled by juniper.

    I will get the AC unit for the room so I can close the door, then 6650 will go live. Really love it.
     
    #2662
  3. FlorianZ

    FlorianZ New Member

    Joined:
    Dec 10, 2019
    Messages:
    2
    Likes Received:
    0
    Damn, I purchased from them as well. Short of physically testing all ports for functionality and power delivery, is there anything I should watch out for to determine whether my unit is broken? Any diagnostics I should be running? Thank you!
     
    #2663
  4. eduncan911

    eduncan911 Member

    Joined:
    Jul 27, 2015
    Messages:
    91
    Likes Received:
    33
    Yep, the 6610 is more ideal for ports and a little cheaper for more features. But it's also more power hungry, louder, and doesn't run an OS I can build (my ultimate goal since I've switched to Coreboot on most of my key devices). Less watts = less heat, which is also another key goal of mine since my closet I keep them is sensitive to a lot of heat (I build my servers for ultra-low power - 96 TB, 16 core server that idles at 45W).
     
    #2664
    Last edited: Dec 11, 2019
  5. rootwyrm

    rootwyrm New Member

    Joined:
    Mar 25, 2017
    Messages:
    8
    Likes Received:
    1
    Oh, the 7450's look pretty damn bad... they're rated 47dBA. I'm fan noise sensitive so 47dBA is an extremely high number to me. (Really, anything over 38dBA @ 1m is irritant level.) For comparison, the 6610's rated 39-48dBA @ 1m. So the 7450 actually is as loud as the 6610. But the 7450 doesn't state a low-side number, so I honestly don't know how quiet it is at low load. Any time it's going to be under high load, I'm already going to have hearing protection on anyways.

    Really there's no reason to not have BGP other than market segmentation on Brocade/Ruckus' part. Contrary to popular belief, unless you're doing some really big tables, BGP is noisy but not high CPU demand. Usually it's RAM. For a smaller environment (couple dozen /22-/32's,) even something like the Juniper EX2200-12CT can handle it with ease. You sure as hell don't want it trying a significant table, but you could very easily do a pretty robust anycast setup on even an ICX6430 without putting a dent in performance. And with the 7000-series on a common ARM architecture, it kinda surprises me nobody just copy-pasted the bgpd over to a lower switch.
     
    #2665
  6. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,482
    Likes Received:
    1,249
    Like anything else it's just market segmentation. Pretty much the only people demanding something like BGP on the bottom end models designed to be stuck in a campus wiring closet are the people in this thread, and of course they are not the main revenue generators for brocade/ruckus/etc. I'd wager even on the 7450, less than 10% of customers are using bgp on the thing.

    Theres also no bgpd daemon to copy, there's no daemons to speak of at all - all Linux does is launch a single monolithic fastiron binary (a huge one), which contains proprietary implementations of pretty much everything minus the standard broadcom sdk stuff to actually talk to the ASIC (and last I looked, the bgp & ospf stack in the binary are licensed from IP infusion). Also worth noting the 7000 series is actually not on a common ARM architecture, there's some outliers like the 7750 which are still PPC (but at least moved to uboot + linux). I believe the new 7850 they also finally moved to native 64bit on the new armv8 A57s versus the 32 bit armv7 stuff on the older 7000 series as well but I haven't had a chance to look at the firmware yet
     
    #2666
  7. BullCreek

    BullCreek New Member

    Joined:
    Jan 5, 2016
    Messages:
    13
    Likes Received:
    4
    Finally took the plunge and got a 6610 - it didn't come with a console cable unfortunately - and I see multiple links to the required brocade cable in these pages, but looking at the pictures when i follow those links, I don't get it. The recommended one on ebay shows a picture of a cable with an rj45 on one end and mini usb on the other, and then also what looks like an adapter that goes from rj45 to db9. I assume i plug the rj45 end of the cable into the switch, but how to i go from the miniusb to db9 that I can plug into my USB to serial adapter or am I totally not understanding how it is supposed to work?

    FWIW, I have a cisco baby blue cable i use at work, and I tried plugging it into the rj45 on the switch via my usb to serial adapter, and I get output, but just squiggles - any chance I could make that work - maybe wrong settings?
     
    #2667
  8. rocketpanda40

    rocketpanda40 New Member

    Joined:
    Dec 12, 2019
    Messages:
    4
    Likes Received:
    2
    #2668
    fohdeesha likes this.
  9. rocketpanda40

    rocketpanda40 New Member

    Joined:
    Dec 12, 2019
    Messages:
    4
    Likes Received:
    2
    Cisco style cable works, just make sure you're 96008n1 with flow control off.
     
    #2669
    T_Minus likes this.
  10. Wolfstar

    Wolfstar Member

    Joined:
    Nov 28, 2015
    Messages:
    78
    Likes Received:
    45
    The cable people are referring to in this thread is for the 7250/7450 switches. The 6610 and 6450/6430 switches all use a standard RJ45 serial port. That said, the way you would use the cable is to plug the mini-USB port into the switch, then the RJ45 into the adapter, then the adapter into your USB-Serial dongle.
     
    #2670
  11. BullCreek

    BullCreek New Member

    Joined:
    Jan 5, 2016
    Messages:
    13
    Likes Received:
    4
    Thanks Wolfstar, that makes sense about the cable not being for the 6610 which doesn't have a miniusb to plug into - and I'm happy to say my cisco cable worked after I got the settings right per rocketpanda40's advice - I'm off to the races now.
     
    #2671
  12. Wolfstar

    Wolfstar Member

    Joined:
    Nov 28, 2015
    Messages:
    78
    Likes Received:
    45
    Can confirm. I'm a contractor for an... exceptionally large organization, and we're deploying 7450s en masse. (Last number I heard was in excess of 2500 switches.) We use them entirely as Layer 2 devices, so not even inter-VLAN routing.
     
    #2672
    fohdeesha likes this.
  13. vangoose

    vangoose Member

    Joined:
    May 21, 2019
    Messages:
    77
    Likes Received:
    25
    Powered on my second 7250. Very surprised it's a POE version.

    Doing firmware upgrade to 8080e and POE firmware is automatically upgraded, don't need to run it inline upgrade

     
    #2673
  14. rootwyrm

    rootwyrm New Member

    Joined:
    Mar 25, 2017
    Messages:
    8
    Likes Received:
    1
    Oh, where there's a will (and some time) there's a way. At the end of the day, it's Linux. If you can't break the security on a Linux appliance, you just aren't trying. I wonder if you might mixing up licensees though; Broadcom was a known IP Infusion licensee (as was F10 now Dell) which lives on the DCX family. I would be a bit surprised if they bailed on Foundry's BGP and OSPF stacks, which both predated and were significantly better than IP Infusion. But, well, let's just say none of the post-Foundry owners are the sharpest crayons in the shed! ¯\_(ツ)_/¯ (And if Broadcom/Brocade want to argue, I swear to god I will find the list of bugs I found on Silkworm, DCX, and DMM.)

    Based on the spec sheet, I'd guess the ICX7750 is an NXP 4040 (e500mc) and the chassis stack uses the sRIO ports to an XAUI interface on the stack port, which means the stack members can operate in hardware lockstep (critical with a non-RTOS.) The RAM amount (3840MB) points pretty strongly to being a 32-bit core; that's 4GB (32-bit maximum) with a 256MB address hole. But it could also be the P5-series (e5500) at below-stock clock. Also means it may have the secure code controller enabled which would make software modification even with the appropriate tools a non-starter.

    Amusingly, we actually know what the ICX7850 is under the hood; it's a BCM58712D. (Clicky linky.) Unfortunately, it's a <sigh> Broadcrap <sigh> so chances of getting actual meaningful documentation on the CPU are between "screw" and "you." Worse, it looks to be an all around mediocre part that tried to cherry pick from Vulcan. The NXP LS1046A looks to have way better I/O on the newer and just plain better A72 core.
     
    #2674
  15. ronclark

    ronclark New Member

    Joined:
    Dec 6, 2019
    Messages:
    9
    Likes Received:
    3
    so i picked up a ICX6610-48P it should be here Monday, so its over kill on the ports for what i need right now. its clean has ears, two power supplies and two fan trays.

    I have a baby blue serial to RJ45, it sounds like i am set for getting into the console.
    I need to get a few more things I need to get hooked up let see if i did my homework right.
    I am running three 649281-B21 ConnectX3 QSFP+ I have two QSFP+ DAC cables so i should be good for two of servers. for the workstation it looks like i need

    HP-Mellanox-655874-B21-QSFP-SFP-Adapter-Kit-655902-001

    Amphenol-571540002-2-meters-SFP-10GbE-Direct-Attach-Passive-Copper-Cable
     
    #2675
  16. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,482
    Likes Received:
    1,249

    it's definitely IP Infusion: (lots of their sigs in the binary as well):

    [​IMG]

    and yeah, I never went anywhere near the silkworm, DCX, FC etc crap, they seemed to have the total opposite engineering mindset as the fastiron guys that came from foundry (which weren't perfect, but at least weren't super authoritarian about every piece of the OS). the 7750 management plane is a freescale P2041 (four e500mc cores on one chip). Oddly a lot of the 7750 datasheets show 8GB of RAM but indeed it's limited to 4GB. Not sure if they were planning on going 64-bit and didn't last minute or what, but to this day a lot of the datasheets show 8GB, lol.

    the whole ICX7xxx series is broadcom, I'd say it's at least a step up from the Marvell ASICs they were using in the 6 series and before, at least in terms of documentation. Been trying to work with up-n-atom on writing new values to the broadcom warpcore registers on the 7250 to alter the serdes datarate to enable 2.5gbps SFPs on the thing (the ASIC fully supports it), but they've made the broadcom shell super annoying to get at (he's made progress, though, have a whole list of ~800 hidden commands I've been meaning to publish, including the broadcom tor shell). I know the old marvell ASICs didn't have half the tools to allow such modifications, and the ones that did required massive license fees just to get documentation for
     
    #2676
    Wolfstar likes this.
  17. cooldude

    cooldude Member

    Joined:
    Feb 20, 2018
    Messages:
    31
    Likes Received:
    0
    I am deciding between juniper ex3300 and 7250...Need 48port poe and 4 1ogig ports.Looking to do routing on switch and vlan isolation.
    Right now using unifi switch fortigate.Fortigate does all routing..
    I cant decide on switch..Any help appreciated
     
    #2677
  18. vangoose

    vangoose Member

    Joined:
    May 21, 2019
    Messages:
    77
    Likes Received:
    25
    Just got my 7250 stack fully configured and migrated all ports from 6450 to it.
    Absolutely love them.
     
    #2678
    fohdeesha likes this.
  19. vangoose

    vangoose Member

    Joined:
    May 21, 2019
    Messages:
    77
    Likes Received:
    25
    Like both but I find ICX cli is simpler and easier to understand.
     
    #2679
    fohdeesha likes this.
  20. artlessknave

    artlessknave New Member

    Joined:
    Mar 16, 2017
    Messages:
    11
    Likes Received:
    2
    do these brocades work with any 10gbe rj45? they seem tempting compared to CRS317-1G-16S+RM, but as far as i can tell, that supports basically every sfp, including rj45 copper 10gbe, while the brocades...do not.
    I'm not sure I wanna spend 350$ for only 8x10gbe ports that can never do 10gbe rj45 when I could spend 500$ for 16 flexible ports (1/10/1.5gbe, fiber/DAC/rj45), particularly if they sound like they are trying to break the sound barrier while being 40 pounds. I only want a "dumb" switch, I don't care about l3, routing, vlans, or any of that. (this would be connected to my "core" lb6m/lb4m pair (stock FW) to bring 10gbe to a different room)
     
    #2680
Similar Threads: Brocade Series
Forum Title Date
Networking Mikrotik compatible with Brocade 57-0000075-01 transceivers? Dec 17, 2019
Networking [SOLVED] Help Needed - Brocade ICX 6450 + Ruckus R720 Nov 25, 2019
Networking Brocade ICX 6610 - what does dhcp-client enable do? Nov 19, 2019
Networking MikroTik CSS326-24G-2S+RM vs Brocade ICX6450-24 Nov 1, 2019
Networking Brocade ICX6450-24P vs Aruba S2500-24P for Homelab Sep 22, 2019

Share This Page