Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Discussion in 'Networking' started by fohdeesha, Jul 12, 2018.

  1. ewer0012

    ewer0012 Member

    Joined:
    Feb 10, 2019
    Messages:
    57
    Likes Received:
    20
    It wasn't, which is why I tried to update it. It doesn't boot from the secondary, either. :(
     
    #2341
  2. Wolfstar

    Wolfstar Member

    Joined:
    Nov 28, 2015
    Messages:
    56
    Likes Received:
    24
    Was it being sold as for-parts? If not I'd hit up the seller since it doesn't work. But yeah, sounds like it might be toast - maybe fohdeesha will know of something to work around there.
     
    #2342
  3. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,352
    Likes Received:
    1,089
    definitely sounds broke to me
     
    #2343
  4. ewer0012

    ewer0012 Member

    Joined:
    Feb 10, 2019
    Messages:
    57
    Likes Received:
    20
    Nope, was sold as "Used".

    Looks like I need to put in for a return :( No wonder they accepted the offer of $100 for it. Too good to be true.

    Thanks for the ideas, I appreciate the help. :)
     
    #2344
  5. Haim Gelfenbeyn

    Haim Gelfenbeyn New Member

    Joined:
    Sep 8, 2019
    Messages:
    1
    Likes Received:
    0
    I just bought ICX6450-48P off eBay! Seems like the firmware is relatively easy to "open": I used BinWalk to extract everything from the image file:

    binwalk -e -M ./Images/ICX64R08030r.bin

    And then you have everything extracted as separate files, including etc/init.sh which actually starts everything when the switch boots. Too bad the actual switch app is just one, statically-linked monolith (and compressed with xz for some reason).

    I wonder how hard it would be to add additional services to the switch (e.g. add/remove files and repack the firmware, then update), and how much space is still available on that flash... AFAIK the firmware is not signed in any way, am I correct? I wonder if anyone already tried doing something like that...
     
    #2345
  6. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,352
    Likes Received:
    1,089
    oh yeah, I have done some insane stupid stuff to these switches firmware. Some of it is covered back midway in the thread, can't remember where. You can also get access to the live linux FS as the switch runs and load in whatever you want there via tftp etc, methods for that are posted somewhere in the first page

    fyi brocade images (at least for linux based switches) are u-boot FIT images with some brocade specific headers in front. Instead of using binwalk which mostly has to guess, you can install u-boot-tools, which comes with the dumpimage binary, which can walk through and take apart the FIT image, you just have to trim out the first 512 bytes (brocade's header): hastebin

    you can then use dumpimage to selectively extract any part of it:

    Code:
    root@testing:~# dumpimage -T flat_dt -p 5 -i trimmed.bin ICX7250-ramdisk.zip
    Extracted:
     Image 5 (ramdisk@1)
      Description:  Ramdisk -rootfs
      Created:      Tue Apr  9 06:25:31 2019
      Type:         RAMDisk Image
      Compression:  lzma compressed
      Data Size:    25430519 Bytes = 24834.49 KiB = 24.25 MiB
      Architecture: ARM
      OS:           Linux
      Load Address: 0x00000000
      Entry Point:  0x00000000
      Hash algo:    crc32
      Hash value:   960fb2cd
    
    then you have the nicely packaged nix filesystem in an lzma compressed ZIP archive. You can do the same to extract the kernel and device trees, make any modifications you want, then use another u-boot-tools utility, mkimage, to repackage it back into a FIT compliant file ready to flash back. some docu here: Confluence

    fun fact: the ICX7150, ICX7250, and ICX7450 all run the same firmware image. They have codenames:

    ICX7150 = MN (Minion)
    ICX7250 = SI (Sica)
    ICX7450 = SP (Spatha)

    As you can see in the FIT dump above, the unified firmware image detects which platform it's booting on, then loads up a unique combination of a device tree, ramdisk, and kernel for that specific platform:

    Code:
     Default Configuration: 'conf@1'
     Configuration 0 (conf@1)
      Description:  Boot MN Linux kernel with FDT blob
      Kernel:       kernel@2
      Init Ramdisk: ramdisk@1
      FDT:          fdt@3
     Configuration 1 (conf@2)
      Description:  Boot SP  Linux kernel with FDT blob
      Kernel:       kernel@1
      Init Ramdisk: ramdisk@1
      FDT:          fdt@1
     Configuration 2 (conf@3)
      Description:  Boot SI Linux kernel with FDT blob
      Kernel:       kernel@1
      Init Ramdisk: ramdisk@1
      FDT:          fdt@2
    if you're wondering how I got the codenames, they're buried in code comments in a preinit.sh script in the above image ramdisk: hastebin
     
    #2346
    Last edited: Oct 14, 2019
    BeTeP and Haim Gelfenbeyn like this.
  7. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,352
    Likes Received:
    1,089
    fun fact #2: they started protecting the linux install underneath with a root password, however they didn't bother hashing it, so you can get the password by just looking at /etc/passwdshadow from the above extracted ramdisk
     
    #2347
    maes likes this.
  8. maes

    maes Member

    Joined:
    Nov 11, 2018
    Messages:
    43
    Likes Received:
    24
    Great, now I'm curious to see if there are any unused IOs anywhere on there. Some of the empty eeprom pads might be promising for I2C or SPI, to allow extra storage (say, SD cards) or more... unusual... options. GPS module, maybe, to give it a full-blown internal stratum-1 ntp server?

    @fohdeesha , in your teardowns, did you ever write down the exact model of the CPU in the 6450? Or is it integrated in one of the switching ASICs?
     
    #2348
    Last edited: Oct 14, 2019
  9. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,352
    Likes Received:
    1,089
    #2349
    maes likes this.
  10. maes

    maes Member

    Joined:
    Nov 11, 2018
    Messages:
    43
    Likes Received:
    24
    Thanks! Shame the kernel available on the Brocade/Arris/Ruckus sourceforge is absolutely prehistoric (looks like 2.6.12-r3!) so that limits a few things, but it might be fun to experiment.

    There's definitely a lot of connectivity to those CPUs, it's a matter of seeing what's traced out and available.
     
    #2350
  11. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,352
    Likes Received:
    1,089
    that's just the super old icx6xxx line, the newer stuff (like v8091) is using at least 4.4.0 looks like:

    Code:
    Linux kernel version "4.4.0 (swrel@l42-ub-ecbld-03) (gcc version 4.9.3 (Buildroot 201gcc version 4.9.3 (Buildroot 2015.11.1) )
     
    #2351
    maes likes this.
  12. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,352
    Likes Received:
    1,089
    quick guide on extracting brocade images:

    -------Extracting Brocade Firmware (v8060 and above)
    ##Remove the brocade-specific header info from the file
    ##For regular images, this is 512 bytes:
    dd if="SPR08080e.bin" of="trimmed.bin" bs=512 skip=1

    ##For UFI images, (v8091 and up only have UFI images available), it's 1360 bytes:
    dd if=SPR08091ufi.bin of="trimmed.bin" bs=1360 skip=1


    #Install u-boot-tools
    apt install u-boot-tools
    ##Debian 9 (and maybe others) have a very old uboot-tools version in the default repo, and does not come with the required dumpimage
    ##In that case, just manually install recent tools:
    wget http://ftp.us.debian.org/debian/pool/main/u/u-boot/u-boot-tools_2019.01+dfsg-7_amd64.deb
    dpkg -i u-boot-tools_2019.01+dfsg-7_amd64.deb

    ##Now use dumpimage to view a list of all the components in the firmware package:
    dumpimage -l trimmed.bin

    ##You'll get a list of everything in the package:
    Code:
    FIT description: Linux kernel and FDT blob
    Created:         Fri Jun  7 00:12:13 2019
     Image 0 (kernel@1)
      Description:  Ruckus Linux SP/SI
      Created:      Fri Jun  7 00:12:13 2019
      Type:         Kernel Image
      Compression:  lzma compressed
      Data Size:    2386204 Bytes = 2330.28 KiB = 2.28 MiB
      Architecture: ARM
      OS:           Linux
      Load Address: 0x61008000
      Entry Point:  0x61008000
      Hash algo:    crc32
      Hash value:   19ecdaa8
     Image 1 (kernel@2)
      Description:  Ruckus Linux MN VER=08.0.91
      Created:      Fri Jun  7 00:12:13 2019
      Type:         Kernel Image
      Compression:  lzma compressed
      Data Size:    2401701 Bytes = 2345.41 KiB = 2.29 MiB
      Architecture: ARM
      OS:           Linux
      Load Address: 0x61008000
      Entry Point:  0x61008000
      Hash algo:    crc32
      Hash value:   8c4ccc81
    ----trimmed for brevity----

    ##Choose a part to extract to a separate file, putting the image number after the -p argument:
    ##Note: -T must be kept set to "-T flat_dt", even if you are extracting a different image type
    dumpimage -T flat_dt -p 0 -i trimmed.bin kernel.zip

    #If you extract firmware device trees and want to see them in human-readable form:
    apt install device-tree-compiler
    fdtdump FDT.bin
     
    #2352
    Last edited: Oct 15, 2019
    maes likes this.
  13. maes

    maes Member

    Joined:
    Nov 11, 2018
    Messages:
    43
    Likes Received:
    24
    I'm only running a icx6540 so 'super old' is appropriate. Fortunately, the 88f6281 is so common even on other hardware, if Brocade hasn't done anything too weird to the old kernel it might be possible to run a far more recent one. I know the sheevaplug (and successors) and a gaggle of NAS boxes use the exact same CPU.

    Still, not quite something I'm too tempted to attempt until I at least have a spare to tinker with.
     
    #2353
  14. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,352
    Likes Received:
    1,089
    heh yup, when I had to write a JTAG config from scratch for openOCD for the 6450 I borrowed heavily from some of the existing sheevaplug configs

    I wouldn't worry about messing around with the OS firmware, it's not like you're re-writing the bootloader. You can flash the most screwed up image possible and if it doesn't work you can just drop into u-boot and flash the stock firmware back. Even if you did flash a new u-boot for some reason and it went wrong, the boot memory on the 6450 is SPI (mx25l25635emi-12g) so you could most likely just use a 10 dollar SPI reader/writer and a chip clip to dump u-boot back on it
     
    #2354
    maes likes this.
  15. Smbaker

    Smbaker New Member

    Joined:
    Oct 9, 2019
    Messages:
    13
    Likes Received:
    6
    Thanks again to fohdeesha for this thread. I was all prepared to spend around $350 on a new Mikrotik and instead I have this wonderful ICX6450-24P for just under two hundred bucks. Following the guide and licensing went without issue.

    I installed a couple digikey 259-1633-ND in place of the factory fans and it's now silent at least to the ability of my hearing. It's not that the stock fans were loud, not compared to some other equipment I've owned, but they were loud enough. I don't even hear the PWM noise that people have complained of in this thread; it's possible it's either above my auditory range or a perfect match for my tinnitus.

    This is going to allow me to get rid of a pair of Cisco SG300 switches that I had, going down to just one switch, with more PoE than I had before, and the 10G ports.
     
    #2355
    fohdeesha likes this.
  16. Roelf Zomerman

    Joined:
    Jan 10, 2019
    Messages:
    34
    Likes Received:
    4
    Given I'm putting up camera's outside the house (and therefore extending the LAN to outside). I was wondering what the best way is to Mac-Lock the ports on the 6450? So, if someone just adds something on the wire, the switch would deny it?

    (I know not the best method, but it's a start... - the cams dont support 802.1x)
     
    #2356
  17. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,352
    Likes Received:
    1,089
    you can use the generic MAC ACLs but it's simpler to use the MAC security feature, which was made for your particular application. basically you specify "secure" MACs and these are allowed to work, nothing else is (either on one port, or globally). If that port sees another mac, it can disable the port, creates a snmp trap, generates a log entry in the switch (as well as syslog server if you have one configured)

    something like:
    int e 1/1/5
    port security
    secure-mac-address 3adc.9f45.7a76
    #if the port has tagged vlans on it, which I doubt yours does with the camera, you put the vlan ID on the end (not necessary for untagged ports)
    secure-mac-address 3adc.9f45.7a76 10
    #tell it to shutdown the port if it sees another mac, and for how long in minutes. 0 means forever until you manually re-enable it
    violation shutdown 5

    #show status
    show port security e 1/1/5

    more details starting on page 189 of fastiron-08030n-securityguide.pdf in the firmware zip from the update guide. If your camera has a sticker on it that includes the MAC address, I advise removing it if relying on this type of security :p
     
    #2357
    capn13 and Roelf Zomerman like this.
  18. Roelf Zomerman

    Joined:
    Jan 10, 2019
    Messages:
    34
    Likes Received:
    4
  19. tojoski

    tojoski New Member

    Joined:
    Sep 22, 2019
    Messages:
    14
    Likes Received:
    1
    So I did some testing on my 6450 last night with the replacement sunon fans and something is amiss.

    The more I look at it, I think these might be knock-offs.. they make a sound almost like old hard drive would.



    Comparing the one I am holding to a used one I found on ebay, the ones that i received seem to be of notably lesser quality, both in construction and the label.

    IMG_20191016_030225.jpg Capture.JPG

    I bought them off of a Amazon, but they were a third-party seller. Anyone else run into this?
     
    #2359
  20. fohdeesha

    fohdeesha Kaini Industries

    Joined:
    Nov 20, 2016
    Messages:
    1,352
    Likes Received:
    1,089
    Anyone need licenses I only have 1.9660494e+39 left
     
    #2360
Similar Threads: Brocade Series
Forum Title Date
Networking Brocade ICX 6610 - what does dhcp-client enable do? Yesterday at 9:48 PM
Networking MikroTik CSS326-24G-2S+RM vs Brocade ICX6450-24 Nov 1, 2019
Networking Brocade ICX6450-24P vs Aruba S2500-24P for Homelab Sep 22, 2019
Networking Brocade VDX 6720 - what do I need to know? Sep 16, 2019
Networking anyone have Brocade CER 2024C latest firmware Sep 14, 2019

Share This Page