Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Discussion in 'Networking' started by fohdeesha, Jul 12, 2018.

  1. fohdeesha

    fohdeesha Kaini Industries

    Nov 20, 2016
    Likes Received:

  2. gustav9797

    gustav9797 New Member

    Dec 30, 2016
    Likes Received:
    Bought a pair of ICX6450-24P from this UK seller:
    Brocade ICX6450-24P 24-Port Gig PoE Switch + 4 X SFP Ports 5055927896927 | eBay
    Accepted 110GBP/each but at dubble shipping rate..
    I'm looking forward to having a solid home network with some server experiment, now that I will get fiber to my house(finally) in a couple of months.
    I hope to replace the fiber converter with one switch and do a "long range stack" to the second switch in a separate building. I hope it works.
    fohdeesha likes this.
  3. Freebsd1976

    Freebsd1976 Member

    Feb 23, 2018
    Likes Received:
    help needed , just buy icx7150-c12P ,connect to serial , putty show following message then got DRAG> ,there seem no firmware on it ,what I can do :
    U-Boot 2016.01-Broadcom XLDK-3.8.1-svn20090 (Oct 04 2016 - 07:22:19 +0800)
    I2C: ready
    DRAM: Reset XGPLL
    Release reset
    DEV ID = 0xb160
    SKU ID = 0xb160
    DDR type: DDR4
    MEMC 0 DDR speed = 800MHz
    PHY revision version: 0x00a2f001
    ddr_init2: Calling soc_and28_shmoo_dram_info_set
    ddr_init2: Calling soc_and28_shmoo_phy_init
    A Series - PHY Initialization (PHY index: 00)
    A01. Turn off CKE
    A02. Configure timing parameters
    A03. Configure PHY PLL
    PHY PLL Configuration
    Fref.............: 50 MHz
    Data rate........: 1600 Mbps
    PLL locked.
    A04. Configure reference voltage
    A05. Compute VDL step size
    VDL calibration complete.
    VDL step size....: 10.775 ps
    UI size..........: 58.000 steps
    A06. Configure ADDR/CTRL VDLs
    A07. Disable Virtual VTT
    A08. ZQ calibration
    P drive..........: 0x0C
    N drive..........: 0x13
    P termination....: 0x0C
    N termination....: 0x00
    P idle...........: 0x0C
    N idle...........: 0x00
    AQ P drive.......: 0x12
    AQ N drive.......: 0x13
    A09. Configure Static Pad Control
    A10. Configure ODT
    A11. Configure Write Pre-/Post-amble
    A12. Configure Auto Idle
    A13. Release PHY control
    A Series - PHY Initialization complete (PHY index: 00)
    Programming controller register
    Enabling mt40a512m8_093_1600
    ddr_init2: MemC initialization complete
    ddr_init2: Calling soc_and28_shmoo_ctl
    Validate Shmoo parameters stored in flash ..... OK
    Press Ctrl-C to run Shmoo ..... skipped
    DDR Tuning Complete
    Running simple memory test ..... OK
    GPIO Reaset Reason is 0x7
    COLD BOOT: Clearing entire memory
    BIST Start 0x0, END 0x1ffffff
    clear_ddr: OK
    DDR Interface Ready
    1022 MiB
    arm_clk=1250MHz, axi_clk=400MHz, apb_clk=100MHz, arm_periph_clk=625MHz
    Flash: 0 Bytes
    NAND: PNOR flash is not present - switch mux back for NAND
    Micron MT29F32G08CBADB, blocks per lun: 850 lun count: 1
    Invalid strap options for this NAND: page=2 type=6
    Overriding invalid strap options: strap_type=8
    2048 KiB blocks, 8 KiB pages, 45B OOB, 8-bit
    NAND: chipsize 4096 MiB
    MMC: iproc_sdhci: 0
    Access set to SECONDARY FL..
    SF: Detected W25Q64CV with page size 256 Bytes, erase size 64 KiB, total 8 MiB
    In: serial
    Out: serial
    Err: serial
    DDR Tuning Complete
    Unlocking L2 Cache ...Done
    Net: Registering BCM xgs eth
    Broadcom XGS Iproc Ethernet driver 0.1
    Using GMAC0
    gmac_mac_init: Chip ID: 0xb160
    r = 0x0, addr = 0x1, devad = 0xffffffff, phy_id = 0x600d84aa
    Add phy reset delay
    Add phy reset delay
    Basic ethernet functionality initialized
    bcm_xgs_gmac-0 [PRIME] ETH MAC: 60:9c:9f:bc:c1:78
    Booted From: Uboot partition 2
    Hit any key to stop autoboot: 0
    NAND read: device 0 offset 0x0, size 0x5000000
    83886080 bytes read: OK
    ## Loading kernel from FIT Image at 65000000 ...
    Using 'conf@1' configuration
    Trying 'kernel@1' kernel subimage
    Description: Broadcom iProc Linux
    Type: Kernel Image
    Compression: uncompressed
    Data Start: 0x650000d4
    Data Size: 66215968 Bytes = 63.1 MiB
    Architecture: ARM
    OS: Linux
    Load Address: 0x61008000
    Entry Point: 0x61008000
    Hash algo: crc32
    Hash value: e4d0f5c7
    Verifying Hash Integrity ... crc32+ OK
    ## Loading fdt from FIT Image at 65000000 ...
    Using 'conf@1' configuration
    Trying 'fdt@1' fdt subimage
    Description: Flattened Device Tree blob - bcm956160.dtb
    Type: Flat Device Tree
    Compression: uncompressed
    Data Start: 0x68f261e8
    Data Size: 7565 Bytes = 7.4 KiB
    Architecture: ARM
    Hash algo: crc32
    Hash value: 02d9852b
    Verifying Hash Integrity ... crc32+ OK
    Booting using the fdt blob at 0x68f261e8
    Loading Kernel Image ... OK
    Using Device Tree in place at 68f261e8, end 68f2af74
    Starting kernel ...
    iproc-pcie 18012000.pcie: no PCIe EP device detected
    iproc-pcie 18012000.pcie: PCIe controller setup failed
    mtd: partition "rootfs" is out of reach -- disabled
    Initializing random number generator... done.
    Starting network...
    Trying to mount ntest on NAND...
    UBI device number 1, total 128 LEBs (266338304 bytes, 254.0 MiB), available 0 LEBs (0 bytes), LEB size 2080768 bytes (2.0 MiB)
    DMA pool size: 33554432
    Broadcom Command Monitor: Copyright (c) 1998-2010 Broadcom Corporation
    Release: sdk-6.5.4 built 20161028 (Fri Oct 28 18:00:30 2016)
    From keven.lin@Indian:/home/keven.lin/SWITCH/Minions/XLDK_381/buildroot/output/build/broadcom-sdk-6.5.4
    Platform: IPROC_CMICD
    OS: Unix (Posix)
    AXI unit 0: Dev 0xb160, Rev 0x11, Chip BCM56160_B0, Driver BCM56160_A0
    SOC unit 0 attached to PCI device BCM56160_B0
    ERROR loading rc script on unit 0
    Board Id: Minions-C12P
    Board Revision: Alpha 1
    Totally, 16ports are installed to port info DB.
    mgmt socket interface name = eth0
    Enable temperature monitor function ... Done
    Minions-C12 config set
    fox diag init
    Init IO expanders ...Done.
    Reset PoE, write control register 0x30, 0xf4
    Release reset PoE PD69208 , write control register 0x30, 0xf6
    Release reset PoE PD69200, write control register 0x30, 0xf7
    Init PoE ... DC-Disconnect Mode
    PoE Software Version: 16.0172.15
    Save system setting
    PoE port 1 enabled
    PoE port 2 enabled
    PoE port 3 enabled
    PoE port 4 enabled
    PoE port 5 enabled
    PoE port 6 enabled
    PoE port 7 enabled
    PoE port 8 enabled
    PoE port 9 enabled
    PoE port 10 enabled
    PoE port 11 enabled
    PoE port 12 enabled
    * FoxDiag: FoxDiag_1.1.6-t3
    * CPLD: Rev3 release 7
    * Platform: Minions-C12P Alpha 1
    * Copyright: Copyright 2016 Foxconn Corporation.

    if I press B when switch poweron ,it will go u-boot :
    Booted From: Uboot partition 2
    Hit any key to stop autoboot: 0
    u-boot> help
    ? - alias for 'help'
    base - print or set address offset
    bdinfo - print Board Info structure
    boot - boot default, i.e., run 'bootcmd'
    bootd - boot default, i.e., run 'bootcmd'
    bootelf - Boot from an ELF image in memory
    bootm - boot application image from memory
    bootp - boot image via network using BOOTP/TFTP protocol
    bootvx - Boot vxWorks from an ELF image
    bootz - boot Linux zImage image from memory
    chpart - change active partition
    cmp - memory compare
    coninfo - print console devices and information
    cp - memory copy
    cplddl - cplddl - To perform cpld download
    crc32 - checksum calculation
    dhcp - boot image via network using DHCP/TFTP protocol
    echo - echo args to console
    editenv - edit environment variable
    env - environment handling commands
    erase - erase FLASH memory
    exit - exit script
    ext2load- load binary file from a Ext2 filesystem
    ext2ls - list files in a directory (default /)
    false - do nothing, unsuccessfully
    fatinfo - print information about filesystem
    fatload - load binary file from a dos filesystem
    fatls - list files in a directory (default /)
    fatsize - determine a file's size
    fatwrite- write file into a dos filesystem
    fdt - flattened device tree utility commands
    flinfo - print FLASH memory information
    go - start application at address 'addr'
    gpio_read- Read from GPIO
    gpio_write- write to GPIO
    hash - compute hash message digest
    help - print command description/usage
    i2c - I2C sub-system
    iminfo - print header information for application image
    imxtract- extract a part of a multi-image
    itest - return true/false on integer compare
    loadb - load binary file over serial line (kermit mode)
    loads - load S-Record file over serial line
    loadx - load binary file over serial line (xmodem mode)
    loady - load binary file over serial line (ymodem mode)
    loop - infinite loop on address range
    md - memory display
    mdc - memory display cyclic
    mdio - MDIO utility commands
    mii - MII utility commands
    mm - memory modify (auto-incrementing address)
    mtdparts- define flash/nand partitions
    mtest - simple RAM read/write test
    mw - memory write (fill)
    mwc - memory write cyclic
    nand - NAND sub-system
    nboot - boot from NAND device
    nfs - boot image via network using NFS protocol
    nm - memory modify (constant address)
    ping - send ICMP ECHO_REQUEST to network host
    printenv- print environment variables
    protect - enable or disable FLASH write protection
    reset - Perform RESET of the CPU
    run - run commands in an environment variable
    saveenv - save environment variables to persistent storage
    setenv - set environment variables
    sf - SPI flash sub-system
    showvar - print local hushshell variables
    sleep - delay execution for some time
    source - run script from memory
    test - minimal test like /bin/sh
    tftp -
    tftpboot- boot image via network using TFTP protocol
    tftpput - TFTP put command, for uploading files to a server
    time - run commands and summarize execution time
    true - do nothing, successfully
    ubi - ubi commands
    ubifsload- load file from an UBIFS filesystem
    ubifsls - list files in a directory
    ubifsmount- mount UBIFS volume
    ubifsumount- unmount UBIFS volume
    usb - USB sub-system
    usbboot - boot from USB device
    version - print monitor, compiler and linker version

    execute env print got these:

    u-boot> env print
    bootargs=console=ttyS0,9600n8 maxcpus=1 mem=496M mtdparts=brcmnand.0:256m(nrootfs),256m(ntest),-(reserve) quiet
    bootcmd=run nand2boot
    dhcp2boot=dhcp ${fileaddr} ${diagImg}; bootm ${fileaddr};
    nand2boot=nand read ${fileaddr} 0x0 0x5000000; bootm ${fileaddr}
    tftp2boot=tftp ${fileaddr} ${diagImg}; bootm ${fileaddr};
    usb2boot=usb start; fatload usb 0:1 ${fileaddr} ${diagImg}; bootm ${fileaddr};
    ver=U-Boot 2016.01-Broadcom XLDK-3.8.1-svn20090 (Oct 04 2016 - 07:22:19 +0800)
    Environment size: 1150/65532 bytes
    Last edited: Aug 10, 2019
  4. Freebsd1976

    Freebsd1976 Member

    Feb 23, 2018
    Likes Received:
    in DIAG> , ? show ,now what I can do with it, or I need returun it to seller ? @fohdeesha
    ? - alias for 'help'
    bcm_shell - Invoke Broadcom Backdoor
    bcmscript - execute uploaded bcm script.
    bypassmux - This command bypass mux mode.
    cpldread - To read cpld register
    cpldtest - To perform cpld test
    cpldupdate - To perform cpld update
    cpldwrite - To write special value to cpld register
    eeprom - EEPROM dump or program command
    fanread - To read fan speed and status
    fantask - To enable/disable fan task for thermal policy
    fantest - This test validates the fantrays can be controlled by I2C.
    fanwrite - To configure DC output for all fan
    fiberLbMode - This command config fiber ports loopback mode.
    fluffydelay - To adjust read delay of Fluffyread
    fluffyeeprom - Fluffy EEPROM dump or program command
    fluffyfwupgrade - Upgrade MCU firware
    fluffyintrdelay - To adjust LRM transceiver interrupt delay
    fluffyioexpinit - To init IOEXP 9536 in Fluffy for debug.
    fluffyioexpread - To read IOEXP 9536 in Fluffy for debug.
    fluffyioexpwrite - To write value to IOEXP 9536 in Fluffy for debug.
    fluffyread - Get MCU info for debug
    fluffyreadonly - Get MCU info for debug
    fluffystatus - To get LRM transceiver present status
    fluffytoggle - To toggle EEPROM between LRM transceiver and Fluffy
    fluffywrite - Set MCU info for debug
    gpioinit - To init GPIO
    gpioread - To read GPIO pin status
    gpioreset - To reset used by GPIO pin
    gpiotest - To test used by GPIO pin
    gpiowrite - To write gpio pin status
    help - print online help
    i2clist - Display all entries of i2c device info table
    i2cread - Get special i2c device id
    i2ctest - Test special i2c device id
    i2cwrite - Set special i2c register
    intrtest - This test interrupt pins from devices to CPU
    ioexpinit - To init io expanders
    ioexpread - To read io expander pin status
    ioexpwrite - To write io expander pin status
    leddump - This test is used to dump LED processor contents
    ledtest - This test validates all led.
    linux_shell - Enter Linux shell for kernel layer debugbing.
    memread - To perform memory read
    memtest - To perform memory test
    memwrite - To perform memory write
    mgmt - To perform management port test
    minicycle - Packets generated by CPU
    nandtest - To perform NAND test
    pdtest - Test Power Source config
    pktTolerance - This command config packet tolerance mode.
    poeafonly - set PoE in at/af mode
    poecmd - transmit a SCP packet with 0x00 as the KEY
    poedownload - Download firware
    poeforcepw - Force power
    poeprogram - transmit a SCP packet with 0x01 as the KEY
    poepwshow - Show power
    poerequest - transmit a SCP packet with 0x02 as the KEY
    poestatus - set PoE in enable/disable
    poetest - test if the PoE system is working correctly
    port - Dump status or counter of all switch ports.
    pseenable - Enable or Disable all PSE ports
    pseoverstress - Enable or Disable PSE ports with overstress ability
    qsberpara - To display QSGMII ber test parameters
    qsbertest - This test is used to measure the link quality and robustness of the system.
    qsbertxdrv - To set TSC-E ber test tx_driver
    qsbertxfir - To set TSC-E ber test tx_fir
    rtcread - Get rtc time
    rtctest - test the rtc function
    rtcwrite - Set rtc time
    sflashtest - To perform SFLASH test
    sfpIoDetection - This command sfp I/O detection mode.
    sfptxdisable - To init io expanders
    sfptxenable - To init io expanders
    smi - PHY test.
    snaketest - This test validates the QSGMII/SFI interfaces between MAC, and PHY.
    swlbtest - Packets generated by CPU
    swsnaketest - This test validates the MDI/QSGMII interfaces between MAC, and PHY.
    syscall - Execute SysCall for kernel layer debugbing.
    tempread - To read temperature
    thermal_threshold - Set thermal threshold
    tscberpara - To display TSC-E ber test parameters
    tscbershow - To display TSC-E ber test result
    tscbertest - This test is used to measure the link quality and robustness of the system.
    tscbertxdrv - To set TSC-E ber test tx_driver
    usb - USB test function
    version - print diagnostic firmware version
    voltagemargin - adjust VCC margin
    voltageread - To read voltage source
    voltagetest - test the margin
    xsmi - PHY test.
  5. kroy

    kroy New Member

    May 17, 2019
    Likes Received:
    Seems my brocade adventure just won't end.

    Got a 6610 today. Model number is ICX-6610-24-PE, Version:07.3.02T7f5

    Before I did anything else loaded it up. And there is absolutely no signs of this switch supporting PoE. Every other switch mentions initializing PoE on booting.

    ICX6610-24 Switch#show inline power
    ICX6610-24 Switch#

    ICX6610-24 Switch(config-if-e1000-1/1/1)#inline power
    Command not applicable for non-POE port 1/1/1.

    ICX6610-24 Switch(config-mif-1/1/1-1/1/24)#inline power
    set inline power cannot be done on this device with stack id 1 and poe slot 1!
    At this point I'm not sure if I should start up another RMA or try upgrading the switch. It's got the L2 firmware on it right now, plus it does appear to actually have one license on it, ICX6610-PREM-LIC-SW, which I don't want to risk wiping with a firmware upgrade (in case I have to RMA)

    @fohdeesha do these things need the L3 software on them to support PoE?

    Edit, and again, think I've been bamboozled. This thing only has a single REVA 250W power supply. Don't they need the 1000W supplies to support POE?
    Last edited: Aug 10, 2019
  6. BobTB

    BobTB New Member

    Jul 19, 2019
    Likes Received:
    I went beserk and bought 5 ICX swithces on ebay in last week, One 6610-24 , two 6430-48p, one 6450-24p and one 6430-24. All have BASE_SOFT_PACKAGE.

    What is actually missing in 6430 models with this license? I know they are only 1GB but this still works for me great.

    Stacking 6610 with 6450 does not work right?

    Today I just received the first one, 6430-24 which was with its config still not reset, all ports except one was disabled, I used the guide in this thread to reset it and all ports work now. I just had to make a serial cable to access the console - this was a pain :) I still think that for 50€ shipped, I could not have gotten a better switch with 24 ports.

    I intend to completely replace my dumb swithces and all poe injectors and stuff all over the house and lab,
  7. BeTeP

    BeTeP Active Member

    Mar 23, 2019
    Likes Received:
    Before sending it back for a moment I considered trying to repair it. One of the PD69208's was visibly burnt but I only needed like a dozen of POE ports and I was hoping to get at least a couple of other seemingly intact PD69208 chips (each responsible for 8 ports) working again.

    So I downloaded a few datasheets and appnotes for the pd69200 + 6x pd69208 chipset. Maybe someone would find these useful.

    PD69208m and PD69200 datasheet
    Application Note AN-211 - Designing an IEEE 802.3af/802.3at/802.3btCompliant PD69208 48-Port PoE System
    PD69200 serial communication protocol user guide
  8. fohdeesha

    fohdeesha Kaini Industries

    Nov 20, 2016
    Likes Received:
    @Freebsd1976 it has a developer/prerelease bootloader on it, I can fix it over a teamviewer session or you can return it, up to you

    @kroy it sounds like indeed they sent you a non-poe model, poe models will identify themselves as poe and will have 1kw PSUs. edit: just saw the model number you posted, that is indeed a non-poe model

    @BobTB all 6430 models are L2 only, they don't do l3 and there's no license available for them. if you need licenses for any of the others PM me

    you actually can stack a 6610 and a 6450, they call it mixed stacking, instructions are in the stacking pdf included in the firmware download from my guide. I'm not sure why you would need to make a serial cable for the 6430, it (and all the other switches you ordered) use the standard cicsco style rj45 pinout and cables are 5 bucks:
    tommybackeast likes this.
  9. BobTB

    BobTB New Member

    Jul 19, 2019
    Likes Received:
    Thank you for the info. As far as for the cable, it was friday afternoon, I got the switch deliverd, and could not wait for a week or so to get the cable from amazon. As I never had such a cable I just had to make it to try to get to the switch somehow :)
  10. juey

    juey Member

    Oct 1, 2018
    Likes Received:
    The ICX6610-24-PE is the non PoE model.
    What you need is the ICX6610-24P-PE model.
  11. fohdeesha

    fohdeesha Kaini Industries

    Nov 20, 2016
    Likes Received:
  12. infoMatt

    infoMatt Member

    Apr 16, 2019
    Likes Received:
    Yea.. smart move...
    The funniest part is the last sentence: "or trade for an Unifi switch"... sure, you're swapping a full L3 switch with horsepower to spare with a tplink-on-a-fancy-case-and-premium-pricetag "dumb" managed L2 switch, known for their recurring problem with passive 24V PoE activating even when not requested, that doesn't have 10G ports except for the 48P... but hey, you've got that point'n'clicky Java webinterface... :(

    Don't shoot me on that, I've got two Ubiquiti AP currently providing WiFi in my house... but let's be honest, they are more on the consumer level than what they want you to think about... And I'm having "buyer's remorse", thinking that I might have just bought two TPlink and flashed 'em with OpenWRT, I could have saved some money... oh well... :rolleyes:
  13. fohdeesha

    fohdeesha Kaini Industries

    Nov 20, 2016
    Likes Received:
    it has my licenses, he didn't even bother blanking out the LID
    tommybackeast likes this.
  14. kroy

    kroy New Member

    May 17, 2019
    Likes Received:
    Ahh yep. I thought it was the *P*E that meant POE.

    Definitely explains it. Thanks
  15. gustav9797

    gustav9797 New Member

    Dec 30, 2016
    Likes Received:
    What are people thinking since they in practice keep selling your licenses, are they doing it deliberately to earn more when selling it or have they forgotten about it. Either way it is a bit sad.
    tommybackeast likes this.
  16. tommybackeast

    tommybackeast Active Member

    Jun 10, 2018
    Likes Received:
    reach out to a /r/homelabsales mod and get that thread deleted, that's bullshit
  17. fohdeesha

    fohdeesha Kaini Industries

    Nov 20, 2016
    Likes Received:
    if it's who I think it is, I gave him licences almost a full year ago, I'm giving him the benefit of the doubt and assuming he just forgot he's supposed to delete them if reselling (please don't leave nasty messages on his post, even if he is being a dick on purpose it won't solve anything)
  18. Roelf Zomerman

    Jan 10, 2019
    Likes Received:
    Hey everyone,

    I know the 7450-48p with premium license can do BGP... anyone know if these licenses are still available also?
    I want to connect my Azure Stack home kits (2) with 40GB to the switch and then use them as the top of rack switches with BGP..
  19. fohdeesha

    fohdeesha Kaini Industries

    Nov 20, 2016
    Likes Received:
    the licenses for the entire icx7xxx are honor based as in they don't actually exist, you just enter an unlock command. I'm not a big fan of the 7450, they usually cost a lot more than the 6610, and no matter how you configure it, it has barely half of the high speed I/O as the 6610 - for instance occupying all 3 expansion slots on the 7450 can get you 2x 40gbe and 4x 10gbe, whereas the 6610 comes out of the box with 2x 40gbe and 16x 10gbE
  20. Roelf Zomerman

    Jan 10, 2019
    Likes Received:
    ah cool.. I'll try to see which commands I need to enter to enable it :) ..

    I bought it for $50, with 2x 40GB modules in the back and 4x 10Gb in the front.. ;)
Similar Threads: Brocade Series
Forum Title Date
Networking Mikrotik compatible with Brocade 57-0000075-01 transceivers? Dec 17, 2019
Networking [SOLVED] Help Needed - Brocade ICX 6450 + Ruckus R720 Nov 25, 2019
Networking Brocade ICX 6610 - what does dhcp-client enable do? Nov 19, 2019
Networking MikroTik CSS326-24G-2S+RM vs Brocade ICX6450-24 Nov 1, 2019
Networking Brocade ICX6450-24P vs Aruba S2500-24P for Homelab Sep 22, 2019

Share This Page