Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

gustav9797

New Member
Dec 30, 2016
16
7
3
22
Bought a pair of ICX6450-24P from this UK seller:
Brocade ICX6450-24P 24-Port Gig PoE Switch + 4 X SFP Ports 5055927896927 | eBay
Accepted 110GBP/each but at dubble shipping rate..
I'm looking forward to having a solid home network with some server experiment, now that I will get fiber to my house(finally) in a couple of months.
I hope to replace the fiber converter with one switch and do a "long range stack" to the second switch in a separate building. I hope it works.
 
  • Like
Reactions: fohdeesha

Freebsd1976

Active Member
Feb 23, 2018
202
30
28
help needed , just buy icx7150-c12P ,connect to serial , putty show following message then got DRAG> ,there seem no firmware on it ,what I can do :
U-Boot 2016.01-Broadcom XLDK-3.8.1-svn20090 (Oct 04 2016 - 07:22:19 +0800)
I2C: ready
DRAM: Reset XGPLL
Release reset
Polling
Locked
DEV ID = 0xb160
SKU ID = 0xb160
DDR type: DDR4
MEMC 0 DDR speed = 800MHz
PHY revision version: 0x00a2f001
ddr_init2: Calling soc_and28_shmoo_dram_info_set
ddr_init2: Calling soc_and28_shmoo_phy_init
A Series - PHY Initialization (PHY index: 00)
A01. Turn off CKE
A02. Configure timing parameters
A03. Configure PHY PLL
PHY PLL Configuration
Fref.............: 50 MHz
Data rate........: 1600 Mbps
PLL locked.
A04. Configure reference voltage
A05. Compute VDL step size
VDL calibration complete.
VDL step size....: 10.775 ps
UI size..........: 58.000 steps
A06. Configure ADDR/CTRL VDLs
A07. Disable Virtual VTT
A08. ZQ calibration
P drive..........: 0x0C
N drive..........: 0x13
P termination....: 0x0C
N termination....: 0x00
P idle...........: 0x0C
N idle...........: 0x00
AQ P drive.......: 0x12
AQ N drive.......: 0x13
A09. Configure Static Pad Control
A10. Configure ODT
A11. Configure Write Pre-/Post-amble
A12. Configure Auto Idle
A13. Release PHY control
A Series - PHY Initialization complete (PHY index: 00)
Programming controller register
Enabling mt40a512m8_093_1600
ddr_init2: MemC initialization complete
ddr_init2: Calling soc_and28_shmoo_ctl
Validate Shmoo parameters stored in flash ..... OK
Press Ctrl-C to run Shmoo ..... skipped
DDR Tuning Complete
Running simple memory test ..... OK
GPIO Reaset Reason is 0x7
COLD BOOT: Clearing entire memory
BIST Start 0x0, END 0x1ffffff
clear_ddr: OK
DDR Interface Ready
1022 MiB
arm_clk=1250MHz, axi_clk=400MHz, apb_clk=100MHz, arm_periph_clk=625MHz
Flash: 0 Bytes
NAND: PNOR flash is not present - switch mux back for NAND
Micron MT29F32G08CBADB, blocks per lun: 850 lun count: 1
Invalid strap options for this NAND: page=2 type=6
Overriding invalid strap options: strap_type=8
2048 KiB blocks, 8 KiB pages, 45B OOB, 8-bit
NAND: chipsize 4096 MiB
MMC: iproc_sdhci: 0
Access set to SECONDARY FL..
SF: Detected W25Q64CV with page size 256 Bytes, erase size 64 KiB, total 8 MiB
In: serial
Out: serial
Err: serial
DDR Tuning Complete
Unlocking L2 Cache ...Done
Net: Registering BCM xgs eth
Broadcom XGS Iproc Ethernet driver 0.1
Using GMAC0
gmac_mac_init: Chip ID: 0xb160
r = 0x0, addr = 0x1, devad = 0xffffffff, phy_id = 0x600d84aa
Add phy reset delay
Add phy reset delay
Basic ethernet functionality initialized
bcm_xgs_gmac-0 [PRIME] ETH MAC: 60:9c:9f:bc:c1:78
Booted From: Uboot partition 2
Hit any key to stop autoboot: 0
NAND read: device 0 offset 0x0, size 0x5000000
83886080 bytes read: OK
## Loading kernel from FIT Image at 65000000 ...
Using 'conf@1' configuration
Trying 'kernel@1' kernel subimage
Description: Broadcom iProc Linux
Type: Kernel Image
Compression: uncompressed
Data Start: 0x650000d4
Data Size: 66215968 Bytes = 63.1 MiB
Architecture: ARM
OS: Linux
Load Address: 0x61008000
Entry Point: 0x61008000
Hash algo: crc32
Hash value: e4d0f5c7
Verifying Hash Integrity ... crc32+ OK
## Loading fdt from FIT Image at 65000000 ...
Using 'conf@1' configuration
Trying 'fdt@1' fdt subimage
Description: Flattened Device Tree blob - bcm956160.dtb
Type: Flat Device Tree
Compression: uncompressed
Data Start: 0x68f261e8
Data Size: 7565 Bytes = 7.4 KiB
Architecture: ARM
Hash algo: crc32
Hash value: 02d9852b
Verifying Hash Integrity ... crc32+ OK
Booting using the fdt blob at 0x68f261e8
Loading Kernel Image ... OK
Using Device Tree in place at 68f261e8, end 68f2af74
Starting kernel ...
iproc-pcie 18012000.pcie: no PCIe EP device detected
iproc-pcie 18012000.pcie: PCIe controller setup failed
mtd: partition "rootfs" is out of reach -- disabled
Initializing random number generator... done.
Starting network...
Trying to mount ntest on NAND...
UBI device number 1, total 128 LEBs (266338304 bytes, 254.0 MiB), available 0 LEBs (0 bytes), LEB size 2080768 bytes (2.0 MiB)
DMA pool size: 33554432
Broadcom Command Monitor: Copyright (c) 1998-2010 Broadcom Corporation
Release: sdk-6.5.4 built 20161028 (Fri Oct 28 18:00:30 2016)
From keven.lin@Indian:/home/keven.lin/SWITCH/Minions/XLDK_381/buildroot/output/build/broadcom-sdk-6.5.4
Platform: IPROC_CMICD
OS: Unix (Posix)
AXI unit 0: Dev 0xb160, Rev 0x11, Chip BCM56160_B0, Driver BCM56160_A0
SOC unit 0 attached to PCI device BCM56160_B0
ERROR loading rc script on unit 0
Board Id: Minions-C12P
Board Revision: Alpha 1
Totally, 16ports are installed to port info DB.
mgmt socket interface name = eth0
Enable temperature monitor function ... Done
Minions-C12 config set
fox diag init
Init IO expanders ...Done.
Reset PoE, write control register 0x30, 0xf4
Release reset PoE PD69208 , write control register 0x30, 0xf6
Release reset PoE PD69200, write control register 0x30, 0xf7
Init PoE ... DC-Disconnect Mode
PoE Software Version: 16.0172.15
Save system setting
PoE port 1 enabled
PoE port 2 enabled
PoE port 3 enabled
PoE port 4 enabled
PoE port 5 enabled
PoE port 6 enabled
PoE port 7 enabled
PoE port 8 enabled
PoE port 9 enabled
PoE port 10 enabled
PoE port 11 enabled
PoE port 12 enabled
Done
****************************************************
* FoxDiag: FoxDiag_1.1.6-t3
* CPLD: Rev3 release 7
* Platform: Minions-C12P Alpha 1
*
* Copyright: Copyright 2016 Foxconn Corporation.
****************************************************
DIAG>


if I press B when switch poweron ,it will go u-boot :
Booted From: Uboot partition 2
Hit any key to stop autoboot: 0
u-boot> help
? - alias for 'help'
base - print or set address offset
bdinfo - print Board Info structure
boot - boot default, i.e., run 'bootcmd'
bootd - boot default, i.e., run 'bootcmd'
bootelf - Boot from an ELF image in memory
bootm - boot application image from memory
bootp - boot image via network using BOOTP/TFTP protocol
bootvx - Boot vxWorks from an ELF image
bootz - boot Linux zImage image from memory
chpart - change active partition
cmp - memory compare
coninfo - print console devices and information
cp - memory copy
cplddl - cplddl - To perform cpld download
crc32 - checksum calculation
dhcp - boot image via network using DHCP/TFTP protocol
echo - echo args to console
editenv - edit environment variable
env - environment handling commands
erase - erase FLASH memory
exit - exit script
ext2load- load binary file from a Ext2 filesystem
ext2ls - list files in a directory (default /)
false - do nothing, unsuccessfully
fatinfo - print information about filesystem
fatload - load binary file from a dos filesystem
fatls - list files in a directory (default /)
fatsize - determine a file's size
fatwrite- write file into a dos filesystem
fdt - flattened device tree utility commands
flinfo - print FLASH memory information
go - start application at address 'addr'
gpio_read- Read from GPIO
gpio_write- write to GPIO
hash - compute hash message digest
help - print command description/usage
i2c - I2C sub-system
iminfo - print header information for application image
imxtract- extract a part of a multi-image
itest - return true/false on integer compare
loadb - load binary file over serial line (kermit mode)
loads - load S-Record file over serial line
loadx - load binary file over serial line (xmodem mode)
loady - load binary file over serial line (ymodem mode)
loop - infinite loop on address range
md - memory display
mdc - memory display cyclic
mdio - MDIO utility commands
mii - MII utility commands
mm - memory modify (auto-incrementing address)
mtdparts- define flash/nand partitions
mtest - simple RAM read/write test
mw - memory write (fill)
mwc - memory write cyclic
nand - NAND sub-system
nboot - boot from NAND device
nfs - boot image via network using NFS protocol
nm - memory modify (constant address)
ping - send ICMP ECHO_REQUEST to network host
printenv- print environment variables
protect - enable or disable FLASH write protection
reset - Perform RESET of the CPU
run - run commands in an environment variable
saveenv - save environment variables to persistent storage
setenv - set environment variables
sf - SPI flash sub-system
showvar - print local hushshell variables
sleep - delay execution for some time
source - run script from memory
test - minimal test like /bin/sh
tftp -
tftpboot- boot image via network using TFTP protocol
tftpput - TFTP put command, for uploading files to a server
time - run commands and summarize execution time
true - do nothing, successfully
ubi - ubi commands
ubifsload- load file from an UBIFS filesystem
ubifsls - list files in a directory
ubifsmount- mount UBIFS volume
ubifsumount- unmount UBIFS volume
usb - USB sub-system
usbboot - boot from USB device
version - print monitor, compiler and linker version
u-boot>

execute env print got these:

u-boot> env print
baudrate=9600
bootImg=Minions_UBOOT_1.1.1.bin
bootargs=console=ttyS0,9600n8 maxcpus=1 mem=496M mtdparts=brcmnand.0:256m(nrootfs),256m(ntest),-(reserve) quiet
bootcmd=run nand2boot
bootdelay=3
cpldImg=minion_syscpld_rev0307.jbc
dhcp2boot=dhcp ${fileaddr} ${diagImg}; bootm ${fileaddr};
diagImg=Minions_DIAG_1.1.4.img
et0phyaddr=1
ethact=bcm_xgs_gmac-0
ethaddr=60:9C:9F:BC:C1:78
ethprime=bcm_xgs_gmac-0
fdt_high=0xffffffff
fileaddr=65000000
filesize=3f29910
ipaddr=10.141.200.22
loadaddr=0x64000000
mtddevname=nrootfs
mtddevnum=0
mtdids=nand0=brcmnand.0
mtdparts=mtdparts=brcmnand.0:256m(nrootfs),256m(ntest),-(reserve)
nand2boot=nand read ${fileaddr} 0x0 0x5000000; bootm ${fileaddr}
nand_erasesize=200000
nand_oobsize=2e8
nand_writesize=2000
partition=nand0,0
serverip=10.141.200.80
stderr=serial
stdin=serial
stdout=serial
tftp2boot=tftp ${fileaddr} ${diagImg}; bootm ${fileaddr};
ubifsImg=Minions_DIAG_1.1.4_rootfs.ubifs
usb2boot=usb start; fatload usb 0:1 ${fileaddr} ${diagImg}; bootm ${fileaddr};
ver=U-Boot 2016.01-Broadcom XLDK-3.8.1-svn20090 (Oct 04 2016 - 07:22:19 +0800)
Environment size: 1150/65532 bytes
u-boot>
 
Last edited:

Freebsd1976

Active Member
Feb 23, 2018
202
30
28
in DIAG> , ? show ,now what I can do with it, or I need returun it to seller ? @fohdeesha
DIAG>help
? - alias for 'help'
bcm_shell - Invoke Broadcom Backdoor
bcmscript - execute uploaded bcm script.
bypassmux - This command bypass mux mode.
cpldread - To read cpld register
cpldtest - To perform cpld test
cpldupdate - To perform cpld update
cpldwrite - To write special value to cpld register
eeprom - EEPROM dump or program command
fanread - To read fan speed and status
fantask - To enable/disable fan task for thermal policy
fantest - This test validates the fantrays can be controlled by I2C.
fanwrite - To configure DC output for all fan
fiberLbMode - This command config fiber ports loopback mode.
fluffydelay - To adjust read delay of Fluffyread
fluffyeeprom - Fluffy EEPROM dump or program command
fluffyfwupgrade - Upgrade MCU firware
fluffyintrdelay - To adjust LRM transceiver interrupt delay
fluffyioexpinit - To init IOEXP 9536 in Fluffy for debug.
fluffyioexpread - To read IOEXP 9536 in Fluffy for debug.
fluffyioexpwrite - To write value to IOEXP 9536 in Fluffy for debug.
fluffyread - Get MCU info for debug
fluffyreadonly - Get MCU info for debug
fluffystatus - To get LRM transceiver present status
fluffytoggle - To toggle EEPROM between LRM transceiver and Fluffy
fluffywrite - Set MCU info for debug
gpioinit - To init GPIO
gpioread - To read GPIO pin status
gpioreset - To reset used by GPIO pin
gpiotest - To test used by GPIO pin
gpiowrite - To write gpio pin status
help - print online help
i2clist - Display all entries of i2c device info table
i2cread - Get special i2c device id
i2ctest - Test special i2c device id
i2cwrite - Set special i2c register
intrtest - This test interrupt pins from devices to CPU
ioexpinit - To init io expanders
ioexpread - To read io expander pin status
ioexpwrite - To write io expander pin status
leddump - This test is used to dump LED processor contents
ledtest - This test validates all led.
linux_shell - Enter Linux shell for kernel layer debugbing.
memread - To perform memory read
memtest - To perform memory test
memwrite - To perform memory write
mgmt - To perform management port test
minicycle - Packets generated by CPU
nandtest - To perform NAND test
pdtest - Test Power Source config
pktTolerance - This command config packet tolerance mode.
poeafonly - set PoE in at/af mode
poecmd - transmit a SCP packet with 0x00 as the KEY
poedownload - Download firware
poeforcepw - Force power
poeprogram - transmit a SCP packet with 0x01 as the KEY
poepwshow - Show power
poerequest - transmit a SCP packet with 0x02 as the KEY
poestatus - set PoE in enable/disable
poetest - test if the PoE system is working correctly
port - Dump status or counter of all switch ports.
pseenable - Enable or Disable all PSE ports
pseoverstress - Enable or Disable PSE ports with overstress ability
qsberpara - To display QSGMII ber test parameters
qsbertest - This test is used to measure the link quality and robustness of the system.
qsbertxdrv - To set TSC-E ber test tx_driver
qsbertxfir - To set TSC-E ber test tx_fir
rtcread - Get rtc time
rtctest - test the rtc function
rtcwrite - Set rtc time
sflashtest - To perform SFLASH test
sfpIoDetection - This command sfp I/O detection mode.
sfptxdisable - To init io expanders
sfptxenable - To init io expanders
smi - PHY test.
snaketest - This test validates the QSGMII/SFI interfaces between MAC, and PHY.
swlbtest - Packets generated by CPU
swsnaketest - This test validates the MDI/QSGMII interfaces between MAC, and PHY.
syscall - Execute SysCall for kernel layer debugbing.
tempread - To read temperature
thermal_threshold - Set thermal threshold
tscberpara - To display TSC-E ber test parameters
tscbershow - To display TSC-E ber test result
tscbertest - This test is used to measure the link quality and robustness of the system.
tscbertxdrv - To set TSC-E ber test tx_driver
usb - USB test function
version - print diagnostic firmware version
voltagemargin - adjust VCC margin
voltageread - To read voltage source
voltagetest - test the margin
xsmi - PHY test.
 

kroy

New Member
May 17, 2019
24
7
3
Seems my brocade adventure just won't end.

Got a 6610 today. Model number is ICX-6610-24-PE, Version:07.3.02T7f5

Before I did anything else loaded it up. And there is absolutely no signs of this switch supporting PoE. Every other switch mentions initializing PoE on booting.

Code:
ICX6610-24 Switch#show inline power
ICX6610-24 Switch#
and

Code:
ICX6610-24 Switch(config-if-e1000-1/1/1)#inline power
Command not applicable for non-POE port 1/1/1.
or

Code:
ICX6610-24 Switch(config-mif-1/1/1-1/1/24)#inline power
set inline power cannot be done on this device with stack id 1 and poe slot 1!
At this point I'm not sure if I should start up another RMA or try upgrading the switch. It's got the L2 firmware on it right now, plus it does appear to actually have one license on it, ICX6610-PREM-LIC-SW, which I don't want to risk wiping with a firmware upgrade (in case I have to RMA)

@fohdeesha do these things need the L3 software on them to support PoE?

Edit, and again, think I've been bamboozled. This thing only has a single REVA 250W power supply. Don't they need the 1000W supplies to support POE?
 
Last edited:

BobTB

New Member
Jul 19, 2019
16
3
3
I went beserk and bought 5 ICX swithces on ebay in last week, One 6610-24 , two 6430-48p, one 6450-24p and one 6430-24. All have BASE_SOFT_PACKAGE.


What is actually missing in 6430 models with this license? I know they are only 1GB but this still works for me great.

Stacking 6610 with 6450 does not work right?

Today I just received the first one, 6430-24 which was with its config still not reset, all ports except one was disabled, I used the guide in this thread to reset it and all ports work now. I just had to make a serial cable to access the console - this was a pain :) I still think that for 50€ shipped, I could not have gotten a better switch with 24 ports.

I intend to completely replace my dumb swithces and all poe injectors and stuff all over the house and lab,
 

BeTeP

Well-Known Member
Mar 23, 2019
509
315
63
I have also bought a 7250 with a burnt out POE board recently - I sent it back.
Before sending it back for a moment I considered trying to repair it. One of the PD69208's was visibly burnt but I only needed like a dozen of POE ports and I was hoping to get at least a couple of other seemingly intact PD69208 chips (each responsible for 8 ports) working again.

So I downloaded a few datasheets and appnotes for the pd69200 + 6x pd69208 chipset. Maybe someone would find these useful.

PD69208m and PD69200 datasheet
Application Note AN-211 - Designing an IEEE 802.3af/802.3at/802.3btCompliant PD69208 48-Port PoE System
PD69200 serial communication protocol user guide
 

fohdeesha

Kaini Industries
Nov 20, 2016
1,853
1,645
113
29
fohdeesha.com
@Freebsd1976 it has a developer/prerelease bootloader on it, I can fix it over a teamviewer session or you can return it, up to you


@kroy it sounds like indeed they sent you a non-poe model, poe models will identify themselves as poe and will have 1kw PSUs. edit: just saw the model number you posted, that is indeed a non-poe model

@BobTB all 6430 models are L2 only, they don't do l3 and there's no license available for them. if you need licenses for any of the others PM me

you actually can stack a 6610 and a 6450, they call it mixed stacking, instructions are in the stacking pdf included in the firmware download from my guide. I'm not sure why you would need to make a serial cable for the 6430, it (and all the other switches you ordered) use the standard cicsco style rj45 pinout and cables are 5 bucks: https://www.amazon.com/HDE-Serial-Ethernet-Rollover-Console/dp/B00979DMSM
 
  • Like
Reactions: tommybackeast

BobTB

New Member
Jul 19, 2019
16
3
3
@BobTB I'm not sure why you would need to make a serial cable for the 6430, it (and all the other switches you ordered) use the standard cicsco style rj45 pinout and cables are 5 bucks: https://www.amazon.com/HDE-Serial-Ethernet-Rollover-Console/dp/B00979DMSM
Thank you for the info. As far as for the cable, it was friday afternoon, I got the switch deliverd, and could not wait for a week or so to get the cable from amazon. As I never had such a cable I just had to make it to try to get to the switch somehow :)
 

juey

Member
Oct 1, 2018
56
14
8
Germany
Seems my brocade adventure just won't end.

Got a 6610 today. Model number is ICX-6610-24-PE, Version:07.3.02T7f5

Before I did anything else loaded it up. And there is absolutely no signs of this switch supporting PoE. Every other switch mentions initializing PoE on booting.

Code:
ICX6610-24 Switch#show inline power
ICX6610-24 Switch#
and

Code:
ICX6610-24 Switch(config-if-e1000-1/1/1)#inline power
Command not applicable for non-POE port 1/1/1.
or

Code:
ICX6610-24 Switch(config-mif-1/1/1-1/1/24)#inline power
set inline power cannot be done on this device with stack id 1 and poe slot 1!
At this point I'm not sure if I should start up another RMA or try upgrading the switch. It's got the L2 firmware on it right now, plus it does appear to actually have one license on it, ICX6610-PREM-LIC-SW, which I don't want to risk wiping with a firmware upgrade (in case I have to RMA)

@fohdeesha do these things need the L3 software on them to support PoE?

Edit, and again, think I've been bamboozled. This thing only has a single REVA 250W power supply. Don't they need the 1000W supplies to support POE?
The ICX6610-24-PE is the non PoE model.
What you need is the ICX6610-24P-PE model.
 

infoMatt

Active Member
Apr 16, 2019
148
53
28
lol <link removed>
Yea.. smart move...
The funniest part is the last sentence: "or trade for an Unifi switch"... sure, you're swapping a full L3 switch with horsepower to spare with a tplink-on-a-fancy-case-and-premium-pricetag "dumb" managed L2 switch, known for their recurring problem with passive 24V PoE activating even when not requested, that doesn't have 10G ports except for the 48P... but hey, you've got that point'n'clicky Java webinterface... :(

Don't shoot me on that, I've got two Ubiquiti AP currently providing WiFi in my house... but let's be honest, they are more on the consumer level than what they want you to think about... And I'm having "buyer's remorse", thinking that I might have just bought two TPlink and flashed 'em with OpenWRT, I could have saved some money... oh well... :rolleyes:
 

fohdeesha

Kaini Industries
Nov 20, 2016
1,853
1,645
113
29
fohdeesha.com
if it's who I think it is, I gave him licences almost a full year ago, I'm giving him the benefit of the doubt and assuming he just forgot he's supposed to delete them if reselling (please don't leave nasty messages on his post, even if he is being a dick on purpose it won't solve anything)
 
Jan 10, 2019
57
12
8
blog.azureinfra.com
Hey everyone,

I know the 7450-48p with premium license can do BGP... anyone know if these licenses are still available also?
I want to connect my Azure Stack home kits (2) with 40GB to the switch and then use them as the top of rack switches with BGP..
 

fohdeesha

Kaini Industries
Nov 20, 2016
1,853
1,645
113
29
fohdeesha.com
the licenses for the entire icx7xxx are honor based as in they don't actually exist, you just enter an unlock command. I'm not a big fan of the 7450, they usually cost a lot more than the 6610, and no matter how you configure it, it has barely half of the high speed I/O as the 6610 - for instance occupying all 3 expansion slots on the 7450 can get you 2x 40gbe and 4x 10gbe, whereas the 6610 comes out of the box with 2x 40gbe and 16x 10gbE