In the 7xxx series stacking can only be done with units that are in the same 'family'... 71xx, 72xx, 74xx, etc. There is no support for stacking 7xxx and 6xxx units.
Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
FYI: The reason is that ICX6XXX uses Marvell, while ICX7XXX uses Broadcom.
so I am looking at the Brocade ICX6610-24 with POE+. I want to replace my Dell 5528p (POE). I already have a VDX6740T. Is it going to be a "pain" to get all the ports licensed like the VDX6740T or getting firmware updates?
I am also primarily looking using the 4x 40Gb ports in the back. need to uplink to the VDX6740T and a Celestica DX010. Can the 4x40Gb be used at the same time? or only 2 of the 4 at a time?
I am also primarily looking using the 4x 40Gb ports in the back. need to uplink to the VDX6740T and a Celestica DX010. Can the 4x40Gb be used at the same time? or only 2 of the 4 at a time?
Last edited:
Read the guide linked in first post, should answer any questions you have regarding licensing and firmware.
2 of the 40Gbps ports are actually 4x10 and that can't be changed. So really the switch has 16x 10G ports and 2x 40G ports.
I have 2 ICX6450-48Ps that after updating the firmware following the fohdeeshadocs instructions to the letter, keep spewing these errors on the terminal:
I also tried flashing the older t variant of the firmware (instead of the u) that I had lying around, same issue.
I have never had this problem before when flashing a mix of 5 other 6450-48Ps and 6450-24Ps.
I saw mentions of this issue in the past in this thread but no solution.
The weird thing is that PoE works fine.
Any solutions or ideas? I also tried reflashing the PoE firmware again but it failed.
Code:
PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
Resetting module in slot 1 again to recover from dev faultI have never had this problem before when flashing a mix of 5 other 6450-48Ps and 6450-24Ps.
I saw mentions of this issue in the past in this thread but no solution.
The weird thing is that PoE works fine.
Any solutions or ideas? I also tried reflashing the PoE firmware again but it failed.
I have an ICX-6450-24P that I have been using without issue for a few years. Recently after a power outage I discover that not only was my UPS no longer working, but the switch didn't power itself back up when the power came back on.
After unplugging it, waiting about 5 seconds and then plugging it back in, it will power back up. It has done this several times always following a power loss.
Is this some sort of expected behavior or could there be a power supply issue?
After unplugging it, waiting about 5 seconds and then plugging it back in, it will power back up. It has done this several times always following a power loss.
Is this some sort of expected behavior or could there be a power supply issue?
What model of UPS do you have? Is it a pure sine wave model or simulated sine wave? I've had a few weird issues before with some enterprise-y gear on simulated sine models that went away after I upgraded them to pure sine wave models (or just ran them off AC power).
It was on a Cyberpower pure sine wave, but it happened again this morning while directly plugged into the wall.
good morning, gentlemen.
I am still reading this thread ( rather insightful and overpacked with knowledge ) and the following picture diagrams my fantasies on how I should build a home lab using brocade, some transceivers and cabling.
at the top left corner you will find four tables defining the cables, transceivers and NICs I should purchase for the lab. I tried to color code the entire diagram, checked it twice for correctness, but certain mistakes still are possible.
I would like to ask for your experience in identifying certain types of cables and transceivers -- this is still not quite clear to me. This is the first time I am going to build anything with fibre.
Any help is greatly appreciated!
many thanks in advance!
best regards,
Dmitry
UPD: not sure if I was right but I went and ordered these guys:
1. Original Brocade Mini-GBIC-Modul 10Gbit SR SFP+ Transceiver -57-0000075-01
2. Avago GBIC 40Gb QSFP+ Transceiver 850nm 21CFR(J)Class1 AFBR-79EQPZ
3. Brocade 40GE SR4 QSFP+ 300m (57-1000296-01), 4x 10GBASE-SR/SW, 40G-QSFP-SR4-INT
4. 0.2M OM3 LC to LC Fibre Optic Patch Cable. Length Options: 0.2 m - 100 m, 10 GB Duplex LC-LC 50/125 um Multimode Fibre Optic Cable LSZH 0.2 m
5. MPO to MPO Fibre Optic Cable 3M EMSECKO 12 Core Female Type B OM3 Multimode Fibre Optic Patch Cable LSZH MPO-MPO Patch Cord for QSFP+ Transceiver MTP Compatible Cabling System, Aqua
...in hope that they are somehow compatible to each other and icx6610 + icx6450
I am still reading this thread ( rather insightful and overpacked with knowledge ) and the following picture diagrams my fantasies on how I should build a home lab using brocade, some transceivers and cabling.
at the top left corner you will find four tables defining the cables, transceivers and NICs I should purchase for the lab. I tried to color code the entire diagram, checked it twice for correctness, but certain mistakes still are possible.
I would like to ask for your experience in identifying certain types of cables and transceivers -- this is still not quite clear to me. This is the first time I am going to build anything with fibre.
Any help is greatly appreciated!
many thanks in advance!
best regards,
Dmitry
UPD: not sure if I was right but I went and ordered these guys:
1. Original Brocade Mini-GBIC-Modul 10Gbit SR SFP+ Transceiver -57-0000075-01
2. Avago GBIC 40Gb QSFP+ Transceiver 850nm 21CFR(J)Class1 AFBR-79EQPZ
3. Brocade 40GE SR4 QSFP+ 300m (57-1000296-01), 4x 10GBASE-SR/SW, 40G-QSFP-SR4-INT
4. 0.2M OM3 LC to LC Fibre Optic Patch Cable. Length Options: 0.2 m - 100 m, 10 GB Duplex LC-LC 50/125 um Multimode Fibre Optic Cable LSZH 0.2 m
5. MPO to MPO Fibre Optic Cable 3M EMSECKO 12 Core Female Type B OM3 Multimode Fibre Optic Patch Cable LSZH MPO-MPO Patch Cord for QSFP+ Transceiver MTP Compatible Cabling System, Aqua
...in hope that they are somehow compatible to each other and icx6610 + icx6450
Last edited:
Running my ICX 6610 48 for a year straight. Two of my ports stopped responding, port state is BLOCKING, but if I connect the cable, the lights blink. tried enable/disable port, but even with the cable connected the port is still showing BLOCKING
Member of L2 VLAN ID 1000, port is untagged, port state is BLOCKING BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled Link Error Dampening is Disabled STP configured to ON, priority is level0, mac-learning is enabled
show span gives me Spanning-tree is not configured on port-vlan 1000
What can I do, I for now can not just reset the switch, a lot of things are running on it in my "production"
Member of L2 VLAN ID 1000, port is untagged, port state is BLOCKING BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled Link Error Dampening is Disabled STP configured to ON, priority is level0, mac-learning is enabled
show span gives me Spanning-tree is not configured on port-vlan 1000
What can I do, I for now can not just reset the switch, a lot of things are running on it in my "production"
The ICX6610 Doesn't support 2.5gb on the sfp+ ports, right? I stuck some copper transceivers in there and got like 500mbit down and 1700mbit up with a 2.5g NIC
I would expect it to be all or nothing but it actually kind of worked? My computer negotiated a 2.5g connection speed and just worked.
I would expect it to be all or nothing but it actually kind of worked? My computer negotiated a 2.5g connection speed and just worked.
The SFP+ transceiver negotiates 10G with the switch and 2.5G on the other side (it kinda translates between the speeds).
The downside is that for almost all commercially available transceivers of this type, performance is very bad.
I've got one from FS and one from H!Fiber that are customized for Cisco and Ubiquity switches respectively. That's definitely not the problem?
It would likely be cheaper to get a 10GB capable 2.5g switch than to get the 3 transceivers I need. I'll probably end up going that route.
If you want symmetric 2.5Gbps performance from such a setup, try getting Aquantia based GBase-T ones. These will give you proper performance. If you get Marvell based ones, you will see asymmetric throughput. Or get a Netgear 2.5/5 Gb switch - it's cheaper and less error prone.
edit: I probably should have read that whole thread before I posted this. The dell transcievers apparently won't go below 5GB
Incidentally, there's someone on ebay right now with a bunch of dell branded WP2PP transceivers for $47 each
This thread seems to indicate they're compatible
Thanks for the info! I'll report back when they get here
Last edited:
I'm in the beginning stages of learning how to operate my icx 6450, its super cool and there's so many features to uncover. Thank you fohdeesha for the comprehensive guide on getting it set up!
One pain so far that I've not been able to overcome: I've been searching this thread and google for a while on how I could either port forward all tcp/udp port 53 communications to my pihole dns service (while excluding the pihole ip to prevent some sort of a redirect loop), or somehow redirect 8.8.8.8 and other public dns ips to my pihole's ip. I haven't been successful in finding the right documentation or the right search terms to learn more about how I might go about that. Could someone point me in the right direction?
A few of my devices, android phone included, have hardcoded DNS ips that bypass my DNS so I can't resolve my local records. The pihole server is also the DHCP server so I assume the DNS ips are set correctly that way and that is the case for most of my devices. I've tried all the hacky suggestions on the device side, but haven't had any luck. The other recommendation seems to be to do it at the router level, which is where I'm currently stuck.
One pain so far that I've not been able to overcome: I've been searching this thread and google for a while on how I could either port forward all tcp/udp port 53 communications to my pihole dns service (while excluding the pihole ip to prevent some sort of a redirect loop), or somehow redirect 8.8.8.8 and other public dns ips to my pihole's ip. I haven't been successful in finding the right documentation or the right search terms to learn more about how I might go about that. Could someone point me in the right direction?
A few of my devices, android phone included, have hardcoded DNS ips that bypass my DNS so I can't resolve my local records. The pihole server is also the DHCP server so I assume the DNS ips are set correctly that way and that is the case for most of my devices. I've tried all the hacky suggestions on the device side, but haven't had any luck. The other recommendation seems to be to do it at the router level, which is where I'm currently stuck.
Last edited:
Anyone know what the deal is with these particular bad blocks in the flash on 7150 / 7250 switches?
There are a few reports of bad blocks, both on 7150-series switches, and they all show these specific blocks:
I recently purchased a ICX 7250-48p the seller indicated had "bad flash." Upon receiving the switch and powering it up, neither primary or secondary would boot. I noticed while trying to boot the secondary image bad sectors were skipped when reading the image from NAND.
Is this a known issue? It seems odd that multiple people have issues with these specific sectors.
I'm prepared dump the flash and replace the flash chip but I don't want to waste time if it isn't needed (even though it would certainly be fun).
Code:
0x05a00000
0x05b00000- https://forums.servethehome.com/ind...erful-10gbe-40gbe-switching.21107/post-286849
- https://forums.servethehome.com/ind...erful-10gbe-40gbe-switching.21107/post-243532
I recently purchased a ICX 7250-48p the seller indicated had "bad flash." Upon receiving the switch and powering it up, neither primary or secondary would boot. I noticed while trying to boot the secondary image bad sectors were skipped when reading the image from NAND.
nand bad showed both 0x05a00000 & 0x05b00000. The switch is seemingly function after re-flashing everything but, of course, the bad sectors remain.Is this a known issue? It seems odd that multiple people have issues with these specific sectors.
I'm prepared dump the flash and replace the flash chip but I don't want to waste time if it isn't needed (even though it would certainly be fun).
Hello,
I bought 3x ICX 7750 switches couple of months ago and 2 of them are working but 1 does not... It shows some errors and do boot loop. I thought update of firmware could do the trick and did it, even exchanged the flash between the switches but without success
Here are what console shows during boot :
Any help would be appreciated
I bought 3x ICX 7750 switches couple of months ago and 2 of them are working but 1 does not... It shows some errors and do boot loop. I thought update of firmware could do the trick and did it, even exchanged the flash between the switches but without success
Here are what console shows during boot :
Code:
Bootloader Version: 10.1.15T205 (Jan 30 2019 - 23:08:02)
SVR: 0x82100120, PVR: 0x80230032
ICX7750:48F,Production-1
SYS CPLD VER: 0xe, Released Ver: 0
Enter 'b' to stop at boot monitor: 0
bootdelay: ===
Booting image from Primary
## Booting kernel from Legacy Image at 86000000 ...
Created: 2018-07-10 11:03:23 UTC
Data Size: 473584 Bytes = 462.5 KiB
Load Address: 00000000
Entry Point: 00000000
Verifying Checksum ... OK
## Verifying Application Image ...
Verifying Checksum ... OK
Loading Kernel Image ... OK
OK
HV#
No vsmgr node in config tree
HV# cpu0/0: failover_register() - registering notifiers...
Mounting file system...
^[[0ndefault eusb_infra enabled
ICX7750 eUSB-Infra Called
Number of eUSB partitions 7 ...
PLATFORM SW
Config partition mounted.
eUSB flash drive is ready No External USB Drive(s) Mounted . Processing packages from primary partition
Installed packages are up to date
Starting up system statistics daemon...
eth0 Link encap:Ethernet HWaddr CC:4E:24:D0:41:68
inet addr:10.11.12.38 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::ce4e:24ff:fed0:4168/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:516 (516.0 B)
Memory:fe4e0000-fe4e0fff
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth0 Link encap:Ethernet HWaddr CC:4E:24:D0:41:68
inet addr:10.11.12.38 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::ce4e:24ff:fed0:4168/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:516 (516.0 B)
Memory:fe4e0000-fe4e0fff
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.255.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Log Level 7: Daemonized systatsd
Systatsd Logging Env is not set, initializing log level to: D
Setting connection to systemlogger
Starting systatsd done
Starting up poe daemon...
poed Logging Env is not set, initializing log level to: I
Setting connection to systemlogger
Starting poed done
Starting up health monitor daemon...
eth0 Link encap:Ethernet HWaddr CC:4E:24:D0:41:68
inet addr:10.11.12.38 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::ce4e:24ff:fed0:4168/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:516 (516.0 B)
Memory:fe4e0000-fe4e0fff
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.255.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth0 Link encap:Ethernet HWaddr CC:4E:24:D0:41:68
inet addr:10.11.12.38 Bcast:10.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::ce4e:24ff:fed0:4168/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:516 (516.0 B)
Memory:fe4e0000-fe4e0fff
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.255.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Hmond Logging Env is not set, initializing log level to: I
Setting connection to systemlogger
Starting hmond done
nohup: appending output to nohup.out
Resources folder check done!
dma_mem_base: 0xf8000000, dma_mem_len: 0x8000000, warm_mem_base: 0xf0200000, warm_mem_len: 0xd00000
Enabling time stamp.
*** Crash dump information is available, please collect it using "supportsave" CLI ***
Starting the FastIron.
USER=root
boot_partition=pri_partition
HOME=/
OLDPWD=/dev
GCOV_PREFIX=/fast_iron
TERM=vt102
PATH=/sbin:/usr/sbin:/bin:/usr/bin
SHELL=/bin/sh
PWD=/
No Shared Library Linked
FIPS Disabled:PORT NOT DISABLED
platform type 61
OS>
Loaded Image SWR08095h from Pri Built on (UTC): Fri Aug 26 08:30:36 2022
Enabling Console Logging
T=0.0: cannot read stacking.boot. The system will be a standalone.
Upgrade from pre 8.0.80 image. Dual mode to VLAN config upgrade needed
Hotplugger Daemon Initialized ..
py_filemgr: py_filemgr_task Message Registration Success
ITC error: message type 9f0005 is being re-registered
set_board_level: gi_board_type = 94[ 29.170402] linux-kernel-bde: map phys range 0xf0000000-0xf0040000 to 0x50fb1000-0x50ff1000
[ 29.172803] linux-kernel-bde: map phys range 0xf8000000-0x0 to 0x50ff1000-0x58ff1000
DMA pool size: 134217728
SOC unit 0 attached to PCI device BCM56852_A2
0:soc_trident2_chip_reset: LCPLL 2 not locked on unit 0 status = 0x00000000
0:soc_do_init: soc_do_init: total cpu and arm cosq 0030 unexpected
0:soc_schan_timeout_check: soc_schan_op: operation attempt timed out
0:_soc_cmicm_schan_op: SchanTimeOut:soc_schan_op operation timed out cmc=0
0:soc_reg_get: soc_reg_get failed for PGW_OBM0_ECC_ENABLE
fdry_system_init: Misc init failed: Operation timed out
ERROR: SOC unit 0 system_init failed
********** BCM SDK init FAILED *************
sh: can't create /proc/softlockup/cancel: nonexistent directory
Failed to cancel WDT
stack: 1074e464 102e2b3c 102d457c 1041c17c 10034df8 13381994 133cc98c
Tuning CFS scheduler parameters...
Compressing Console file
Copying fitrace errorlog file to flash
CORE_PATTERN:PID=1153 UID=0 GID=0 sig=11
Sun Dec 11 00:36:05 UTC 2022: Dumping core file to /tmp.gz, this will take couple of minutes ...
Segmentation fault (core dumped)
Sun Dec 11 00:36:31 UTC 2022: Core file collected as /tmp.gz, processing the core file ...
Sun Dec 11 00:36:31 UTC 2022: Removing the oldest core file:core_1153_FastIron-6_2022-08-29_14-24-46.gz of size 4844 kbytes
seq 7
Sun Dec 11 00:36:31 UTC 2022: Calculating checksum: core_1153_FastIron-8_2022-12-11_00-36-31.gz
Sun Dec 11 00:36:31 UTC 2022: Checking integrity of core_1153_FastIron-8_2022-12-11_00-36-31.gz
Sun Dec 11 00:36:36 UTC 2022: Moving core_1153_FastIron-8_2022-12-11_00-36-31.gz of size 4832 KB to flash, with available space of 655856 KB...
[ 68.387304] WDG_INFO: Coredump Task is running, Pid: 1200, Name: coredump_utilit, Probe Time: 1670719003
Sun Dec 11 00:36:55 UTC 2022: Core saved as /fast_iron/cores/core_1153_FastIron-8_2022-12-11_00-36-31.gz.
Moving system log files to persistent memory
Copying SZ log files to persistent memory..
Done.
Core collection and system log collection completed, rebooting the switch ...
Stopping File Manager daemon...
Stopped File Manager daemon
Stopping health monitor daemon...
Stopped health monitor daemon
Stopping poe daemon...
Stopped poe daemon
Stopping system statistics daemon...
Stopped systatsd.
The system is going down NOW!
Sent SIGTERM to all processes
Sent SIGKILL to all processes
Requesting system reboot
[ 85.736190] Restarting system.
Bootloader Version: 10.1.15T205 (Jan 30 2019 - 23:08:02)
SVR: 0x82100120, PVR: 0x80230032