Quite a bit earlier in this thread another user had a 7750-48f that did the same; never heard if it was fixed or dead. You can try reflashing uboot and the primary following the guide on page 1. I had a 7150 with bad flash that would boot loop, but I forget the error output in that case. I wound up using the nand scrub commands in uboot then reflashing it and that sorted it out. You may have a dead ASIC and the OS times out waiting for a response from it.Hello,
I bought 3x ICX 7750 switches couple of months ago and 2 of them are working but 1 does not... It shows some errors and do boot loop. I thought update of firmware could do the trick and did it, even exchanged the flash between the switches but without success
Any help would be appreciated
Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
0:soc_trident2_chip_reset: LCPLL 2 not locked on unit 0 status = 0x00000000Hello,
I bought 3x ICX 7750 switches couple of months ago and 2 of them are working but 1 does not... It shows some errors and do boot loop. I thought update of firmware could do the trick and did it, even exchanged the flash between the switches but without success
Here are what console shows during boot :
Any help would be appreciatedCode:Bootloader Version: 10.1.15T205 (Jan 30 2019 - 23:08:02) SVR: 0x82100120, PVR: 0x80230032 ICX7750:48F,Production-1 SYS CPLD VER: 0xe, Released Ver: 0 Enter 'b' to stop at boot monitor: 0 bootdelay: === Booting image from Primary ## Booting kernel from Legacy Image at 86000000 ... Created: 2018-07-10 11:03:23 UTC Data Size: 473584 Bytes = 462.5 KiB Load Address: 00000000 Entry Point: 00000000 Verifying Checksum ... OK ## Verifying Application Image ... Verifying Checksum ... OK Loading Kernel Image ... OK OK HV# No vsmgr node in config tree HV# cpu0/0: failover_register() - registering notifiers... Mounting file system... ^[[0ndefault eusb_infra enabled ICX7750 eUSB-Infra Called Number of eUSB partitions 7 ... PLATFORM SW Config partition mounted. eUSB flash drive is ready No External USB Drive(s) Mounted . Processing packages from primary partition Installed packages are up to date Starting up system statistics daemon... eth0 Link encap:Ethernet HWaddr CC:4E:24:D0:41:68 inet addr:10.11.12.38 Bcast:10.255.255.255 Mask:255.0.0.0 inet6 addr: fe80::ce4e:24ff:fed0:4168/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:516 (516.0 B) Memory:fe4e0000-fe4e0fff lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) eth0 Link encap:Ethernet HWaddr CC:4E:24:D0:41:68 inet addr:10.11.12.38 Bcast:10.255.255.255 Mask:255.0.0.0 inet6 addr: fe80::ce4e:24ff:fed0:4168/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:516 (516.0 B) Memory:fe4e0000-fe4e0fff lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.255.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Log Level 7: Daemonized systatsd Systatsd Logging Env is not set, initializing log level to: D Setting connection to systemlogger Starting systatsd done Starting up poe daemon... poed Logging Env is not set, initializing log level to: I Setting connection to systemlogger Starting poed done Starting up health monitor daemon... eth0 Link encap:Ethernet HWaddr CC:4E:24:D0:41:68 inet addr:10.11.12.38 Bcast:10.255.255.255 Mask:255.0.0.0 inet6 addr: fe80::ce4e:24ff:fed0:4168/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:516 (516.0 B) Memory:fe4e0000-fe4e0fff lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.255.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) eth0 Link encap:Ethernet HWaddr CC:4E:24:D0:41:68 inet addr:10.11.12.38 Bcast:10.255.255.255 Mask:255.0.0.0 inet6 addr: fe80::ce4e:24ff:fed0:4168/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:516 (516.0 B) Memory:fe4e0000-fe4e0fff lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.255.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Hmond Logging Env is not set, initializing log level to: I Setting connection to systemlogger Starting hmond done nohup: appending output to nohup.out Resources folder check done! dma_mem_base: 0xf8000000, dma_mem_len: 0x8000000, warm_mem_base: 0xf0200000, warm_mem_len: 0xd00000 Enabling time stamp. *** Crash dump information is available, please collect it using "supportsave" CLI *** Starting the FastIron. USER=root boot_partition=pri_partition HOME=/ OLDPWD=/dev GCOV_PREFIX=/fast_iron TERM=vt102 PATH=/sbin:/usr/sbin:/bin:/usr/bin SHELL=/bin/sh PWD=/ No Shared Library Linked FIPS Disabled:PORT NOT DISABLED platform type 61 OS> Loaded Image SWR08095h from Pri Built on (UTC): Fri Aug 26 08:30:36 2022 Enabling Console Logging T=0.0: cannot read stacking.boot. The system will be a standalone. Upgrade from pre 8.0.80 image. Dual mode to VLAN config upgrade needed Hotplugger Daemon Initialized .. py_filemgr: py_filemgr_task Message Registration Success ITC error: message type 9f0005 is being re-registered set_board_level: gi_board_type = 94[ 29.170402] linux-kernel-bde: map phys range 0xf0000000-0xf0040000 to 0x50fb1000-0x50ff1000 [ 29.172803] linux-kernel-bde: map phys range 0xf8000000-0x0 to 0x50ff1000-0x58ff1000 DMA pool size: 134217728 SOC unit 0 attached to PCI device BCM56852_A2 0:soc_trident2_chip_reset: LCPLL 2 not locked on unit 0 status = 0x00000000 0:soc_do_init: soc_do_init: total cpu and arm cosq 0030 unexpected 0:soc_schan_timeout_check: soc_schan_op: operation attempt timed out 0:_soc_cmicm_schan_op: SchanTimeOut:soc_schan_op operation timed out cmc=0 0:soc_reg_get: soc_reg_get failed for PGW_OBM0_ECC_ENABLE fdry_system_init: Misc init failed: Operation timed out ERROR: SOC unit 0 system_init failed ********** BCM SDK init FAILED ************* sh: can't create /proc/softlockup/cancel: nonexistent directory Failed to cancel WDT stack: 1074e464 102e2b3c 102d457c 1041c17c 10034df8 13381994 133cc98c Tuning CFS scheduler parameters... Compressing Console file Copying fitrace errorlog file to flash CORE_PATTERN:PID=1153 UID=0 GID=0 sig=11 Sun Dec 11 00:36:05 UTC 2022: Dumping core file to /tmp.gz, this will take couple of minutes ... Segmentation fault (core dumped) Sun Dec 11 00:36:31 UTC 2022: Core file collected as /tmp.gz, processing the core file ... Sun Dec 11 00:36:31 UTC 2022: Removing the oldest core file:core_1153_FastIron-6_2022-08-29_14-24-46.gz of size 4844 kbytes seq 7 Sun Dec 11 00:36:31 UTC 2022: Calculating checksum: core_1153_FastIron-8_2022-12-11_00-36-31.gz Sun Dec 11 00:36:31 UTC 2022: Checking integrity of core_1153_FastIron-8_2022-12-11_00-36-31.gz Sun Dec 11 00:36:36 UTC 2022: Moving core_1153_FastIron-8_2022-12-11_00-36-31.gz of size 4832 KB to flash, with available space of 655856 KB... [ 68.387304] WDG_INFO: Coredump Task is running, Pid: 1200, Name: coredump_utilit, Probe Time: 1670719003 Sun Dec 11 00:36:55 UTC 2022: Core saved as /fast_iron/cores/core_1153_FastIron-8_2022-12-11_00-36-31.gz. Moving system log files to persistent memory Copying SZ log files to persistent memory.. Done. Core collection and system log collection completed, rebooting the switch ... Stopping File Manager daemon... Stopped File Manager daemon Stopping health monitor daemon... Stopped health monitor daemon Stopping poe daemon... Stopped poe daemon Stopping system statistics daemon... Stopped systatsd. The system is going down NOW! Sent SIGTERM to all processes Sent SIGKILL to all processes Requesting system reboot [ 85.736190] Restarting system. Bootloader Version: 10.1.15T205 (Jan 30 2019 - 23:08:02) SVR: 0x82100120, PVR: 0x80230032
0:soc_do_init: soc_do_init: total cpu and arm cosq 0030 unexpected
smells like dead ASIC
I had same thoughts
Somebody suggested the problem might be related to that switches were stacked before I got them and this one still expecting another switch to tell him what to do but I doubt it.I tried to went into base OS to eventually see something extra and succeeded but didn't knew what to do there LOL
At least now I have some spare PSU and fan's
BTW I realized VDX6740T which I bought month ago have the same PSU as ICX7750 instead of VDX6740 SFP variant.
Hello,
Long-time lurker, now I need some expert assistance. I have TWO ICX-6610's configured and licensed thanks to Fodeesha's docs. They are up and running with a 40G fiber link between them. Now is the time to merge this 'homelab' network into my 'home network', in two phases.
Phase I - Ubiquiti Dream Machine UDM - ICX-6610 Integration
My home network consists of a UDM (as the gateway, router, firewall and network controller) and two Unifi remote access points. My ISP terminates at the UDM. I have 4 subnets (192.168.1.X, 192.168.10.x, 192.168.20.x, 192.168.30.x) defined and ONE wireless SSID that operates via the two wireless AP's. Most of my devices connect wirelessly. These subnets are to reduce broadcast traffic, all other traffic is open between subnets. Future plans include a DMZ and GUEST subnets with isolation...
Now I'd like to physically connect ONE of the 6610's to my UDM. I need to have all the subnets successfully transit from one 6610 to the other via the Stack Trunk. I also need all the devices physically connected to both 6610's (they are in all 4 of the subnets) to gain access to the Internet via the UDM. I'd love to move the subnet-to-subnet routine to the 6610, if that's easy. My confusion rests in migrating my subnets, to VLANs and setting up the appropriate TRUNKS and TAGGING to accomplish my goals. My fun begins once the network is operational, so I'd like to skip the network configuration experimentation.
This seems straightforward, but I'm not a network guy. So I'm looking for a network guy. I'm very comfortable with the Brocade web interface and I can cut/paste CLI instructions as necessary. Be gentle.
Phase 2 - New ISP Transition
After my home network is playing nice at Brocade speed. I'd like to keep my internal network managed by the UDM (DHCP and Network controller) but change my ISP to a fiber terminated in a modem that connects to a port on one of the 6610's. DHCP staying on UDM. 6610's connecting everything together.
Again, something I'm sure experts do all the time. New to me, and just complicated enough that I need expert advice to get it right the first time.
Thanks in advance for the help!
Long-time lurker, now I need some expert assistance. I have TWO ICX-6610's configured and licensed thanks to Fodeesha's docs. They are up and running with a 40G fiber link between them. Now is the time to merge this 'homelab' network into my 'home network', in two phases.
Phase I - Ubiquiti Dream Machine UDM - ICX-6610 Integration
My home network consists of a UDM (as the gateway, router, firewall and network controller) and two Unifi remote access points. My ISP terminates at the UDM. I have 4 subnets (192.168.1.X, 192.168.10.x, 192.168.20.x, 192.168.30.x) defined and ONE wireless SSID that operates via the two wireless AP's. Most of my devices connect wirelessly. These subnets are to reduce broadcast traffic, all other traffic is open between subnets. Future plans include a DMZ and GUEST subnets with isolation...
Now I'd like to physically connect ONE of the 6610's to my UDM. I need to have all the subnets successfully transit from one 6610 to the other via the Stack Trunk. I also need all the devices physically connected to both 6610's (they are in all 4 of the subnets) to gain access to the Internet via the UDM. I'd love to move the subnet-to-subnet routine to the 6610, if that's easy. My confusion rests in migrating my subnets, to VLANs and setting up the appropriate TRUNKS and TAGGING to accomplish my goals. My fun begins once the network is operational, so I'd like to skip the network configuration experimentation.
This seems straightforward, but I'm not a network guy. So I'm looking for a network guy. I'm very comfortable with the Brocade web interface and I can cut/paste CLI instructions as necessary. Be gentle.
Phase 2 - New ISP Transition
After my home network is playing nice at Brocade speed. I'd like to keep my internal network managed by the UDM (DHCP and Network controller) but change my ISP to a fiber terminated in a modem that connects to a port on one of the 6610's. DHCP staying on UDM. 6610's connecting everything together.
Again, something I'm sure experts do all the time. New to me, and just complicated enough that I need expert advice to get it right the first time.
Thanks in advance for the help!
No problem, the stacked ICX6610 will behave as one big virtual switch with centralised management. No extra settings necessary.
Yes, that's relatively simple.
First, you need to create 4 VLANs. Then you create 4 VEs. Those VEs are your gateways for inter-vlan and inter-subnet routing. Those VEs need to be assigned as gateways for the respective subnet in your DHCP server.
that'a a bit more challenging, but possible.
Great!
I reviewed Patrick Henry's video on VLAN's and another on VLAN Routing with Router-Interface. These explainer series videos are awesome!
Here's my first VLAN, VE and router-interface:
vlan 4
untag e 2/1/12 (port to attach a switch that is all subnet 4)
router-interface ve 4
exit
int ve 4
ip address 192.168.4.1/24
On the UDM this IP address is the gateway for subnet 4. If this looks correct I can repeat for each of the other 4 subnets, although they won't have the untag port assigned, this is the only subnet with it's own switch.
Last edited:
Update: I have one tagged port where the UDM will connect for the other three VLANS.
Is the VLAN-VLAN traffic allowed by default in the Brocade or is that the purpose of the IP ADDRESS command to move that to the gateway?
Since the UDM is also the DHCP server, is there anything on the Brocade I need to do so that all new devices get their IP from the UDM?
Is the VLAN-VLAN traffic allowed by default in the Brocade or is that the purpose of the IP ADDRESS command to move that to the gateway?
Since the UDM is also the DHCP server, is there anything on the Brocade I need to do so that all new devices get their IP from the UDM?
VLANs are generally isolated from each other.
VLAN-VLAN traffic works by routing via / through the VEs
VLANs are layer 2 networks, imagine each VLAN as a separate switch (or physical network, if your background goes back far enough to remember those). Each has its own IP subnets.
Moving traffic between layer 2 networks is done at layer 3 (routing), which means each VLAN needs one or more gateway addresses that land on routers that have interfaces in all the necessary VLANs. The ICX can certainly be such a router, and each VE you create is a layer 3 interface into a VLAN.
You create two VEs (Virtual Interface) and assign them to the VLANs (VE2 -> VLAN2, VE4 -> VLAN4)
Choose IPs for those VEs and set them as Gateways for the devices in the VLANs
for example
VLAN 2: router-interface VE2
VE2 IP: 10.2.0.3
Subnet: 10.2.0.0/16
Gateway / Default route for devices in VLAN2: 10.2.0.3
VLAN 4: router-interface VE4
VE4 IP: 10.4.0.3
Subnet: 10.4.0.0/16
Gateway / Default route for devices in VLAN 4:
10.4.0.3
Last edited:
And just to be clear: if you don't implement any Access Control Lists, all traffic will be permitted between layer 3 networks in the switch. As @NablaSquaredG has posted, putting the VEs in place, and setting the client systems to use the ICX as their default gateway, will be sufficient for them to talk across the VLANs.
One correction: those subnet masks should be /16 (or even /24), not /8. With /8 masks those two subnets will be considered to be in a single network.
One correction: those subnet masks should be /16 (or even /24), not /8. With /8 masks those two subnets will be considered to be in a single network.
OK,
@kpfleming thanks for the default. I was feeling like I needed an ALLOW rule or the equivalent somewhere...
So now I have my subnets/VLANs communicating.
Is this the right way to designate the route out to the WAN? (with failover)
ip route 10.4.0.0/16 ethernet 1/1/6 1 (with my WAN connected on port 6)
ip route 10.4.0.0/16 ehternet 1/1/12 3 (alternate WAN connected on port 12)
I realize this is an example, and /16 is a huge network. Mine are /24 networks, but I want to get the concept correct.
@kpfleming thanks for the default. I was feeling like I needed an ALLOW rule or the equivalent somewhere...
So now I have my subnets/VLANs communicating.
Is this the right way to designate the route out to the WAN? (with failover)
ip route 10.4.0.0/16 ethernet 1/1/6 1 (with my WAN connected on port 6)
ip route 10.4.0.0/16 ehternet 1/1/12 3 (alternate WAN connected on port 12)
I realize this is an example, and /16 is a huge network. Mine are /24 networks, but I want to get the concept correct.
No, not quite.
If your WAN router has an IP address in a VLAN that is on port 6, and the ICX has a VE in that VLAN with an address from the same subnet, then you'd use something like "ip route 0.0.0.0/0 addr.of.WAN.gateway". The route table entries are based on destination addresses, not source addresses.
If your WAN router has an IP address in a VLAN that is on port 6, and the ICX has a VE in that VLAN with an address from the same subnet, then you'd use something like "ip route 0.0.0.0/0 addr.of.WAN.gateway". The route table entries are based on destination addresses, not source addresses.
Wasted 2 days with an XL710 before I found out it's busted with this switch. Same symptoms as the other people in this thread.
Untagged VLANs work.
With tagged VLANs, DHCP works, nothing else enters the NIC. Everything else is a CRC error.
I'm going to dig in with Intel's debug tools and see if I can find anything.
E: Generated some debug logs and I'll see if Intel can help. Otherwise going to try and frankenstein a QSFP+ sniffer out of an old FPGA accelerator and see what's actually happening on the wire.
Untagged VLANs work.
With tagged VLANs, DHCP works, nothing else enters the NIC. Everything else is a CRC error.
I'm going to dig in with Intel's debug tools and see if I can find anything.
E: Generated some debug logs and I'll see if Intel can help. Otherwise going to try and frankenstein a QSFP+ sniffer out of an old FPGA accelerator and see what's actually happening on the wire.
Last edited:
Hi everyone,
thanks for putting this thread with a TON of information together. I must admit that I have not read the complete thread, so please forgive me if some of my questions have already been answered:
1. I am interested in the Brocade ICX6450-24P:
thanks for putting this thread with a TON of information together. I must admit that I have not read the complete thread, so please forgive me if some of my questions have already been answered:
1. I am interested in the Brocade ICX6450-24P:
- My understanding is that the latest supported firmware is 08030u. Is that correct?
- Is the noise level with the stock fans OK if I am in another room?
- I would use it in an in-wall media enclosure (see picture here). Do I need to create additional airflow?
- It looks like it goes for $150-$250 on eBay. Anything special I need to pay attention to? (Of course making sure I buy the P-model that I need)
- I currently run the Brocade ICX7150-C12P which came with the house. I assume the web UI and CLI are identical?
- I am still on the fence between this switch and a similar switch from Unifi/Ubiquiti and Mikrotik. I assume the L3 capabilities are much better with the Brocade, correct?
- There is a big price swing on eBay between $250 and $700. What is an OK price at the moment for PoE or non-PoE?
- The post says "non-PoE models easily converted to 12v DC power". What would be the benefit of that? Reduced power consumption?
- My understanding is the this one supports the firmware up to 08095k, which is the same as my current device.
- I am still on the fence between this switch and a similar switch from Unifi/Ubiquiti and Mikrotik. I assume the L3 capabilities are much better with the Brocade, correct?
- Looks like this one consumes double the power of the one above. Would you pick this one or the 6450?
ICX 6k series max version 8030; ICX 7k series supported through latest I believe. Web UI will have some differences; v9 is modernized.
As far as L3 the differences will only matter if you’re planning on doing inter-VLAN routing on the switch.
I had similar issues going between an XL710 in 4x10 mode and a 7250 using a generic breakout DAC that was listed as "compatible" by the eBay seller. I switched it out for an Intel-coded 10Gtek QSFP+ 40G SR4 optic, MTP to 4xLC breakout fiber, and Brocade SFP+ optics on the switch side. So far the CRC errors have disappeared. I also have an Intel coded breakout DAC coming soon from FS.com that I'm going to try as well.
I chose these (a pair) since they are a more modern hardware platform, although still EoL. The 7250 also has twice as many SFP+ slots, should that be of interest to you now or in the future.
@thimplicity
both ICX6450-24P and 7250-24P are nice. Just keep in mind that 6450-24P is like half the price of a 7250-24P (24 port SKUs are usually more expensive on the second hand market, because they need less power).
If you don't care about money, take the 7250-24P. Otherwise you're most likely going to be happy with the 6450-24P, except if you're planning to do weird network stunts... But also, if you were planning weird networks stunts, you probably wouldn't be asking whether you should choose the 7250 or 6450
both ICX6450-24P and 7250-24P are nice. Just keep in mind that 6450-24P is like half the price of a 7250-24P (24 port SKUs are usually more expensive on the second hand market, because they need less power).
If you don't care about money, take the 7250-24P. Otherwise you're most likely going to be happy with the 6450-24P, except if you're planning to do weird network stunts... But also, if you were planning weird networks stunts, you probably wouldn't be asking whether you should choose the 7250 or 6450