Just for constancy, all of my hardware runs at 115200, and I hate the slow interface as well. It is not the end of the world.
Also, it is one of those, I want it to work this way why can't it kind of thinking.
Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Also, it is one of those, I want it to work this way why can't it kind of thinking.
Thanks for the informative (and very long...haha) thread with good info.
If we wanted a 48-port POE switch today, which would you pick between the 6610-48P and the 7250-48P?
The 6610 seems to have the advantage of the 4 x 40Gbe QSFP ports (although not really sure how popular 40Gbe is these days) and slightly cheaper second-hand.
However, the 7250 is non-EOL, and still getting software updates.
Are there any reasons (apart from cost) you'd still get the 6610 today?
If we wanted a 48-port POE switch today, which would you pick between the 6610-48P and the 7250-48P?
The 6610 seems to have the advantage of the 4 x 40Gbe QSFP ports (although not really sure how popular 40Gbe is these days) and slightly cheaper second-hand.
However, the 7250 is non-EOL, and still getting software updates.
Are there any reasons (apart from cost) you'd still get the 6610 today?
I'd only get the 6610 for its 40G ports if I don't have any other 40G-capable switch. After acquiring the 6610 I've obtained quite a few other switches that are 40G capable.
Be aware its 40G ports are hard configured: 2 are 40G, 2 are split, and they can't be changed. Their configuration isn't a problem for stacking but can be a hassle if you need a different topology for using them as access/trunks to non-6610 devices.
Be aware its 40G ports are hard configured: 2 are 40G, 2 are split, and they can't be changed. Their configuration isn't a problem for stacking but can be a hassle if you need a different topology for using them as access/trunks to non-6610 devices.
Only if I had a console server or connection manager. That would make my life easier 15 console cables connected to a raspberry pi are a mess. (any recommendations?)
You use Linux, based on your user name? You can associate a host with a comment in your command line history and search for the host name. If you're running Windows PuTTY is a perfectly fine way to use connection strings to a host.
I don't use a Pi to perform KVM functions so don't know how you launch your terminals. If you can parse the host name/device where you want to connect you can have some .rc file that will provide the correct settings. Lots of possibilities.
I don't use a Pi to perform KVM functions so don't know how you launch your terminals. If you can parse the host name/device where you want to connect you can have some .rc file that will provide the correct settings. Lots of possibilities.
I just want to know whether I can use 8x10G ports on Brocade ICX7250 for downstream other than uplinking and stacking. Can I use those 8 x 10G ports to connect my servers to them and have internal routing between servers?
Or can I use 2 x 10G for uplink to a router (e.g. Opnsese/PFsense) and other 6 x 10Gb connects to my 3 servers with LACP (for example: 2 x ports will be LACP’d and connect to a 10Gb optical adapter in one of my servers and so on)
Or can I use 2 x 10G for uplink to a router (e.g. Opnsese/PFsense) and other 6 x 10Gb connects to my 3 servers with LACP (for example: 2 x ports will be LACP’d and connect to a 10Gb optical adapter in one of my servers and so on)
You can. They may be called uplinks, but they're just regular switch ports. No magic to them.
I use LINUX for all my work, putty is available, but not as clean as the windows version. I just open a terminal and ssh into the Pi and run picocom to connect to the serial port and work from there (i.e. picocom -b 115200 /dev/ttyUSB24). The Pi is connected to the 15X USB to serial cables attached to all of my devices.
I create an alias to the /dev/ttyUSB*1-200) for each device.
I use a udev rule for the serial number of the USB serial that creates a symlink to the actual /dev/ttyUSB(1-200) port.
When I mentioned suggesting a device, I was thinking of an actual muxer like what I have used in the data center years ago for serial devices. I would, at that time, just telnet into the muxer and then select the tty that I needed to connect to.
Cyclades consoles can be had for reasonable prices on eBay. For example, a 48 port unit: AVOCENT Cyclades ACS6048 - 520-839-501 48-Port Console Server | eBay
Or a 16 port (I have this one, I'll never have 16 serial devices in my home lab at a single time) Avocent Cyclades ACS6016 16-Port Dual AC Power Console Server 520-659-501 132017885223 | eBay
Or a 16 port (I have this one, I'll never have 16 serial devices in my home lab at a single time) Avocent Cyclades ACS6016 16-Port Dual AC Power Console Server 520-659-501 132017885223 | eBay
I've had an ACS6048 in my home rack for years, I like it. super reliable, it's just uboot+ linux with a shit ton of RS232 ports and a decent webUI/custom CLI basically. although I don't remember paying that much
Hi guys. I opened a new thread regarding some L3 questions and moving parts or all of my inter VLAN routing tfrom pfSense to the switch.
If you have time to help, heres the link. Thanks!
If you have time to help, heres the link. Thanks!
Hey guys, I'm brand new to enterprise switches and was able to snag an ICX 7250-24 off ebay recently. Running through the setup today with fohdeesha's guide and this is what I'm running into. I think this was the second time running through the steps because I thought I missed something. Maybe I'm missing something obvious, but I'm getting this error when attempting to boot from primary:
Below is the full log including commands sent. IPs have been changed back to fohdeesha's defaults for security--I don't know if this is necessary, but I'm not having any issues connecting to tftp so I don't think the IPs are the problem.
Any thoughts? Thanks in advance.
Code:
ICX7250-24 Copper (Non-POE), PVT1
SYS CPLD VER: 0x6, Released Ver: 0
Enter 'b' to stop at boot monitor: 0
bootdelay: ===
Booting image from Primary
NAND read: device 0 offset 0x0, size 0x2000000
................................ 33554432 bytes read: OK
Wrong Image Format for bootm command
ERROR: can't get kernel image!
could not boot from primary, no valid image; trying to boot from secondary
Code:
Brocade Bootloader: 10.1.06T215 (Dec 14 2015 - 13:10:57)
Validate Shmoo parameters stored in flash ..... OK
Restoring Shmoo parameters from flash .....
Running simple memory test ..... OK
ICX7250-24 Copper (Non-POE), PVT1
SYS CPLD VER: 0x6, Released Ver: 0
Enter 'b' to stop at boot monitor: 0
ICX7250-Boot> setenv ipaddr 192.168.1.50
ICX7250-Boot> setenv netmask 255.255.255.0
ICX7250-Boot> setenv serverip 192.168.1.8
ICX7250-Boot> setenv image_name ICX7xxx/SPR08090mc.bin
ICX7250-Boot> setenv uboot ICX7xxx/spz10118.bin
ICX7250-Boot> update_uboot
Loading image to Uboot Partition 2
Using bcmiproc_eth-0 device
TFTP from server 192.168.1.8; our IP address is 192.168.1.50
Filename 'ICX7xxx/spz10118.bin'.
Load address: 0x61007dc0
Loading: ######################################################
done
Bytes transferred = 786944 (c0200 hex)
sf erase 0x0 0x100000
copying uboot image to flash, it will take sometime...
sf write 0x61007fc0 0x0 0xc0000
TFTP to Flash Done.
ICX7250-Boot> saveenv
ICX7250-Boot> reset
resetting ...
Ruckus Networks Bootloader: 10.1.18T215 (Jul 13 2020 - 01:53:05)
Validate Shmoo parameters stored in flash ..... OK
Restoring Shmoo parameters from flash .....
Running simple memory test ..... OK
ICX7250-24 Copper (Non-POE), PVT1
SYS CPLD VER: 0x6, Released Ver: 0
Enter 'b' to stop at boot monitor: 0
ICX7250-Boot> factory set-default
Execution of "factory set-default" will remove all user data like config, keys etc.
Do you want to continue? (Y/N) Y
Command executed successfully
ICX7250-Boot> update_primary
Updating PRIMARY image over TFTP
Using bcmiproc_eth-0 device
TFTP from server 192.168.1.8; our IP address is 192.168.1.50
Filename 'ICX7xxx/SPR08090mc.bin'.
Load address: 0x1000000
Loading: #################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
###########
done
Bytes transferred = 33554432 (2000000 hex)
img_sz=0x2000000 block_count = 0x20
NAND erase.spread: device 0 offset 0x0, size 0x2000000
Erasing at 0x1f00000 -- 100% complete.
OK
NAND write: device 0 offset 0x0, size 0x2000000
................................ 33554432 bytes written: OK
TFTP to Flash Done.
ICX7250-Boot> setenv ipaddr
ICX7250-Boot> saveenv
ICX7250-Boot> boot_primary
resetting ...
Brocade Bootloader: 10.1.06T215 (Dec 14 2015 - 13:10:57)
Validate Shmoo parameters stored in flash ..... OK
Restoring Shmoo parameters from flash .....
Running simple memory test ..... OK
ICX7250-24 Copper (Non-POE), PVT1
SYS CPLD VER: 0x6, Released Ver: 0
Enter 'b' to stop at boot monitor: 0
bootdelay: ===
Booting image from Primary
NAND read: device 0 offset 0x0, size 0x2000000
................................ 33554432 bytes read: OK
Wrong Image Format for bootm command
ERROR: can't get kernel image!
could not boot from primary, no valid image; trying to boot from secondary
BOOTING image from Secondary
NAND read: device 0 offset 0x4000000, size 0x2000000
................................................................ 33554432 bytes read: OK
## Booting kernel from Legacy Image at 70000200 ...
Image Name: Image
ARM Linux Kernel Image (uncompressed)
Data Size: 31678084 Bytes = Load Address: 61008000
Entry Point: 61008000
Verifying Checksum ... OK
Loading Kernel Image ... OK
OK
Starting kernel ...
Config partition mounted
FIPS reset is enable.
Removing startup config & backup
Removing stacking.boot
Removing SSL_CERT_FILE
Removing SSH_RSA_HOSTKEY_FILE
Removing SSH_DSA_HOSTKEY_FILE
Creating TUN device
Starting the FastIron.
FIPS Disabled:PORT NOT DISABLED
platform type 81
OS>Applying factory defaults..
INFO: startup config data is not available, try to read from backup
INFO: startup config data in the backup area is not available
set_board_level: gi_board_type = 108SOC unit 0 attached to PCI device BCM56344_A0
Initializing the parallel detect
Pre Parsing Config Data ...
INFO: empty config data in the primary area, try to read from backup
INFO: empty config data in the backup area also
Parsing Config Data ...
INFO: empty config data in the primary area, try to read from backup
INFO: empty config data in the backup area also
Hotplugger Daemon Initialized ..
Successfuly set IngressClassPort on CPU port to 2
System initialization completed...console going online.
Copyright (c) 1996-2016 Brocade Communications Systems, Inc. All rights reserved.
UNIT 1: compiled on May 19 2016 at 01:44:58 labeled as SPR08030h
(31678660 bytes) from Secondary SPR08030h.bin
SW: Version 08.0.30hT213
Compressed Boot-Monitor Image size = 786944, Version:10.1.06T215 (spz10118)
HW: Stackable ICX7250-24
==========================================================================
UNIT 1: SL 1: ICX7250-24 24-port Management Module
Serial #:DUH3817N019
License: ICX7250_L3_SOFT_PACKAGE (LID: fwjINGMpFGO)
License Compliance: ICX7250-PREM-LIC-SW is Compliant
P-ASIC 0: type B344, rev 01 Chip BCM56344_A0
==========================================================================
UNIT 1: SL 2: ICX7250-SFP-Plus 8-port 80G Module
==========================================================================
1000 MHz ARM processor ARMv7 88 MHz bus
8192 KB boot flash memory
2048 MB code flash memory
2048 MB DRAM
STACKID 1 system uptime is 16 second(s)
The system : started=cold start
My stack unit ID = 1, bootup role = alone
ICX7250-24 Router>
ICX7250-24 Router>
PS 1, Internal Power supply detected.
PS 1, Internal Power supply is up.
Stack unit 1 Power supply 1 is up
TFTP session timed out
TFTP session timed out
TFTP session timed out
TFTP session timed out
TFTP session timed out
TFTP session timed out
TFTP session timed out
TFTP session timed out
TFTP session timed outAny thoughts? Thanks in advance.
Have you tried pinging the gateway IP of the other vlan from the device. ex. Device on vlan16 ping switch ip on vlan 32. If that works then you should have vlan ip routing working. Another would be to check your netmask settings on each device.@kapone I'll try anything at this point!
I've got a VLAN 16 and a VLAN 32 now. Same setup -- x.x.[32/16].10 IP's for two different devices, one on each subnet. Neither can ping the other at this moment.
Anything else I can check? Is there a way I could have inadvertently disabled inter-VLAN routing?
You could also run a "show ip route" and check your routing table to see if there are any strange routes.
Another thing is in the vlan settings, do you have a "router-interface" line.
example
vlan 16 by port
router-interface ve 16
where interface "ve 16" is your interface configured with the gateway IP of your vlan.
So, I recently switched from my Brocade switch to a Cisco one. I was having problems mostly with multicast, mDNS/Bonjour stuff specifically. I would have Google Homes or Chromecasts not showing up in lists to cast to, light switches which wouldn't turn on and off, Airprint printers which would show up sometimes and then not others, and other weird behavior I otherwise couldn't explain. It was not all the time, and sometimes it worked fine.
Initially, I thought this was a result of how Brocade had implemented PIM, as I had phones and PCs in a separate L2 segment. For science, I merged all of my devices into the same vlan and the problem persisted. I tried turning PIM off and just used IGMP. I tried using Ubiquiti, Aruba and finally Cisco APs as I thought maybe one brand or another's WiFi would solve the problem with their various multicast features. The same weird behavior still existed no matter the vendor. Aruba and Cisco worked better and was more stable than Ubiquiti, but that is probably a result of tunneling/CAPWAPing the traffic to a controller, so the MAC addresses of all of the various devices were literally on the same port on the switch, However, if I ran them in Flexconnect or Bridging mode, the multicast behavior was just as unreliable as the Ubiquiti deployment. I even thought my Android phone's IP stack was just hokey (I was having other problems, too. Bluetooth and Wifi calling never worked reliably, as an example), so I switched to an iPhone. Problems persisted.
Finally, I kicked the Brocade 7450 switch (which had replaced an HP E3800 switch because it had better 10GbE density) for a Cisco 3850 switch. I turned PIM on the Cisco and enabled the mdns-sd gateway. The problem is more-or-less solved now. Things populate properly for casting all of the time, the AirPrint printers always show up, I can be in a L3 hop away and things still work, and I can turn my wifi lights on and off reliably. The only problem I have is that Apple Home occasionally does not like when I roam between APs and thinks things are disconnected on my iPhone, but then my iPad is working fine and It is connected to the same AP. I may just work towards CAPWAPing my traffic again and calling this a success.
In addition, the local network is performing measurably better. Take a look at this graph, as an example. This is a SmokePing chart monitoring my core switch. Can you see where I transitioned from the Brocade to the Cisco?
And just so you don't think something is wrong with my SmokePing instance, PRTG tells a similar story, and is running on different metal:
I've kept my ICX 7150 in production as an access switch (L2 only) in my Garage and it seems to do that job fairly well. I have an AP running on it and it seems to be behaving fine for that purpose. It's latency graphs looks better than the 7450 I had in production with L3 services (OSPF, PIM, etc) running.
But the latency is so much better on my Cisco switch that I even took my VDX out of production. This is an enterprise access/aggregation switch performing 40-50% better than a Datacenter switch in terms of latency, if Ping times are to be believed. The VDX always did this weird ping spike thing every day or so that I could never explain...pinging THROUGH it didn't correlate to an increase in latency so I never really gave it much mind but it was an annoying anomaly nonetheless.
As much as I like the value of what Brocade/Ruckus has to offer, clearly it can be worth paying more money to get better stability and performance...
Figured I would share, maybe my experience will help someone make a purchasing decision later.
If anyone wants a Brocade ICX 7450, 6450 or a VDX 6740 (SFP+, not copper, lower power consumption version) I have one of each sitting in a pile right now I would be willing to get rid of for a reasonable price (shipping costs are the issue really).
Initially, I thought this was a result of how Brocade had implemented PIM, as I had phones and PCs in a separate L2 segment. For science, I merged all of my devices into the same vlan and the problem persisted. I tried turning PIM off and just used IGMP. I tried using Ubiquiti, Aruba and finally Cisco APs as I thought maybe one brand or another's WiFi would solve the problem with their various multicast features. The same weird behavior still existed no matter the vendor. Aruba and Cisco worked better and was more stable than Ubiquiti, but that is probably a result of tunneling/CAPWAPing the traffic to a controller, so the MAC addresses of all of the various devices were literally on the same port on the switch, However, if I ran them in Flexconnect or Bridging mode, the multicast behavior was just as unreliable as the Ubiquiti deployment. I even thought my Android phone's IP stack was just hokey (I was having other problems, too. Bluetooth and Wifi calling never worked reliably, as an example), so I switched to an iPhone. Problems persisted.
Finally, I kicked the Brocade 7450 switch (which had replaced an HP E3800 switch because it had better 10GbE density) for a Cisco 3850 switch. I turned PIM on the Cisco and enabled the mdns-sd gateway. The problem is more-or-less solved now. Things populate properly for casting all of the time, the AirPrint printers always show up, I can be in a L3 hop away and things still work, and I can turn my wifi lights on and off reliably. The only problem I have is that Apple Home occasionally does not like when I roam between APs and thinks things are disconnected on my iPhone, but then my iPad is working fine and It is connected to the same AP. I may just work towards CAPWAPing my traffic again and calling this a success.
In addition, the local network is performing measurably better. Take a look at this graph, as an example. This is a SmokePing chart monitoring my core switch. Can you see where I transitioned from the Brocade to the Cisco?
And just so you don't think something is wrong with my SmokePing instance, PRTG tells a similar story, and is running on different metal:
I've kept my ICX 7150 in production as an access switch (L2 only) in my Garage and it seems to do that job fairly well. I have an AP running on it and it seems to be behaving fine for that purpose. It's latency graphs looks better than the 7450 I had in production with L3 services (OSPF, PIM, etc) running.
But the latency is so much better on my Cisco switch that I even took my VDX out of production. This is an enterprise access/aggregation switch performing 40-50% better than a Datacenter switch in terms of latency, if Ping times are to be believed. The VDX always did this weird ping spike thing every day or so that I could never explain...pinging THROUGH it didn't correlate to an increase in latency so I never really gave it much mind but it was an annoying anomaly nonetheless.
As much as I like the value of what Brocade/Ruckus has to offer, clearly it can be worth paying more money to get better stability and performance...
Figured I would share, maybe my experience will help someone make a purchasing decision later.
If anyone wants a Brocade ICX 7450, 6450 or a VDX 6740 (SFP+, not copper, lower power consumption version) I have one of each sitting in a pile right now I would be willing to get rid of for a reasonable price (shipping costs are the issue really).
Attachments
Last edited: