Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)
- Thread starter fohdeesha
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Yes, the OPNSense box can ping 192.168.10.1 (and even telnet in and manage the switch at that IP when I temporarily enabled the telnet server). As I'd expect, when I disconnected the PC, I got a routing loop (ping says TTL exceeded, and traceroute confirms) when I tried to traceroute from the OPNSense box to 192.168.10.1 (192.168.253.10 -> 192.168.253.1 -> back out the switch default route to 192.168.253.10 -> etc.). Reconnecting the PC made 192.168.10.1/24 reappear in the switch's routing table and made the routing loop go away (ping worked normally again).
A few more notes:
- `show lldp neighbors` shows my PC connected (with the correct MAC on the correct port)
- `show ip dhcp-server statistics` shows an equal number of received DHCP-DISCOVER and sent DHCP-OFFER packets. Both numbers increment every time I disconnect and reconnect my PC. No other DHCP packet types have more than 0 sent/received.
- I get the same behavior when I connect my PC to a port with no untagged vlan associated (thus using the default vlan 1). When I set a static IP on my PC of 192.168.1.10/24 (default gateway 192.168.1.1), all I see in wireshark is my PC broadcasting out ARP packets looking for 192.168.1.1 (please tell 192.168.1.10).
do you have a standalone NIC (could even by 10/100Mb) that you can test in this PC or another PC? Sounds like some sort of MAC LUT arp issue - which could be hardware too.
You probably tried this but if you didn't, add another port to VLAN 10 move your PC there and make sure the issue follows. If it were me I'd try a block away from port 1, like 9 or better yet 17 just in case your issue is on a block of ports from a single chip in the switch.
Megafacepalm o'clock: good call, it works perfectly on another machine. That inspired me to consider the hardware more carefully, leading me to download the latest copy of the RealTek 2.5G NIC drivers (from RealTek, not my motherboard manufacturer).
And guess what? It works now! Thanks so much for sticking around for this troubleshooting journey and making sure that I covered all of my bases.
Another great thought, I did try that. Now that I updated the drivers, I'm having the same (successful) experience on any of the ports that I try.
Phew. At least I learned something there. Thanks again!
p.s. And just to confirm that everything is extra super working now, I set up my Engenius AP to use tagged VLANs for different SSIDs, and I confirmed that a wifi client connecting to each SSID gets a DHCP IP from the correct pool from the switch. It's probably time to save a backup of all of these configs before I start adding access lists and IPv6 and any other fun things that will break a currently working setup...
just something to keep in mind:
many folks in this thread (incl. @fohdeesha) discourage using the switch's DHCP server cause its borked in some ways. If you run into issues around DHCP - you may want to consider the possibility the DHCP server is not happy before the client... You're mostly super simple right now but that may not be the case as your journey continues. My advice, take the leap now since IP address management is foundational as your explorations get more advanced.
for me, I run a pair of Centos 7 vm's with ISC-Bind/ISC-DHCP and the stack forwards the requests.
Thanks! I'd seen some posts about this from ~2018 and wasn't sure if it was still the case. At the moment the only hardware that I have for this is a Fitlet2 that's running OpnSense, and I'd like to avoid running a hypervisor on my internet-facing firewall "appliance" (mostly for simplicity and stability, I have no doubt it could be done sufficiently securely). I'll probably pick up a small server sometime soon and run DHCP there, but the switch should be okay for now.
Yes, I believe the chow mux errors occur when the units boot up with the breakout ports connected, it attempts the basic 4x10gb stack mux thinking a 6610 is on the other side, and of course can't because it's actually servers. As for the switch not liking the links going up and down - that certainly shouldn't be the case. I know I have some 6610's here I've used with breakout cables that didn't need to have stuff plugged in during boot for them to work. However I have seen that fix some people's link issues where they have a stubborn switch or breakout cable. Not sure what the root cause is
for what it's worth all the DHCP server issues we had were in the 8030 firmware (icx6xxx series), I've heard (and seen in a lot of release notes) that the DHCP server functionality has been greatly improved in 8080, 8090, and 8095, and I believe you said you have a 7250 so it can run these. worth a try. To potentially save yourself a lot of troubleshooting time, just remember in the back of your head if some random IOT devices/rokus/chromecasts suddenly don't get a DHCP lease, it's not your wifi, it's the switch's DHCP serv
Thanks! Yes, I'm running my 7250 on 8095d (from your new guide - thank you for that!). Do you happen to remember if the DHCP symptom was that the switch knew about the lease but the client never configured its IP, or did the switch not have a lease for the client at all?
nope, no support for made up PoE implementations. spec only (802.3af, 802.3at, etc)
honestly don't remember, if you search the thread I might have given those details when I was originally reporting it
thats what I thought. thanks. These damn Mikrotik and Ubuiqiti devices using passive POE... oh well
Don’t know about Mikrotik, but Ubiquity does sell 802.3af adapters to handle their wonky legacy devices (like my just-barely-missed-the-PoE-cutoff AC-Lite):
Accessories
store.ui.com
I'm running 08.0.92bT211 - is that good also?
EDIT: I'm running L2 only - no L3 on the 2 stacked switches.
Thanks. Im aware of those...the problem is the 0.5a max current. I run my mikrotik Omni via poe on a pole on my roof and the omni then powers an ubiquiti lbe ac gen2, lbe-lr and a mikrotik sxt. Unfortunately if I used one of those adapters,it wouldn't have enough power to run the whole setup.Don’t know about Mikrotik, but Ubiquity does sell 802.3af adapters to handle their wonky legacy devices (like my just-barely-missed-the-PoE-cutoff AC-Lite):
Accessories
ICX7250-24 Fan/Temp info!
Hi folks, I've spent some time testing out different 40x40mm fans in my ICX7250-24, with the hopes of finding something quieter than stock but that still keeps the ASIC cool.
The contenders
The testbench
A Brocade ICX7250-24 running FastIron 09.0.00 with a minimal config and no ports connected (except management). A Zabbix VM polls the switch every 15 seconds for chassis temp information (OID 1.3.6.1.4.1.1991.1.1.1.1.18.0, multiplied by 0.5). I let the switch idle until the temperature stabilized, then took the raw data and wrangled it into Excel. Between changing out fans, I let the ASIC cool down to approximately 50-55ºC. For these tests, the switch was laying on a desk, and the room temperature was approximately 21.5ºC. No additional "ASIC heatsink fan" was used.
The results
Clearly the stock Foxconn fan does a fantastic job of keeping the chassis cool (peak 64ºC), albeit at the expense of being noisy. The two Sanyo Denki fans provided adequate cooling; the B3013 (peak 79ºC) was slightly quieter than the G3013 (peak 76ºC), but was hotter by about 3ºC. The Delta FFB (peak 86.5ºC) was quieter yet, but was about 7ºC hotter than the B3013. Finally, the Delta EFB was the quietest of the bunch, but comes in last place at a peak temp of 94.5ºC.
Analysis
If you're looking for a good, middle-of-the-ground replacement fan, the Sanyo Denki fans will suit your needs. There is a slight noise, but they're significantly quieter than the stock fan. I would recommend the B3013 as it is a bit quieter than the G3013.
If noise is your primary concern, the Delta FFB will be slightly audible over room noise or home HVAC, but the switch will run hotter. Under heavy workloads, the FFB may alternate between Speed 1 and 2.
Avoid the Delta EFB, as it does not keep the switch cool enough (in my opinion) to justify the slight reduction in noise over the FFB. It will also very likely alternate between Speed 1 and 2.
Please note that I did not have any ethernet or SFP+ modules connected, so the switch truly was idle during these tests. Your results may vary depending on configuration and interface utilization. This post is informational only - modify your hardware at your own risk!
To connect the fans to the system board, I purchased a 3-pin fan extender cable, cut and stripped it, and connected the leads to each fan as appropriate (fan red to board red, fan black to board yellow, fan blue/yellow to board black). This allows me to fully reverse the fan modification easily by simply unplugging and swapping out the modded fan for the original.
Closing
I hope all of this info helps - please let me know if you have any questions. I will try my best to answer! In a day or two, I will probably select a fan (likely the B3013 or FFB) and rack the switch, so I won't easily be able to perform any further tests or comparisons.
Cheers!
Hi folks, I've spent some time testing out different 40x40mm fans in my ICX7250-24, with the hopes of finding something quieter than stock but that still keeps the ASIC cool.
The contenders
Mfr/model | Size | Air flow | Static Pressure | Noise | Fan speed | Price |
|---|---|---|---|---|---|---|
| Delta FFB0412VHN-F00 | 40mm x 28mm D | 15.8 cfm | 0.509 in H2O | 45 dBA | 9500 rpm | $14.31 |
| Sanyo Denki 109P0412G3013 | 40mm x 28mm D | 14.8 cfm | 0.719 in H2O | 42 dBA | 11500 rpm | $10.51 |
| Sanyo Denki 109P0412B3013 | 40mm x 28mm D | 13.4 cfm | 0.574 in H2O | 40 dBA | 10300 rpm | $10.24 |
| Delta EFB0412VHD-F00 | 40mm x 20mm D | 10.1 cfm | 0.416 in H2O | 32.5 dBA | 9000 rpm | $12.57 |
| Foxconn PIA040H12P [STOCK] | 40mm x 28mm D | 23 cfm (?) |
The testbench
A Brocade ICX7250-24 running FastIron 09.0.00 with a minimal config and no ports connected (except management). A Zabbix VM polls the switch every 15 seconds for chassis temp information (OID 1.3.6.1.4.1.1991.1.1.1.1.18.0, multiplied by 0.5). I let the switch idle until the temperature stabilized, then took the raw data and wrangled it into Excel. Between changing out fans, I let the ASIC cool down to approximately 50-55ºC. For these tests, the switch was laying on a desk, and the room temperature was approximately 21.5ºC. No additional "ASIC heatsink fan" was used.
The results
Clearly the stock Foxconn fan does a fantastic job of keeping the chassis cool (peak 64ºC), albeit at the expense of being noisy. The two Sanyo Denki fans provided adequate cooling; the B3013 (peak 79ºC) was slightly quieter than the G3013 (peak 76ºC), but was hotter by about 3ºC. The Delta FFB (peak 86.5ºC) was quieter yet, but was about 7ºC hotter than the B3013. Finally, the Delta EFB was the quietest of the bunch, but comes in last place at a peak temp of 94.5ºC.
Analysis
If you're looking for a good, middle-of-the-ground replacement fan, the Sanyo Denki fans will suit your needs. There is a slight noise, but they're significantly quieter than the stock fan. I would recommend the B3013 as it is a bit quieter than the G3013.
If noise is your primary concern, the Delta FFB will be slightly audible over room noise or home HVAC, but the switch will run hotter. Under heavy workloads, the FFB may alternate between Speed 1 and 2.
Avoid the Delta EFB, as it does not keep the switch cool enough (in my opinion) to justify the slight reduction in noise over the FFB. It will also very likely alternate between Speed 1 and 2.
Please note that I did not have any ethernet or SFP+ modules connected, so the switch truly was idle during these tests. Your results may vary depending on configuration and interface utilization. This post is informational only - modify your hardware at your own risk!
To connect the fans to the system board, I purchased a 3-pin fan extender cable, cut and stripped it, and connected the leads to each fan as appropriate (fan red to board red, fan black to board yellow, fan blue/yellow to board black). This allows me to fully reverse the fan modification easily by simply unplugging and swapping out the modded fan for the original.
Closing
I hope all of this info helps - please let me know if you have any questions. I will try my best to answer! In a day or two, I will probably select a fan (likely the B3013 or FFB) and rack the switch, so I won't easily be able to perform any further tests or comparisons.
Cheers!
So found out the wires
- Blue is PWM
- White or yellow is TACH
- Black is ground
- Red is 12v
Brand: Nidec
Manufacturer: Nidec Corporation
Fan model: W40S12BS4A5-57
Size: 40 * 40 * 28mm
Voltage: 12V
Working voltage: 4.5V-13.8V
Rated current: 0.73A
Air volume: 24CFM
Noise: 52.1db
Speed measurement: support (4-wire) support PWM automatic speed regulation function
Bearing: double ball bearing
Last edited:
Has anyone tried one of those 4-pin "fan simulators" from ebay, even if you had to jumper the pins to match this? They are cheap enough that I might try one and run 3x Noctuas off the remaining 2 headers