Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[CIR-Engineering]

I just noted that FastIron 10.0.10f is out and it is supporting the ICX 7150 by this version.
But starting from FastIron 10.0.20 it will not support ICX 7150 any longer.

Commscope Technical Content Portal

docs.commscope.com

No, I think this is a wrong conclusion. FastIron 10.0.20 already exists (released in July and October).
FastIron 10.0.20 seems to be a separate release train from 10.0.10

Ruckus' firmware situation is a bit of a clusterf... at the moment

What version do you guys like now? I have a ICX7450-32ZP that I want to update, but I tried 09.0.x a while ago and was unfamiliar with some of the syntax required to accomplish what I needed to do. Lacking time I dropped back to 08.0.95m. I was about to update to SPR08095q, but am wondering if I should switch to 10.0.x now? I still have zero time to learn new code.

 

[jmdomini]

Picked up two ICX7250's on EBay. They look well worn but are functional. The one issue that concerns me is the temperature. While this shows as "normal" in the web interface, it's seems mighty toasty. The fans occasionally rev up, but mostly run at low speed despite the high temps.

Does this look normal for a 7250? If not, is there a recommended replacement fan that will do the job without being too noisy.

Sorry if this has been asked and answered already. It's hard to sort through a 512 page thread.

Fan controlled temperature:
Rule 1/2 (MGMT THERMAL PLANE): 92.9 deg-C
Rule 2/2 (AIR OUTLET NEAR PSU): 49.0 deg-C

Fan speed switching temperature thresholds:
Rule 1/2 (MGMT THERMAL PLANE):
Speed 1: NM<-----> 93 deg-C
Speed 2: 82<----->105 deg-C (shutdown)
Rule 2/2 (AIR OUTLET NEAR PSU):
Speed 1: NM<-----> 58 deg-C
Speed 2: 49<----->105 deg-C (shutdown)

Fan 1 Air Flow Direction: Front to Back
Fan 2 Air Flow Direction: Front to Back
Slot 1 Current Temperature: 92.9 deg-C (Sensor 1), 49.0 deg-C (Sensor 2)
Warning level.......: 100.0 deg-C
Shutdown level......: 105.0 deg-C
Boot Prom MAC : cc4e.24b4.bc70
Management MAC: cc4e.24b4.bc70

 

[frameshift18]

I'd personally think that is too high and feels like something is off, unless your ambient is like 40C lol. Airflow blocked? Maybe some heatsinks were dislodged in shipping? Although with that high of an outlet temp I'd suspect an airflow problem at first glance.


For reference one of my 7250-48P with ~32W PoE load, ~20C ambient, two stock fans, third disconnected:

Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 ok, speed (auto): [[1]]<->2
Fan 3 failed

Fan controlled temperature:
        Rule 1/2 (MGMT THERMAL PLANE): 75.0 deg-C
        Rule 2/2 (AIR OUTLET NEAR PSU): 26.5 deg-C

Fan speed switching temperature thresholds:
        Rule 1/2 (MGMT THERMAL PLANE):
                Speed 1: NM<-----> 95       deg-C
                Speed 2:        85<----->105 deg-C (shutdown)
        Rule 2/2 (AIR OUTLET NEAR PSU):
                Speed 1: NM<-----> 41       deg-C
                Speed 2:        34<----->105 deg-C (shutdown)

Fan 1 Air Flow Direction:  Front to Back
Fan 2 Air Flow Direction:  Front to Back
Fan 3 Air Flow Direction:  Front to Back
Slot 1 Current Temperature: 75.0 deg-C (Sensor 1), 26.5 deg-C (Sensor 2)

 

[jmdomini]

I'd personally think that is too high and feels like something is off, unless your ambient is like 40C lol. Airflow blocked? Maybe some heatsinks were dislodged in shipping? Although with that high of an outlet temp I'd suspect an airflow problem at first glance.

Thanks for confirming. I ordered some replacement fans, hopefully that will do the trick.

 

[kapone]

Thanks for confirming. I ordered some replacement fans, hopefully that will do the trick.

This is not just a fan issue...

Slot 1 Current Temperature: 92.9 deg-C (Sensor 1), 49.0 deg-C (Sensor 2)

That is seriously hot and quite unusual. You may wanna open up the chassis to inspect the heatsinks to see if something popped loose or something. And unless they're stuck on with thermal glue (which is quite possible), a repaste with fresh thermal compound usually helps.

 

[jmdomini]

This is not just a fan issue...

Slot 1 Current Temperature: 92.9 deg-C (Sensor 1), 49.0 deg-C (Sensor 2)

That is seriously hot and quite unusual. You may wanna open up the chassis to inspect the heatsinks to see if something popped loose or something. And unless they're stuck on with thermal glue (which is quite possible), a repaste with fresh thermal compound usually helps.

That's not good to hear! Thing is, I took them apart to clean out the dust and I don't recall seeing any heatsinks. I hope I just wasn't paying close enough attention and the seller didn't rip them off.

Not looking forward to un-racking these guys and putting back my old switches for a second time. Unfortunately didn't notice the temp until after I racked these guys.

 

[kapone]

That's not good to hear! Thing is, I took them apart to clean out the dust and I don't recall seeing any heatsinks. I hope I just wasn't paying close enough attention and the seller didn't rip them off.

Not looking forward to un-racking these guys and putting back my old switches for a second time. Unfortunately didn't notice the temp until after I racked these guys.

There should be plenty of heatsinks...

 

[CTurtle]

Cables are not the issue, first thing I did was connect the AP to the POE injector using a whole new set of cables, and when that worked out I reuse the same known good cables to try in the switch again, which failed.

Show log show nothing, really. I have both ports 13 and 14 with POE powered devices, and these don't even show up in the log after a cold boot:

SSH@puft-prince#show inline power

Power Capacity:         Total is 748000 mWatts. Current Free is 748000 mWatts.

Power Allocations:      Requests Honored 0 times


Port   Admin   Oper    ---Power(mWatts)---  PD Type  PD Class  Pri  Fault/
        State   State   Consumed  Allocated                          Error
--------------------------------------------------------------------------
1/1/13 On      Off            0          0  n/a      n/a         3  n/a
1/1/14 On      Off            0          0  n/a      n/a         3  n/a

Jan 21 14:02:46:I:System: Interface ethernet 1/1/47, state up
Jan 21 14:02:45:I:System: Interface ethernet 1/1/23, state upI 
Jan 21 14:02:45:I:System: Stack unit 1 POE  Power supply 1  with 748000 mwatts capacity is up
Jan 21 14:02:45:I:System: Interface ethernet 1/1/24, state up
Jan 21 14:02:45:I:System: Interface ethernet 1/2/2, state up
Jan 21 14:02:45:I:System: Interface ethernet 1/1/15, state up
Jan 21 14:02:43:I:System: Interface ethernet 1/3/3, state up
Jan 21 14:02:43:I:System: Interface ethernet 1/3/1, state up
Jan 21 14:02:43:I:System: Interface ve 1, state up
Jan 21 14:02:43:I:System: Cold start
Jan 21 14:02:17:I:System: Port init success Stack unit 1 Port 1/2/1 Lane 0 T 0 R 0 Type 0:  00x00000x0000 00x00000x0000 00x00000x0000 00x00000x0000 00x00000x0000 00x00000x0000 00x00000x0000 00x00000x0000 00x00000x0000 00x00000x0000 00x00000x0000 00x00000x0000 00x00000x000
Jan 21 14:00:15:I:DHCPC: protocol disabled by user
Jan 21 14:00:15:I:NTP: The system clock is not synchronized to any time source.
Jan 21 14:00:15:I:NTP: The system clock is not synchronized and does not have a reference configured.

Well, I'm embarrassed I didn't think of that immediately, it is a good part of my day job to troubleshoot similar things, but unfortunately that was not the culprit.

I used a Netgear GS108PEv3 switch I had laying around as a POE source, it is supposed to only do 802.3af, so no passive POE in theory, and the AP worked fine connected to it. As a further step I connected another POE powered device (A DLink switch that does POE passthrough) which I have working in my home lan powered by POE, so definitely working correctly, and did a cold boot of the brocade with the switch on port 13 and the AP on port 14... neither got detected by the switch.

At this point the only thing I know for certain is that the issue is within the brocade switch itself.

I know it has been a long time, but given I've replaced my two 6610s with a couple of 6450s in this installation, I was left momentarily with both a fully functional ICX6610-48P and this one that seems to be lacking on the PoE part. I opened both of them, removed the PoE board from both and installed the one from the bad switch into the good one, just to assert the issue stems from that board and not something else in the switch.

It did fail, though in a slightly different way for some reason. Instead of not having PoE, all the ports have PoE power fed to them, although the switch knows nothing about it. Turning on or off inline power does nothing to change it and show inline power states there are no PoE negotiations happening, it says the full power is available, even though it is happily feeding a couple of APs.

Next I moved to the bad switch, installed the PoE board from the good switch and yeah, it works great. So now I'm sure the problem is, in fact, in the PoE board of the affected switch, and so my only recourse is to use one of the switches without PoE, in fact without the PoE board lest I fry some connected device that knows not of ethernet port isolation.

Just wanted to share the update

 

[andrea87]

That's not good to hear! Thing is, I took them apart to clean out the dust and I don't recall seeing any heatsinks. I hope I just wasn't paying close enough attention and the seller didn't rip them off.

Not looking forward to un-racking these guys and putting back my old switches for a second time. Unfortunately didn't notice the temp until after I racked these guys.

Temps are way too high. What fans are you running inside that switch? make sure they're either original or can push a significant amount of air. The 24 PoE version draws around 50W idle, I would expect these temps with low flow fans (like noctua).

Is it the PoE or non PoE version?



This is the inside of my icx7250-24p, I've added a 60x60x10 fan on the ASIC. The two 40x40 fans are a bit different than what shown, I'm running delta ffb 40x40x28 which are audible but not bothersome in a network rack.

For comparison, my switch runs at 65/37°C with ambient air in the closet at 20°C

Spoiler: show cha

SSH@ICX7250-24P>show cha
The stack unit 1 chassis info:

Power supply 1 (AC - PoE) present, status ok
Power supply 2 not present

Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 ok, speed (auto): [[1]]<->2

Fan controlled temperature:
Rule 1/2 (MGMT THERMAL PLANE): 65.6 deg-C
Rule 2/2 (AIR OUTLET NEAR PSU): 37.0 deg-C

Fan speed switching temperature thresholds:
Rule 1/2 (MGMT THERMAL PLANE):
Speed 1: NM<-----> 93 deg-C
Speed 2: 82<----->105 deg-C (shutdown)
Rule 2/2 (AIR OUTLET NEAR PSU):
Speed 1: NM<-----> 58 deg-C
Speed 2: 49<----->105 deg-C (shutdown)

Fan 1 Air Flow Direction: Front to Back
Fan 2 Air Flow Direction: Front to Back
Slot 1 Current Temperature: 65.6 deg-C (Sensor 1), 37.0 deg-C (Sensor 2)
Slot 2 Current Temperature: NA
Warning level.......: 100.0 deg-C
Shutdown level......: 105.0 deg-C

 

[Avamander]

make sure they're either original or can push a significant amount of air. The 24 PoE version draws around 50W idle, I would expect these temps with low flow fans (like noctua).

It also depends if the fans are pushing or pulling air. With Noctua replacements (even if they're set to maximum) one might have to change the direction to have proper airflow.

 

[jmdomini]

There should be plenty of heatsinks...

Opened them up and heatsinks are there, however the thermal compound is dried out and crunchy. Only have enough for that single larger heatsink on one switch, so that too is on order. The smaller ones look like they are on with tape. I ordered some heatsink tape, unsure if I should pry those off and reset while I'm add it.

I don't think the fans are OEM, they are these....https://tinyurl.com/35t3adb8

 

[fohdeesha]

saw 8095q was released around christmas, updated the guide 8095q, S4810 MAC issue, de-halloween · Fohdeesha/lab-docu@b03ee84

not much relevant new, niche bug fixes as usual

 

[rjrobert]

I'm finally jumping onto the enterprise networking train and picked up a pretty nice 6450-48P and have been going through the @fohdeesha (thanks for all of the work that goes into creating such an awesome guide, btw -- you're a rock star!), but unable to get past this error message during the update_primary step.



I have confirmed that the TFTP server is running and I can download the files onto other machines on my network.

I have also tried setting up TFTP servers on MacOS, Linux, and Windows, and I done some searching, but haven't found anything helpful so far. I'm not really sure what else to try at this point -- anyone have any ideas?

 

[TonyArrr]

I'm finally jumping onto the enterprise networking train and picked up a pretty nice 6450-48P and have been going through the @fohdeesha (thanks for all of the work that goes into creating such an awesome guide, btw -- you're a rock star!), but unable to get past this error message during the update_primary step.

View attachment 41238

Just checking definitely connected the management port of the switch to the network, not one of the regular ports?
And can you ping the s

I have confirmed that the TFTP server is running and I can download the files onto other machines on my network.

I have also tried setting up TFTP servers on MacOS, Linux, and Windows, and I done some searching, but haven't found anything helpful so far. I'm not really sure what else to try at this point -- anyone have any ideas?

any output in the logs for your TFTP server?

 

[rjrobert]

Just checking definitely connected the management port of the switch to the network, not one of the regular ports?
And can you ping the s



any output in the logs for your TFTP server?

Yeah, definitely connected to the management port -- always good to double check. I can also ping my TFTP server from the console.

Here are the most recent logs from my windows box.

Connection received from 192.168.1.50 on port 1052 [08/01 16:41:36.856]
Read request for file <ICX64xx/ICX64R08030u.bin>. Mode octet [08/01 16:41:36.856]
OACK: <timeout=5,blksize=1468,> [08/01 16:41:36.856]
Using local port 57706 [08/01 16:41:36.856]
Ack block 19 ignored (received twice) [08/01 16:41:56.886]
Ack block 29 ignored (received twice) [08/01 16:42:04.896]
Ack block 34 ignored (received twice) [08/01 16:42:09.899]
TIMEOUT waiting for Ack block #35 [08/01 16:42:32.938]

No firewalls enabled anywhere either. I am able connect to this TFTP server from other machines and download the files without issue, too. It seems like it connects to the server, but there's an issue when trying to download.

I came across this post that seemed relevant since my bootloader appears to be version 7.4, but the >enable level is locked down from the previous owner, so I cannot access the >configure terminal level to complete these steps.

 

[fohdeesha]

Yeah, definitely connected to the management port -- always good to double check. I can also ping my TFTP server from the console.

Here are the most recent logs from my windows box.


No firewalls enabled anywhere either. I am able connect to this TFTP server from other machines and download the files without issue, too. It seems like it connects to the server, but there's an issue when trying to download.

I came across this post that seemed relevant since my bootloader appears to be version 7.4, but the >enable level is locked down from the previous owner, so I cannot access the >configure terminal level to complete these steps.

What's the output of printenv in the bootloader

 

[fohdeesha]

Also triple check the .50 IP you gave the switch isn't already assigned to something else on your network, chance it to a different IP just in case

 

[rjrobert]

What's the output of printenv in the bootloader

And to your other point, I've used a few other IPs for the switch to make sure there was no collision.

 

[TonyArrr]

View attachment 41239

And to your other point, I've used a few other IPs for the switch to make sure there was no collision.

Here are the most recent logs from my windows box.

No firewalls enabled anywhere either. I am able connect to this TFTP server from other machines and download the files without issue, too. It seems like it connects to the server, but there's an issue when trying to download.

that’s more or less how my scanner would act when I accidentally had it set statically to an address that was also a dhcp reservation… start a connection, run for a (very) little while, then just vanish. These addresses you’re using are definitely outside the pool of addresses your dhcp server hands out, and not assigned to any reservations right?

It might be worth just connecting your computer and the switch directly and putting static IPs on both, then retrying, takes the entirety of the rest of your network out of the picture as possible causes.

edit: if doing this, you reconnect your computer and the switch back to the rest of your network after running

factory set-default
reset

I came across this post that seemed relevant since my bootloader appears to be version 7.4, but the >enable level is locked down from the previous owner, so I cannot access the >configure terminal level to complete these steps.

If the boot loader is too old to run factory set-default, you could follow fohdeesha’s much earlier advice to reset the enable password. But if you really have to use the webUI, move the firmware bins to the root folder of your TFTP server cause as I remember it, the webUI was interesting with slashes…

 

[rjrobert]

that’s more or less how my scanner would act when I accidentally had it set statically to an address that was also a dhcp reservation… start a connection, run for a (very) little while, then just vanish. These addresses you’re using are definitely outside the pool of addresses your dhcp server hands out, and not assigned to any reservations right?

It might be worth just connecting your computer and the switch directly and putting static IPs on both, then retrying, takes the entirety of the rest of your network out of the picture as possible causes.

edit: if doing this, you reconnect your computer and the switch back to the rest of your network after running

factory set-default
reset

If the boot loader is too old to run factory set-default, you could follow fohdeesha’s much earlier advice to reset the enable password. But if you really have to use the webUI, move the firmware bins to the root folder of your TFTP server cause as I remember it, the webUI was interesting with slashes…

Yeah, I tried connecting the switch to my laptop (which is running TFTP) with static IPs, and turned off my wifi for good measure, with the same results -- it always seems to make the connection initially but timeout during the transfer.

It's too old for the factory set-default, apparently -- it gives an error when I try it.

I'll take a look for the steps to reset the enable password as that might be my only option.