Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[chewbacca8403]

I'm looking for a PoE replacement board for the 6610-48P. Any good leads on where I might find one? I've looked on ebay and am considering just buying another to have for parts / replacements but if I could get a board for cheaper I'd like to do that instead.

I assume it was from a close lighting strike a few weeks ago. I've got the infamous "Resetting module..." loop on the console and one of the 4 "pods" has no PoE. After examining the PoE board one of the chips has a burn mark on the top of it.

 

[Freebsd1976]

ok , a big thank ,will report back,try it but not success,there are message

Bytes transferred = 29826604 (1c71e2c hex)
no current device selected
TFTP to Flash Done.


ICX7250-Boot> boot_primary
Error: The header crc ffffffff and the calculated  crc 4007851a mismatch
Fatal: Firmware integrity checksum failed
Booting image from Primary
no current device selected
Wrong Image Format for bootm command
ERROR: can't get kernel image!


ICX7250-Boot> tftpboot
Using bcmiproc_eth-0 device
TFTP from server 192.168.1.12; our IP address is 192.168.1.49
Filename 'SPR08080e.bin'.
Load address: 0x61007dc0
done
Bytes transferred = 29826604 (1c71e2c hex)
Wrong Image Format for tftpboot command
ERROR: can't get kernel image!

update_primary the reset , still boot to diag firmware. reenter u-boot , input boot_primary or tftpboot not success

 

[infoMatt]

I'm looking for a PoE replacement board for the 6610-48P. Any good leads on where I might find one? I've looked on ebay and am considering just buying another to have for parts / replacements but if I could get a board for cheaper I'd like to do that instead.

I assume it was from a close lighting strike a few weeks ago. I've got the infamous "Resetting module..." loop on the console and one of the 4 "pods" has no PoE. After examining the PoE board one of the chips has a burn mark on the top of it.

Personally, I've never seen any spare board auction on ebay (not that I've searched for them explicitly, but still...); I think that your best bet is to grab another unit and swap boards, or if you don't need PoE, just remove the faulty board so you won't have all the errors on log.

 

[Freebsd1976]

the whole update_primary as following， IMHO it show

no current device selected
TFTP to Flash Done.

I think this maybe the reason , it seems just download firmware but not write to nand,question is how to write nand? these command maybe helpful?
mtdparts- define flash/nand partitions
mii - MII utility commands
nand - NAND sub-system

ICX7250-Boot>setenv netmask 255.255.255.0
ICX7250-Boot> setenv serverip 192.168.1.12
ICX7250-Boot> setenv image_name SPR08080e.bin
ICX7250-Boot> saveenv
ICX7250-Boot> update_primary
Updating PRIMARY image over TFTP
Using bcmiproc_eth-0 device
TFTP from server 192.168.1.12; our IP address is 192.168.1.49
Filename 'SPR08080e.bin'.
Load address: 0x1000000
Loading: #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################
done
Bytes transferred = 29826604 (1c71e2c hex)
no current device selected
TFTP to Flash Done.

 

[chewbacca8403]

Personally, I've never seen any spare board auction on ebay (not that I've searched for them explicitly, but still...); I think that your best bet is to grab another unit and swap boards, or if you don't need PoE, just remove the faulty board so you won't have all the errors on log.

Didn't see any spare board auctions or anything in google. Got the switch originally for the PoE so looks like a new switch is coming my way.... Thanks!

 

[AcrobaticWoodpecker]

from what I've found it's because there's wildly different sound levels from these things depending on configuration. If it's indeed a rev C, it should not be that loud, and shouldn't require two PSUs to spin down (that's just the rev A supplies). make sure the PSU and fan tray are in the left chassis slots (facing the rear of the switch). Also make sure they're actually the E (exhaust models), they should have green stickers on the psu and fan tray, not orange. the orange intake models are much louder. Hearing it from several rooms over is absurd though, I can't hear a stack of 2x 6610s on the other side of the drywall behind me

I performed some (non-scientific) sound level tests using a dB meter app on my phone (I don't know how accurate the app is, this is purely for comparative purposes). With the phone two feet from the rear of the switch (again, these tests are not representative of sound levels in any one environment, they are purely to compare the different switch configurations) and the switch on a non-conductive surface I recorded a two minute average with recording beginning after boot was completed. The switch was reset to factory default prior to testing with zero licensing and zero connected devices. The results are in the table below along with my show chassis output below that (command was run after last test). Let me know if these numbers seem normal.

From the Brocade spec sheet: From 39.6 dB (24 ports, 1 fan, 1 PSU)

So these numbers don't seem right at all. Did they measure with the switch in a deep freezer?

	PS Slot 1	PS Slot 2
Chassis Fan Slot 1​	63.9dB	64.2dB*
Chassis Fan Slot 2​	63.7dB	63.9dB

* Factory configuration (I think?)

ICX6610-24 Router>show chassis
The stack unit 1 chassis info:

Power supply 1 not present
Power supply 2 (AC - Regular) present, status ok
        Model Number:   23-0000144-02
        Serial Number:  0HE
        Firmware Ver:    C
Power supply 2 Fan Air Flow Direction:  Front to Back

Fan 1 ok, speed (auto): [[1]]<->2
Fan 2 not present

Fan controlled temperature: 47.0 deg-C

Fan speed switching temperature thresholds:
                Speed 1: NM<----->80       deg-C
                Speed 2:       75<-----> 87 deg-C (shutdown)

Fan 1 Air Flow Direction:  Front to Back
MAC 1 Temperature Readings:
        Current temperature : 38.5 deg-C
CPU Temperature Readings:
        Current temperature : 38.5 deg-C
sensor A Temperature Readings:
        Current temperature : 25.0 deg-C
sensor B Temperature Readings:
        Current temperature : 33.0 deg-C
sensor C Temperature Readings:
        Current temperature : 22.0 deg-C
sensor D Temperature Readings:
        Current temperature : 19.0 deg-C
stacking card Temperature Readings:
        Current temperature : 47.0 deg-C
        Warning level.......: 84.0 deg-C
        Shutdown level......: 87.0 deg-C
Boot Prom MAC : cc4e.24bd.1d0c
Management MAC: cc4e.24bd.1d0c

 

[infoMatt]

From the Brocade spec sheet: From 39.6 dB (24 ports, 1 fan, 1 PSU)

So these numbers don't seem right at all. Did they measure with the switch in a deep freezer?

	PS Slot 1	PS Slot 2
Chassis Fan Slot 1​	63.9dB	64.2dB*
Chassis Fan Slot 2​	63.7dB	63.9dB

* Factory configuration (I think?)

I've made the conclusion that it's due mostly to aging fans that makes considerably more noise than when they were brand new... My 6450, albeit described in this thread as "quiet", it's screaming like hell, I can hear it from the other side of the house (with thiccc brick wall, EU/IT)... And it's buzzing presumeably due to the hard PWM on-off modulation... lubed the fans, no difference. I can hear it also wearing my closed-back cans playing music.
The fans inside are Foxconn PIA040H12L. I haven't tested in a scientific manner, but to my ears it's louder than a stack of Cisco 3850 in my $dayjob lab.

Also, it draws ~50% more power than what is written in the datasheet (even stranger, plugged in 230V mains)... Maybe it's time to change some capacitors... After this lockdown I'll try to look at those.

 

[Wolfstar]

Yeah... that seems wrong for a 6450. Mine's relatively quiet - I can't pick it out over the general quiet noise of the rack at least - and it pulls maybe 40-45w tops, and that's with a pair of APs and a switch getting power fed from it.

 

[AcrobaticWoodpecker]

Also why is the stacking card the hottest component in the switch when there's nothing even connected to it? Could this be a case of bad thermal compound?

 

[infoMatt]

Yeah... that seems wrong for a 6450. Mine's relatively quiet - I can't pick it out over the general quiet noise of the rack at least - and it pulls maybe 40-45w tops, and that's with a pair of APs and a switch getting power fed from it.

Mine pulls ~33-34W at idle (all port active, all port PoE enabled, nothing connected)... it should be 22W max... ICX6450-24P.
We'll see...

Also why is the stacking card the hottest component in the switch when there's nothing even connected to it? Could this be a case of bad thermal compound?

Maybe because it has the worse airflow above it, and thus it can't dissipate the heat very well. Usually, these type of devices have their heatsinks glued onto the chips with thermal conductive glue, not thermal compound... But I've never touched a 6610...

 

[AcrobaticWoodpecker]

Maybe because it has the worse airflow above it, and thus it can't dissipate the heat very well. Usually, these type of devices have their heatsinks glued onto the chips with thermal conductive glue, not thermal compound... But I've never touched a 6610...

It is in the worst spot for air flow and the heatsinks are attached to the chips via thermal adhesive. I removed the stacking card to see if the 'Fan controlled temperature' would default to the next-highest temperature but the switch wouldn't boot to completion and I didn't disable stacking before removal.

 

[Freebsd1976]

anyone who have icx7250 , could you help me, share your u-boot printenv and show flash cmd output ? I think my switch env maybe missing or just wrong

 

[Wolfstar]

Mine pulls ~33-34W at idle (all port active, all port PoE enabled, nothing connected)... it should be 22W max... ICX6450-24P.
We'll see...

Okay, so I'm pretty sure you're getting normal levels. I'm guessing you're using a Kill-a-Watt or something similar yes? Because that's going to give you pre-conversion wattage, and not absolute minimum wattage from a DC power source. At such low wattages, even that percentage is a low absolute number and is easily at least part going to be conversion efficiency of the PSU. I thought you were talking that it should be 40 and you were getting almost 80 or something like that.

 

[fohdeesha]

anyone who have icx7250 , could you help me, share your u-boot printenv and show flash cmd output ? I think my switch env maybe missing or just wrong

I can reset your bootloader and fix this for you, I got your teamviewer PM but I didn't have time tonight. I should be able to tomorrow night.

Also everyone, there's new 8030 firmware for the icx6xxx series, 8030u - Ruckus ICX FastIron 08.0.30u Software Release (.zip) | Software Downloads | Ruckus Wireless Support

release notes here, fixes start on page 29 - https://fohdeesha.com/data/other/brocade/08030u_ReleaseNotes_v1.pdf

I'll update my config/update guide when I get a chance, but it's nothing critical. the most impactful bug they fixed (and the only one I've noticed) is that when running "show version" on 8030t, the startup date is wildly wrong, like 2036

 

[Freebsd1976]

I can reset your bootloader and fix this for you, I got your teamviewer PM but I didn't have time tonight. I should be able to tomorrow night.

Also everyone, there's new 8030 firmware for the icx6xxx series, 8030u - Ruckus ICX FastIron 08.0.30u Software Release (.zip) | Software Downloads | Ruckus Wireless Support

release notes here, fixes start on page 29 - https://fohdeesha.com/data/other/brocade/08030u_ReleaseNotes_v1.pdf

I'll update my config/update guide when I get a chance, but it's nothing critical. the most impactful bug they fixed (and the only one I've noticed) is that when running "show version" on 8030t, the startup date is wildly wrong, like 2036

big thanks , will keep the pc on, switch off(too lound ), waiting for tomorrow night

 

[infoMatt]

Okay, so I'm pretty sure you're getting normal levels. I'm guessing you're using a Kill-a-Watt or something similar yes? Because that's going to give you pre-conversion wattage, and not absolute minimum wattage from a DC power source. At such low wattages, even that percentage is a low absolute number and is easily at least part going to be conversion efficiency of the PSU. I thought you were talking that it should be 40 and you were getting almost 80 or something like that.

Yes, a DIN rail power meter.
Absolutely, it will include the power loss of the supply, but it is also what is stated on the datasheet... It lists AC power, because for those switches the primary supply is the internal one; at least, that's what I supposed reading the datasheet.
https://webresources.ruckuswireless.com/pdf/datasheets/ds-icx-6430-6450.pdf page 11.

ICX6450-24P idle should be 21W, I've just checked. Mine measured 33-34W with an horrible 0.5ish PF, and is this figure that to me seems very strange. Ok, it's a ~200W supply, and it's running at about 10% load, but that's the power usage of it for basically all the time (-0/+10W or so), and according to EU power efficiency regulations, it shouldn't have such a low power factor; the only exceptions if memory serves are those little adapters for charging phones, shavers and what not, with a limit of just a couple of watts of maximum rated power.

To me, something doesn't add up...

 

[britinpdx]

ICX6450-24P idle should be 21W, I've just checked. Mine measured 33-34W

FWIW, as I have both the non-POE and POE units handy, I just checked idle power using a Kill-A-Watt once units have booted up and "settled down".

The ICX6450-24 idles at ~24W and the ICX6450-24P idles at ~34W, so roughly in line with what you have seen.

 

[infoMatt]

FWIW, as I have both the non-POE and POE units handy, I just checked idle power using a Kill-A-Watt once units have booted up and "settled down".

The ICX6450-24 idles at ~24W and the ICX6450-24P idles at ~34W, so roughly in line with what you have seen.

Thank you! But then... why bother writing on the datasheet the "wrong" power usage?
Usually in the IT world, those figures are waaayy overestimated; this is the first example of underrating on the DS.

Oh well...

 

[LodeRunner]

Yes, a DIN rail power meter.
Absolutely, it will include the power loss of the supply, but it is also what is stated on the datasheet... It lists AC power, because for those switches the primary supply is the internal one; at least, that's what I supposed reading the datasheet.
https://webresources.ruckuswireless.com/pdf/datasheets/ds-icx-6430-6450.pdf page 11.

ICX6450-24P idle should be 21W, I've just checked. Mine measured 33-34W with an horrible 0.5ish PF, and is this figure that to me seems very strange. Ok, it's a ~200W supply, and it's running at about 10% load, but that's the power usage of it for basically all the time (-0/+10W or so), and according to EU power efficiency regulations, it shouldn't have such a low power factor; the only exceptions if memory serves are those little adapters for charging phones, shavers and what not, with a limit of just a couple of watts of maximum rated power.

To me, something doesn't add up...

I can't think of any power supplies that are particularly efficient at 10% load, generally in the 50-80% band is maximum efficiency. I mean, they could be crap too, but even the best PSU's at 10% load are not going to be very efficient.

 

[infoMatt]

I can't think of any power supplies that are particularly efficient at 10% load, generally in the 50-80% band is maximum efficiency. I mean, they could be crap too, but even the best PSU's at 10% load are not going to be very efficient.

Actually, they are pretty good PSU! They are made by Delta IIRC... and in case of the 6450, it's as deep as the switch itself, and ~ 20cm wide... it's a decent sized "brick"

But then... why using a 200-250W PSU if the board will pull ~40W max? I don't think that they made a "stack power cable" (as for example Cisco does on the 3650/3850 stacks) for sharing power; the only use for the DC connector at the back is to connect every unit to a dedicated external (stack-)PSU.