Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[MTFPLEX]

I think this is answered somewhere 5-10 pages back in this thread.
Looks like a dead POE board.
My first switch came with an error like this. When I opened it up, there was a scorch mark on the POE board. When I took the board out and flipped it over, there was a microchip that had let out the magic smoke. The eBay seller sent me a replacement and everything worked with that one.

Do you remember how to take out the board? Do I just remove the screws and unplug it then slide it out?

 

[Tegan]

Do you remember how to take out the board? Do I just remove the screws and unplug it then slide it out?

I think there were just a couple screws holding the board in place and then some unsecured rows of header pins making the electrical connection between the POE board to the main board.
Apparently I didn't take any good photos of the interior, just photos of the damage.
EDIT: I didn't catch which model you have. I have an ICX6610-24P, and I didn't have any challenges disassembling it.

 

[NablaSquaredG]

PSA: DON'T disassembly ICX7650-48ZP. It's a nightmare.

It's a two layer design, but the standoffs rotate when you try to remove the top PCBs. Not an issue for the PoE board (it's plugged in from the top), but for the management board.

The management board is kinda plugged in like an M.2 card in a slot, but with 4 standoffs. That means if the standoffs start to loosen when you remove the top screws holding the management board into place.
Once the standoffs have loosened, you cannot remove the management board. You still need some very special pliers to hold the standoffs into place...

 

[CIR-Engineering]

Argh. I'm done for today but I am getting this error again while booting and in log. PoE seems works fine though. Is this something to be worried about or not?

PoE Severe Error: Unexpected controller reset encountered on PoE module.

Power supply 1  detected.
Power supply 1  is up.
PoE: Stack unit 1 Power supply 1  with 748000 mwatts capacity is up

Power supply 2  detected.
Power supply 2  is up.
PoE: Stack unit 1 Power supply 2  with 748000 mwatts capacity is up
PoE Severe Error: Unexpected controller reset encountered on PoE module.
PoE Info: PoE module 1 of Unit 1 on ports 1/1/1 to 1/1/32 detected. Initializing....
PoE Info: PoE module 1 of Unit 1 initialization is done.

 

[NablaSquaredG]

PoE Severe Error: Unexpected controller reset encountered on PoE module.

Can you return this switch? There is clearly a hardware defect.

 

[86turbodsl]

Having trouble with my 10g ports on a 6610. I had a dual lag connection with my storage server and it was working fine. Left the original 1g connection on it. Came back an hour later and the 10g ports were not connected anymore. Reverted back to the 1g connection and it works but nothing I do can make the 2 10g connections come back up. Moved the sfp adapters around, tried a 10g connection to another switch, works fine. A patch cable looped works fine. Tried clearing everything in cache, rebooting warm and cold. Any suggestions?

Edit. I am using an HP branded mellanox connectx3 dual card. Seems there might be issues with that card.

 

[CIR-Engineering]

Can you return this switch? There is clearly a hardware defect.

I can return it, but I’ll never find another ICX7450-32ZP. The PoE is working fine too. It seems to reset once during each boot and then is fine.

 

[MTFPLEX]

I think there were just a couple screws holding the board in place and then some unsecured rows of header pins making the electrical connection between the POE board to the main board.
Apparently I didn't take any good photos of the interior, just photos of the damage.
EDIT: I didn't catch which model you have. I have an ICX6610-24P, and I didn't have any challenges disassembling it.

I have a ICX6450-48P, after removing the PoE board, I can't find anything wrong with the board, however I did get electrocuted after accidently touching the power supply, whoops! When I boot the switch without the PoE board, the fans run at max speed.

 

[MTFPLEX]

I have a ICX6450-48P, after removing the PoE board, I can't find anything wrong with the board, however I did get electrocuted after accidently touching the power supply, whoops! When I boot the switch without the PoE board, the fans run at max speed.

Here is the PCB

 

[CIR-Engineering]

Can you return this switch? There is clearly a hardware defect.

I'm not sure if there is a defect. It never throws the error if I power cycle, only when I do "reload" at the CLI. It's been a "FUN" evening. I looked and looked for shorted caps and didn't find any. If I must, I'll remove the heat sinks and look under there. This is an unusual switch and the 2.5 GbE ports seem to be on a different "card" as the OS puts it during boot. I think it may be an erroneous error because after the reset the PoE ALWAYS works fine. I hope someone else chimes in who has this switch, but they are so rare. Hopefully some other members who participate on this thread grabbed the other 12 that sold on eBay in the past week. I hope I don't have unlucky 13... but I got the third one.



I'm putting her back together, reading, and going to bed. Long day with this thing, and I thought it would be cake.

 

[CIR-Engineering]

Can you return this switch? There is clearly a hardware defect.

Well, I don't know when it decides to throw that error or not. After I disassembled it and looked for shorted caps, I noticed some small metal shavings here and there. I hit everywhere with the air compressor and after that no more error. Now it's back. I wish I knew if this were terminal or not. The switch seems to run as expected in every way.

 

[fohdeesha]

AAAAAAAAAAAAHHHHHHHHHHHHHHHHHHH!!!!!

Reading multiple places:

SSH@ICX7450(config-if-e1000-1/1/1)#interface ethernet 1/1/31 to 1/1/32
SSH@ICX7450(config-mif-1/1/31-1/1/32)#speed-duplex auto
This CLI operation is not applicable to 2.5G port 1/1/31
This CLI operation is not applicable to 2.5G port 1/1/32
SSH@ICX7450(config-mif-1/1/31-1/1/32)#

What do I need to do

check page 45 in fastiron-08095-managementguide.pdf from the zip of my guide. on newer OS versions I believe that grouping has changed per the example footnote on that page:

In ICX 7450-32ZP, 2.5G ports can be configured only in pairs or set of pairs e.g. (1/1/25 -1/1/26) (1/1/27 -1/1/28) (1/1/25-1/1/32) etc

I'm also pretty sure this earlier implementation of 2.5gbE (before the IEEE standard for it was even finished) does not support auto-neg at all, which is why it's telling you "not applicable" when you tried to set it to auto. it also states that they are indeed set by default to 2500-full, so that's why they weren't linking with your other stuff:

"On ICX 7450-32ZP, default speed is 2500-full on MultiGig ports"

so speed-duplex 1000-full should solve your issue regarding linking to other normal 1gbe stuff

I'd also ignore the PoE error if it just happens once during boot but works fine. It could be triggered by having devices plugged into the ports that have strange resistance on the pairs and do something to the PoE controller before the link is properly negotiated after fully booting. Try booting it with nothing plugged into it at all if you're curious

 

[TonyArrr]

…however I did get electrocuted after accidently touching the power supply, whoops!

You’re very dedicated, to be posting here from heaven

Real talk though, I hope you’ve been by a ER to get a EKG/ECG. Shocks off ac/dc power supplies are well and truely dangerous, and one of the biggest reasons is that other than just immediately killing you, they can also just screw with your heart rhythm so at first you feel fine, but your heart enters fibrillation later.

It’s serious enough that at work, any shock means the hospital visit, and we are to remain awake and with someone at all times for 24 hours afterwards. They even keep us on the clock the whole time.

Please be safe! I know statistically you’ll probably be fine, but if you’re not fine you will be veeeeerrrrrrrryyyyyyyy not fine

 

[CIR-Engineering]

check page 45 in fastiron-08095-managementguide.pdf from the zip of my guide. on newer OS versions I believe that grouping has changed per the example footnote on that page:

In ICX 7450-32ZP, 2.5G ports can be configured only in pairs or set of pairs e.g. (1/1/25 -1/1/26) (1/1/27 -1/1/28) (1/1/25-1/1/32) etc

I'm also pretty sure this earlier implementation of 2.5gbE (before the IEEE standard for it was even finished) does not support auto-neg at all, which is why it's telling you "not applicable" when you tried to set it to auto. it also states that they are indeed set by default to 2500-full, so that's why they weren't linking with your other stuff:

"On ICX 7450-32ZP, default speed is 2500-full on MultiGig ports"

so speed-duplex 1000-full should solve your issue regarding linking to other normal 1gbe stuff

I'd also ignore the PoE error if it just happens once during boot but works fine. It could be triggered by having devices plugged into the ports that have strange resistance on the pairs and do something to the PoE controller before the link is properly negotiated after fully booting. Try booting it with nothing plugged into it at all if you're curious

Last night I did try booting up with nothing plugged into any Ethernet port, just serial, and it does still throw the error. Thoughts? Should I keep it? I also checked all pins in all RJ45 connectors and also blew them out with the air compressor.

Thanks for confirming no auto on the 2.5 GbE ports. I’ll stop going crazy over that... EDIT Well, I tried these:

ICX7450-32ZP Router(config)#interface ethernet 1/1/25 to 1/1/32
ICX7450-32ZP Router(config-mif-1/1/25-1/1/32)#speed auto
This CLI operation is not applicable to 2.5G port 1/1/25
This CLI operation is not applicable to 2.5G port 1/1/26
This CLI operation is not applicable to 2.5G port 1/1/27
This CLI operation is not applicable to 2.5G port 1/1/28
This CLI operation is not applicable to 2.5G port 1/1/29
This CLI operation is not applicable to 2.5G port 1/1/30
This CLI operation is not applicable to 2.5G port 1/1/31
This CLI operation is not applicable to 2.5G port 1/1/32
ICX7450-32ZP Router(config-mif-1/1/25-1/1/32)#speed-duplex 1000-full

This DID in fact work, but I want to use these are 2.5 GbE anyway.

ICX7450-32ZP Router(config-mif-1/1/25-1/1/32)#speed-duplex 2500-full

So I tried going to auto from 1000-full and failed.  Reset to 2.5GbE worked fine though.

 

[MTFPLEX]

You’re very dedicated, to be posting here from heaven

I don't think I touched anything high voltage (120v), instead I think I hit something around 12v as my hand kinda just felt funny for a few seconds and felt fine after removing contact.

 

[CIR-Engineering]

I don't think I touched anything high voltage (120v), instead I think I hit something around 12v as my hand kinda just felt funny for a few seconds and felt fine after removing contact.

I’m not a doctor, but unless you have a pre existing condition, a zap from 120AC or lower is probably not an issue. I was going to say however that you are indeed writing here so I don’t think you were “electrocuted.”

I work with high voltage regularly, up to 35k VDC. You need to always be cautious with electricity. With high voltage you always use your right hand and keep your left in your back pocket. That way if you get hit, at least you won’t have a path directly through your heart.

Whatever voltage got you was above 12V. You can’t even feel 12V unless you discharge a large capacitor somehow or water is involved (put a 9-volt battery on your tongue is an old way to test the battery).

I’ve seen two bad accidents; I saw a man indeed get electrocuted from 660VAC. Then in collage we worked on very early hybrid cars. I saw a kid get hit with a battery and to this day he has heart issues from it.

Be careful man.

 

[CIR-Engineering]

check page 45 in fastiron-08095-managementguide.pdf from the zip of my guide. on newer OS versions I believe that grouping has changed per the example footnote on that page:

In ICX 7450-32ZP, 2.5G ports can be configured only in pairs or set of pairs e.g. (1/1/25 -1/1/26) (1/1/27 -1/1/28) (1/1/25-1/1/32) etc

I'm also pretty sure this earlier implementation of 2.5gbE (before the IEEE standard for it was even finished) does not support auto-neg at all, which is why it's telling you "not applicable" when you tried to set it to auto. it also states that they are indeed set by default to 2500-full, so that's why they weren't linking with your other stuff:

"On ICX 7450-32ZP, default speed is 2500-full on MultiGig ports"

so speed-duplex 1000-full should solve your issue regarding linking to other normal 1gbe stuff

I'd also ignore the PoE error if it just happens once during boot but works fine. It could be triggered by having devices plugged into the ports that have strange resistance on the pairs and do something to the PoE controller before the link is properly negotiated after fully booting. Try booting it with nothing plugged into it at all if you're curious

Last night I did try booting up with nothing plugged into any Ethernet port, just serial, and it does still throw the error. Thoughts? Should I keep it? I also checked all pins in all RJ45 connectors and also blew them out with the air compressor.

One more thing and thanks again for chiming in fohdeesha, you are the man! I don't like that the error is intermittent. I can't find a pattern to it. This morning, booted with no error on the PoE restart. Do you think that's significant?

 

[donedeal19]

Run a system diagnostics test, dm diag and dm alt-diag. This should run a full diagnostics on reboot. Watch the status led lights as the test will runs link speeds on every port. And then, show environment fan, power, and sensor for health monitoring.

 

[CIR-Engineering]

Run a system diagnostics test, dm diag and dm alt-diag. This should run a full diagnostics on reboot. Watch the status led lights as the test will runs link speeds on every port. And then, show environment fan, power, and sensor for health monitoring.

CRAP

What does this even mean?

1/2/2   374945528       374945528       0               ##PASS
1/2/3   371165213       371165213       0               ##PASS
1/2/4   366856907       366856907       0               ##PASS
1/3/1   [  409.240204] [BrcdSoftlockup]: Tracker queued..
416501837       416501837       0               ##PASS
1/4/1   419099154       419099154       0               ##PASS


FATAL ERROR: System did not pass diagnosis
SYSTEM STOPPED, Please Check with Ruckus Networks Support
Typing 'r' would reset the system
##FAILED DIAG
[  411.030781] [BrcdSoftlockup]: sim_softwatchdog thread is detached on core=1

I've also been seeing this during boot and now wonder if it is significant?

                        SOC unit 0 attached to PCI device BCM56548_A0
0:soc_do_init: soc_do_init: total cpu and arm cosq 0030 unexpected
Initializing the Port pair for the P2 board
u=0 p=50 Using SDK PHY driver for BCM84756/7/9
u=0 p=51 Using SDK PHY driver for BCM84756/7/9
Setting the CMIC registers for the link scan
SOC unit 1 attached to PCI device BCM56548_A0
1:soc_do_init: soc_do_init: total cpu and arm cosq 0030 unexpected
u=1 p=42 Using SDK PHY driver for BCM84756/7/9
u=1 p=43 Using SDK PHY driver for BCM84756/7/9
Setting the CMIC registers for the link scan
Info: SendEventCommon: scp task Not inited by FI

Sigh. I've wanted this switch for so long....

 

[R3Z3N]

^Well, kinda like dating a dime if the mental stability isn't there. Do you REALLY want it, and if so how long can you last before you lose it...

Just forget you found a broken model, and go find what works.