Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

MTFPLEX

New Member
Apr 28, 2024
6
0
1
I think this is answered somewhere 5-10 pages back in this thread.
Looks like a dead POE board.
My first switch came with an error like this. When I opened it up, there was a scorch mark on the POE board. When I took the board out and flipped it over, there was a microchip that had let out the magic smoke. The eBay seller sent me a replacement and everything worked with that one.
Do you remember how to take out the board? Do I just remove the screws and unplug it then slide it out?
 

Tegan

New Member
Jan 31, 2011
16
9
3
Do you remember how to take out the board? Do I just remove the screws and unplug it then slide it out?
I think there were just a couple screws holding the board in place and then some unsecured rows of header pins making the electrical connection between the POE board to the main board.
Apparently I didn't take any good photos of the interior, just photos of the damage.
EDIT: I didn't catch which model you have. I have an ICX6610-24P, and I didn't have any challenges disassembling it.
 

Attachments

Last edited:

NablaSquaredG

Destroyer of Mellanox switches
Aug 17, 2020
1,436
879
113
PSA: DON'T disassembly ICX7650-48ZP. It's a nightmare.

It's a two layer design, but the standoffs rotate when you try to remove the top PCBs. Not an issue for the PoE board (it's plugged in from the top), but for the management board.

The management board is kinda plugged in like an M.2 card in a slot, but with 4 standoffs. That means if the standoffs start to loosen when you remove the top screws holding the management board into place.
Once the standoffs have loosened, you cannot remove the management board. You still need some very special pliers to hold the standoffs into place...
 

CIR-Engineering

I am a functional adult?
Jan 14, 2021
150
58
28
48
Chicago USA
www.cir-engineering.com
Argh. I'm done for today but I am getting this error again while booting and in log. PoE seems works fine though. Is this something to be worried about or not?

PoE Severe Error: Unexpected controller reset encountered on PoE module.

Code:
Power supply 1  detected.
Power supply 1  is up.
PoE: Stack unit 1 Power supply 1  with 748000 mwatts capacity is up

Power supply 2  detected.
Power supply 2  is up.
PoE: Stack unit 1 Power supply 2  with 748000 mwatts capacity is up
PoE Severe Error: Unexpected controller reset encountered on PoE module.
PoE Info: PoE module 1 of Unit 1 on ports 1/1/1 to 1/1/32 detected. Initializing....
PoE Info: PoE module 1 of Unit 1 initialization is done.
 

86turbodsl

Member
Feb 24, 2020
90
32
18
Having trouble with my 10g ports on a 6610. I had a dual lag connection with my storage server and it was working fine. Left the original 1g connection on it. Came back an hour later and the 10g ports were not connected anymore. Reverted back to the 1g connection and it works but nothing I do can make the 2 10g connections come back up. Moved the sfp adapters around, tried a 10g connection to another switch, works fine. A patch cable looped works fine. Tried clearing everything in cache, rebooting warm and cold. Any suggestions?

Edit. I am using an HP branded mellanox connectx3 dual card. Seems there might be issues with that card.
 
Last edited:

MTFPLEX

New Member
Apr 28, 2024
6
0
1
I think there were just a couple screws holding the board in place and then some unsecured rows of header pins making the electrical connection between the POE board to the main board.
Apparently I didn't take any good photos of the interior, just photos of the damage.
EDIT: I didn't catch which model you have. I have an ICX6610-24P, and I didn't have any challenges disassembling it.
I have a ICX6450-48P, after removing the PoE board, I can't find anything wrong with the board, however I did get electrocuted after accidently touching the power supply, whoops! When I boot the switch without the PoE board, the fans run at max speed.
 

CIR-Engineering

I am a functional adult?
Jan 14, 2021
150
58
28
48
Chicago USA
www.cir-engineering.com
Can you return this switch? There is clearly a hardware defect.
I'm not sure if there is a defect. It never throws the error if I power cycle, only when I do "reload" at the CLI. It's been a "FUN" evening. I looked and looked for shorted caps and didn't find any. If I must, I'll remove the heat sinks and look under there. This is an unusual switch and the 2.5 GbE ports seem to be on a different "card" as the OS puts it during boot. I think it may be an erroneous error because after the reset the PoE ALWAYS works fine. I hope someone else chimes in who has this switch, but they are so rare. Hopefully some other members who participate on this thread grabbed the other 12 that sold on eBay in the past week. I hope I don't have unlucky 13... but I got the third one.

IMG_1011.jpg

I'm putting her back together, reading, and going to bed. Long day with this thing, and I thought it would be cake.
 
Last edited:

CIR-Engineering

I am a functional adult?
Jan 14, 2021
150
58
28
48
Chicago USA
www.cir-engineering.com
Can you return this switch? There is clearly a hardware defect.
Well, I don't know when it decides to throw that error or not. After I disassembled it and looked for shorted caps, I noticed some small metal shavings here and there. I hit everywhere with the air compressor and after that no more error. Now it's back. I wish I knew if this were terminal or not. The switch seems to run as expected in every way.
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,789
3,168
113
33
fohdeesha.com
AAAAAAAAAAAAHHHHHHHHHHHHHHHHHHH!!!!!

Reading multiple places:


Code:
SSH@ICX7450(config-if-e1000-1/1/1)#interface ethernet 1/1/31 to 1/1/32
SSH@ICX7450(config-mif-1/1/31-1/1/32)#speed-duplex auto
This CLI operation is not applicable to 2.5G port 1/1/31
This CLI operation is not applicable to 2.5G port 1/1/32
SSH@ICX7450(config-mif-1/1/31-1/1/32)#
What do I need to do :Do_O:eek:
check page 45 in fastiron-08095-managementguide.pdf from the zip of my guide. on newer OS versions I believe that grouping has changed per the example footnote on that page:

In ICX 7450-32ZP, 2.5G ports can be configured only in pairs or set of pairs e.g. (1/1/25 -1/1/26) (1/1/27 -1/1/28) (1/1/25-1/1/32) etc

I'm also pretty sure this earlier implementation of 2.5gbE (before the IEEE standard for it was even finished) does not support auto-neg at all, which is why it's telling you "not applicable" when you tried to set it to auto. it also states that they are indeed set by default to 2500-full, so that's why they weren't linking with your other stuff:

"On ICX 7450-32ZP, default speed is 2500-full on MultiGig ports"

so speed-duplex 1000-full should solve your issue regarding linking to other normal 1gbe stuff

I'd also ignore the PoE error if it just happens once during boot but works fine. It could be triggered by having devices plugged into the ports that have strange resistance on the pairs and do something to the PoE controller before the link is properly negotiated after fully booting. Try booting it with nothing plugged into it at all if you're curious
 
  • Like
Reactions: CIR-Engineering

TonyArrr

Active Member
Sep 22, 2021
141
75
28
Straylia
…however I did get electrocuted after accidently touching the power supply, whoops!
You’re very dedicated, to be posting here from heaven ;)

Real talk though, I hope you’ve been by a ER to get a EKG/ECG. Shocks off ac/dc power supplies are well and truely dangerous, and one of the biggest reasons is that other than just immediately killing you, they can also just screw with your heart rhythm so at first you feel fine, but your heart enters fibrillation later.

It’s serious enough that at work, any shock means the hospital visit, and we are to remain awake and with someone at all times for 24 hours afterwards. They even keep us on the clock the whole time.

Please be safe! I know statistically you’ll probably be fine, but if you’re not fine you will be veeeeerrrrrrrryyyyyyyy not fine
 
  • Like
Reactions: MTFPLEX

CIR-Engineering

I am a functional adult?
Jan 14, 2021
150
58
28
48
Chicago USA
www.cir-engineering.com
check page 45 in fastiron-08095-managementguide.pdf from the zip of my guide. on newer OS versions I believe that grouping has changed per the example footnote on that page:

In ICX 7450-32ZP, 2.5G ports can be configured only in pairs or set of pairs e.g. (1/1/25 -1/1/26) (1/1/27 -1/1/28) (1/1/25-1/1/32) etc

I'm also pretty sure this earlier implementation of 2.5gbE (before the IEEE standard for it was even finished) does not support auto-neg at all, which is why it's telling you "not applicable" when you tried to set it to auto. it also states that they are indeed set by default to 2500-full, so that's why they weren't linking with your other stuff:

"On ICX 7450-32ZP, default speed is 2500-full on MultiGig ports"

so speed-duplex 1000-full should solve your issue regarding linking to other normal 1gbe stuff

I'd also ignore the PoE error if it just happens once during boot but works fine. It could be triggered by having devices plugged into the ports that have strange resistance on the pairs and do something to the PoE controller before the link is properly negotiated after fully booting. Try booting it with nothing plugged into it at all if you're curious
Last night I did try booting up with nothing plugged into any Ethernet port, just serial, and it does still throw the error. Thoughts? Should I keep it? I also checked all pins in all RJ45 connectors and also blew them out with the air compressor.

Thanks for confirming no auto on the 2.5 GbE ports. I’ll stop going crazy over that... EDIT Well, I tried these:

Code:
ICX7450-32ZP Router(config)#interface ethernet 1/1/25 to 1/1/32
ICX7450-32ZP Router(config-mif-1/1/25-1/1/32)#speed auto
This CLI operation is not applicable to 2.5G port 1/1/25
This CLI operation is not applicable to 2.5G port 1/1/26
This CLI operation is not applicable to 2.5G port 1/1/27
This CLI operation is not applicable to 2.5G port 1/1/28
This CLI operation is not applicable to 2.5G port 1/1/29
This CLI operation is not applicable to 2.5G port 1/1/30
This CLI operation is not applicable to 2.5G port 1/1/31
This CLI operation is not applicable to 2.5G port 1/1/32
ICX7450-32ZP Router(config-mif-1/1/25-1/1/32)#speed-duplex 1000-full

This DID in fact work, but I want to use these are 2.5 GbE anyway.

ICX7450-32ZP Router(config-mif-1/1/25-1/1/32)#speed-duplex 2500-full

So I tried going to auto from 1000-full and failed.  Reset to 2.5GbE worked fine though.
 
Last edited:

MTFPLEX

New Member
Apr 28, 2024
6
0
1
You’re very dedicated, to be posting here from heaven ;)
I don't think I touched anything high voltage (120v), instead I think I hit something around 12v as my hand kinda just felt funny for a few seconds and felt fine after removing contact.
 

CIR-Engineering

I am a functional adult?
Jan 14, 2021
150
58
28
48
Chicago USA
www.cir-engineering.com
I don't think I touched anything high voltage (120v), instead I think I hit something around 12v as my hand kinda just felt funny for a few seconds and felt fine after removing contact.
I’m not a doctor, but unless you have a pre existing condition, a zap from 120AC or lower is probably not an issue. I was going to say however that you are indeed writing here so I don’t think you were “electrocuted.”

I work with high voltage regularly, up to 35k VDC. You need to always be cautious with electricity. With high voltage you always use your right hand and keep your left in your back pocket. That way if you get hit, at least you won’t have a path directly through your heart.

Whatever voltage got you was above 12V. You can’t even feel 12V unless you discharge a large capacitor somehow or water is involved (put a 9-volt battery on your tongue is an old way to test the battery).

I’ve seen two bad accidents; I saw a man indeed get electrocuted from 660VAC. Then in collage we worked on very early hybrid cars. I saw a kid get hit with a battery and to this day he has heart issues from it.

Be careful man.
 
Last edited:
  • Like
Reactions: MTFPLEX

CIR-Engineering

I am a functional adult?
Jan 14, 2021
150
58
28
48
Chicago USA
www.cir-engineering.com
check page 45 in fastiron-08095-managementguide.pdf from the zip of my guide. on newer OS versions I believe that grouping has changed per the example footnote on that page:

In ICX 7450-32ZP, 2.5G ports can be configured only in pairs or set of pairs e.g. (1/1/25 -1/1/26) (1/1/27 -1/1/28) (1/1/25-1/1/32) etc

I'm also pretty sure this earlier implementation of 2.5gbE (before the IEEE standard for it was even finished) does not support auto-neg at all, which is why it's telling you "not applicable" when you tried to set it to auto. it also states that they are indeed set by default to 2500-full, so that's why they weren't linking with your other stuff:

"On ICX 7450-32ZP, default speed is 2500-full on MultiGig ports"

so speed-duplex 1000-full should solve your issue regarding linking to other normal 1gbe stuff

I'd also ignore the PoE error if it just happens once during boot but works fine. It could be triggered by having devices plugged into the ports that have strange resistance on the pairs and do something to the PoE controller before the link is properly negotiated after fully booting. Try booting it with nothing plugged into it at all if you're curious
Last night I did try booting up with nothing plugged into any Ethernet port, just serial, and it does still throw the error. Thoughts? Should I keep it? I also checked all pins in all RJ45 connectors and also blew them out with the air compressor.
One more thing and thanks again for chiming in fohdeesha, you are the man! I don't like that the error is intermittent. I can't find a pattern to it. This morning, booted with no error on the PoE restart. Do you think that's significant?
 
Last edited:

donedeal19

Member
Jul 10, 2013
46
15
8
Run a system diagnostics test, dm diag and dm alt-diag. This should run a full diagnostics on reboot. Watch the status led lights as the test will runs link speeds on every port. And then, show environment fan, power, and sensor for health monitoring.
 
  • Like
Reactions: CIR-Engineering

CIR-Engineering

I am a functional adult?
Jan 14, 2021
150
58
28
48
Chicago USA
www.cir-engineering.com
Run a system diagnostics test, dm diag and dm alt-diag. This should run a full diagnostics on reboot. Watch the status led lights as the test will runs link speeds on every port. And then, show environment fan, power, and sensor for health monitoring.
CRAP o_O

What does this even mean?

Code:
1/2/2   374945528       374945528       0               ##PASS
1/2/3   371165213       371165213       0               ##PASS
1/2/4   366856907       366856907       0               ##PASS
1/3/1   [  409.240204] [BrcdSoftlockup]: Tracker queued..
416501837       416501837       0               ##PASS
1/4/1   419099154       419099154       0               ##PASS


FATAL ERROR: System did not pass diagnosis
SYSTEM STOPPED, Please Check with Ruckus Networks Support
Typing 'r' would reset the system
##FAILED DIAG
[  411.030781] [BrcdSoftlockup]: sim_softwatchdog thread is detached on core=1
I've also been seeing this during boot and now wonder if it is significant?

Code:
                        SOC unit 0 attached to PCI device BCM56548_A0
0:soc_do_init: soc_do_init: total cpu and arm cosq 0030 unexpected
Initializing the Port pair for the P2 board
u=0 p=50 Using SDK PHY driver for BCM84756/7/9
u=0 p=51 Using SDK PHY driver for BCM84756/7/9
Setting the CMIC registers for the link scan
SOC unit 1 attached to PCI device BCM56548_A0
1:soc_do_init: soc_do_init: total cpu and arm cosq 0030 unexpected
u=1 p=42 Using SDK PHY driver for BCM84756/7/9
u=1 p=43 Using SDK PHY driver for BCM84756/7/9
Setting the CMIC registers for the link scan
Info: SendEventCommon: scp task Not inited by FI
Sigh. I've wanted this switch for so long....:(
 

R3Z3N

New Member
Jan 29, 2024
29
6
3
^Well, kinda like dating a dime if the mental stability isn't there. Do you REALLY want it, and if so how long can you last before you lose it...

Just forget you found a broken model, and go find what works.