[Solved] Brocade ICX6610-48P PoE Errors

KFJ

New Member
Mar 11, 2019
10
5
3
I bought a used ICX-6610 from eBay but it turns out the PoE board was shot (had a hole in an IC and a blown transistor on the PoE board). Despite this, I was able to update the firmware and image and PoE firmware but the error persisted and of course the error blocked the output of commands like 'show chassis'.

The switch was just displaying the following in a loop:

Code:
PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
Resetting module in slot 1 again to recover from dev fault
The seller graciously sent another used PoE board which I installed yesterday. The switch powers on and I can get a console, but the console doesn't load past "Enter 'b' to stop at boot monitor" regardless if I press b per usual or not. The switch never fully boots.

Any tips or tricks to reflash the PoE board or reset the switch from the current state? I know how to reflash when the switch actually boots, but I have no idea what to do if pressing 'b' does nothing or if I cannot get a prompt.

I still have the old PoE board - at least that let me boot the switch, but I'd kinda like to have the PoE working, but it is optional (I'll have to get another PoE switch).
 

BoredSysadmin

Active Member
Mar 2, 2019
625
213
43
Have you tried to update the switch f/w: FCX / ICX6610 - Fohdeesha Docs ?
pinging @fohdeesha

PoE f/w updating is curious. It's detailed in the guide, but you should know that it happens in the background and takes 10-15 mins 6610 and the only way to check the progress is to do a "show log"
 

KFJ

New Member
Mar 11, 2019
10
5
3
Have you tried to update the switch f/w: FCX / ICX6610 - Fohdeesha Docs ?
pinging @fohdeesha

PoE f/w updating is curious. It's detailed in the guide, but you should know that it happens in the background and takes 10-15 mins 6610 and the only way to check the progress is to do a "show log"
Thanks boredSysadmin.

Yes, I did update the switch, including the PoE firmware using @Fodeesha's docs. However, there was a blown transistor on the original PoE board and an IC with a burn hole in it (not me, bought used). Surprisingly, the switch was functional and I was able to run commands even with the blown board - but the console was flooded with the PoE errors I posted above and of course the PoE did not supply any Power over Ethernet due to the blown board. I configured the switch and was using it with a couple of dev laptops and all was fine except no PoE.

So I replaced the PoE board and when I power on the switch and hook a laptop to the console port with Putty I get the familiar console prompt and I can press 'i' or 'b' but then nothing happens, the switch does absolutely nothing new: displays the press 'i' and press 'b' prompts and nothing else but fans. If I don't press 'i' or 'b' then the switch still does nothing (except keep the fans on full blast). I have had it powered on for 2 hours and the console never updates past press 'i' or 'b'.

My thinking is the 'new' PoE board needs to be flashed to the latest PoE firmware version as was the 'old' damaged board that I had updated the PoE firmware on. But now I can't flash anything without a functioning switch or console - the console is not doing anything now, just displaying "press 'b' for boot" no matter if I press 'b' or not.
 

Dreece

Active Member
Jan 22, 2019
506
158
43
you still holding it @BoredSysadmin ?

reading about all these problems with the poe brocades just warrants a sticky on this sub-forum "Health Warning - Check used POE Brocades before purchasing"
 

BoredSysadmin

Active Member
Mar 2, 2019
625
213
43
I've bought a used Brocade 6610-24P PoE from the bay and so far no issues to report. I've witnessed fohdeesha performed small miracles with these switches I and I'm a believer that he'll guide KFJ towards salvation fixing his switch
 
  • Like
Reactions: Dreece

Dreece

Active Member
Jan 22, 2019
506
158
43
It isn't so much to do with being fixable, just people should be aware that there are known concerns regarding used PoE Brocades and if people arent into ripping open hardware then they should check their potential purchase over before handing over the cash.
 
  • Like
Reactions: fohdeesha

KFJ

New Member
Mar 11, 2019
10
5
3
I shutdown the switch, and removed the 'new' PoE board. Same issue as before - console won't load past "Enter 'b'" or "Interrupted by entering 'b'" I let it each attempt (not pressing b and pressing b) run for 20 minutes - no change just fan at full blast.

I then shutdown the switch and inserted the 'old' PoE board - same issue as before - console won't load past "Enter 'b'" or "Interrupted by entering 'b'" .

I think she might be bricked :(

Attached image shows the three attempts separated by about 20 minutes each. The first two are with the 'new' PoE board removed. The last one is with the new PoE board.
 

Attachments

BoredSysadmin

Active Member
Mar 2, 2019
625
213
43
I assume that you're pressing b really really quickly? it shows up in prompt very soon after power on and the window is easy to miss.
but I guess you already know this since you've upgraded it per the guide.
 

KFJ

New Member
Mar 11, 2019
10
5
3
I assume that you're pressing b really really quickly? it shows up in prompt very soon after power on and the window is easy to miss.
but I guess you already know this since you've upgraded it per the guide.
Yes, in the attached image it shows 'Interrupted by pressing b'. Prior to installing the 'new' board the switch would boot per usual (without pressing 'b') but throw a loop of PoE errors (every 1 to 10 seconds) and I could press 'b' and enter the boot prompt (at startup only of course). I configured the switch, it could reach my router and the network but I wanted PoE to work (as advertised and as paid for), so I contacted the seller, we agreed on replacing the board after I opened up the switch and saw the blown transistor and burnt IC. The first time I booted the switch with only the console port used I received the PoE errors and did not smell smoke, I do not believe it is anything I did. Regardless of the constant PoE errors I was able to update the firmware and image and the PoE firmware to latest (803t?) and the switch was usable other than the PoE errors. After I replaced the board, the switch will not boot past the 'Enter 'b' prompt if I do not press 'b' or 'Interrupted by pressing 'b' if I do press 'b'.

I am going to do a close inspection of the other boards to see if I missed something or if I accidentally left a shovel on the main board or something likely non-obvious.

Edit: NICE! I noticed the RAM wasn't fully seated, so I re-seated it. Maybe I nudged it, but I am now in the boot monitor! Gonna try the new PoE board in a few minutes.
 

KFJ

New Member
Mar 11, 2019
10
5
3
PIR: Replacing the PoE board resolved the PoE errors experienced with the original PoE board (see attached). The issue with the switch not booting with the new board (or any board or no board) was due to the DRAM module not being fully seated, likely caused by OP when removing the connector from the bottom of the original PoE board. Reseating the DRAM module resolved the issue and the switch is currently running with the configuration and firmware updates applied prior to replacing the PoE board.

Thank you STH members for your help and insights. I guess it wasn't a shovel or a brick after all.
 

Attachments

BoredSysadmin

Active Member
Mar 2, 2019
625
213
43
It isn't so much to do with being fixable, just people should be aware that there are known concerns regarding used PoE Brocades and if people arent into ripping open hardware then they should check their potential purchase over before handing over the cash.
Looks like not all Brocade PoE switches are to blame for issues after all :)
 
  • Like
Reactions: Dreece

KFJ

New Member
Mar 11, 2019
10
5
3
it's a RAM stick just like it sounds (well, laptop style DDR2 ECC if I remember right), here's a spare pic I had: https://i.imgur.com/l8QLkoH.jpg
Exactly. Thanks @fohdeesha. That is the RAM near the middle of your pic.

I had noticed the DRAM wasn't seated properly because the latch that is supposed to hold the DIMM in place was not in the correct position and one side of the DIMM was showing like 1/16th of an inch more pins. Simply applying pressure reset the DRAM. I have no idea if it was loose before or not and I just made it looser or not, but that was one of the issues.

In case someone has a similar issue with the PoE errors and has a replacement board: the PoE board is the larger one near the bottom of the pic with the wire harness in the middle. I recommend removing the 4 screws holding the PoE board down, and carefully prying/lifting the board from multiple positions prior to removing the wire harness from the bottom of the PoE board. There are 4 sets of double-ended header pins that are about 1 inch long on the bottom side near the bottom edge of the board that need to not get bent when removing the PoE board. The header pins are not soldered down on either end, just loosely fit is enough I guess.