Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

infoMatt

Active Member
Apr 16, 2019
222
100
43
Is my ICX6450-48P salvageable? I checked the caps on the powersupply and a few other components but didn't really find anything wrong. There's about three fuses on the PSU hidden behind some heat shrinking tube.

POE doesnt work and throws the following error:

ICX6450-48P Router(config)#PoE Info: Resetting module in slot 1....completed.
PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
Resetting module in slot 1 again to recover from dev fault
Partially. Most likely a static discharge or power surge on one of the ethernet cables has fried the PoE controller chips on the board... I've seen some photos in which the chips have a hole punched through the plastic moulding of their cases :eek:
The best case scenario for you is to open it and remove the PoE daughter board, and use it as a non-PoE unit :( unless you can return it to your seller and make it swap for you with another switch...

99,99% the PoE part has to be deemed non recoverable :(

I also noticed a strange issue with the switch where if e1/1/45 to 1/1/48 aren't shut down the switch seems to 'lag' or get stuck and it reflects that latency in the pings sent to the switch

Reply from 10.111.122.222: bytes=32 time=471ms TTL=63
Reply from 10.111.122.222: bytes=32 time=1ms TTL=63
Reply from 10.111.122.222: bytes=32 time=1ms TTL=63
Reply from 10.111.122.222: bytes=32 time=466ms TTL=63
This is VERY strange, it's the first time I see something like this... are you sure that the lags go away as you shut those two interfaces? Have you got anything plugged on those ports?
Are you pinging a host on your network or the gateway address of the switch itself (ie. the VE ip address)?
 

mythosmc

New Member
Apr 1, 2020
3
0
1
100% positive on the shutting the ports down thing because i noticed the same latency while consoled in! I can record a video if you'd like =)

I slowly incremented the disabling across all ports.. i.e. int e 1/1/1 to 1/1/20 ->disa then 21 to 30 until i got to those last 4 ports


Looks like you're right my POE card may be toast.. I didn't see it until I used a black light.

 
Last edited:

infoMatt

Active Member
Apr 16, 2019
222
100
43
100% positive on the shutting the ports down thing because i noticed the same latency while consoled in! I can record a video if you'd like =)

I slowly incremented the disabling across all ports.. i.e. int e 1/1/1 to 1/1/20 ->disa then 21 to 30 until i got to those last 4 ports
No, I believe you! I was just asking because to me it seemed very very strange... Well, we might need to call out for @fohdeesha, as personally I have no clue on what might be causing a latency so high on those ports...
BTW, are these interfaces plugged to something?
To me what seems strange is the latency you're experiencing on the console... it should be completely separated from the data plane.

Those latencies are measured from PC to switch or from PC to another device of the network?

I think that these delays might be caused by the bad PoE board... it might be trying to poll some power statistics or status on those interfaces and hangs by the non responsive fryed chips... Would you mind trying again with the PoE daughter board disconnected?
 

mythosmc

New Member
Apr 1, 2020
3
0
1
No, I believe you! I was just asking because to me it seemed very very strange... Well, we might need to call out for @fohdeesha, as personally I have no clue on what might be causing a latency so high on those ports...
BTW, are these interfaces plugged to something?
To me what seems strange is the latency you're experiencing on the console... it should be completely separated from the data plane.

Those latencies are measured from PC to switch or from PC to another device of the network?

I think that these delays might be caused by the bad PoE board... it might be trying to poll some power statistics or status on those interfaces and hangs by the non responsive fryed chips... Would you mind trying again with the PoE daughter board disconnected?
Sure, i'll give it a shot now - only port that has anything plugged into it is mgmt1

Same results with the POE card out:

 
Last edited:

infoMatt

Active Member
Apr 16, 2019
222
100
43
Ok, I'm out of ideas :(

Just for the sake of completeness: mgmt port is connected directly to the CPU, it is not part of the switched ports... maybe it causes some delay? I've onestly not tryed it yet... I might give it a shot tomorrow (now in Italy it's 1:30 AM...) on my 6450 24P, basically because now I'm curious :p
In any case, it doesn't explain why when you shut those ports the latencies go down...
 
  • Like
Reactions: fohdeesha

fohdeesha

Kaini Industries
Nov 20, 2016
2,728
3,078
113
33
fohdeesha.com
Sure, i'll give it a shot now - only port that has anything plugged into it is mgmt1

Same results with the POE card out:

that's indeed not normal and I can't reproduce it here. Can you paste the output of "show run" and "show version", and double-confirm nothing is plugged into those ports?
 

epicurean

Active Member
Sep 29, 2014
785
80
28
Is there any any down side to enabling POE on all the ports of my ICX 6450-48 POE, instead of just the ones that actually need POE?
 

infoMatt

Active Member
Apr 16, 2019
222
100
43
Is there any any down side to enabling POE on all the ports of my ICX 6450-48 POE, instead of just the ones that actually need POE?
Not that I can think of, but il will complain at boot time that it doesn't have enough power budget to enable PoE on the high gruup of ports (ot at least, the one with less priority if you've configured it, otherwise, the lower the interface number, the more priority it gets in case of power budget exhaustion).

It will honor any PD request even on those, unless, as stated, other devices on the interface with higher priority consume all the available budget.
 

Bjorn Smith

Well-Known Member
Sep 3, 2019
876
481
63
49
r00t.dk
Hi,

Quick question.

I have a ICX6450-24 POE - but I am not using any POE and will never get to use it - do I need to do anything special if I remove the POE board?

Thanks
 

epicurean

Active Member
Sep 29, 2014
785
80
28
Not that I can think of, but il will complain at boot time that it doesn't have enough power budget to enable PoE on the high gruup of ports (ot at least, the one with less priority if you've configured it, otherwise, the lower the interface number, the more priority it gets in case of power budget exhaustion).

It will honor any PD request even on those, unless, as stated, other devices on the interface with higher priority consume all the available budget.
And it will also not blow any non POE devices connected to the switch as well?
 

infoMatt

Active Member
Apr 16, 2019
222
100
43
Hi,

Quick question.

I have a ICX6450-24 POE - but I am not using any POE and will never get to use it - do I need to do anything special if I remove the POE board?

Thanks
I think that unplugging woud be enough... but the real question is: why bother?
The switch will do its job the same even with the board inserted... maybe maybe you can shave off ~1W of power consumption, but is it worth the effort?

And it will also not blow any non POE devices connected to the switch as well?
Absolutely not.
Those ports are 802.3af/at compliant, meaning that they will perform a so called "PD Detection" wherever a cable is plugged in; if the devices doesn't match the requirement for a powered device (if memory serves me, a specific resistance value between two pairs), it won't supply any power... you can unplug a suveillance camera and plug in a computer without touching anything config-wise, and it won't melt your PC network card. ;)
 
  • Like
Reactions: epicurean

Bjorn Smith

Well-Known Member
Sep 3, 2019
876
481
63
49
r00t.dk
maybe maybe you can shave off ~1W of power consumption
Any watt saved is worth the effort - both from an economical point of view - I pay $0.35 per KW - but also from an environmental point of view - if its not needed and easily removed, its worth it :)

Furthermore it might make my switch run just a teeny bit cooler :)

I know money wise it will take a while before I can buy a cup of cofee, but every bit counts :)
 

infoMatt

Active Member
Apr 16, 2019
222
100
43
Ok, I'm out of ideas :(

Just for the sake of completeness: mgmt port is connected directly to the CPU, it is not part of the switched ports... maybe it causes some delay? I've onestly not tryed it yet... I might give it a shot tomorrow (now in Italy it's 1:30 AM...) on my 6450 24P, basically because now I'm curious :p
In any case, it doesn't explain why when you shut those ports the latencies go down...
Just tryed it... I was thinking of it, but my mind the last night didn't connect all the dots...

MGMT1 is a CPU port, but it cannot be added on any VLAN; it can have an IP (v4 and/or v6), but it cannot overlap any subned defined on other VE interfaces... and, it won't route traffic to any switched VLAN.

Any watt saved is worth the effort - both from an economical point of view - I pay $0.35 per KW - but also from an environmental point of view - if its not needed and easily removed, its worth it :)

Furthermore it might make my switch run just a teeny bit cooler :)

I know money wise it will take a while before I can buy a cup of cofee, but every bit counts :)
I know, I live in Italy where energy (for private citizen at least) has one of the higher cost of all European countries, but probably the most energy hogging part of the PoE when idle is the +48V rail of the PSU, rather than the board itself.
The stock fans alone when running at low speed account for ~3-4W at the plug.... you should run fanless then :p
In every house there are far far more energy-wasting devices that can be turn off or unplugged... I tried with a power meter, an old compact stereo (early '90s) was drawing non-stop 10-11W (with a ridiculous power factor of 0.2) just being on standby mode...

You can try it, if you have a power meter you can evaluate the difference, but I doubt it would be even measurable... looking at a mere energy consumption standpoint, you should have bought the -24, rather than the -24P... It has even no fans (or just one, I can't remember at the moment).
 
Last edited:

fohdeesha

Kaini Industries
Nov 20, 2016
2,728
3,078
113
33
fohdeesha.com
no special requirements when removing the PoE board, just remove it and plug the switch back in. I'm sure you could sell it to one of the members here who got fried PoE boards if you want to deal with shipping (would need to be the same port count switch)
 
  • Like
Reactions: tommybackeast

Wolfstar

Active Member
Nov 28, 2015
159
83
28
48
Ok, I'm out of ideas :(

Just for the sake of completeness: mgmt port is connected directly to the CPU, it is not part of the switched ports... maybe it causes some delay? I've onestly not tryed it yet... I might give it a shot tomorrow (now in Italy it's 1:30 AM...) on my 6450 24P, basically because now I'm curious :p
In any case, it doesn't explain why when you shut those ports the latencies go down...
Keep in mind that - in all likelihood - the ASIC driving those ports is the same ASIC. It's typical to have 4 ports to an ASIC on gigabit switches. @mythosmc if you've ever tried to apply QoS on four ports of a Layer 3 switch and seen it apply to three others, that's why.

If I had to guess, I'd say that the ASIC on those four ports is bad and it's causing issues, possibly flooding the TCAM with garbage. If you enable those ports, ping the switch, then disable them and check the logs (show log), what are you seeing between the port enable and disable?

It's entirely possible too that whatever fried the PoE board on that switch came through one of those four ports, and managed to kill the ASIC at the same time.
 

infoMatt

Active Member
Apr 16, 2019
222
100
43
Keep in mind that - in all likelihood - the ASIC driving those ports is the same ASIC. It's typical to have 4 ports to an ASIC on gigabit switches. @mythosmc if you've ever tried to apply QoS on four ports of a Layer 3 switch and seen it apply to three others, that's why.

If I had to guess, I'd say that the ASIC on those four ports is bad and it's causing issues, possibly flooding the TCAM with garbage. If you enable those ports, ping the switch, then disable them and check the logs (show log), what are you seeing between the port enable and disable?

It's entirely possible too that whatever fried the PoE board on that switch came through one of those four ports, and managed to kill the ASIC at the same time.
Looking at the board, it seems that there are 6 chips (in case of the 24 port) near the ports, so your theory might be correct... Just looking, without a schematic, I cannot say however if those are switch chips or simply ethernet interfaces frontend to RGMII/SerDes connection to the main switch chip.

In every case, it can't explain the lag on the console configuration, that is managed by a completely different processor than the data plane.
 

CorvetteGS

Member
Jan 20, 2014
40
5
8
Atlanta, GA
Hi everyone, I'm looking to utilize my ICX 7150 as a layer 3 router and moving all inter-vlan routing off my edgerouter for my home network. I was wondering if anyone has any experience with this in regards to IPv6? My current configuration on the edgerouter uses DHCPv6-PD to request a /60 prefix from my ISP and distributes the prefixes to my various VLAN interfaces using SLAAC.

I assume I have to use DHCPv6-PD still on my edgerouter to request the /60 prefix from the ISP as it would be the only device with the WAN interface, but if I am using a transit VLAN to connect to the ICX, how would I distribute the various prefixes amongst my VLANs? Thanks in advance for any help provided!
 

epicurean

Active Member
Sep 29, 2014
785
80
28
no special requirements when removing the PoE board, just remove it and plug the switch back in. I'm sure you could sell it to one of the members here who got fried PoE boards if you want to deal with shipping (would need to be the same port count switch)
Does that mean POE module for 24P wont work for 48p?