Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[fohdeesha]

Just seeing who's paying attention

 

[fohdeesha]

I will have some actual (private) good news in a week or so, for anyone with an MLX/CER/VDX

 

[itronin]

ok, here's how to enable icx6610 fan swaps. Connected over serial, enter the OS console using the key command (ctrl + y, let go, press m). Once in console run the following commands:

OS>toggle aprilfools 1
OS>thiswillliterallyneverwork
OS>unlessthelawsofthermodynamics
OS>vastlychange

I should not eat cereal on the morning of April 2. the milk came out of my nose... too effing funny.

 

[tommybackeast]

ok, here's how to enable icx6610 fan swaps. Connected over serial, enter the OS console using the key command (ctrl + y, let go, press m). Once in console run the following commands:

OS>toggle aprilfools 1
OS>thiswillliterallyneverwork
OS>unlessthelawsofthermodynamics
OS>vastlychange

well played, very well done...

 

[infoMatt]

Is my ICX6450-48P salvageable? I checked the caps on the powersupply and a few other components but didn't really find anything wrong. There's about three fuses on the PSU hidden behind some heat shrinking tube.

POE doesnt work and throws the following error:

ICX6450-48P Router(config)#PoE Info: Resetting module in slot 1....completed.
PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
Resetting module in slot 1 again to recover from dev fault

Partially. Most likely a static discharge or power surge on one of the ethernet cables has fried the PoE controller chips on the board... I've seen some photos in which the chips have a hole punched through the plastic moulding of their cases
The best case scenario for you is to open it and remove the PoE daughter board, and use it as a non-PoE unit unless you can return it to your seller and make it swap for you with another switch...

99,99% the PoE part has to be deemed non recoverable

I also noticed a strange issue with the switch where if e1/1/45 to 1/1/48 aren't shut down the switch seems to 'lag' or get stuck and it reflects that latency in the pings sent to the switch

Reply from 10.111.122.222: bytes=32 time=471ms TTL=63
Reply from 10.111.122.222: bytes=32 time=1ms TTL=63
Reply from 10.111.122.222: bytes=32 time=1ms TTL=63
Reply from 10.111.122.222: bytes=32 time=466ms TTL=63

This is VERY strange, it's the first time I see something like this... are you sure that the lags go away as you shut those two interfaces? Have you got anything plugged on those ports?
Are you pinging a host on your network or the gateway address of the switch itself (ie. the VE ip address)?

 

[mythosmc]

100% positive on the shutting the ports down thing because i noticed the same latency while consoled in! I can record a video if you'd like =)

I slowly incremented the disabling across all ports.. i.e. int e 1/1/1 to 1/1/20 ->disa then 21 to 30 until i got to those last 4 ports


Looks like you're right my POE card may be toast.. I didn't see it until I used a black light.

 

[infoMatt]

100% positive on the shutting the ports down thing because i noticed the same latency while consoled in! I can record a video if you'd like =)

I slowly incremented the disabling across all ports.. i.e. int e 1/1/1 to 1/1/20 ->disa then 21 to 30 until i got to those last 4 ports

No, I believe you! I was just asking because to me it seemed very very strange... Well, we might need to call out for @fohdeesha, as personally I have no clue on what might be causing a latency so high on those ports...
BTW, are these interfaces plugged to something?
To me what seems strange is the latency you're experiencing on the console... it should be completely separated from the data plane.

Those latencies are measured from PC to switch or from PC to another device of the network?

I think that these delays might be caused by the bad PoE board... it might be trying to poll some power statistics or status on those interfaces and hangs by the non responsive fryed chips... Would you mind trying again with the PoE daughter board disconnected?

 

[mythosmc]

No, I believe you! I was just asking because to me it seemed very very strange... Well, we might need to call out for @fohdeesha, as personally I have no clue on what might be causing a latency so high on those ports...
BTW, are these interfaces plugged to something?
To me what seems strange is the latency you're experiencing on the console... it should be completely separated from the data plane.

Those latencies are measured from PC to switch or from PC to another device of the network?

I think that these delays might be caused by the bad PoE board... it might be trying to poll some power statistics or status on those interfaces and hangs by the non responsive fryed chips... Would you mind trying again with the PoE daughter board disconnected?

Sure, i'll give it a shot now - only port that has anything plugged into it is mgmt1

Same results with the POE card out:

 

[infoMatt]

Ok, I'm out of ideas

Just for the sake of completeness: mgmt port is connected directly to the CPU, it is not part of the switched ports... maybe it causes some delay? I've onestly not tryed it yet... I might give it a shot tomorrow (now in Italy it's 1:30 AM...) on my 6450 24P, basically because now I'm curious
In any case, it doesn't explain why when you shut those ports the latencies go down...

 

[fohdeesha]

Sure, i'll give it a shot now - only port that has anything plugged into it is mgmt1

Same results with the POE card out:

that's indeed not normal and I can't reproduce it here. Can you paste the output of "show run" and "show version", and double-confirm nothing is plugged into those ports?

 

[epicurean]

Is there any any down side to enabling POE on all the ports of my ICX 6450-48 POE, instead of just the ones that actually need POE?

 

[infoMatt]

Is there any any down side to enabling POE on all the ports of my ICX 6450-48 POE, instead of just the ones that actually need POE?

Not that I can think of, but il will complain at boot time that it doesn't have enough power budget to enable PoE on the high gruup of ports (ot at least, the one with less priority if you've configured it, otherwise, the lower the interface number, the more priority it gets in case of power budget exhaustion).

It will honor any PD request even on those, unless, as stated, other devices on the interface with higher priority consume all the available budget.

 

[Bjorn Smith]

Hi,

Quick question.

I have a ICX6450-24 POE - but I am not using any POE and will never get to use it - do I need to do anything special if I remove the POE board?

Thanks

 

[epicurean]

Not that I can think of, but il will complain at boot time that it doesn't have enough power budget to enable PoE on the high gruup of ports (ot at least, the one with less priority if you've configured it, otherwise, the lower the interface number, the more priority it gets in case of power budget exhaustion).

It will honor any PD request even on those, unless, as stated, other devices on the interface with higher priority consume all the available budget.

And it will also not blow any non POE devices connected to the switch as well?

 

[infoMatt]

Hi,

Quick question.

I have a ICX6450-24 POE - but I am not using any POE and will never get to use it - do I need to do anything special if I remove the POE board?

Thanks

I think that unplugging woud be enough... but the real question is: why bother?
The switch will do its job the same even with the board inserted... maybe maybe you can shave off ~1W of power consumption, but is it worth the effort?

And it will also not blow any non POE devices connected to the switch as well?

Absolutely not.
Those ports are 802.3af/at compliant, meaning that they will perform a so called "PD Detection" wherever a cable is plugged in; if the devices doesn't match the requirement for a powered device (if memory serves me, a specific resistance value between two pairs), it won't supply any power... you can unplug a suveillance camera and plug in a computer without touching anything config-wise, and it won't melt your PC network card.

 

[Bjorn Smith]

maybe maybe you can shave off ~1W of power consumption

Any watt saved is worth the effort - both from an economical point of view - I pay $0.35 per KW - but also from an environmental point of view - if its not needed and easily removed, its worth it

Furthermore it might make my switch run just a teeny bit cooler

I know money wise it will take a while before I can buy a cup of cofee, but every bit counts

 

[infoMatt]

Ok, I'm out of ideas

Just for the sake of completeness: mgmt port is connected directly to the CPU, it is not part of the switched ports... maybe it causes some delay? I've onestly not tryed it yet... I might give it a shot tomorrow (now in Italy it's 1:30 AM...) on my 6450 24P, basically because now I'm curious
In any case, it doesn't explain why when you shut those ports the latencies go down...

Just tryed it... I was thinking of it, but my mind the last night didn't connect all the dots...

MGMT1 is a CPU port, but it cannot be added on any VLAN; it can have an IP (v4 and/or v6), but it cannot overlap any subned defined on other VE interfaces... and, it won't route traffic to any switched VLAN.

Any watt saved is worth the effort - both from an economical point of view - I pay $0.35 per KW - but also from an environmental point of view - if its not needed and easily removed, its worth it

Furthermore it might make my switch run just a teeny bit cooler

I know money wise it will take a while before I can buy a cup of cofee, but every bit counts

I know, I live in Italy where energy (for private citizen at least) has one of the higher cost of all European countries, but probably the most energy hogging part of the PoE when idle is the +48V rail of the PSU, rather than the board itself.
The stock fans alone when running at low speed account for ~3-4W at the plug.... you should run fanless then
In every house there are far far more energy-wasting devices that can be turn off or unplugged... I tried with a power meter, an old compact stereo (early '90s) was drawing non-stop 10-11W (with a ridiculous power factor of 0.2) just being on standby mode...

You can try it, if you have a power meter you can evaluate the difference, but I doubt it would be even measurable... looking at a mere energy consumption standpoint, you should have bought the -24, rather than the -24P... It has even no fans (or just one, I can't remember at the moment).

 

[Bjorn Smith]

ooking at a mere energy consumption standpoint, you should have bought the -24, rather than the -24P

I know But the -P was cheaper

 

[fohdeesha]

no special requirements when removing the PoE board, just remove it and plug the switch back in. I'm sure you could sell it to one of the members here who got fried PoE boards if you want to deal with shipping (would need to be the same port count switch)

 

[fohdeesha]

also big thanks to @infoMatt for answering so many questions in here, they're even the right answers to top it off!