Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[ms264556]

If they decide to do something about it they should send him an awesome giftbasket, he sold me on ruckus assces points and icx switches

Me too! I bought 2 ICX 7150s based 100% on the availability of his documentation.

 

[kpfleming]

I also bought (used) 7150s after finding the helpful information here. In addition, I actually contacted Ruckus to see if I could purchase a support contract so that I'd have a proper channel for reporting issues (their web forum is not that), and I was told they won't sell a support contract except for used equipment. Seems like a silly restriction since I'm not asking for them to take responsibility for the operation of the hardware, only the software, but it's their choice.

 

[Nardokor]

Hey everyone,

I'm pretty new to the higher end of networking. I bought an ICX6610 48 port a few months ago, one power supply and one extra fan if it matters. It worked fine until the other day when it turned off and won't finish booting. On applying power, the two fans ramp to full speed and the orange LEDs in the front SFP ports light up. The data below comes out of the serial monitor port at the same time. I have some idea of what this means, but not if it's likely to be fixable. Are there any other troubleshooting steps that could give me more information or am I better off just looking for a new one?

Thanks in advance.

$
ICX Boot Code Version 10.1.00 (grz10100)
Enter 'a' to stop at memory test
Enter 'b' to stop at boot monitor
BOOT INFO: load monitor from boot flash, cksum = 71f1
BOOT INFO: verify flash files.......

Exception Type 0100 (Critical Input), $con
02029030(msr)
00000000(dar)
00800000(esr)
000a9120(pc)
0008f8b0(lr)
0008e9e4
00041be8
00040118
End of Trace
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abo▒$
ICX Boot Code Version 10.1.00 (grz10100)
Enter 'a' to stop at memory test
Enter 'b' to stop at boot monitor
BOOT INFO: load monitor from boot flash, cksum = 71f1
BOOT INFO: verify flash files.......

Exception Type 0100 (Critical Input), $con
02029030(msr)
00000000(dar)
00800000(esr)
000a9120(pc)
0008f8b0(lr)
0008e9e4
00041be8
00040118
End of Trace
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abort
Error flash copy, err#: 7
4. Abo▒$
ICX Boot Code Version 10.1.00 (grz10100)
Enter 'a' to stop at memory test
Enter 'b' to stop at boot monitor

 

[ms264556]

I actually contacted Ruckus to see if I could purchase a support contract so that I'd have a proper channel for reporting issues (their web forum is not that)

They provide so little support to their web forum that it's a complete waste of time.
Better to ask here or reddit so you can be directed at resources like Fohdeesha's pages, rather than get ignored (most likely) or be told to buy a new switch/AP.
Their latest forum "upgrade" doesn't even let you sort by active discussions. Useless.

(I also tried and failed to buy a support contract, but this is normal for enterprise gear.)

 

[klui]

I bought an ICX6610 48 port a few months ago, one power supply and one extra fan if it matters. It worked fine until the other day when it turned off and won't finish booting.

Go to the first page in this thread and follow the setup procedure from the top for your switch. If that doesn't work your switch's flash is probably toast.

 

[Lone Wolf]

Hi there!

It looks like I've joined the family! Two Tuesdays ago I saw a link to this thread on Reddit. I spent some time reading through this thread then immediately bought an ICX6450-48P off of ebay! I got it for the low price of $139 Canadian. No complaints there! What drew me to this particular switch was the 10gbE ports, the PoE, and the comment that the fan is "nearly silent" since this will be in my living room for now. I would prefer the 24 port version but none were available so 48 port it is!

It arrived by the Friday. It was in almost perfect condition save for a slight scratch across the top of the switch. Very clean inside too, components looked in good shape. The date stamped on the inside of the case was September 2012. I turned it on, and this thing was anything but "nearly silent" lol. I ended up pulling out my trusty rusty analog Radio Shack decibel meter. At three feet it was 80dB then when the fans spun down it dropped to 74dB! Disconnecting two of the fans it read 70dB at start, dropping to 65dB. Not living room friendly lol. I got a console cable on it and saw that the firmware version was 07.4.00dT31. So I hoped it was due to being an old firmware.

I started working through the guide but found the commands listed didn't work - I assume because of being an older firmware. I dug through the '?' menu until I found the commands that did what was needed, but I was unable to find a command to update the firmware with tftp. I had a look at the webui and saw there was an option to update the Primary with tftp. No option for uboot, so I told it to update the Primary and rebooted. Lo and behold the commands worked as listed. The best part: the fan is a LOT quieter now! I started the guide from the top and updated uboot, then went on to updating the PoE firmware and all the rest, and now I have a fully working, fully licensed 6450-48P. This is awesome! I'm going to learn a lot with this switch!

I've read through the first 100 pages of this thread, and the last 55 so far. I've learned a lot from everyone here, and enjoyed reading your adventures. So felt I should post mine! Fohdeesha, this is an incredible resource and I've contributed to your liquor fund. It's much appreciated!

My next step is to get a replacement fan. The single fan is still too loud for a living room so I'm looking into getting a quieter fan.

I'm also looking into getting a couple 10gbE cards. I was looking at the Mellanox MCX311A cards, but about all I can find on ebay come from China so I'll be waiting a month. Any alternative cards one can recommend that I might actually be able to find?

I also saw a comment to not use heatpaste on the heatsinks? And that the clips break easily? Being a 10 year old switch I thought it might be good to freshen it up, but not if it's going to cause problems.

This has been fun so far, thanks all!

 

[aarcane]

I have not tried physically reversing the fans...but...I do know that the fan tray that connects (the two "tandem" fans each), to the switch has resistors in the wire/connector (on the fan side). That's how it knows if you put in a fan tray that's "front to back" or "back to front". You'll have to hack that as well.

Does anybody know the resister values, or have a guide for physically reversing the fan trays? Or perhaps more importantly, the psu? I could only get I models, but I need E for my rack configuration. I could just swap the fans and leave them reporting I, but that feels wrong and dirty.

 

[Serhan]

Which UPS do you use for models with POE? I tried APC 1500G and Vertiv PSA5 to power ICX7450-24P with no luck. These are 900W units as far as I know. I have three Omada POE APs connected to the switch. I measure 65 W at the wall and also on the UPS, both ups power on, but whenever I disconnect the mains, the switch shuts down.

 

[fohdeesha]

Hi there!

It looks like I've joined the family! Two Tuesdays ago I saw a link to this thread on Reddit. I spent some time reading through this thread then immediately bought an ICX6450-48P off of ebay! I got it for the low price of $139 Canadian. No complaints there! What drew me to this particular switch was the 10gbE ports, the PoE, and the comment that the fan is "nearly silent" since this will be in my living room for now. I would prefer the 24 port version but none were available so 48 port it is!

It arrived by the Friday. It was in almost perfect condition save for a slight scratch across the top of the switch. Very clean inside too, components looked in good shape. The date stamped on the inside of the case was September 2012. I turned it on, and this thing was anything but "nearly silent" lol. I ended up pulling out my trusty rusty analog Radio Shack decibel meter. At three feet it was 80dB then when the fans spun down it dropped to 74dB! Disconnecting two of the fans it read 70dB at start, dropping to 65dB. Not living room friendly lol. I got a console cable on it and saw that the firmware version was 07.4.00dT31. So I hoped it was due to being an old firmware.

I started working through the guide but found the commands listed didn't work - I assume because of being an older firmware. I dug through the '?' menu until I found the commands that did what was needed, but I was unable to find a command to update the firmware with tftp. I had a look at the webui and saw there was an option to update the Primary with tftp. No option for uboot, so I told it to update the Primary and rebooted. Lo and behold the commands worked as listed. The best part: the fan is a LOT quieter now! I started the guide from the top and updated uboot, then went on to updating the PoE firmware and all the rest, and now I have a fully working, fully licensed 6450-48P. This is awesome! I'm going to learn a lot with this switch!

I've read through the first 100 pages of this thread, and the last 55 so far. I've learned a lot from everyone here, and enjoyed reading your adventures. So felt I should post mine! Fohdeesha, this is an incredible resource and I've contributed to your liquor fund. It's much appreciated!

My next step is to get a replacement fan. The single fan is still too loud for a living room so I'm looking into getting a quieter fan.

I'm also looking into getting a couple 10gbE cards. I was looking at the Mellanox MCX311A cards, but about all I can find on ebay come from China so I'll be waiting a month. Any alternative cards one can recommend that I might actually be able to find?

I also saw a comment to not use heatpaste on the heatsinks? And that the clips break easily? Being a 10 year old switch I thought it might be good to freshen it up, but not if it's going to cause problems.

This has been fun so far, thanks all!

thank you for the donation follow my guide linked in the original post here, starting from the beginning, don't skip anything. it won't matter what version your switch comes with, as the first thing it has you do is flash the latest software from the bootloader. If you're getting "commands not found" or missing, you skipped something somewhere. you shouldn't have ever even been in the actual OS yet, you should have been in the bootloader

 

[Freebsd1976]

anyone use icx7150-48zp? which is better icx7250-24p or icx7150-48zp?

 

[Lone Wolf]

thank you for the donation follow my guide linked in the original post here, starting from the beginning, don't skip anything. it won't matter what version your switch comes with, as the first thing it has you do is flash the latest software from the bootloader. If you're getting "commands not found" or missing, you skipped something somewhere. you shouldn't have ever even been in the actual OS yet, you should have been in the bootloader

I did follow your guide command for command, step by step, skipping nothing. Booted the switch with a console cable, smashed 'b' until it dropped me to a prompt. Exactly as written - or I tried. But they were invalid commands. I rebooted the switch a couple of times to be sure I didn't miss the bootloader. I assume because of being on really old firmware? (version 07.4.00dT31 ) For example "setenv" was an unknown command so I couldn't set the ip. "update_primary" wasn't valid either. I used the '?' menu to pull up lists of commands and dug through the menus until I found different commands that let me set the IP. Once I did that, update_primary still didn't work because it was an unknown command but I could do it through the webui. Once I updated the primary and rebooted and smashed 'b' to get to the prompt, the commands in your guide worked exactly as listed and I was able to fully update and set up the switch. The guide was fantastic once I got to newer firmware, and it showed me what I needed to do to get to a newer firmware to be able to use the guide, so I consider it a resounding success! I'm only guessing it's due to older firmware, all I know is that the commands listed were unknown until I got to newer firmware. In the end tho, the switch works exactly as you described and your guide was very clear and easy to follow!

 

[kpfleming]

anyone use icx7150-48zp? which is better icx7250-24p or icx7150-48zp?

It's hard to define 'better' for switches that are dramatically different. If you need 25-48 ports, or multi-gigabit Ethernet, or both, then the 48ZP is clearly 'better' since it has those and the 24P does not.

 

[Vesalius]

anyone use icx7150-48zp? which is better icx7250-24p or icx7150-48zp?

I use an icx7150-48zp and it's exactly as @kpflemming said. It's better for my needs as I wanted multigig POE for my ruckus r750&r650's and 10g in one switch. I also needed few more than 24 ports.

 

[Freebsd1976]

I use an icx7150-48zp and it's exactly as @kpflemming said. It's better for my needs as I wanted multigig POE for my ruckus r750&r650's and 10g in one switch. I also needed few more than 24 ports.

How about The power consumption of Your 7150-48zp ? It seems higher than 7250 in the data sheet

 

[fohdeesha]

I did follow your guide command for command, step by step, skipping nothing. Booted the switch with a console cable, smashed 'b' until it dropped me to a prompt. Exactly as written - or I tried. But they were invalid commands. I rebooted the switch a couple of times to be sure I didn't miss the bootloader. I assume because of being on really old firmware? (version 07.4.00dT31 ) For example "setenv" was an unknown command so I couldn't set the ip. "update_primary" wasn't valid either. I used the '?' menu to pull up lists of commands and dug through the menus until I found different commands that let me set the IP. Once I did that, update_primary still didn't work because it was an unknown command but I could do it through the webui. Once I updated the primary and rebooted and smashed 'b' to get to the prompt, the commands in your guide worked exactly as listed and I was able to fully update and set up the switch. The guide was fantastic once I got to newer firmware, and it showed me what I needed to do to get to a newer firmware to be able to use the guide, so I consider it a resounding success! I'm only guessing it's due to older firmware, all I know is that the commands listed were unknown until I got to newer firmware. In the end tho, the switch works exactly as you described and your guide was very clear and easy to follow!

strange, the update commands have been in the production bootloaders since day one (which includes the 07.4.00 era bootloaders). I wonder if you got one of the development bootloaders I've ran into in the past, happened with one or two other members here who got their switches off ebay (which turned out to come from brocade's HQ in california)

 

[Vesalius]

How about The power consumption of Your 7150-48zp ? It seems higher than 7250 in the data sheet

Not sure as I have not measured it. Would assume it's higher given the 16 multigig ports.

 

[kpfleming]

Not sure as I have not measured it. Would assume it's higher given the 16 multigig ports.

Right; again, it's a 48-port multi-gig capable switch, it's going to draw more power than a 24-port non-multi-gig switch.

 

[Lone Wolf]

strange, the update commands have been in the production bootloaders since day one (which includes the 07.4.00 era bootloaders). I wonder if you got one of the development bootloaders I've ran into in the past, happened with one or two other members here who got their switches off ebay (which turned out to come from brocade's HQ in california)

Anything is possible. I know the first time I booted it up I did miss hitting 'b' at the time right time as I was talking with a co-worker. I rebooted and made sure I hit 'b' and got a prompt right away. When the commands didn't work, I rebooted yet again and watched to make sure I hit 'b' in time. Since the commands didn't work I had to update the firmware as I described above. When the switch rebooted after update I hit 'b' the same as before and the commands worked. So something changed lol.

With the 07.4.00 era bootloaders, did the fans stay loud or did they get quieter after it booted? Did the development ones stay loud? My fans dropped down to 70dB after boot, not the 37dB it listed in the manual. I'm curious now if I did get a development one lol

 

[Jason Antes]

strange, the update commands have been in the production bootloaders since day one (which includes the 07.4.00 era bootloaders). I wonder if you got one of the development bootloaders I've ran into in the past, happened with one or two other members here who got their switches off ebay (which turned out to come from brocade's HQ in california)

I've toured that HQ with a good friend at Brocade. It's quite the HQ.

 

[YEAHHWAY]

Every time I get to the point where I want to use the following commands:

enable
copy tftp license 192.168.1.8 ICX6610-FCX/1-6610-ports.xml unit 1
copy tftp license 192.168.1.8 ICX6610-FCX/2-6610-adv.xml unit 1
copy tftp license 192.168.1.8 ICX6610-FCX/3-6610-macsec.xml unit 1

I get the following error message:

Copy Software License from TFTP to Flash copy stack Error: tftp timeout.

I have tried disabling the Windows firewall and allowing the tftpd64.exe through the firewall.

Has anyone else run into this problem?