Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[SuperServer]

So my icx is pulling this error on start up not sure if anyone had this issue before or how to fix it. I hit b but cant stop the boot loader with B it just reenters back into it.

ICX Boot Code Version 10.1.00 (grz10100)
Enter 'a' to stop at memory test
Enter 'b' to stop at boot monitor
.BOOT ERR: load monitor failed (ERR0)
$

Boot(Mem-Test)>mt
1: [20000000] {Incremental}
Failed at 0: expected 0, actual 400

Ok so i purchased a new 6610... because you know... live the good life you have to go back to it. Quickly got it updated and set up. So 2 quick things...

If i do the sho inline power it doesnt show me any statistics it comes back blank. However if i go to the web gui or CLI i can manually turn on and off the poe and see that it supposedly is on. Is this normal since i dont have anything plugged in?

Second the new switch is psu ver a... super loud so i decided to put the version B i had from the bad switch with the fan module. When i put it it smells like burnt electric and the switch will never post. So it seems the psu is causing the error i was receiving?

 

[Matt G]

as stated in previous posts, its a raw serial connection on a usb connector, not an actual USB port, so that will not work. You need the cable I previously linked: Mini-USB(M) to RJ45(M), 8ft, Brocade Console Cable, + 9pin(F) to RJ45(F) adapter | eBay

Can you exchange the rj45(M) for an rj45(F) on these cables and just extend it with a normal rj45(M) to USB type C serial cable? I don't want to buy a serial adapter when I've gotten the complete usb cable for simplicity.

 

[BThunderW]

Could someone shed some light on my issue? I have a 2.5G GPON OLT module plugged into one of the SFP+ ports. The module will not negotiate a connection unless the port is set to 1G. My FTTH connection is 1.5G, trying to get it to utilize the full connection speed. Is there a way to set the port speed to something other than 1G or 10G?

 

[u238]

If you just want to add redundancy to your dumb L2 switches connection to the stack spanning-tree should do what you want. Turn on 802.1w for the vlan your dumb-switch traffic is interfacing with on the stack and plug your redundant links into different stack switches on that vlan and that should be it. If you lose the stack switch with the currently active link RSTP will re-learn the topology and unblock another link to take over.

You almost definitely want to use 802.1w here instead of OG STP, as I understand it it's a lot faster at handling newly plugged devices. Also, beware of "rstp" vs "802.1w" while configuring -- "rstp" is a Brocade proprietary early implementation of the spec, it might not play nicely with other rstp capable devices.

Interesting, I didn't consider just using spanning tree. I'll give it a shot. Thanks!

 

[u238]

Could someone shed some light on my issue? I have a 2.5G GPON OLT module plugged into one of the SFP+ ports. The module will not negotiate a connection unless the port is set to 1G. My FTTH connection is 1.5G, trying to get it to utilize the full connection speed. Is there a way to set the port speed to something other than 1G or 10G?

Pretty sure it's 1G, 10G or nothin.

 

[arglebargle]

So good news and bad news re: tapping 12v directly to drive the fan(s). The good: it works fine, as expected. The bad: the switch apparently looks for actual power draw on the fan header to determine fan health. Even if the tach line is hooked up the switch reports that the fan is failed if there's no power draw. I think I knew this already but I was still hoping I'd get some joy by hooking up the tach wire, oh well.

I used 3M snap-on quick-disconnect T taps for the 12V feed and cannibalized an old molex fan adapter that split the tach wire off so it's easy to disconnect everything and plug the fan back into the onboard header. I regret tapping the internal supply directly instead of the distribution board but I don't have a contactless multimeter to figure out voltages on the output wires. I'd tap in there so the mod supports both internal and external PSUs if I were doing this again.

FYI: the Sunon 12v 40x40x20mm maglev V3 fans are perfect for this mod, they're cheap, almost silent and move a decent amount of air -- I definitely recommend them. I wouldn't use them without running directly off 12V though, they draw too little power at low speed for the switch to recognize that they're still alive and you'll get "fan failed; fan on; fan failed; fan on" notification/log spam.

(Fan notes: I know that the Sunon VX model will work as a drop in replacement without issue without the 12v mod, I'm 99.9% certain that the V1 will too, I'm unsure if the V2 draws enough power at low speed to be detected as working and as above the V3 will work but spam your logs with fan failed notifications. One of these days I'll buy a cheap scope and cobble together a filter for the PWM circuit and try all of them again for confirmation.)

The whole thing took about 10 minutes, the most involved part was modifying the power/tach splitter power lines with some spare 16ga wire so I could crimp on the quick disconnect spades, the fan wire on my DI(wh)Y splitter is way too small gauge to get a good crimp on its own.

Edit: Pictures: https://photos.app.goo.gl/AC2iMCVJqNdJu9aT6

 

[arglebargle]

Could someone shed some light on my issue? I have a 2.5G GPON OLT module plugged into one of the SFP+ ports. The module will not negotiate a connection unless the port is set to 1G. My FTTH connection is 1.5G, trying to get it to utilize the full connection speed. Is there a way to set the port speed to something other than 1G or 10G?

I think you need one of the ICX7000 Z series switches that have 1/2.5Gbps ports. IIRC the 7150 and 7450 have Z models, I don't remember if there's a 7250 Z switch. You'll probably also need a media converter to 2.5Gbps copper because I'm pretty sure only the copper ports have 2.5Gbps support.

 

[arglebargle]

Interesting, I didn't consider just using spanning tree. I'll give it a shot. Thanks!

Just out of curiosity does the ONT have a switch built in? Mine is just a single port media converter, I haven't seen an ONT/switch combo yet.

 

[u238]

Just out of curiosity does the ONT have a switch built in? Mine is just a single port media converter, I haven't seen an ONT/switch combo yet.

Yeah. It has a 3 port switch built in. I wish it was a simple ONT but the ISP doesn't allow any other devices. The model is an HG8045Q.

 

[xm4rcell0x]

Hi guys, i'm new on this forum I just buy an icx-6450P from ebay, but i cannot start PuTTY session....
i also buy this cable from amazon. I don't know why it doesn't work.... already tried all the possible settings .
Here a link to a video on MEGA
thanks to all

 

[BThunderW]

Gah, that's unfortunate. I guess I'll have to be content with measly 1Gbps

 

[itronin]

Possible 2post RM kit solution for ICX6610

I don't know if these are sturdy enough to support a 6610 in a 2 post rack but at $26.00 I'm going to give it a shot to see if it works. At a minimum it will keep my switches from sliding back on the shelf they are resting on.

Brocade ICX6610-48P-PE Switch Rack Mount RMK Kit Corners Brackets | eBay

itr

 

[J0hn]

Hi guys, i'm new on this forum I just buy an icx-6450P from ebay, but i cannot start PuTTY session....
i also buy this cable from amazon. I don't know why it doesn't work.... already tried all the possible settings .
Here a link to a video on MEGA
thanks to all

I just had a similar problem with a icx-6450P from ebay. It ended up being the cable. (First red flag was a warning on the Amazon Page about "Fakes")

This is the cable that didn't work (someone else used it in this thread without problems)
https://www.amazon.com/gp/product/B...n_title_o04_s00?ie=UTF8&psc=1&tag=servecom-20

This cable worked - but also has the FTDI chip
https://www.amazon.com/gp/product/B07BK6P67K/ref=ppx_yo_dt_b_asin_title_o03_s00?ie=UTF8&psc=1

fohdeesha suggested this cable which has a different chipset, but I had already ordered the second cable
https://www.amazon.com/Prolific-PL2323RA-Rollover-Fortinet-Juniper/dp/B078LX59LW?tag=servecom-20

 

[fohdeesha]

I regret tapping the internal supply directly instead of the distribution board but I don't have a contactless multimeter to figure out voltages on the output wires. I'd tap in there so the mod supports both internal and external PSUs if I were doing this again

if you ever do it again, you can just tap into the yellow and black wires on the harness going from the distro board to the switch mainboard - those are still plain 12V, but are fed from either the internal PSU or external PSU. these guys - https://i.imgur.com/p4u6lei.jpg

either one of the pair per positive/negative works, they're just paired up for current carrying capacity (they're still the same rail)

 

[arglebargle]

Possible 2post RM kit solution for ICX6610

I don't know if these are sturdy enough to support a 6610 in a 2 post rack but at $26.00 I'm going to give it a shot to see if it works. At a minimum it will keep my switches from sliding back on the shelf they are resting on.

Brocade ICX6610-48P-PE Switch Rack Mount RMK Kit Corners Brackets | eBay

itr

Those are the same front brackets that came attached to both of my ICX 6610's, I'm tempted to pick up a kit so I have rear brackets.

Actually, looking closely at those the rear brackets are *really* close to being the same as the front. I wonder if I could buy one set and punch a couple of holes to convert the extra front brackets into rears.

Hi guys, i'm new on this forum I just buy an icx-6450P from ebay, but i cannot start PuTTY session....
i also buy this cable from amazon. I don't know why it doesn't work.... already tried all the possible settings .
Here a link to a video on MEGA
thanks to all

I just had a similar problem with a icx-6450P from ebay. It ended up being the cable. (First red flag was a warning on the Amazon Page about "Fakes")

This is the cable that didn't work (someone else used it in this thread without problems)
https://www.amazon.com/gp/product/B...n_title_o04_s00?ie=UTF8&psc=1&tag=servecom-20

I have two of these and they work flawlessly, maybe try booting a Ubuntu live USB and run a very simple client like screen to double check that it isn't some weird Windows/driver issue? Windows can be stupid about drivers without making it obvious. If that doesn't work there's either something wrong with the cable or possibly the console port on the switch.

if you ever do it again, you can just tap into the yellow and black wires on the harness going from the distro board to the switch mainboard - those are still plain 12V, but are fed from either the internal PSU or external PSU. these guys - https://i.imgur.com/p4u6lei.jpg

either one of the pair per positive/negative works, they're just paired up for current carrying capacity (they're still the same rail)

Thanks! While I was installing the T-taps I realized that the yellow and black wires on the distro board were probably the same voltage as the supply, I just didn't want to chance it without confirmation. The cable routing and case clearance will be better if you tap the distro board as well, it's a little crowded tapping the PSU where I did.

I think I'm going to get an o-scope and figure out how to filter the PWM signal at some point. I know I don't need to, the replacement fans will probably last at least as long as the switches do, but it's a good excuse to learn some basic electrical skills (I don't think I ever even learned how to use a scope properly TBH) and solve a problem at the same time. If you've got any recommendations for <$75 scopes or an inexpensive and good contactless meter I'm all ears.

 

[SuperServer]

So i was able to find out the reason the sho inline power CLI didnt work... power supply sent was the 250.Placed in the 1000 watt and it came up with poe error show below. So i swapped it with another 6610 that got a power surge (big board on top in front by the actual 1G ports. Booted up had new info (2nd code) on start up but was again having same error (3rd code). Rebbot again.... I get this after the parse data returns info (4th code) and i am now updating.

Issues with PoE boards on this switch seem to be rather common not sure the cause of this be it design issue or just a common failure in used switches in general but perhaps this portion can be added to the main post to assist some others in step by step process of knowing you are getting some where in your troubleshooting lol.

sho inline power

Power Capacity:         Total is 748000 mWatts. Current Free is 748000 mWatts.

Power Allocations:      Requests Honored 0 times


 Port   Admin   Oper    ---Power(mWatts)---  PD Type  PD Class  Pri  Fault/
        State   State   Consumed  Allocated                          Error
--------------------------------------------------------------------------
  1/1/1 Off     Off            0          0  n/a      n/a         3  n/a
  1/1/2 Off     Off            0          0  n/a      n/a         3  n/a
  1/1/3 Off     Off            0          0  n/a      n/a         3  n/a
  1/1/4 Off     Off            0          0  n/a      n/a         3  n/a
  1/1/5 Off     Off            0          0  n/a      n/a         3  n/a
  1/1/6 Off     Off            0          0  n/a      n/a         3  n/a
  1/1/7 Off     Off            0          0  n/a      n/a         3  n/a
  1/1/8 Off     Off            0          0  n/a      n/a         3  n/a
  1/1/9 Off     Off            0          0  n/a      n/a         3  n/a
 1/1/10 Off     Off            0          0  n/a      n/a         3  n/a
 1/1/11 Off     Off            0          0  n/a      n/a         3  n/a
 1/1/12 Off     Off            0          0  n/a      n/a         3  n/a
 1/1/13 Off     Off            0          0  n/a      n/a         3  n/a
 1/1/14 Off     Off            0          0  n/a      n/a         3  n/a
--More--, next page: Space, next line: Return key, quit: Control-cPoE Info: Resetting module in slot 1....completed.
PoE Error: Device 0 failed to start on PoE module.
PoE Error: Device 1 failed to start on PoE module.
Resetting module in slot 1 again to recover from dev fault

PoE: Stack unit 1 Power supply 2  with 748000 mwatts capacity is up
PoE Info: Adding new 55V capacity of 748000 mW, total capacity is 748000, total free capacity is 748000
PoE Info: PoE module 1 of Unit 1 on ports 1/1/1 to 1/1/48 detected. Initializing....
PoE Event Trace Log Buffer for 2000 log entries allocated
PoE Event Trace Logging enabled...
PoE Info: Programming Ruckus defaults. Step 1: Writing port defaults on module in slot 1....
PoE Info: Programming Ruckus Defaults: Step 2: Writing PM defaults on module in slot 1.
PoE Info: Programming Ruckus defaults. Step 3: Writing user byte 0xf3 on module in slot 1.
PoE Info: Programming Ruckus defaults. Step 4: Saving settings on module in slot 1.
PoE Info: Programming Ruckus defaults....completed.
PoE Info: PoE module 1 of Unit 1 initialization is done.

#inline power install-firmware stack-unit 1 tftp 192.168.2.134 fcx_poeplus_02.1.0.b004.fw
Error! stack unit 1 does not have POE module

PoE: Stack unit 1 Power supply 2  with 748000 mwatts capacity is up
PoE Info: Adding new 55V capacity of 748000 mW, total capacity is 748000, total free capacity is 748000

Sewer>PoE Info: PoE module 1 of Unit 1 on ports 1/1/1 to 1/1/48 detected. Initializing....
PoE Event Trace Log Buffer for 2000 log entries allocated
PoE Event Trace Logging enabled...
PoE Info: PoE module 1 of Unit 1 initialization is done.

 

[arglebargle]

If anyone's in the market for one of the little 6450-C12 switches I've been watching one on eBay for a bit: No bids yet, 16hrs left, starting at $105 --

Brocade ICX6450-C12-PD 12-Port 1 GBE Switch 4 PoE Ports 42111710614 | eBay

I'd take it myself but I've got six switches and no money left in the toys budget for a bit. It didn't seem deals forum worthy since it's a single unit and an auction to boot. Anyway, good luck to anyone interested.

Note: This is the L3 version of the C12 switch, the L2's go for ~$100 all the time, the L3's not so much.

 

[fohdeesha]

Issues with PoE boards on this switch seem to be rather common not sure the cause of this be it design issue or just a common failure in used switches in general but perhaps this portion can be added to the main post to assist some others in step by step process of knowing you are getting some where in your troubleshooting lol.

I wouldn't say common, but PoE failures certainly happen more than any other failure type with switches connected to devices outdoors and in the weather (security cams etc).

What's strange is we went a good 9 months in this thread with only a single report of a bad PoE board, now in the last 3 weeks I've seen 5 or 6. I'm thinking a seller or IT recycler got a bad batch of them and is either selling them directly, or they got distributed out to the major resellers on ebay and that's why they're suddenly popping up. never seen this many bad PoE boards in one go, and I've bought a LOT of these

 

[SuperServer]

I wouldn't say common, but PoE failures certainly happen more than any other failure type with switches connected to devices outdoors and in the weather (security cams etc).

What's strange is we went a good 9 months in this thread with only a single report of a bad PoE board, now in the last 3 weeks I've seen 5 or 6. I'm thinking a seller or IT recycler got a bad batch of them and is either selling them directly, or they got distributed out to the major resellers on ebay and that's why they're suddenly popping up. never seen this many bad PoE boards in one go, and I've bought a LOT of these

Purchased my first one thru G-electronics or whatever on ebay. Packed like crazy no issues with box unwrap and the 10g ports have a massive dent in the corner got it to work dropped me $40 and said sorry. Got this guy looks typical used item but the poe board is bad and the wrong power supply sent them a message yesterday (saturday) will see what becomes of it. Would highly advise not to purchase through them unless you are willing to take risks.

The only issue i am having with this board now is that it hits 100% complete on the firmware poe update and then from then on out i can never write mem again lol. Makes literally no sense at all. Is there a way to see the poe firmware some where?

 

[arglebargle]

Purchased my first one thru G-electronics or whatever on ebay. Packed like crazy no issues with box unwrap and the 10g ports have a massive dent in the corner got it to work dropped me $40 and said sorry. Got this guy looks typical used item but the poe board is bad and the wrong power supply sent them a message yesterday (saturday) will see what becomes of it. Would highly advise not to purchase through them unless you are willing to take risks.

The only issue i am having with this board now is that it hits 100% complete on the firmware poe update and then from then on out i can never write mem again lol. Makes literally no sense at all. Is there a way to see the poe firmware some where?

Man, you need to get friendly with the eBay return system the next time this happens. Any time you get something from an eBay seller that isn't *exactly* as described/shown in the auction pictures reach out and ask for a price adjustment or just start a return right then and there. You have *all* of the leverage in this negotiation thanks to eBay's policies. If the item doesn't arrive _exactly as described/pictured and undamaged in transit you can file for a refund through ebay and the seller is on the hook not just for your refund but for return shipping too.

I've had a couple of things arrived packed badly and damaged in shipping and in every case I've negotiated at least the cost of the repair parts from the seller, if not an extra $20-30 for my time spent replacing parts. I had one guy try to dick me over and run the 30 day return clock down on me (he kept telling me he needed to get authorization to refund the damage cost) but as soon as I filed for a return he pivoted and I got a partial refund the same day.

Shipping on a lot of the things we're buying is 25-50% the cost of the auction, being able to dangle a 25-50% loss on top of the refund over sellers who ship badly will get you what you want and fast. Remember, they have to re-sell the item again, this time damaged and at a lower price, as well.

Also, if something arrives damaged (like bent rack ears that need to be replaced) you can pocket the partial refund and then DIY a repair for your own use (I've fixed rack ears with a crowbar and a bolt before.) The goods lost resale value due to the damage, don't just eat it yourself.

Obviously none of this applies (except shipping damage!) if you're buying "As-Is, for parts" equipment.