Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[63z5mgng]

I also just got an ICX7250-48p and ran into a little glitch after following the upgrade instructions. Mine failed to boot the SPR08090mc.bin primary image and instead failed back to the secondary with 08030.

Turns out this was because the second reset reverted back to the original uboot image. It seems there are two uboot partitions as well. I was updating the second one. The first "reset" brought it up as expected, but the second reset (after the update_primary) was reverting back to the other uboot image.

Cure for this, found in the "Ruckus FastIron 08.0.80 Software Upgrade Guide" page 29, is to repeat the "update_uboot" command after the first reset or powercycle. Sure enough, that one will report "Loading image to Uboot Partition 1".


Allen

I didn't check that. Great catch.

 

[tinfoil3d]

Tried to stack up two 7150 with 15+m rj45 and fs and mt SFP+ modules, didn't want to come up at all. When I was planning this house I didn't know what SFP is yet. Seems like I'll have to get creative and pull fiber either through very narrow pipe used for air ventilation or just upstairs directly.

 

[Phlesh]

I just picked up a 6610-48P after thorough research and a huge help from multiple folks in this community. Really appreciate this information and the detailed guides!

I followed @fohdeesha instructions all the way up to the point where I've disconnected the management network and plugged in one of the front network ports. What's weird is that, so far, I can't get the switch to connect to my network in any way.

What's the best guide for doing diagnostic network commands on the switch itself to try to figure out the issue? I found that I can do "show ip address" to see that the static IP is set correctly, but that's about as far as I've gotten.

 

[LodeRunner]

Output of "sh int br" and "sh run" would help. You can post them here in code tags wrapped in spoiler tags to make them collapse.
Off hand, subnet mismatch, VLAN tag vs untag issue (or port not part of the VLAN that has the VE with assigned IP), no default route, or a port in ERR-DISABLE state come to mind.

 

[S!rius]

Hello together

I've been trying to access my 7250 via console for 2 days now and I'm getting desperate. I have a Cisco DB9 cable which I have converted according to the following instructions.
https://www.reddit.com/r/Brocade/comments/b71xtq
Unfortunately the putty session does not respond. After that I found this article and tried it too, without success.
https://www.reddit.com/r/Brocade/comments/k25l5u
I also searched the internet for the original Brocade cable, but it is not available anywhere.

Should I assume that the 7250 mini-usb port is broken?

What else can I try?

 

[juju]

I am trying to setup my 7250 with pfsense as bgp router for kubernetes. Can't get bgp to work on pfsense. Connected to the pfsense via a layer 3 transit. Anyone has any pointers on how to set this up? Is this a route distribution issue? Do I need to do anything on the 7250 to allow bgp traffic?

 

[AllenB]

Hello together

I've been trying to access my 7250 via console for 2 days now and I'm getting desperate. I have a Cisco DB9 cable which I have converted according to the following instructions.
https://www.reddit.com/r/Brocade/comments/b71xtq
Unfortunately the putty session does not respond. After that I found this article and tried it too, without success.
https://www.reddit.com/r/Brocade/comments/k25l5u
I also searched the internet for the original Brocade cable, but it is not available anywhere.

Should I assume that the 7250 mini-usb port is broken?

What else can I try?

View attachment 20548

Hi S!rius,

I had to make the same sort of cable for mine, and can perhaps shed some light. First off, unless your Cisco cable uses different colors than mine, those are not the right pins. Here's the mapping I've used (successfully):

Green (Cisco) -> White (USB)
Yellow (Cisco) -> Blue (USB) (orange on the Cisco end should also work; both are grounds)
Red (Cisco) -> Green (USB)

Edit: My USB wire colors were different than what I found online and may be different than yours, so you might want to ignore this part of mine.

The important thing is that on the Cisco side, the middle two wires (pins 4 & 5) are ground. Then to each side of those (pins 3 and 6) are transmit and receive. Not sure offhand which is which, but on my cable they are green and red.

You can test your Putty setup by temporarily connecting pins 3 and 6 of the Cisco cable (ignoring the USB side). With that in place, anything you type in Putty should come right back and display on the screen.

If that doesn't work, you've got a problem on the Putty/serial port/Cisco cable side. If it does work, it's probably just a matter of getting the correct wires connected to the USB cable.

Hope that helps a bit.

Cheers,
Allen

 

[tinfoil3d]

Might be a stupid question. How do you reset PoE without scripting all the ssh login, config terminal, interface e .... and inline power, no inline power in expect script?

 

[klui]

What's your expectation of how it should work?

 

[tinfoil3d]

What's your expectation of how it should work?

If that's at me, I expected there is some kind of non-interactive shortcut command for that.
Suppose PoE camera is unresponsive and needs hard reset, I thought I can easily call that in ssh from a monitoring script without much pain.

 

[klui]

It's already non-interactive. Issue the command and you're done. I don't understand.

 

[tinfoil3d]

Oh, my bad. I read about batch buffer and stuff but that's totally useless.
In fact I figured out you can simply feed your batch right into ssh stdin and that would do the trick.
So something like printf 'enable\nconf t\ninterface e 2/1/2\nno inline power\ninline power' | ssh switch would do the trick. No need for anything else. I couldn't figure this out at first because no ; or && supported in shell.

 

[juju]

When I drop this to a 6450, I've decided to just do router-on-a-stick with a Mikrotik RB4011, which on its single SFP+ port can route 10gb/s on that one port (I've tested it) and supports BGP. I'll then have it redistribute the routes via OSPF to the rest of the network, which the 6450 will manage. All of my nodes have a separate interface to my storage VLAN, so that won't hit the Mikrotik, so the Mikrotik will really only be routing traffic that enters/leaves the cluster...and 10gb/s should be plenty enough for this for me.

@CED6688 - I am trying to do this on a 7250 and have no idea how to set it up. I have pfsense running frr bgp and a kubernetes cluster that cant reach it via metallb. Any pointers how to set this up?

 

[63z5mgng]

Hello together

I've been trying to access my 7250 via console for 2 days now and I'm getting desperate. I have a Cisco DB9 cable which I have converted according to the following instructions.
https://www.reddit.com/r/Brocade/comments/b71xtq
Unfortunately the putty session does not respond. After that I found this article and tried it too, without success.
https://www.reddit.com/r/Brocade/comments/k25l5u
I also searched the internet for the original Brocade cable, but it is not available anywhere.

Should I assume that the 7250 mini-usb port is broken?

What else can I try?

View attachment 20548

Here is what I did. Maybe is helpful for you.
https://www.reddit.com/r/Brocade/comments/qunj4c

 

[Phlesh]

Output of "sh int br" and "sh run" would help. You can post them here in code tags wrapped in spoiler tags to make them collapse.
Off hand, subnet mismatch, VLAN tag vs untag issue (or port not part of the VLAN that has the VE with assigned IP), no default route, or a port in ERR-DISABLE state come to mind.

Thanks @LodeRunner. See below:

Spoiler: sh int br

ICX6610-48P Router(config-vlan-1)#sh int br

Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1/1/1 Up Forward Full 1G None No 1 0 cc4e.2413.74cc
1/1/2 Down None None None None No 1 0 cc4e.2413.74cc
1/1/3 Down None None None None No 1 0 cc4e.2413.74cc
1/1/4 Down None None None None No 1 0 cc4e.2413.74cc
1/1/5 Down None None None None No 1 0 cc4e.2413.74cc
1/1/6 Down None None None None No 1 0 cc4e.2413.74cc
1/1/7 Down None None None None No 1 0 cc4e.2413.74cc
1/1/8 Down None None None None No 1 0 cc4e.2413.74cc
1/1/9 Down None None None None No 1 0 cc4e.2413.74cc
1/1/10 Down None None None None No 1 0 cc4e.2413.74cc
1/1/11 Down None None None None No 1 0 cc4e.2413.74cc
1/1/12 Down None None None None No 1 0 cc4e.2413.74cc
1/1/13 Down None None None None No 1 0 cc4e.2413.74cc
1/1/14 Down None None None None No 1 0 cc4e.2413.74cc
1/1/15 Down None None None None No 1 0 cc4e.2413.74cc
1/1/16 Down None None None None No 1 0 cc4e.2413.74cc
1/1/17 Down None None None None No 1 0 cc4e.2413.74cc
1/1/18 Down None None None None No 1 0 cc4e.2413.74cc
1/1/19 Down None None None None No 1 0 cc4e.2413.74cc
1/1/20 Down None None None None No 1 0 cc4e.2413.74cc
1/1/21 Down None None None None No 1 0 cc4e.2413.74cc
1/1/22 Down None None None None No 1 0 cc4e.2413.74cc
1/1/23 Down None None None None No 1 0 cc4e.2413.74cc
1/1/24 Down None None None None No 1 0 cc4e.2413.74cc
1/1/25 Down None None None None No 1 0 cc4e.2413.74cc
1/1/26 Down None None None None No 1 0 cc4e.2413.74cc
1/1/27 Down None None None None No 1 0 cc4e.2413.74cc
1/1/28 Down None None None None No 1 0 cc4e.2413.74cc
1/1/29 Down None None None None No 1 0 cc4e.2413.74cc
1/1/30 Down None None None None No 1 0 cc4e.2413.74cc
1/1/31 Down None None None None No 1 0 cc4e.2413.74cc
1/1/32 Down None None None None No 1 0 cc4e.2413.74cc
1/1/33 Down None None None None No 1 0 cc4e.2413.74cc
1/1/34 Down None None None None No 1 0 cc4e.2413.74cc
1/1/35 Down None None None None No 1 0 cc4e.2413.74cc
1/1/36 Down None None None None No 1 0 cc4e.2413.74cc
1/1/37 Down None None None None No 1 0 cc4e.2413.74cc
1/1/38 Down None None None None No 1 0 cc4e.2413.74cc
1/1/39 Down None None None None No 1 0 cc4e.2413.74cc
1/1/40 Down None None None None No 1 0 cc4e.2413.74cc
1/1/41 Down None None None None No 1 0 cc4e.2413.74cc
1/1/42 Down None None None None No 1 0 cc4e.2413.74cc
1/1/43 Down None None None None No 1 0 cc4e.2413.74cc
1/1/44 Down None None None None No 1 0 cc4e.2413.74cc
1/1/45 Down None None None None No 1 0 cc4e.2413.74cc
1/1/46 Down None None None None No 1 0 cc4e.2413.74cc
1/1/47 Down None None None None No 1 0 cc4e.2413.74cc
1/1/48 Down None None None None No 1 0 cc4e.2413.74cc
1/2/1 Down None None None None No 1 0 cc4e.2413.74cc
1/2/2 Down None None None None No 1 0 cc4e.2413.74cc
1/2/3 Down None None None None No 1 0 cc4e.2413.74cc
1/2/4 Down None None None None No 1 0 cc4e.2413.74cc
1/2/5 Down None None None None No 1 0 cc4e.2413.74cc
1/2/6 Down None None None None No 1 0 cc4e.2413.74cc
1/2/7 Down None None None None No 1 0 cc4e.2413.74cc
1/2/8 Down None None None None No 1 0 cc4e.2413.74cc
1/2/9 Down None None None None No 1 0 cc4e.2413.74cc
1/2/10 Down None None None None No 1 0 cc4e.2413.74cc
1/3/1 Down None None None None No 1 0 cc4e.2413.74cc
1/3/2 Down None None None None No 1 0 cc4e.2413.74cc
1/3/3 Down None None None None No 1 0 cc4e.2413.74cc
1/3/4 Down None None None None No 1 0 cc4e.2413.74cc
1/3/5 Down None None None None No 1 0 cc4e.2413.74cc
1/3/6 Down None None None None No 1 0 cc4e.2413.74cc
1/3/7 Down None None None None No 1 0 cc4e.2413.74cc
1/3/8 Down None None None None No 1 0 cc4e.2413.74cc
mgmt1 Down None None None None No None 0 cc4e.2413.74cc

Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
ve1 Up N/A N/A N/A None N/A N/A N/A cc4e.2413.74cc

Spoiler: sh run

ICX6610-48P Router(config-vlan-1)#sh run
Current configuration:
!
ver 08.0.30uT7f3
!
stack unit 1
module 1 icx6610-48p-poe-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
stack-trunk 1/2/1 to 1/2/2
stack-trunk 1/2/6 to 1/2/7
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
!
!
!
!
ip dhcp-client disable
!
!
!
!
!
!
!
!
!
!
interface ve 1
ip address 192.168.255.222 255.255.255.0
!
!
!
!
!
!
!
!
!
end

 

[LodeRunner]

@Phlesh So to be clear, your home subnet (everything on your network) is 192.168.255.0/24? Because if your home network is really 192.168.0.0/24 or anything else really, then that's your problem. Otherwise, that config is as barebones as it gets and it should work. Is the one port that's connected going to your workstation or another switch? If it's another switch, how is the port on the far end configured (tagged or untagged)?

 

[S!rius]

Hi S!rius,

I had to make the same sort of cable for mine, and can perhaps shed some light. First off, unless your Cisco cable uses different colors than mine, those are not the right pins. Here's the mapping I've used (successfully):

Green (Cisco) -> White (USB)
Yellow (Cisco) -> Blue (USB) (orange on the Cisco end should also work; both are grounds)
Red (Cisco) -> Green (USB)

Edit: My USB wire colors were different than what I found online and may be different than yours, so you might want to ignore this part of mine.

The important thing is that on the Cisco side, the middle two wires (pins 4 & 5) are ground. Then to each side of those (pins 3 and 6) are transmit and receive. Not sure offhand which is which, but on my cable they are green and red.

You can test your Putty setup by temporarily connecting pins 3 and 6 of the Cisco cable (ignoring the USB side). With that in place, anything you type in Putty should come right back and display on the screen.

If that doesn't work, you've got a problem on the Putty/serial port/Cisco cable side. If it does work, it's probably just a matter of getting the correct wires connected to the USB cable.

Hope that helps a bit.

Cheers,
Allen

This was exactly what I needed! The cable now works. Thanks for the support.

 

[Phlesh]

@Phlesh So to be clear, your home subnet (everything on your network) is 192.168.255.0/24? Because if your home network is really 192.168.0.0/24 or anything else really, then that's your problem. Otherwise, that config is as barebones as it gets and it should work. Is the one port that's connected going to your workstation or another switch? If it's another switch, how is the port on the far end configured (tagged or untagged)?

@LodeRunner

I have a Ubiquiti USG (soon to be put to pasture) that has multiple VLAN's on it already. The default VLAN is the subnet as shown, yes (*.255/24). I have already verified I can ping from, say, my laptop on one of the other VLAN's in the network to devices in this *.255 subnet, so there should be no firewall issues at play here.

The port is connecting to another switch (which, again, is a Ubiquiti switch soon to be retired). That port is configured with a profile that sets "All" VLAN's to be tagged, or so it seems to me. I spent hours on this last night trying to determine whether that's indeed the case, and it seems to be. But I'm not sure how I can inspect frames, etc. to be able to debug it further.

 

[Phlesh]

@LodeRunner

I have a Ubiquiti USG (soon to be put to pasture) that has multiple VLAN's on it already. The default VLAN is the subnet as shown, yes (*.255/24). I have already verified I can ping from, say, my laptop on one of the other VLAN's in the network to devices in this *.255 subnet, so there should be no firewall issues at play here.

The port is connecting to another switch (which, again, is a Ubiquiti switch soon to be retired). That port is configured with a profile that sets "All" VLAN's to be tagged, or so it seems to me. I spent hours on this last night trying to determine whether that's indeed the case, and it seems to be. But I'm not sure how I can inspect frames, etc. to be able to debug it further.

Hmm, maybe some new information here.

I tried again this morning to ping from the ICX to the Unifi switch (which is in the same *.255/24 subnet), and it worked! I also pinged another device in that subnet.

However, I can't ping any device in the other VLAN's. So, there is something still either wrong with tagging or there's a firewall rule I don't understand, I think?

 

[infoMatt]

However, I can't ping any device in the other VLAN's. So, there is something still either wrong with tagging or there's a firewall rule I don't understand, I think?

You are missing a default route on the switch. It doesn't know where it has to forward traffic to that doesn't belong to its L2 segment.