Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[fohdeesha]

Did @fohdeesha make public the “special things“ he was going to make public? Would be a shame if we closed the thread before that ...

when I said page 200 I meant uhh page 205

 

[kh78]

Hi all,

Does anyone how to enable 'ingress filtering' for vlan tags on a port (on an ICX 6610)?

I know this typically has to be enabled as a separate setting in Cisco switches, but not sure about FastIron.

What I'm trying to achieve is:
VLAN Aware End Point, has tagged interfaces on VLANs 25, 200, 322 and 1000.
This is connected to e 1/1/1 which is a tagged port on the 6610 for VLANs 1000 and 322. VLANs 25 and 200 are not present on this port (neither in a tagged or untagged/dual-mode fashion).
I want e 1/1/1 to drop any traffic from the end point that is tagged as VLAN 25 or 200, whilst accepting VLANs 1000 and 322.

I have this configured, other than how to tell it to discard traffic tagged for VLANs that aren't configured on the e 1/1/1 interface.

This link implies (i think) that what I'm trying to achieve happens by default:

Commscope Technical Content Portal

docs.ruckuswireless.com

I disagree though, as it's playing havoc with DHCP on VLAN 200 when the endpoint is plugged into e 1/1/1.
(The reasoning as to why, is so that in the event of failure of a different piece of equip, they can move the cable from e 1/1/1 to a different port with VLAN 200, where the end point needs to pickup the DHCP role).

Any tips?

 

[kapone]

Hi all,

Does anyone how to enable 'ingress filtering' for vlan tags on a port (on an ICX 6610)?

I know this typically has to be enabled as a separate setting in Cisco switches, but not sure about FastIron.

What I'm trying to achieve is:
VLAN Aware End Point, has tagged interfaces on VLANs 25, 200, 322 and 1000.
This is connected to e 1/1/1 which is a tagged port on the 6610 for VLANs 1000 and 322. VLANs 25 and 200 are not present on this port (neither in a tagged or untagged/dual-mode fashion).
I want e 1/1/1 to drop any traffic from the end point that is tagged as VLAN 25 or 200, whilst accepting VLANs 1000 and 322.

I have this configured, other than how to tell it to discard traffic tagged for VLANs that aren't configured on the e 1/1/1 interface.

This link implies (i think) that what I'm trying to achieve happens by default:

Commscope Technical Content Portal

docs.ruckuswireless.com

I disagree though, as it's playing havoc with DHCP on VLAN 200 when the endpoint is plugged into e 1/1/1.
(The reasoning as to why, is so that in the event of failure of a different piece of equip, they can move the cable from e 1/1/1 to a different port with VLAN 200, where the end point needs to pickup the DHCP role).

Any tips?

Maybe I'm not understanding this correctly...

- The "device" connected to e 1/1/1 is VLAN aware and has VLANS 25, 200, 322 and 1000 defined.
- e 1/1/1 only has VLANS 1000 and 232.

Which means...the switch is not sending any VLAN 25, 200 traffic to e 1/1/1.

What am I missing? What would you filter??

 

[kh78]

Well, exactly as I originally wrote basically:
I want e 1/1/1 to drop any traffic from the end point that is tagged as VLAN 25 or 200, whilst accepting VLANs 1000 and 322.
(The above could be clarified to include: "and also drop any untagged traffic, or traffic tagged with any vlan except 1000 and 322")

I get where you are coming from (i.e. that the end point isn't responding to traffic originated from somewhere within VLAN 25 or 200 because, as you say, no such traffic would egress from e 1/1/1 on the switch to the end point). That is not to say though that the end point isn't originating it's own tagged traffic and sending it, resulting in ingress traffic on the e 1/1/1 interface.

In this case the end point is a firewall with NTP/DHCP/All-Other--Usual-Suspects, so I'm trying to ensure that whilst plugged in to e 1/1/1, if it tries to generate any untagged traffic or traffic on a vlan other than 1000 or 322, then that traffic will be dropped by the switch when it hits the e 1/1/1 interface. I'm thinking this is meant to be happening by default on FastIron, but it isn't.

I'm going to reboot the switch and see if it continues the same behaviour, it's just weird what it's doing at the moment.

 

[kapone]

Well, exactly as I originally wrote basically:
I want e 1/1/1 to drop any traffic from the end point that is tagged as VLAN 25 or 200, whilst accepting VLANs 1000 and 322.
(The above could be clarified to include: "and also drop any untagged traffic, or traffic tagged with any vlan except 1000 and 322")

I get where you are coming from (i.e. that the end point isn't responding to traffic originated from somewhere within VLAN 25 or 200 because, as you say, no such traffic would egress from e 1/1/1 on the switch to the end point). That is not to say though that the end point isn't originating it's own tagged traffic and sending it, resulting in ingress traffic on the e 1/1/1 interface.

In this case the end point is a firewall with NTP/DHCP/All-Other--Usual-Suspects, so I'm trying to ensure that whilst plugged in to e 1/1/1, if it tries to generate any untagged traffic or traffic on a vlan other than 1000 or 322, then that traffic will be dropped by the switch when it hits the e 1/1/1 interface. I'm thinking this is meant to be happening by default on FastIron, but it isn't.

I'm going to reboot the switch and see if it continues the same behaviour, it's just weird what it's doing at the moment.

That should happen automatically.

 

[fohdeesha]

That happens automatically, when you tag a port in a vlan (or multiple vlans) it's going to ignore any other traffic outside that vlan including untagged (unless you use the dual-mode command)

 

[EngineerNate]

I'm struggling to see why you'd configure the downstream device such that the switch has to constantly ignore traffic from it.

 

[kh78]

Thanks all, it's working correctly and as expected after a reboot, seems to have just got a bit hung up after the vlans were moved around on the port.

As to your question EngineerNate, it's so that in the event of a hardware failure of the primary firewall, a cable can be moved to alternative switch port, allowing the 4G fw/router to temporarily take over the same functions normally provided by the primary firewall. This can be done by the users without any other reconfig needing to be done etc.

It's all a bit of a sticky tape solution to solving the problem that should be done via an HA FW pair really. This setup isn't as good, but it does provide a clear and simple (i.e. user actionable) failover response that keeps things operational enough for their needs until the real problem could be resolved.

When all is running normally, that same 4G router that needs to provide failover services directly to the LAN, is configured to act as a secondary upstream gateway for the primary firewall as part of a multi-wan gateway group (i.e. it's routing traffic that has already been NAT'd by the primary firewall, so it's got to be on a seperate VLAN at that point to get it to where the 4G router is). Hence, I needed it be dropping any traffic tagged as one of the LAN side VLANs when it's plugged into it's normal switch port, but have it able to operate on the LAN side vlans without reconfig, in the event of a failure on the primary FW.

 

[EngineerNate]

So your endpoint connects to fw1 via the main port in question, but has an extra set of vlans setup as a secondary (normally inactive) uplink. Those vlans are configured on another switch port that's in a vlan with the 4g router. When you swap it, those vlans go active and your primary ones go down. Because you need two vlans on the port you can't just make both ports access ports.

I think I understand. The alternate solution (barring true ha) would be putting the 4g router on its own dedicated switch with identical vlan numbering so your endpoint device wouldn't need the extra vlans, but that's a lot of hardware to dedicate for a single failover port.

 

[kh78]

So your endpoint connects to fw1 via the main port in question, but has an extra set of vlans setup as a secondary (normally inactive) uplink. Those vlans are configured on another switch port that's in a vlan with the 4g router. When you swap it, those vlans go active and your primary ones go down. Because you need two vlans on the port you can't just make both ports access ports.

I think I understand. The alternate solution (barring true ha) would be putting the 4g router on its own dedicated switch with identical vlan numbering so your endpoint device wouldn't need the extra vlans, but that's a lot of hardware to dedicate for a single failover port.

Yep, that's about the summary of it.

 

[TheCodeLife]

Looks like nuking the NAND and starting over all the way at the bottom with SPS8060 GA and then step upgrading has fixed it for me? I don't know if your unit was as far gone as mine, or if it worked with no issues despite the TPM failure.

@LodeRunner I still haven't found a way to dump the NAND successfully to USB. I have only found a way to copy small amounts of data from the NAND using tftp. I can only access the unlocked bootloader which has limited capability. I'd still really like to save the TPM keys, but I haven't found a way to discover where they are located. I would appreciate any thoughts on that you might have though.

 

[csementuh]

That happens automatically, when you tag a port in a vlan (or multiple vlans) it's going to ignore any other traffic outside that vlan including untagged (unless you use the dual-mode command)

This is only on the 64XX switch though right?

On my 7250 I don't have to use dual-mode, it assumes default can pass untagged if tagged ports are specified for another VLAN, or at least it lets you set default VLAN as untagged. My 6450 won't do this and it forces you to set dual-mode by port which is pretty annoying.

I also noticed the 6450 LAG doesn't work the same way as the 7250. You have to set it up a bit differently.. You have to make the LAG, add ports and then you have to make one of the ports a "primary-port". Then you make changes to the primary-port and it changes all the LAG ports. It won't allow me to make a "LAG Virtual Interface" that is then managed like a port. On the 7250 it works this way and gives you a new "lg1" interface that shows 2G, 3G, 4G, etc speed which looks pretty. The 6450 seems to be working, but looks a bit different.

Why is the syntax of command and features different? Just 72XX series switches that much newer in OS? Any other weird/different things I should check for?

Also, FYI my new 6450-24P uses a constant 35ish watts (no inline power on), which is about 10 higher than I expected. I can live with that, but for another 15 watts I also may have lived with another 6450-48P or a 7250-48P (50 watts) for the cheaper price of the switches. lol Ohh well.

 

[LodeRunner]

@TheCodeLife So a thought I had, that ended based on my available time and expertise was to try to build an ARM based Debian image. You would need to pull the device tree data from a Ruckus image and then build a new FIT image and try to TFTP boot that. Or somehow figure out how to compile the needed tpm-tools package for that specific build of BusyBox.

What I discovered by having a factory new device to compare with my broken one was that the mfg-wrapped-key.pem (or whatever the file name is) file is unique to each device; aside from having the certificate and key in it, the header has the device serial number and other unique information that I am sure gets used.

On the switch that was misbehaving, after the NAND scrub and step upgrade, it's now a working switch that will occasionally through a certificate invalid warning on the serial console, but is otherwise fine (previously it kernel panicked with any traffic on data ports). When I copied the TPM files from the factory new switch to it and issued certificate related commands, that resulted in an immediate kernel panic. Removing the files and booting again, the switch accts fine, apart from the serial console messages.

I don't know if this breaks the ability to use it is a port extender, which is my ultimate goal and driving reason to find a 7750 to setup as the control bridge for campus fabric.

 

[sean]

Or somehow figure out how to compile the needed tpm-tools package for that specific build of BusyBox.

I've never used the 7x50 series, but I have successfully cross-compiled something (strace) for the 6450. I used musl-cross-make and tried to match the kernel a closely as possible with 2.6.35. There were some issues with the kernel uapi assuming glibc that had to be patched. After booting with the telnet server enabled, I can ship over the statically compiled binary and run it. Strace had zero external dependencies while tpm-tools does not however.

 

[StarsAndBars]

I am trying to stack my two 6610s, and I have followed the docs, but I am obviously missing something. Where did I go wrong and how do I correct it?

 

[csementuh]

Anyone have a set of rack ears they can sell me for a 7250? They're different than 64XX :-(

 

[fohdeesha]

I am trying to stack my two 6610s, and I have followed the docs, but I am obviously missing something. Where did I go wrong and how do I correct it?

post the output of "show run", looks like you removed a trunk-port command from one of the stack units

 

[StarsAndBars]

Thank you, fohdeesha!

stack unit 1
module 1 icx6610-48p-poe-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
priority 128
stack-port 1/2/1 1/2/6
stack unit 2
module 1 icx6610-48p-poe-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
stack-trunk 2/2/6 to 2/2/7
stack-port 2/2/1 2/2/6
stack enable
stack mac 748e.f8ce.2e9c

 

[fohdeesha]

Thank you, fohdeesha!

stack unit 1
module 1 icx6610-48p-poe-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
priority 128
stack-port 1/2/1 1/2/6
stack unit 2
module 1 icx6610-48p-poe-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
stack-trunk 2/2/6 to 2/2/7
stack-port 2/2/1 2/2/6
stack enable
stack mac 748e.f8ce.2e9c

enable
conf t
stack unit 1
stack-trunk 1/2/6 to 1/2/7
exit
exit
write mem


might need to reload the stack after, not sure

 

[StarsAndBars]

Thanks again, still getting an error though. I run these commands on unit 1, correct?

telnet@ICX6610-1#conf t
telnet@ICX6610-1(config)#stack unit 1
telnet@ICX6610-1(config-unit-1)#stack-trunk 1/2/6 to 2/2/7
Error! 2/2/7 is not in unit 1.
telnet@ICX6610-1(config-unit-1)#