[SOLVED] Help Needed - Brocade ICX 6450 + Ruckus R720

K D

Well-Known Member
Dec 24, 2016
1,431
309
83
30041
After several months of putting it off due to schedule issues, I finally got around to rebuilding my home network based on Brocade ICX 6450 + Ruckus R720 + PFSense combo.

I am currently running a UBNT Based network with 6 switches , 2 AP-Pros, USG and Cloud Key Gen2 +.

After experimenting with the ICX 6450s, I decided to migrate everything (Home+Lab) into one stack.

Here is the new configuration

Step 1.
4 ICX 6450s stacked and 4 VLANS created and appropriate ports assigned to them untagged
VLAN 10 Servers
VLAN 20 Hypervisors
VLAN 30 Home Network
VLAN 90 IPMI
VLAN 99 Transit
I'm able to place devices in all the VLANs and connect between them seamlessly.

Step 2.
pfSense on vlan 99. All firewall rules and static routes defined.
With static IPs assigned, able to access internet from every vlan except VLAN 90

Step 3.
Bring up new Windows 2016 Domain Controllers in VLAN 10 running DNS and DHCP. Define DHCP Scopes for VLAN 10, 20 and 30 in the DHCP server.
add ip helper to VLAN 20 and 30.
Code:
interface ve 20
 ip address 172.16.20.1 255.255.255.0
 ip helper-address 1 172.16.10.11
 ip helper-address 2 172.16.10.12
!
interface ve 30
 ip address 172.16.30.1 255.255.255.0
 ip helper-address 1 172.16.10.11
 ip helper-address 2 172.16.10.12
devices are able to get IP addresses.

Step 3.
enable POE on one interface and set it to VLAN 10 untagged and 20 & 30 tagged

Code:
interface ethernet 1/1/20
    dual-mode  10
    inline power
    !
  
    vlan 10 name DNET by port
    tagged ethe 1/1/20
    router-interface ve 10
    !
    vlan 20 name HYPERVISOR by port
    tagged ethe 1/1/20
    untagged ethe 1/1/1 ethe 1/1/3
    router-interface ve 20
    !
    vlan 30 name HOME by port
    tagged ethe 1/1/20
    router-interface ve 30

Step 4.

Flash unleashed firmware on ruckus R720.
Setup new Wifi network (LAB). No changes to defaults.
Setup new Wifi network (HOME). No changes to defaults.
Able to connect devices to both Wifi Networks. Both get IP addresses from the VLAN 10 subnet

Step 5.
For HOME wifi network, change Access VLAN to 30 in the advanced settings page in the r720 admin page
.

Here's where I'm stuck.

If I connect to the LAB wifi network, there are no issues and everything works fine.

The trouble starts when I connect to the HOME wifi network.
IP Address is not assigned via DHCP.
the R720 access point reboots.
Network becomes unstable - Unable to reach the gateway ip address for any of the vlans.
I have to reload the switches to get back network access.


Any pointers a to what to look for? Is it a faulty access point? Have I messed up something else?

Paging the resident brocade guru @fohdeesha
 

j_h_o

Active Member
Apr 21, 2015
475
112
43
California, US
When you say you get an IP address from the VLAN 10 subnet, you mean from the DHCP server on VLAN 10? Or in that range?

You need to create different scopes (with different IP blocks) on the DHCP server; the Brocade will hint at which iface it received the DHCP request from, and your client(s) should get an IP from the appropriate pool. Your DHCP scope should specify the router/gateway as 172.16.30.1 so that it hits the Brocade vlan interface.

  1. When you connect a client to the Home WiFi network, do you get a DHCP IP in the range 172.16.30.0/24?
  2. Remove WiFi from the equation. Set a port as access VLAN30, plug in Ethernet and see if that works. What IP do you get, and what is the gateway returned to you via DHCP? Can you access the internet?
  3. Do you have any Brocade ACLs in place? Can you temporarily disable them, so you can confirm DHCP and routing is working correctly?
 
  • Like
Reactions: K D

K D

Well-Known Member
Dec 24, 2016
1,431
309
83
30041
When you say you get an IP address from the VLAN 10 subnet, you mean from the DHCP server on VLAN 10? Or in that range?
That's correct. IP Address assigned by DHCP server in the VLAN 10 Subnet.

You need to create different scopes (with different IP blocks) on the DHCP server; the Brocade will hint at which iface it received the DHCP request from, and your client(s) should get an IP from the appropriate pool. Your DHCP scope should specify the router/gateway as 172.16.30.1 so that it hits the Brocade vlan interface.
That's how it is set up

When you connect a client to the Home WiFi network, do you get a DHCP IP in the range 172.16.30.0/24?
No IP assignment from DHCP.

Remove WiFi from the equation. Set a port as access VLAN30, plug in Ethernet and see if that works. What IP do you get, and what is the gateway returned to you via DHCP? Can you access the internet?
Taking wifi out of the equation, everything works as it should. DHCP assignments are correct with the address from the correct subnet being assigned.

Do you have any Brocade ACLs in place? Can you temporarily disable them, so you can confirm DHCP and routing is working correctly?
No ACLs setup.


This is my routing table.

Code:
        Destination        Gateway         Port          Cost          Type Uptime
1       0.0.0.0/0          172.16.16.2     ve 16         1/1           S    1h1m
2       172.16.10.0/24     DIRECT          ve 10         0/0           D    1h1m
3       172.16.16.0/29     DIRECT          ve 16         0/0           D    1h1m
4       172.16.20.0/24     DIRECT          ve 20         0/0           D    1h1m
5       172.16.30.0/24     DIRECT          ve 30         0/0           D    1h1m
6       172.16.90.0/24     DIRECT          ve 90         0/0           D    1h1m

Within a couple of minutes of joining connecting to the HOME wifi network, I'm unable to ping any device from anywhere. Almost as if the switch has locked up. Only a reboot fixes the issue.

Switch Firmware:
Code:
    UNIT 1: compiled on Feb 13 2019 at 17:44:29 labeled as ICX64R08030t
                (9868556 bytes) from Primary ICX64R08030t.bin
        SW: Version 08.0.30tT313
    UNIT 2: compiled on Feb 13 2019 at 17:44:29 labeled as ICX64R08030t
                (9868556 bytes) from Primary ICX64R08030t.bin
        SW: Version 08.0.30tT313
    UNIT 3: compiled on Feb 13 2019 at 17:44:29 labeled as ICX64R08030t
                (9868556 bytes) from Primary ICX64R08030t.bin
        SW: Version 08.0.30tT313
    UNIT 4: compiled on Feb 13 2019 at 17:44:29 labeled as ICX64R08030t
                (9868556 bytes) from Primary ICX64R08030t.bin
        SW: Version 08.0.30tT313
R720 Firmware
Code:
R720_200.7.10.202.92.bl7
 

fohdeesha

Kaini Industries
Nov 20, 2016
1,942
1,773
113
29
fohdeesha.com
I'm about to head to bed so I only took a quick glance but it sounds like it's an AP config issue, like it's bridging vlans or otherwise somehow causing a broadcast storm, I'm not sure what else would cause the switches to lock up. I've not used ruckus wifi gear so I have no clue how they set up multiple vlans - if there's anyway to eliminate dual mode so everything is tagged (eg make vlan 10 tagged as well on the ruckus ap) I would start there
 

itronin

Well-Known Member
Nov 24, 2018
412
250
63
Denver, Colorado
Same issue.
Do you have a single ethernet port plugged in or both on the AP's? I'm guessing you only have 1 port plugged in since I do not see a LAG configuration in your posts.

FWIW this past weekend I replaced my Cisco WLC and AP's with Ruckus R710's running unleashed. I'm using an ICX6610 though. I had some unexpected behavior trying to use both ethernet ports on the R710, my initial thoughts were STP related since AP switch ports starting going into disable state (even though I had LAG's defined). I moved to a single port connection on my AP's until I have time (and downtime; home vlan=prod for me) to troubleshoot. Briefly troubleshooting I noticed some notes about the power usage requirements on the POE ports required to use BOTH ethernet ports on the AP.

Differences:
qty 3 x R710's
Single ICX 6610 at the moment
I created a VLAN for all the AP's and their unleashed traffic. "AP VLAN"
I'm serving reserved IP's to each AP VIA DHCP using dhcp forwarding from the AP VLAN.
I am using a static IP in the AP VLAN for the unleashed management IP.
Running ISC Bind.

I plan on setting up a dedicated VLAN for guest traffic and forwarding that out through a separate FW interface until I have time to learn about the guest capabilities in unleashed.

Edit - One thing I did notice in ruckus forums is that LAG must be configured via CLI for each AP.
 
Last edited:

b1g_bake

New Member
Jun 27, 2019
5
0
1
I have an ICX6450 and a ruckus r600 on unleashed as well. I had the opposite issue. I could get DHCP to WLAN clients on the VLAN30 guest network, but could not get it to the VLAN20 main network that the r600 got it's address from. I found out that on the AP side the main ssid needs to be set as "1" for the vlan to use the untagged vlan coming from the switch. On the brocade side I tagged the port with both VLAN20 and VLAN 30. Then I went to the port and issued "dual-mode 20". Both those settings got clients on the main ssid DHCP addresses. I'm using pfsense to handle DHCP for me and the 6450 is only doing L2 duties for me currently.
 

K D

Well-Known Member
Dec 24, 2016
1,431
309
83
30041
I'm about to head to bed so I only took a quick glance but it sounds like it's an AP config issue, like it's bridging vlans or otherwise somehow causing a broadcast storm, I'm not sure what else would cause the switches to lock up. I've not used ruckus wifi gear so I have no clue how they set up multiple vlans - if there's anyway to eliminate dual mode so everything is tagged (eg make vlan 10 tagged as well on the ruckus ap) I would start there
The management VLAN has to be untagged per the unleashed documentation. I did try disabling dual-mode and setting the port to be tagged on both. I couldn't connect to the AP when I did that.
 

K D

Well-Known Member
Dec 24, 2016
1,431
309
83
30041
Solved. There was a broadcast storm happening that was crashing the network in this configuration. I enabled spanning tree on the vlans and he issue was resolved.

Thanks to all of you for the pointers. Next step is to start working on ACLs.
 

b1g_bake

New Member
Jun 27, 2019
5
0
1
Glad to see it was solved. STP seemed to be enabled for me by default when setting up vlans on the ICX6450
 

fohdeesha

Kaini Industries
Nov 20, 2016
1,942
1,773
113
29
fohdeesha.com
Solved. There was a broadcast storm happening that was crashing the network in this configuration. I enabled spanning tree on the vlans and he issue was resolved.

Thanks to all of you for the pointers. Next step is to start working on ACLs.

I had a feeling, I would find out what was actually causing the storm in the first place and remedy the loop / possible loop rather than just blocking with stp
 

K D

Well-Known Member
Dec 24, 2016
1,431
309
83
30041
I had a feeling, I would find out what was actually causing the storm in the first place and remedy the loop / possible loop rather than just blocking with stp
I would like to do that but have no clue how to proceed. I'm good at following instructions though.
 

fohdeesha

Kaini Industries
Nov 20, 2016
1,942
1,773
113
29
fohdeesha.com
well, I think we've probably narrowed it down at least to something related to the AP, do you have any wifi devices that connect to the AP, but are also connected to wired ethernet, and could be potentially bridging them? I know Sonos stuff does this (and hopes and prays STP will stop it from being too aggressive)
 

K D

Well-Known Member
Dec 24, 2016
1,431
309
83
30041
No. It was just my iPhone connecting to the WiFi to test it out. Nothing else on the wifi. For now, only wired devices connected are 2 servers per vlan.

I know what you mean about the sonos setup. Mine gave me headaches when I first set them up. All my sonos devices have wifi disabled and are connected to a wired network that is physically separate from this one.
 

fohdeesha

Kaini Industries
Nov 20, 2016
1,942
1,773
113
29
fohdeesha.com
so just to clarify, with the wifi AP plugged in an active, everything is fine, it only started the broadcast storm once you connected with your iphone? connecting with a laptop or something else didn't do it?

Can you make it happen then post the output of "show span" - that will show what ports it's seen the loop happening on and blocked
 

epicurean

Active Member
Sep 29, 2014
661
42
28
I have an ICX6450 and a ruckus r600 on unleashed as well. I had the opposite issue. I could get DHCP to WLAN clients on the VLAN30 guest network, but could not get it to the VLAN20 main network that the r600 got it's address from. I found out that on the AP side the main ssid needs to be set as "1" for the vlan to use the untagged vlan coming from the switch. On the brocade side I tagged the port with both VLAN20 and VLAN 30. Then I went to the port and issued "dual-mode 20". Both those settings got clients on the main ssid DHCP addresses. I'm using pfsense to handle DHCP for me and the 6450 is only doing L2 duties for me currently.
I have a similar setup with Pfsense, to a ICX6610, and a ruckus r710 AP. The r710 is on port 42 of the 6610 serving vlan 1, 50 and 30. How exactly do I tag it? Right now I cannot get any ip address on the vlan networks
 
Last edited:

itronin

Well-Known Member
Nov 24, 2018
412
250
63
Denver, Colorado
I have a similar setup with Pfsense, to a ICX6610, and a ruckus r710 AP. The r710 is on port 42 of the 6610 serving vlan 1, 50 and 30. How exactly do I tag it? Right now I cannot get any ip address on the vlan networks
Are you serving DHCP from pfsense? Does pfsense have a network interface in each VLAN?
IIRC pfsense must have an interface in each VLAN for it to serve addresses.

vlan 50
tagged ethe 42

vlan 30
tagged ethe 42

ya got me on vlan 1 - not sure how to tag it, for some reason I have always believed that vlan 1 was magic on brocades and if you are using it then its always untagged traffic and you can add tags for other vlans by using dual-mode 1 for that port which so for Port 42 vlans 1,3,50 it would be 1U, 50T and 30T

I try and avoid using vlan 1 on any platform as I find it can limit flexibility but that's just me.
 

K D

Well-Known Member
Dec 24, 2016
1,431
309
83
30041
I have a similar setup with Pfsense, to a ICX6610, and a ruckus r710 AP. The r710 is on port 42 of the 6610 serving vlan 1, 50 and 30. How exactly do I tag it? Right now I cannot get any ip address on the vlan networks
I am not using VLAN1 for anything. All ports as assigned to other VLANs.

I have 2 windows servers performing DNS and DHCP. You will need to specify the DHCP server address as ip helper address for the virtual interface. Example

Code:
interface ve 20
 ip address 172.16.20.1 255.255.255.0
 ip helper-address 1 172.16.10.1
 ip helper-address 2 172.16.10.2
 

epicurean

Active Member
Sep 29, 2014
661
42
28
Thanks KD. I am not familiar with command line . is it possible to do the same thing within the web interface?