Switch inter-vlan routing with only WAN traffic going to a firewall

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Nikotine

Member
Mar 17, 2021
35
0
6
Hold on to your ethernet cables! I think I have it working as intended.

Let's start with the traceroutes.
VLAN 9 with WAN access:
Code:
pi@rpi3-sdr:~ $ traceroute -i enxb827eb45a8ec 9.9.9.9
traceroute to 9.9.9.9 (9.9.9.9), 30 hops max, 60 byte packets
1  10.25.9.2 (10.25.9.2)  1.311 ms  1.725 ms  2.245 ms
2  WRT1900AC.lan (172.16.1.2)  0.491 ms  0.371 ms  0.407 ms
3  192.168.0.1 (192.168.0.1)  1.709 ms  2.470 ms  3.117 ms
4  <redacted>  15.344 ms  18.718 ms  19.465 ms
5  * * *
6  dD5E0FA71.access.telenet.be (213.224.250.113)  21.903 ms  22.433 ms  23.317 ms
7  * * *
8  * * *
9  PCH-level3-Amsterdam1.Level3.net (4.68.72.246)  94.928 ms  96.780 ms  96.964 ms
10  dns9.quad9.net (9.9.9.9)  33.544 ms !X  32.523 ms !X  18.518 ms !X
pi@rpi3-sdr:~ $
VLAN 100 without WAN access:
Code:
pi@rpi3-sdr:~ $ traceroute -i enxb827eb45a8ec 9.9.9.9
traceroute to 9.9.9.9 (9.9.9.9), 30 hops max, 60 byte packets
1  10.25.100.2 (10.25.100.2)  5.066 ms  4.822 ms  4.835 ms
2  WRT1900AC.lan (172.16.1.2)  4.151 ms  4.888 ms  5.354 ms
3  WRT1900AC.lan (172.16.1.2)  5.531 ms  5.658 ms  5.789 ms
pi@rpi3-sdr:~ $
Does this look correct?
Any slow hops I should be worried about?
Those 5ms hops for VLAN 100 are not good, but this is a ping to 172.16.1.2:
Code:
pi@rpi3-sdr:~ $ ping -I enxb827eb45a8ec 172.16.1.2
PING 172.16.1.2 (172.16.1.2) from 10.25.100.229 enxb827eb45a8ec: 56(84) bytes of data.
64 bytes from 172.16.1.2: icmp_seq=1 ttl=63 time=6.68 ms
64 bytes from 172.16.1.2: icmp_seq=2 ttl=63 time=0.545 ms
64 bytes from 172.16.1.2: icmp_seq=3 ttl=63 time=0.541 ms
64 bytes from 172.16.1.2: icmp_seq=4 ttl=63 time=0.582 ms
64 bytes from 172.16.1.2: icmp_seq=5 ttl=63 time=0.587 ms
64 bytes from 172.16.1.2: icmp_seq=6 ttl=63 time=0.587 ms
64 bytes from 172.16.1.2: icmp_seq=7 ttl=63 time=0.637 ms
64 bytes from 172.16.1.2: icmp_seq=8 ttl=63 time=0.533 ms
64 bytes from 172.16.1.2: icmp_seq=9 ttl=63 time=0.562 ms
^C
--- 172.16.1.2 ping statistics ---
9 packets transmitted, 9 received, 0% packet loss, time 313ms
rtt min/avg/max/mdev = 0.533/1.250/6.684/1.921 ms
pi@rpi3-sdr:~ $
The first ping is slow, the rest is quick.

Changes I made compared to the previous config:
  1. Before I had three VLANs on the OpenWRT router, the same as on the switch: VLAN 172 "transit", VLAN 9 and VLAN 100. Each had a separate interface with DHCP enabled.
    Now I have just one VLAN on the OpenWRT router: VLAN 172. The three interface are linked to that one VLAN. Each interface had DHCP enabled.
    config interface 'TRANSIT'
    option proto 'static'
    option ifname 'eth0.172'
    list ipaddr '172.16.1.2/24'

    config interface 'VLAN9'
    option ifname 'eth0.172'
    option proto 'static'
    option ipaddr '10.25.9.1'
    option netmask '255.255.255.0'

    config interface 'VLAN100'
    option proto 'static'
    option netmask '255.255.255.0'
    option ifname 'eth0.172'
    option ipaddr '10.25.100.1'
  2. Before the three VLANs on the OpenWRT router were linked via a tagged port to the Brocade switch. Now it is untagged as only VLAN 172 needs to pass.
  3. Before port 1/1/1 was tagged in every VLAN on the Brocade switch, now it is only untagged in VLAN 172.
  4. I added ip helper-address 1 172.16.1.2 to VLAN 9 and VLAN 100 on the switch to point to the DHCP server.
  5. Finally, and this took a while to figure out, I had to change the static route on the OpenWRT router. Before it was this:
    Code:
    config route
    option target '10.25.0.0'
    option gateway '172.16.1.1'
    option netmask '255.255.0.0'
    option interface 'TRANSIT'
    Now I created two separate routes. I could see that OpenWRT received the DHCP requests and offered an IP address, but it would never reach the switch without these routes:
    Code:
    config route option target '10.25.9.0'
    option gateway '172.16.1.1'
    option netmask '255.255.255.0'
    option interface 'VLAN9'
    
    config route
    option target '10.25.100.0'
    option gateway '172.16.1.1'
    option netmask '255.255.255.0'
    option interface 'VLAN100'

The firewall rule on OpenWRT to only allow 10.25.9.0/24 to WAN remained unchanged.
Also the route on the switch ip route 0.0.0.0/0 172.16.1.2 stays the same.

ICX6450-48P-Router(config)#sh run
Current configuration:
!
ver 08.0.30tT313
!
stack unit 1
module 1 icx6450-48p-poe-port-management-module
module 2 icx6450-sfp-plus-4port-40g-module
!
global-stp
!
!
!
vlan 9 name main by port
untagged ethe 1/1/2 to 1/1/36 ethe 1/2/1 to 1/2/4
router-interface ve 9
!
vlan 100 name IPcams by port
untagged ethe 1/1/37 to 1/1/48
router-interface ve 100
!
vlan 172 name TRANSIT by port
untagged ethe 1/1/1
router-interface ve 172
!
vlan 4095 name DEFAULT-VLAN by port
!
!
!
!
!
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
default-vlan-id 4095
enable telnet authentication
hostname ICX6450-48P-Router
ip dhcp-client disable
ip dns server-address 172.16.1.2
ip route 0.0.0.0/0 172.16.1.2
!
no telnet server
username root password .....
snmp-server community ..... ro
!
!
clock summer-time
clock timezone gmt GMT+01
!
!
ntp
disable serve
server 172.16.1.2
!
!
web-management https
web-management refresh front-panel 30
web-management session-timeout 3600
!
!
!
interface ethernet 1/1/37
inline power priority 1 power-by-class 3
!
interface ve 9
ip address 10.25.9.2 255.255.255.0
ip helper-address 1 172.16.1.2
!
interface ve 100
ip address 10.25.100.2 255.255.255.0
ip helper-address 1 172.16.1.2
!
interface ve 172
ip address 172.16.1.1 255.255.255.0
!
!
!
!
!
!
!
!
!
end
 
Last edited:

kapone

Well-Known Member
May 23, 2015
1,095
642
113
That's an improvement from a routing perspective. Now it's routing the way we want it to route. However...

Any slow hops I should be worried about?
Those 5ms hops for VLAN 100 are not good, but this is a ping to 172.16.1.2:
That's a real slow hop. We'll have to think about what's causing that. Try the same from a different device, not the RPi. It may be something unique to it.

Your "new" VLAN configuration is...just different...from the old one. Functionally, they should be identical. But it seems to help the routing. However I'm confused... Are these interfaces or VLANs on the wrt?

Code:
config interface 'TRANSIT'
option proto 'static'
option ifname 'eth0.172'
list ipaddr '172.16.1.2/24'

config interface 'VLAN9'
option ifname 'eth0.172'
option proto 'static'
option ipaddr '10.25.9.1'
option netmask '255.255.255.0'

config interface 'VLAN100'
option proto 'static'
option netmask '255.255.255.0'
option ifname 'eth0.172'
option ipaddr '10.25.100.1'
Now I created two separate routes. I could see that OpenWRT received the DHCP requests and offered an IP address, but it would never reach the switch without these routes:
Those two routes again are functionally identical to a single route 10.25.0.0/16, but may be the wrt was tripping on the /16 mask.
 

Nikotine

Member
Mar 17, 2021
35
0
6
Your "new" VLAN configuration is...just different...from the old one. Functionally, they should be identical. But it seems to help the routing. However I'm confused... Are these interfaces or VLANs on the wrt?
They are interfaces, there is just one VLAN 172 (device eth0.172). Yeah, confusing, I kept the interface names.

Those two routes again are functionally identical to a single route 10.25.0.0/16, but may be the wrt was tripping on the /16 mask.
Please note that the former static route was for the transit interface (I had by mistake pasted the firewall rule in my post above, I have corrected this now).
Now I had to create them for the two other interfaces.

I'll do some traceroute tests with other devices.
 

coxhaus

Active Member
Jul 7, 2020
109
36
28
I think it looks better. Can you post a trace route from a client on the 10.25.100.x network to 9.9.9.9? I want to make sure untagging does not cause an issue.

A L3 switch needs to calculate the route on the first packet from L3 routing. Then all other packets use that calculated route at line speed. 6ms does seem high. Maybe the switch is slow at processing L3 routes.

I still like my structure better.
 
Last edited:

Nikotine

Member
Mar 17, 2021
35
0
6
I think it looks better. Can you post a trace route from a client on the 10.25.100.x network to 9.9.9.9? I want to make sure untagging does not cause an issue.
VLAN 100 without WAN access:
Code:
pi@rpi3-sdr:~ $ traceroute -i enxb827eb45a8ec 9.9.9.9
traceroute to 9.9.9.9 (9.9.9.9), 30 hops max, 60 byte packets
1  10.25.100.2 (10.25.100.2)  5.066 ms  4.822 ms  4.835 ms
2  WRT1900AC.lan (172.16.1.2)  4.151 ms  4.888 ms  5.354 ms
3  WRT1900AC.lan (172.16.1.2)  5.531 ms  5.658 ms  5.789 ms
pi@rpi3-sdr:~ $
I noticed an issue with DNS.
Traceroute went really slow, step by step.
Now I'm giving clients the OpenWRT router's IP as DNS server with option 6 of DHCP.
Traceroute goes really quick now.
I'll do some more testing.
 

hibby50

New Member
Mar 17, 2022
3
0
1
Hi, I'm trying to do this exact same config, and I'm running into an issue. I can't ping interfaces on the openwrt router. Per your config above for example, if I have a device on the switch and try to ping 10.25.9.1 (the ip for vlan 9 on openwrt) I get in a routing loop. This makes sense to me because switch is routing it to openwrt and openwrt is routing it back to the switch from the static route added in openwrt.

Is there a way to stop this behavior? I tried to add a /32 route to openwrt but that was not successful. @Nikotine does this pinging one of the openwrt interfaces work for you?