LB6M (brocade firmware) Trouble Routing Vlans

monte1299

New Member
May 2, 2018
4
2
3
40
Hello all!

I recently purchased a Quanta LB6m and flashed it to brocade firmware using fohdeesha's awesome instructions. For the past several days, I've been reading the manual and trying to figure out a good way to design my home network now that I have 10G. It's been relatively slow going, but it's a learning process.

I've started this diagram so I can keep the details straight as I work through the issues one by one. Here is the network right now (the numbered ports are the ports on the lb6m). Ignore all of the FreeNAS/ESXi business for now. I'm just trying to get basic inter-vlan routing going.

upload_2018-7-13_10-14-14.png

I've set up vlans 2 and 3 in addition to the default vlan1. Here's the show vlan command output:

brocade(config)#show vlan
Total PORT-VLAN entries: 3
Maximum PORT-VLAN entries: 64
PORT-VLAN 1, Name DEFAULT-VLAN, Priority level0, Spanning tree Off
Untagged Ports: 6 7 8 9 10 11 12 13 14 15 16 17
Untagged Ports: 18 19 20 21 22 23 24 25 26 27 28
Tagged Ports: None
Uplink Ports: None
DualMode Ports: None

PORT-VLAN 2, Name iscsi0, Priority level0, Spanning tree Off
Untagged Ports: 1 3
Tagged Ports: None
Uplink Ports: None
DualMode Ports: None

PORT-VLAN 3, Name iscsi1-10g, Priority level0, Spanning tree Off
Untagged Ports: 2 4 5
Tagged Ports: None
Uplink Ports: None
DualMode Ports: None

brocade(config)#


Next, I created a router-interface for each vlan. Here's the show int br:
brocade(config)#show int br

Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1 Up Forward Full 10G None No 2 0 54ab.3a51.a66e
2 Up Forward Full 10G None No 3 0 54ab.3a51.a66e
3 Down None None None None No 2 0 54ab.3a51.a66e
4 Up Forward Full 10G None No 3 0 54ab.3a51.a66e
5 Up Forward Full 10G None No 3 0 54ab.3a51.a66e
6 Up Forward Full 10G None No 1 0 54ab.3a51.a66e
7 Down None None None None No 1 0 54ab.3a51.a66e
8 Down None None None None No 1 0 54ab.3a51.a66e
9 Down None None None None No 1 0 54ab.3a51.a66e
10 Down None None None None No 1 0 54ab.3a51.a66e
11 Down None None None None No 1 0 54ab.3a51.a66e
12 Down None None None None No 1 0 54ab.3a51.a66e
13 Down None None None None No 1 0 54ab.3a51.a66e
14 Down None None None None No 1 0 54ab.3a51.a66e
15 Down None None None None No 1 0 54ab.3a51.a66e
16 Down None None None None No 1 0 54ab.3a51.a66e
17 Down None None None None No 1 0 54ab.3a51.a66e
18 Down None None None None No 1 0 54ab.3a51.a66e
19 Down None None None None No 1 0 54ab.3a51.a66e
20 Down None None None None No 1 0 54ab.3a51.a66e
21 Down None None None None No 1 0 54ab.3a51.a66e
22 Down None None None None No 1 0 54ab.3a51.a66e
23 Down None None None None No 1 0 54ab.3a51.a66e
24 Down None None None None No 1 0 54ab.3a51.a66e
25 Down None None None None No 1 0 54ab.3a51.a66e
26 Up Forward Full 100M None No 1 0 54ab.3a51.a66e
27 Down None None None None No 1 0 54ab.3a51.a66e
28 Up Forward Full 1G None No 1 0 54ab.3a51.a66e
mgmt1 Up None Full 1G None No None 0 54ab.3a51.a66e

Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
ve2 Up N/A N/A N/A None N/A N/A N/A 54ab.3a51.a66e
ve3 Up N/A N/A N/A None N/A N/A N/A 54ab.3a51.a66e
ve1 Up N/A N/A N/A None N/A N/A N/A 54ab.3a51.a66e
brocade(config)#


and here's the routing table on the lb6m. I added a static route to my netgear router for internet bound traffic. The default vlan routes were added automatically when I created the router-interfaces.


brocade#show ip route
Total number of IP routes: 4, avail: 12282 (out of max 12286)
B:BGP D:Connected R:RIP S:Static O:OSPF *:Candidate default
Destination NetMask Gateway Port Cost Type
0.0.0.0 0.0.0.0 192.168.1.1 v1 1 S
1 192.168.1.0 255.255.255.0 0.0.0.0 v1 1 D
2 192.168.10.0 255.255.255.0 0.0.0.0 v2 1 D
3 192.168.11.0 255.255.255.0 0.0.0.0 v3 1 D
brocade#



Ignore that link 3 is down. I'm trying to simplify my situation. All I'm trying to do is get my FreeNAS box (Helios, 192.168.11.3, vlan 3) to ping anything on vlan 1 (192.168.1.0/24). So on Helios, I shut down all interfaces except the one connected to 192.168.11.3. Then I added a default route to 192.168.11.1 (vme3 on vlan 3). Here's the routing table from Helios:

root@helios:~ # netstat -nr
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 192.168.11.1 UGS cxl1
127.0.0.1 link#5 UH lo0
192.168.10.3 link#1 UHS lo0
192.168.11.0/24 link#2 U cxl1
192.168.11.3 link#2 UHS lo0

Internet6:
Destination Gateway Flags Netif Expire
::/96 ::1 UGRS lo0
::1 link#5 UH lo0
::ffff:0.0.0.0/96 ::1 UGRS lo0
fe80::/10 ::1 UGRS lo0
fe80::%lo0/64 link#5 U lo0
fe80::1%lo0 link#5 UHS lo0
ff02::/16 ::1 UGRS lo0
root@helios:~ #


Finally, here's my running config on the LB6m. Can anyone see why Helios cannot ping anything on vlan1 (192.168.1.0/24)?

brocade#show run
Current configuration:
!
ver 07.4.00pT203
fan-speed 2
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 2 name iscsi0 by port
untagged ethe 1 ethe 3
router-interface ve 2
!
vlan 3 name iscsi1-10g by port
untagged ethe 2 ethe 4 to 5
router-interface ve 3
!
!
!
!
aaa authentication enable default local
aaa authentication login default local
boot sys fl pri
enable telnet authentication
enable aaa console
hostname brocade
ip show-portname
ip route 0.0.0.0 0.0.0.0 192.168.1.1
!
no telnet server
username root password .....

!
!
interface ve 1
ip address 192.168.1.50 255.255.255.0
!
interface ve 2
ip address 192.168.10.1 255.255.255.0
!
interface ve 3
ip address 192.168.11.1 255.255.255.0
!
!
!
!
!
!
!
!
!
end

brocade#
 

fohdeesha

Kaini Industries
Nov 20, 2016
1,802
1,556
113
29
fohdeesha.com
FreeNAS can't ping things in vlan 1, because every device in vlan 1 needs a route BACK to it. the devices in 192.168.1.xxx will get an ICMP packet, see it's from the subnet 192.168.11.xxx, realize it's not in their subnet, so they'll hit their default gateway, which im assuming is your internet router at 192.168.1.1.

The proper way of doing intervlan routing on a switch like this with an internet router in the mix, is have all device (including vlan 1) have their gateway set to the switches VE. Then have another separate transit subnet from the switch to your internet router, like 172.16.0.1/29. You can't have other devices in this transit-only subnet/link. something like this:






If you want your internet router to still serve DHCP, you'll need to enable ip-helper for the DHCP broadcast traffic to make it from each isolated vlan back and from the internet router, which is a couple commands on the LB6M. You can see there's a reason most people stick to one flat VLAN at home :)
 

monte1299

New Member
May 2, 2018
4
2
3
40
Makes perfect sense - thank you! I was trying to do a little test with my setup BEFORE I went to my hosts on vlan1 and set the default route to ve1. Without a route back, none of these hosts can respond to the ping.

Ahh, good stuff! It's all part of the learning process. Thank you again for your help!
 

French Chamallow

New Member
Mar 15, 2019
17
3
3
FreeNAS can't ping things in vlan 1, because every device in vlan 1 needs a route BACK to it. the devices in 192.168.1.xxx will get an ICMP packet, see it's from the subnet 192.168.11.xxx, realize it's not in their subnet, so they'll hit their default gateway, which im assuming is your internet router at 192.168.1.1.

The proper way of doing intervlan routing on a switch like this with an internet router in the mix, is have all device (including vlan 1) have their gateway set to the switches VE. Then have another separate transit subnet from the switch to your internet router, like 172.16.0.1/29. You can't have other devices in this transit-only subnet/link. something like this:






If you want your internet router to still serve DHCP, you'll need to enable ip-helper for the DHCP broadcast traffic to make it from each isolated vlan back and from the internet router, which is a couple commands on the LB6M. You can see there's a reason most people stick to one flat VLAN at home :)

can you please tell me how in pfsense perform the operation described at the top right of the image?

"on this router set a static route of 192.168.0.0/8 with a next hop of 172.16.0.2"