Hello all!
I recently purchased a Quanta LB6m and flashed it to brocade firmware using fohdeesha's awesome instructions. For the past several days, I've been reading the manual and trying to figure out a good way to design my home network now that I have 10G. It's been relatively slow going, but it's a learning process.
I've started this diagram so I can keep the details straight as I work through the issues one by one. Here is the network right now (the numbered ports are the ports on the lb6m). Ignore all of the FreeNAS/ESXi business for now. I'm just trying to get basic inter-vlan routing going.
I've set up vlans 2 and 3 in addition to the default vlan1. Here's the show vlan command output:
brocade(config)#show vlan
Total PORT-VLAN entries: 3
Maximum PORT-VLAN entries: 64
PORT-VLAN 1, Name DEFAULT-VLAN, Priority level0, Spanning tree Off
Untagged Ports: 6 7 8 9 10 11 12 13 14 15 16 17
Untagged Ports: 18 19 20 21 22 23 24 25 26 27 28
Tagged Ports: None
Uplink Ports: None
DualMode Ports: None
PORT-VLAN 2, Name iscsi0, Priority level0, Spanning tree Off
Untagged Ports: 1 3
Tagged Ports: None
Uplink Ports: None
DualMode Ports: None
PORT-VLAN 3, Name iscsi1-10g, Priority level0, Spanning tree Off
Untagged Ports: 2 4 5
Tagged Ports: None
Uplink Ports: None
DualMode Ports: None
brocade(config)#
Next, I created a router-interface for each vlan. Here's the show int br:
brocade(config)#show int br
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1 Up Forward Full 10G None No 2 0 54ab.3a51.a66e
2 Up Forward Full 10G None No 3 0 54ab.3a51.a66e
3 Down None None None None No 2 0 54ab.3a51.a66e
4 Up Forward Full 10G None No 3 0 54ab.3a51.a66e
5 Up Forward Full 10G None No 3 0 54ab.3a51.a66e
6 Up Forward Full 10G None No 1 0 54ab.3a51.a66e
7 Down None None None None No 1 0 54ab.3a51.a66e
8 Down None None None None No 1 0 54ab.3a51.a66e
9 Down None None None None No 1 0 54ab.3a51.a66e
10 Down None None None None No 1 0 54ab.3a51.a66e
11 Down None None None None No 1 0 54ab.3a51.a66e
12 Down None None None None No 1 0 54ab.3a51.a66e
13 Down None None None None No 1 0 54ab.3a51.a66e
14 Down None None None None No 1 0 54ab.3a51.a66e
15 Down None None None None No 1 0 54ab.3a51.a66e
16 Down None None None None No 1 0 54ab.3a51.a66e
17 Down None None None None No 1 0 54ab.3a51.a66e
18 Down None None None None No 1 0 54ab.3a51.a66e
19 Down None None None None No 1 0 54ab.3a51.a66e
20 Down None None None None No 1 0 54ab.3a51.a66e
21 Down None None None None No 1 0 54ab.3a51.a66e
22 Down None None None None No 1 0 54ab.3a51.a66e
23 Down None None None None No 1 0 54ab.3a51.a66e
24 Down None None None None No 1 0 54ab.3a51.a66e
25 Down None None None None No 1 0 54ab.3a51.a66e
26 Up Forward Full 100M None No 1 0 54ab.3a51.a66e
27 Down None None None None No 1 0 54ab.3a51.a66e
28 Up Forward Full 1G None No 1 0 54ab.3a51.a66e
mgmt1 Up None Full 1G None No None 0 54ab.3a51.a66e
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
ve2 Up N/A N/A N/A None N/A N/A N/A 54ab.3a51.a66e
ve3 Up N/A N/A N/A None N/A N/A N/A 54ab.3a51.a66e
ve1 Up N/A N/A N/A None N/A N/A N/A 54ab.3a51.a66e
brocade(config)#
and here's the routing table on the lb6m. I added a static route to my netgear router for internet bound traffic. The default vlan routes were added automatically when I created the router-interfaces.
brocade#show ip route
Total number of IP routes: 4, avail: 12282 (out of max 12286)
B:BGP D:Connected R:RIP S:Static O:OSPF *:Candidate default
Destination NetMask Gateway Port Cost Type
0.0.0.0 0.0.0.0 192.168.1.1 v1 1 S
1 192.168.1.0 255.255.255.0 0.0.0.0 v1 1 D
2 192.168.10.0 255.255.255.0 0.0.0.0 v2 1 D
3 192.168.11.0 255.255.255.0 0.0.0.0 v3 1 D
brocade#
Ignore that link 3 is down. I'm trying to simplify my situation. All I'm trying to do is get my FreeNAS box (Helios, 192.168.11.3, vlan 3) to ping anything on vlan 1 (192.168.1.0/24). So on Helios, I shut down all interfaces except the one connected to 192.168.11.3. Then I added a default route to 192.168.11.1 (vme3 on vlan 3). Here's the routing table from Helios:
root@helios:~ # netstat -nr
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default 192.168.11.1 UGS cxl1
127.0.0.1 link#5 UH lo0
192.168.10.3 link#1 UHS lo0
192.168.11.0/24 link#2 U cxl1
192.168.11.3 link#2 UHS lo0
Internet6:
Destination Gateway Flags Netif Expire
::/96 ::1 UGRS lo0
::1 link#5 UH lo0
::ffff:0.0.0.0/96 ::1 UGRS lo0
fe80::/10 ::1 UGRS lo0
fe80::%lo0/64 link#5 U lo0
fe80::1%lo0 link#5 UHS lo0
ff02::/16 ::1 UGRS lo0
root@helios:~ #
Finally, here's my running config on the LB6m. Can anyone see why Helios cannot ping anything on vlan1 (192.168.1.0/24)?
brocade#show run
Current configuration:
!
ver 07.4.00pT203
fan-speed 2
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 2 name iscsi0 by port
untagged ethe 1 ethe 3
router-interface ve 2
!
vlan 3 name iscsi1-10g by port
untagged ethe 2 ethe 4 to 5
router-interface ve 3
!
!
!
!
aaa authentication enable default local
aaa authentication login default local
boot sys fl pri
enable telnet authentication
enable aaa console
hostname brocade
ip show-portname
ip route 0.0.0.0 0.0.0.0 192.168.1.1
!
no telnet server
username root password .....
!
!
interface ve 1
ip address 192.168.1.50 255.255.255.0
!
interface ve 2
ip address 192.168.10.1 255.255.255.0
!
interface ve 3
ip address 192.168.11.1 255.255.255.0
!
!
!
!
!
!
!
!
!
end
brocade#
I recently purchased a Quanta LB6m and flashed it to brocade firmware using fohdeesha's awesome instructions. For the past several days, I've been reading the manual and trying to figure out a good way to design my home network now that I have 10G. It's been relatively slow going, but it's a learning process.
I've started this diagram so I can keep the details straight as I work through the issues one by one. Here is the network right now (the numbered ports are the ports on the lb6m). Ignore all of the FreeNAS/ESXi business for now. I'm just trying to get basic inter-vlan routing going.
I've set up vlans 2 and 3 in addition to the default vlan1. Here's the show vlan command output:
brocade(config)#show vlan
Total PORT-VLAN entries: 3
Maximum PORT-VLAN entries: 64
PORT-VLAN 1, Name DEFAULT-VLAN, Priority level0, Spanning tree Off
Untagged Ports: 6 7 8 9 10 11 12 13 14 15 16 17
Untagged Ports: 18 19 20 21 22 23 24 25 26 27 28
Tagged Ports: None
Uplink Ports: None
DualMode Ports: None
PORT-VLAN 2, Name iscsi0, Priority level0, Spanning tree Off
Untagged Ports: 1 3
Tagged Ports: None
Uplink Ports: None
DualMode Ports: None
PORT-VLAN 3, Name iscsi1-10g, Priority level0, Spanning tree Off
Untagged Ports: 2 4 5
Tagged Ports: None
Uplink Ports: None
DualMode Ports: None
brocade(config)#
Next, I created a router-interface for each vlan. Here's the show int br:
brocade(config)#show int br
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1 Up Forward Full 10G None No 2 0 54ab.3a51.a66e
2 Up Forward Full 10G None No 3 0 54ab.3a51.a66e
3 Down None None None None No 2 0 54ab.3a51.a66e
4 Up Forward Full 10G None No 3 0 54ab.3a51.a66e
5 Up Forward Full 10G None No 3 0 54ab.3a51.a66e
6 Up Forward Full 10G None No 1 0 54ab.3a51.a66e
7 Down None None None None No 1 0 54ab.3a51.a66e
8 Down None None None None No 1 0 54ab.3a51.a66e
9 Down None None None None No 1 0 54ab.3a51.a66e
10 Down None None None None No 1 0 54ab.3a51.a66e
11 Down None None None None No 1 0 54ab.3a51.a66e
12 Down None None None None No 1 0 54ab.3a51.a66e
13 Down None None None None No 1 0 54ab.3a51.a66e
14 Down None None None None No 1 0 54ab.3a51.a66e
15 Down None None None None No 1 0 54ab.3a51.a66e
16 Down None None None None No 1 0 54ab.3a51.a66e
17 Down None None None None No 1 0 54ab.3a51.a66e
18 Down None None None None No 1 0 54ab.3a51.a66e
19 Down None None None None No 1 0 54ab.3a51.a66e
20 Down None None None None No 1 0 54ab.3a51.a66e
21 Down None None None None No 1 0 54ab.3a51.a66e
22 Down None None None None No 1 0 54ab.3a51.a66e
23 Down None None None None No 1 0 54ab.3a51.a66e
24 Down None None None None No 1 0 54ab.3a51.a66e
25 Down None None None None No 1 0 54ab.3a51.a66e
26 Up Forward Full 100M None No 1 0 54ab.3a51.a66e
27 Down None None None None No 1 0 54ab.3a51.a66e
28 Up Forward Full 1G None No 1 0 54ab.3a51.a66e
mgmt1 Up None Full 1G None No None 0 54ab.3a51.a66e
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
ve2 Up N/A N/A N/A None N/A N/A N/A 54ab.3a51.a66e
ve3 Up N/A N/A N/A None N/A N/A N/A 54ab.3a51.a66e
ve1 Up N/A N/A N/A None N/A N/A N/A 54ab.3a51.a66e
brocade(config)#
and here's the routing table on the lb6m. I added a static route to my netgear router for internet bound traffic. The default vlan routes were added automatically when I created the router-interfaces.
brocade#show ip route
Total number of IP routes: 4, avail: 12282 (out of max 12286)
B:BGP D:Connected R:RIP S:Static O:OSPF *:Candidate default
Destination NetMask Gateway Port Cost Type
0.0.0.0 0.0.0.0 192.168.1.1 v1 1 S
1 192.168.1.0 255.255.255.0 0.0.0.0 v1 1 D
2 192.168.10.0 255.255.255.0 0.0.0.0 v2 1 D
3 192.168.11.0 255.255.255.0 0.0.0.0 v3 1 D
brocade#
Ignore that link 3 is down. I'm trying to simplify my situation. All I'm trying to do is get my FreeNAS box (Helios, 192.168.11.3, vlan 3) to ping anything on vlan 1 (192.168.1.0/24). So on Helios, I shut down all interfaces except the one connected to 192.168.11.3. Then I added a default route to 192.168.11.1 (vme3 on vlan 3). Here's the routing table from Helios:
root@helios:~ # netstat -nr
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default 192.168.11.1 UGS cxl1
127.0.0.1 link#5 UH lo0
192.168.10.3 link#1 UHS lo0
192.168.11.0/24 link#2 U cxl1
192.168.11.3 link#2 UHS lo0
Internet6:
Destination Gateway Flags Netif Expire
::/96 ::1 UGRS lo0
::1 link#5 UH lo0
::ffff:0.0.0.0/96 ::1 UGRS lo0
fe80::/10 ::1 UGRS lo0
fe80::%lo0/64 link#5 U lo0
fe80::1%lo0 link#5 UHS lo0
ff02::/16 ::1 UGRS lo0
root@helios:~ #
Finally, here's my running config on the LB6m. Can anyone see why Helios cannot ping anything on vlan1 (192.168.1.0/24)?
brocade#show run
Current configuration:
!
ver 07.4.00pT203
fan-speed 2
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 2 name iscsi0 by port
untagged ethe 1 ethe 3
router-interface ve 2
!
vlan 3 name iscsi1-10g by port
untagged ethe 2 ethe 4 to 5
router-interface ve 3
!
!
!
!
aaa authentication enable default local
aaa authentication login default local
boot sys fl pri
enable telnet authentication
enable aaa console
hostname brocade
ip show-portname
ip route 0.0.0.0 0.0.0.0 192.168.1.1
!
no telnet server
username root password .....
!
!
interface ve 1
ip address 192.168.1.50 255.255.255.0
!
interface ve 2
ip address 192.168.10.1 255.255.255.0
!
interface ve 3
ip address 192.168.11.1 255.255.255.0
!
!
!
!
!
!
!
!
!
end
brocade#