Drag to reposition cover

Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

eduncan911

The New James Dean
Jul 27, 2015
638
486
63
eduncan911.com
As I've never used them before, what are people's opinions on OM3 LC/UPC multimode 10Gpbs straight-through patch panels? (Hoping I got the terms right, just learned what all of these things mean)

Is there a loss of signal or risk of weak connections?

I'm building one of those custom 14U mobile (wheels) server rack out of 2x4s and plywood. I'm looking for something in the rear of the rack to connect 2 to 4 LC/UPC OM3 cables. Like this:


I have at least two 40 foot runs (plus room routing) that I've decided it's pretty cheap just to buy some transceivers and an OM3 cable for. However, I don't want to tie my rack down to one spot. Plus, it leaves 10g for the next house occupants when we sale in a few years (there's two studies).

I'll already have CAT6a keystones for a rear panel, with a few open slots. So was hoping to find some LC/UPC OM3 straight through connectors for it.
 
Last edited:

LodeRunner

Active Member
Apr 27, 2019
427
177
43
Weird, I was never able to get the C1/C2 ports to work with mine. Maybe they're just borked
In 'sh int br' do you have int 1/2/1 and 1/2/2?
I used them as a LACP pair until I got a 10g core and ran fiber. They were configured like any other port on the 7150-c12, just with no PoE.

What's the output of 'sh run int e 1/2/1 e 1/2/2'?

Do you have any VLANs? Have you tried adding the interfaces to them?
 

ipmifreely

New Member
Jan 16, 2021
5
0
1
dead ports:

so i configured 3x laggs. 2 were working 1 port of one of the lags was not- i deleted all my vlans and two of the LAGs 3 vlans now all 4 ports are not working of the 2 lags i deleted are not working (no lights when eathernet pugged in)

have i somehow killed 4 ports on this switch? how could i have killed them?

here is my "show run"

Code:
ver 08.0.30uT7f3
!
stack unit 1
  module 1 icx6610-48p-poe-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
  stack-trunk 1/2/1 to 1/2/2
  stack-trunk 1/2/6 to 1/2/7
!
!
!
lag Proxmox1 dynamic id 1
ports ethernet 1/1/9 to 1/1/10
primary-port 1/1/9
lacp-timeout long
!
lag Proxmox2 dynamic id 2
ports ethernet 1/1/11 to 1/1/12
primary-port 1/1/11
lacp-timeout long
!
lag freenas1 dynamic id 11
ports ethernet 1/1/3 to 1/1/6
primary-port 1/1/3
lacp-timeout long
deploy
!                                                                
lag freenas2 dynamic id 12                                      
!
!
vlan 1 name management by port
tagged ethe 1/1/9 to 1/1/13 ethe 1/1/15 #13 & 15 are my workaround since 9 to 11 seem dead...
untagged ethe 1/1/3 to 1/1/8 ethe 1/1/14 ethe 1/1/16 to 1/1/22
router-interface ve 1
!
vlan 2 name lan by port
tagged ethe 1/1/9 to 1/1/12
untagged ethe 1/1/25 to 1/1/48
!
vlan 3 name carp by port
tagged ethe 1/1/9 to 1/1/12
!
vlan 4 name wifi by port
tagged ethe 1/1/9 to 1/1/13 ethe 1/1/15
untagged ethe 1/1/23 to 1/1/24
!
vlan 5 name Cameras by port
tagged ethe 1/1/9 to 1/1/12
!
vlan 6 name webServices by port
tagged ethe 1/1/9 to 1/1/12                                    
!
vlan 7 by port
tagged ethe 1/1/9 to 1/1/12
!
vlan 10 name DEFAULT-VLAN by port
!
vlan 3000 name SaskTel by port
tagged ethe 1/1/1 to 1/1/2 ethe 1/1/9 to 1/1/12
!
!
!
!
!
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
default-vlan-id 10
hostname GrassySwtich1
ip dhcp-client disable
!
no telnet server
username root password .....
!                                                                
!
!
!
!
!
!
!
!
interface ethernet 1/1/9
dual-mode  1
disable
!
interface ethernet 1/1/10
dual-mode  1
disable
!
interface ethernet 1/1/11
dual-mode  1
disable
!
interface ethernet 1/1/12
dual-mode  1
disable                                                        
!
#this is just for testing
interface ethernet 1/1/13
dual-mode  1
!
#this is just for testing
interface ethernet 1/1/15
dual-mode  1
!
interface ve 1
ip address 192.168.1.2 255.255.255.0
!
!
!
!
!
!
!
!
!
end
and here is an example /etc/network/interfaces from a proxmox host:

Code:
auto lo
iface lo inet loopback

auto enp5s0f1
iface enp5s0f1 inet manual
#see bond0

auto enp5s0f0
iface enp5s0f0 inet manual

auto enp7s0f0
iface enp7s0f0 inet manual
#see bond0

auto enp7s0f1
iface enp7s0f1 inet manual
#see bond0

auto bond0
iface bond0 inet manual
        bond-slaves enp5s0f0 enp5s0f1 enp7s0f0 enp7s0f1
        bond-miimon 100
        bond-mode 802.3ad
        bond-xmit-hash-policy layer2+3

auto bond0.2
iface bond0.2 inet manual
#lan

auto bond0.3
iface bond0.3 inet manual
#carp

auto bond0.4
iface bond0.4 inet manual
#wifi

auto bond0.3000
iface bond0.3000 inet manual
#SASKTEL-WAN

auto bond0.5
iface bond0.5 inet manual
#cameras

auto bond0.6
iface bond0.6 inet manual
#webservices

auto bond0.7
iface bond0.7 inet manual
#work

auto bond0.8
iface bond0.8 inet manual
#IPMI

auto vmbr0
iface vmbr0 inet static
        address 192.168.1.20/24
        gateway 192.168.1.1
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
#managment

auto vmbr1
iface vmbr1 inet manual
        bridge-ports bond0.3000
        bridge-stp off
        bridge-fd 0
#wan

auto vmbr2
iface vmbr2 inet manual
        bridge-ports bond0.2
        bridge-stp off
        bridge-fd 0
#lan

auto vmbr3
iface vmbr3 inet manual
        bridge-ports bond0.3
        bridge-stp off
        bridge-fd 0
#carp

auto vmbr4
iface vmbr4 inet manual
        bridge-ports bond0.4
        bridge-stp off
        bridge-fd 0
#wifi

auto vmbr5
iface vmbr5 inet manual
        bridge-ports bond0.5
        bridge-stp off
        bridge-fd 0
#cameras

auto vmbr6
iface vmbr6 inet manual
        bridge-ports bond0.6
        bridge-stp off
        bridge-fd 0
#webservices

auto vmbr7
iface vmbr7 inet manual
        bridge-ports bond0.7
        bridge-stp off
        bridge-fd 0
#work

auto vmbr8
iface vmbr8 inet manual
        bridge-ports bond0.8
        bridge-stp off
        bridge-fd 0
#IPMI
 

LodeRunner

Active Member
Apr 27, 2019
427
177
43
When you remove a LAG on an ICX it disables the ports to prevent loops. You'll have to go to each affected port and issue 'enable'. Note in each interface definition it has 'disable'. A disabled port will not link, hence no lights.
 

tubs-ffm

Active Member
Sep 1, 2013
122
39
28
I exchanged on my 7250-24P the two fans to Delta EFB0412VHD-F00. This was a huge improvement for noise but now I got a temperature issue. I am waiting for the Sunon MF60101V3-1000U-A99 fan I ordered to put on top of the ASIC. Unfortunately, I could not find any other alternative with thin height and have to be patient due to long delivery time.
To follow-up on this topic.

Additionally, to the two Delta EFB0412VHD-F00 in the chassis today I installed the Sunon MF60101V3-1000U-A99 on top of the ASIC. I connected the Sunon in parallel to one of the chassis fans so it also will slow down in fan mode 1. After running a couple of hours in idle mode at room temperature I get these stable temperatures. Look OK to me. Unfortunately, I did not note the temperatures in the original setup with Foxconn fans.

Code:
Fan controlled temperature:
        Rule 1/2 (MGMT THERMAL PLANE): 62.4 deg-C
        Rule 2/2 (AIR OUTLET NEAR PSU): 42.5 deg-C
Just in case someone is asking. I am not planning to use heavy PoE load. Two devices only.
 

dswartz

Active Member
Jul 14, 2011
588
73
28
When you remove a LAG on an ICX it disables the ports to prevent loops. You'll have to go to each affected port and issue 'enable'. Note in each interface definition it has 'disable'. A disabled port will not link, hence no lights.
Yeah, that confused me the first time I had this happen :)
 

plexisaurus

New Member
Jan 14, 2021
6
2
3
To follow-up on this topic.

Additionally, to the two Delta EFB0412VHD-F00 in the chassis today I installed the Sunon MF60101V3-1000U-A99 on top of the ASIC. I connected the Sunon in parallel to one of the chassis fans so it also will slow down in fan mode 1. After running a couple of hours in idle mode at room temperature I get these stable temperatures. Look OK to me. Unfortunately, I did not note the temperatures in the original setup with Foxconn fans.

Code:
Fan controlled temperature:
        Rule 1/2 (MGMT THERMAL PLANE): 62.4 deg-C
        Rule 2/2 (AIR OUTLET NEAR PSU): 42.5 deg-C
Just in case someone is asking. I am not planning to use heavy PoE load. Two devices only.
The Delta I bought from Mouser. The Sunon I ordered from RS.



My plan is first to connect this fan in parallel to the housing fans. So, it will run at fan level 1 anyhow slower and at a different noise level. If this is not enough, I will connect it to permanently 12 V. This is my plan. No idea if it will work. I can report later.

For me the housing fans were expensive as I could not find anything at local shops and I had to order oversee with high shipping costs. But this Sunon I could find locally and it was not more than the value of a beer.

If all will not work, the 7250-24P will go back to eBay where it comes from. Too bad, I really like the spec. But this was part of my plan.
Just installed the MF60101V1-1000U-G99 today in parallel with the 3 sunon housing fans in my 7250-48. ASIC temps are about 2-3c better than stock fans at 52.4c and noise level hasn't increased over the sunon mf40201vx-1000u-g99 40mm fans. Very quiet at level 1.

I also installed a temporary cover made of thin ( mostly transparent to thermal) packing tape to check component/psu temps with my thermal camera. Psu and hotter ICs were at most low 40s.

Overall, very happy with result. All 4 fans were like $28+shipping, but it will save that or more in reduced energy use over life of switch.

Update: switch installed in wiring rack under stairs. ASIC temp has settled at 60c after 24hours under light load
 

Attachments

Last edited:

mrizzo

New Member
Feb 6, 2021
10
1
3
Hi all, I am looking at getting a Brocade ICX6610 for a rack that will have a 10Gb uplink. All incoming traffic is going to be tunneled via GRE from a third party DDoS protection service.

Can the Brocade handle a tunnel (just regular GRE, no encryption) that will have inbound traffic peaking at a few Gbps? I was looking at the Mikrotik CRS354 at first, but from my research none of the tunneling is offloaded from the CPU so the performance is bad.
 

tommybackeast

Active Member
Jun 10, 2018
290
111
43
Brocade firmware Question: 7250 and 7150 currently on 8080 - if i wish to go to 8092; can I upgrade from 8080 to 8092 -or- must I upgrade to 8090 and then 8092 ? thanks
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,557
2,751
113
31
fohdeesha.com
Hi all, I am looking at getting a Brocade ICX6610 for a rack that will have a 10Gb uplink. All incoming traffic is going to be tunneled via GRE from a third party DDoS protection service.

Can the Brocade handle a tunnel (just regular GRE, no encryption) that will have inbound traffic peaking at a few Gbps? I was looking at the Mikrotik CRS354 at first, but from my research none of the tunneling is offloaded from the CPU so the performance is bad.
yes, I actually have an icx6610 doing exactly this in NYC (terminating a GRE tunnel from a ddos provider). it's all done in hardware at line rate. note than enabling gre tunnels disables a couple counter features like ipv6 ACLs. full details starting on page 103 of fastiron-08030mb-l3guide.pdf
 

mrizzo

New Member
Feb 6, 2021
10
1
3
yes, I actually have an icx6610 doing exactly this in NYC (terminating a GRE tunnel from a ddos provider). it's all done in hardware at line rate. note than enabling gre tunnels disables a couple counter features like ipv6 ACLs. full details starting on page 103 of fastiron-08030mb-l3guide.pdf
Thank you so much! Glad to hear that I can do this without issue.

Question about the licensing - I just bought a BNIB one on eBay ( ICX6610-48-PI ) which has the Premium license. I saw somebody selling a license for the 10G upgrade on ebay so I bought that as well, but the listing says that the advanced license features are now included in the premium license. Can you confirm that or will I need to find an advances license to use GRE?
 

fohdeesha

Kaini Industries
Nov 20, 2016
2,557
2,751
113
31
fohdeesha.com
Thank you so much! Glad to hear that I can do this without issue.

Question about the licensing - I just bought a BNIB one on eBay ( ICX6610-48-PI ) which has the Premium license. I saw somebody selling a license for the 10G upgrade on ebay so I bought that as well, but the listing says that the advanced license features are now included in the premium license. Can you confirm that or will I need to find an advances license to use GRE?
you don't need an advanced license, it's merged into premium, and as the first sentence of this thread says, the port license you just spent $$$ on are free :)
 

tubs-ffm

Active Member
Sep 1, 2013
122
39
28
I must do something fundamental wrong in the L3 routing. I placed my question in a separate thread (see below). But maybe here in this Brocade thread I can find more guys that are familiar with FastIron.

What I want to achieve is to move the L3 routing between two networks (LAN and DMZ) from the firewall to the brocade switch. In theory this should work, but not with my config as linked below.

Any Idea where the mistake is?


I have a basic question to inter (V)LAN routing and the related network topology with two routers in it.

I am a home user and today my set-up is a router-on-a-stick configuration. One L2 switch to manage VLANs. All ACLs and routing done on the firewall. So far, all OK. For performance reasons I would like to move the routing between the networks called "LAN" and "DMZ" to a L3 switch. For the other VLANs this is not required as there is no routing to or from other subnets (Guest_WLAN, IoT).

Is the routing including ACLs between LAN and DMZ possible on the L3 switch with a network topology as shown on the picture?
Or must I move the DMZ and LAN network completely to the L3 switch and create "transport networks" and static routes between router and L3 switch?

View attachment 17392
Any idea what's wrong with my config?

My real config is using different networks than my illustration in the post above:

LAN: 192.168.2.0/24
DMZ: 192.168.10.0/24

Code:
Current configuration:
!
ver 08.0.92eT213
!
stack unit 1
  module 1 icx7250-24p-poe-port-management-module
  module 2 icx7250-sfp-plus-8port-80g-module
  stack-port 1/2/1
  stack-port 1/2/3
!
!
!
!
!
vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
vlan 10 name DMZ by port
tagged ethe 1/1/9 ethe 1/2/6 ethe 1/2/8
untagged ethe 1/1/3 ethe 1/1/11 to 1/1/12 ethe 1/2/5 ethe 1/2/7
router-interface ve 10
!
vlan 20 name IoT by port
tagged ethe 1/1/5 ethe 1/1/7 ethe 1/1/9
!
vlan 30 name Guest by port
tagged ethe 1/1/5 ethe 1/1/7 ethe 1/1/9
untagged ethe 1/1/10
!
!
!
!
!
!
!
!
!
!
!
!
!
!
optical-monitor
optical-monitor non-ruckus-optic-enable
aaa authentication web-server default local
aaa authentication login default local
enable telnet authentication
enable aaa console
hostname icx7250
ip dhcp-client disable
ip dns domain-list test.lan
ip dns server-address 192.168.2.1 192.168.2.15
ip route 0.0.0.0/0 192.168.2.1
ip route 0.0.0.0/0 192.168.10.1
!
telnet timeout 10
no telnet server
username admin password .....
!
!
snmp-server contact Administrator
snmp-server location Server Room
!
!
clock timezone gmt GMT+09
!
!
ntp
disable serve
server 192.168.2.1
!
!
web-management https
web-management frame bottom
web-management page-menu
web-management session-timeout 1200
!
!
manager registrar
!
!
!
!
!
!
!
!
!
interface ethernet 1/1/1
port-name OPNsense-2
!
interface ethernet 1/1/2
disable
!
interface ethernet 1/1/3
port-name OPNsense-3
!
interface ethernet 1/1/4
disable
!
interface ethernet 1/1/5
port-name OPNsense-4
!
interface ethernet 1/1/6
disable
!
interface ethernet 1/1/7
port-name WLAN-AP
!
interface ethernet 1/1/8
disable
!
interface ethernet 1/1/9
port-name Trunk-Office
!
interface ethernet 1/1/10
port-name Work-PC
!
interface ethernet 1/1/11
port-name Server-DMZ
!
interface ethernet 1/1/12
port-name PC-DMZ
!
interface ethernet 1/2/1
port-name PC-LAN
!
interface ethernet 1/2/3
port-name Server-LAN
!
interface ethernet 1/2/5
port-name PC-DMZ
!
interface ethernet 1/2/6
port-name PC-Trunk
!
interface ethernet 1/2/7
port-name Server-DMZ
!
interface ethernet 1/2/8
port-name Server-Trunk
!
interface ve 1
ip access-group lan_out in
ip address 192.168.2.2 255.255.255.0
!
interface ve 10
ip access-group dmz_out in
ip address 192.168.10.2 255.255.255.0
!
!
ip access-list extended lan_out
remark allow LAN to switch management
sequence 10 permit tcp 192.168.2.0 0.0.0.255 host 192.168.2.2 eq ssh
sequence 20 permit tcp 192.168.2.0 0.0.0.255 host 192.168.2.2 eq http
sequence 30 permit tcp 192.168.2.0 0.0.0.255 host 192.168.2.2 eq ssl
remark allow LAN to DMZ
sequence 40 permit icmp 192.168.2.0 0.0.0.255 192.168.10.0 0.0.0.255
sequence 50 permit tcp 192.168.2.0 0.0.0.255 192.168.10.0 0.0.0.255 eq http
sequence 60 permit tcp 192.168.2.0 0.0.0.255 192.168.10.0 0.0.0.255 eq ssl
sequence 70 permit tcp 192.168.2.0 0.0.0.255 192.168.10.0 0.0.0.255 eq ssh
sequence 80 permit tcp 192.168.2.0 0.0.0.255 192.168.10.0 0.0.0.255 eq ftp
sequence 90 permit tcp 192.168.2.0 0.0.0.255 host 192.168.10.20 eq 8006
sequence 100 permit tcp 192.168.2.0 0.0.0.255 host 192.168.10.20 eq 26
sequence 110 permit tcp 192.168.2.0 0.0.0.255 host 192.168.10.40 eq smtp
sequence 120 permit tcp 192.168.2.0 0.0.0.255 192.168.10.0 0.0.0.255 eq 8083
sequence 130 permit tcp 192.168.2.0 0.0.0.255 192.168.10.0 0.0.0.255 eq 9090
sequence 140 permit tcp 192.168.2.0 0.0.0.255 host 192.168.10.60 eq 5001
sequence 150 permit tcp 192.168.2.0 0.0.0.255 host 192.168.10.60
sequence 160 permit udp 192.168.2.0 0.0.0.255 host 192.168.10.60
remark deny all other to DMZ
sequence 170 deny ip 192.168.2.0 0.0.0.255 192.168.10.0 0.0.0.255
remark allow every else to everywhere
sequence 180 permit ip any any
!
ip access-list extended dmz_out
remark allow DMZ to LAN
sequence 10 permit tcp 192.168.10.0 0.0.0.255 host 192.168.2.15 eq dns
sequence 20 permit udp 192.168.10.0 0.0.0.255 host 192.168.2.15 eq dns
sequence 30 permit tcp host 192.168.10.10 host 192.168.2.15 eq ldap
sequence 40 permit tcp host 192.168.10.20 host 192.168.2.15 eq ldap
sequence 50 permit tcp host 192.168.10.10 host 192.168.2.15 eq ldaps
sequence 60 permit tcp host 192.168.10.20 host 192.168.2.15 eq ldaps
sequence 70 permit tcp host 192.168.10.10 host 192.168.2.15 eq microsoft-ds
sequence 80 permit tcp host 192.168.10.20 host 192.168.2.30 eq 2525
sequence 90 permit tcp host 192.168.10.40 host 192.168.2.30 eq 2525
sequence 100 permit tcp host 192.168.10.40 host 192.168.2.30 eq smtp
remark deny all other to LAN
sequence 110 deny ip 192.168.10.0 0.0.0.255 192.168.2.0 0.0.0.255
remark allow every else to everywhere
sequence 120 permit ip any any
!
!
!
no lldp run
!
!
ip ssh  idle-time 0
!
!
!
!
!
end
 
  • Like
Reactions: tommybackeast

Roelf Zomerman

Active Member
Jan 10, 2019
140
26
28
blog.azureinfra.com
did you configure your DHCP to provide 10.2 and 2.2 as the default gateway for your clients (per DHCP scope) ?

and you do not need 2x 0.0.0.0 routes as your brocade switch should be the router itself.. it will forward the traffic either to 192.168.2.1 (over interface ve1) or to 192.168.10.1 (over interface ve10) / choose one ..


(Client to Client example)
Client [10.x]--->>Brocade[192.168.10.2]--->>[192.168.2.2]--->>client2[192.168.2.x]

(internet example 1 - if you wish to keep the 10.1 route and drop the 2.1 route)
Client [10.x]--->>Brocade[192.168.10.2]--->>router[192.168.10.1]--->>internet

(internet example 2 - if you wish to keep the 2.1 route and drop the 10.1 route)
Client [10.x]--->>Brocade[192.168.10.2]--->>[192.168.2.2]-->>router[192.168.2.1]--->>internet

also make sure your router knows the route back to both interfaces.... either through the route on an interface or the static added route..

let's take 192.168.2.0/24 as the primary subnet (internet example 2) -.. then you'd need to add: 192.168.10.0/24 --> next-hop - 192.168.2.2 and your Brocade only has 1 default route of 0.0.0.0/0 --> 192.168.2.2

you cannot create an asynchronous routing - where a packet from 192.168.10.x to internet is routed through the Brocade to 192.168.2.2 - to 192.168.2.1 to internet.. and back via internet->192.168.10.1--> 192.168.10.x , bypassing the brocade router completely - most firewalls will block this and a lot of protocols cannot handle this.. so make sure your routing outbound passes the same routers as the return traffic

if you want to retain dual routing tables (also possible) - your clients will essentially have 192.168.2.1 and 192.168.10.1 as their default gateway (for 0.0.0.0/0 traffic) and you will manually have to add a route for 192.168.10.0/24 next-hop 192.168.2.2 on the 192.168.2.0/24 clients and 192.168.2.0/24 next-hop 192.168.10.2 for the 10.x clients..
 
Last edited:
  • Like
Reactions: tommybackeast

tubs-ffm

Active Member
Sep 1, 2013
122
39
28
did you configure your DHCP to provide 10.2 and 2.2 as the default gateway for your clients (per DHCP scope) ?

and you do not need 2x 0.0.0.0 routes as your brocade switch should be the router itself.. it will forward the traffic either to 192.168.2.1 (over interface ve1) or to 192.168.10.1 (over interface ve10) / choose one ..
Thank you. I will go through your text in detail the next days.


The idea behind the two routes was that I wanted to use the bandwidth of three available ports at the OPNsense firewall by avoiding VLAN and LAG on firewall side. So, I connected LAN and DMZ network to separate interfaces on the firewall (untacked port at Bracade). This basically is the same configuration I am running with brocade as L2 switch and the firewall as only router. What I did for testing was adding ve on Brocade side to both VLAN, creating of ACLs, adding the routes mentioned above and manually change the default gateway on the clients to the brocade IP. Traffic to and from DMZ was supposed to take the ve10 and traffic two and from the LAN was supposed to take the other ve1. But this seems to be not possible in this way.

But for sure. If my idea was not good or wrong, any advice for best practice is welcome.
 

ipmifreely

New Member
Jan 16, 2021
5
0
1
When you remove a LAG on an ICX it disables the ports to prevent loops. You'll have to go to each affected port and issue 'enable'. Note in each interface definition it has 'disable'. A disabled port will not link, hence no lights.
ah ha! thank you. duh. now i can get rid of my hacky workarounds.
 

tubs-ffm

Active Member
Sep 1, 2013
122
39
28
if you want to retain dual routing tables (also possible) - your clients will essentially have 192.168.2.1 and 192.168.10.1 as their default gateway (for 0.0.0.0/0 traffic) and you will manually have to add a route for 192.168.10.0/24 next-hop 192.168.2.2 on the 192.168.2.0/24 clients and 192.168.2.0/24 next-hop 192.168.10.2 for the 10.x clients..
Sorry, to answer by two posts. But I first needed some time to understand your answer. This was very helpful for me.


What you described with dual routing tables seems to be exactly what I wanted to achieve. Now I know how to name it. Now it also is clear to me what I did wrong: Default gateway for clients to switch (.10.2 and 2.2) instead to firewall (10.1 and 2.1) and no next hop to the switch on the clients.

The need for a manual configuration of the next hop on the clients would be possible for me but seems to be not the "elegant way". The more common approach, if my understanding is correct, would be a set-up like shown below:

Networks 192.168.2.0/24 and 192.168.10.0/24 are present only on the L3 switch and the the L3 switch is default gateway for the clients. Default route on the switch for 0.0.0.0/0 trough tunnel network 192.168.1.0/30 to the firewall. For VLAN 20, 30 and 40 the switch is doing L2 switching only. No ve and no routing for these.

Or any better idea?

Network2.PNG