Layer 3 Switch w/ PFSense

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Pakna

Member
May 7, 2019
50
3
8
Usually what I have seen is it is related to the gateway and routing. In a layer 3 switch network you want the L3 switch to be the gateway for all clients. There will be a gateway for each network on the L3 switch. Then you want the default route on the switch pointing to the router. Then there should be routing statements on the router pointing to the L3 switch for all disconnected networks. More than likely, you need to open all disconnected networks in the firewall so they have internet access. I would check the gateway for each of the clients not able to ping. There should be a L3 path to the devices.

Adding multiple links between the router and the L3 switch can cause spanning tree problems. I only use 1 link but that will require the L3 switch to do DHCP or a device other than the router. The easy fix is to use 1 connection from pfsense to the L3 switch.
Thanks - my post may have been a bit longer but I didn't want to exclude any important details.

To my understanding, all of these points are satisified - including that I am using just a single transit link (not multiple transit networks). The only true problem is that B cannot ping its gateway on pfSense....which is the source of my frustration. I can ping the pfSense gateway from the switch (if I telnet to the switch).

I can also ping the switch end of the transit network from pfSense (i.e. the transit SVI or, routing interface in Dell parlance).
I can even ping the the Workstation B's SVI from pfSense (which I am using as the configured gateway for B).

All this tells me:
- the switch inter-VLAN routing is working
- the pfSense gateway is correctly configured
- the pfSense static route is correctly configured

By rights, it should work - but it doesn't and I can't really see a reason why it won't. I am also out of creative ideas what else to try.

EDIT: Correct me if I am wrong but you seem to note that I am defining more than 1 transit network? I don't see how I would be doing that - and that is certainly not my intention. Can you explain this a bit?
 

coxhaus

Active Member
Jul 7, 2020
109
36
28
If your routing was working correctly, you could ping.

You have configuration error somewhere. The simple fix is 1 connection. My way is creating a VLAN with a /30 mask between pfsense and the L3 switch. Route all traffic across this VLAN. I ran pfsense using a Cisco L3 switch for over a year this way. I dropped pfsense a few years ago. I still use a Cisco L3 switch.
 
Last edited:

Pakna

Member
May 7, 2019
50
3
8
If your routing was working correctly, you could ping.

You have configuration error somewhere. The simple fix is 1 connection.
Agreed. Given the current symptoms, do you suspect the problem is on switch or firewall side?

Where do I have more than one transit network? There is only one transit network - and Workstation B is using that one.
Are you suggesting that I should bring in the Workstation A into its own VLAN, without DHCP (so same as Workstation B) and then try to ping the pfsense gateway?
 
Last edited:

phil9878

Member
Apr 2, 2021
39
8
8
Agreed. Given the current symptoms, do you suspect the problem is on switch or firewall side?

Where do I have more than one transit network? There is only one transit network - and Workstation B is using that one.
Are you suggesting that I should bring in the Workstation A into its own VLAN, without DHCP (so same as Workstation B) and then try to ping the pfsense gateway?
I looked at your setup

First, and unrelated, you should drop VLAN1 and use a distinct VLAN for management
Second, and unrelated, use a /30 mask for the transit as it is not supposed to have any hosts on it

Now, for your problem, these are the most probable causes/fixes
- ensure Workstation B has 10.0.15.2 (Switch SVI) as gateway (sometimes, windows doesn't save that setting and you should properly verify it through ipconfig command for example)
- ensure pfSense has an allow rule on the VLAN 300 transit interface for traffic having VLAN 15 as source. By default, it will drop all traffic. The most convienient would be to create an alias named Private_Networks_RFC1918 and specify 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16. You then allow traffic on VLAN 300, from VLAN 15 to any network expect Private_Networks_RFC1918 (invert match rule). Do not add any explicit deny rule so that you can get the default deny rule logged for your initial debugging
- if you want to explicitly ping the VLAN 300 gateway on pfSense, you must add a specific allow rule for it on VLAN 300 interface. I do not recommend it, except for debugging
- if you use pfSense as your DNS server, you must allow access to port 53 on the VLAN 15 interface in pfSense
- on the VLAN 15 interface in pfSense, no allow rules should be needed except if you need DNS or forwarding broadcast traffic (10.0.15.255 and 255.255.255.255)
- ensure the VLAN 300 port is untagged (Access mode) on the switch and not member of any trunks
- communication between VLAN 1 and VLAN 15 hosts cannot be done if VLAN 1 hosts gateway doesn't point to your switch VLAN 1 SVI (asymmetric routing). Only the switch can be a gateway for any inter-VLAN routing

But I really suspect either a gateway issue in Workstation B or a missing firewall rule in pfSense

Check pfSense logs for any "default deny rule" dropped traffic entries on VLAN 300 and VLAN 15. For that, just add the allow rule to non RFC1918 domains and do not explicitely add any deny rule. Default deny rule is always logged
 

coxhaus

Active Member
Jul 7, 2020
109
36
28
As I said the simple setup is do all your local routing in the L3 switch and use pfsense for internet traffic only with 1 connection.

This setup also allows you to use multiple L3 switches deep in your network. Using DHCP on pfsense does not.
 

coxhaus

Active Member
Jul 7, 2020
109
36
28
I have used a 24 bit mask to connect pfsense and a Cisco L3 switch. The trick is to use the L3 switch as the gateway for all clients on the /24 mask not the router as using the router's gateway will not work. The client's traffic will forward to the L3 switch where all local networks are located so local routing will happen on the L3 switch. All unknow traffic will be forwarded through the L3 switch's default route to pfsense and out to the internet. Then return routing will point to the L3 switch on pfsense. And of course, do not block any of this with the firewall.
 

Pakna

Member
May 7, 2019
50
3
8
I looked at your setup

First, and unrelated, you should drop VLAN1 and use a distinct VLAN for management
Second, and unrelated, use a /30 mask for the transit as it is not supposed to have any hosts on it

Now, for your problem, these are the most probable causes/fixes
- ensure Workstation B has 10.0.15.2 (Switch SVI) as gateway (sometimes, windows doesn't save that setting and you should properly verify it through ipconfig command for example)
- ensure pfSense has an allow rule on the VLAN 300 transit interface for traffic having VLAN 15 as source. By default, it will drop all traffic. The most convienient would be to create an alias named Private_Networks_RFC1918 and specify 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16. You then allow traffic on VLAN 300, from VLAN 15 to any network expect Private_Networks_RFC1918 (invert match rule). Do not add any explicit deny rule so that you can get the default deny rule logged for your initial debugging
- if you want to explicitly ping the VLAN 300 gateway on pfSense, you must add a specific allow rule for it on VLAN 300 interface. I do not recommend it, except for debugging
- if you use pfSense as your DNS server, you must allow access to port 53 on the VLAN 15 interface in pfSense
- on the VLAN 15 interface in pfSense, no allow rules should be needed except if you need DNS or forwarding broadcast traffic (10.0.15.255 and 255.255.255.255)
- ensure the VLAN 300 port is untagged (Access mode) on the switch and not member of any trunks
- communication between VLAN 1 and VLAN 15 hosts cannot be done if VLAN 1 hosts gateway doesn't point to your switch VLAN 1 SVI (asymmetric routing). Only the switch can be a gateway for any inter-VLAN routing

But I really suspect either a gateway issue in Workstation B or a missing firewall rule in pfSense

Check pfSense logs for any "default deny rule" dropped traffic entries on VLAN 300 and VLAN 15. For that, just add the allow rule to non RFC1918 domains and do not explicitely add any deny rule. Default deny rule is always logged
Thanks for the time looking at my setup in more details - appreciate it.

As for unrelated advice you had made, I agree with it completely and I'll apply the changes as my testing progresses but was trying to make simplest config work end-to-end. I can report back on that later.

I've ensured Workstation B has the VLAN 15 SVI as its gateway - Windows are really annoying with this but as you mention, ipconfig confirms this value is correctly set. Also confirmed VLAN 300 port is in Access mode and not member of any trunks.

As for the pfSense configuration, here is the part where I am confused - based on the previous posts on this threads (and similar blogs related to what I am trying to do here), my understanding was there shouldn't be _any_ VLAN definitions on the pfSense with this setup and didn't create any other VLAN-specific interface.
The current config is as it's on the diagram - for firewall end of transit net, I have defined just the FIBRE1 interface with static IP 172.16.0.1 (to be honest, I don't fully understand how that works with packets tagged with VLAN 300 or why I even needed to go through that trouble if I didn't have a VLAN-aware pfSense interface but.....I went with it anyway). The switch default IP route targets the IP of the FIBRE1 and I am allowing any and all traffic on that FIBRE1 port.

I suspect there is a problem the way this is configured, so suggest we skip the rest of your points to see if this is the reason why it's not working?

Here is how I thought traffic would be routed:
- traffic from Workstation B gets to VLAN 15 SVI gateway
- VLAN 15 SVI doesn't recognise the IP address and uses inter-VLAN routing to reach its next hop, which is the VLAN 300 SVI
- VLAN 300 SVI also doesn't recognise the IP address, tags the packet with PVID 300 and sends it to pfSense
- pfsense gets the packet on FIBRE1 port, doesn't recognise the IP address but uses its routing table to figure out where the packet should egress (that would be FIBRE0 port)
- packet reaches the destination on FIBRE0 net and the inverse process begins for the response
 

kapone

Well-Known Member
May 23, 2015
1,095
642
113
Here is how I thought traffic would be routed:
- traffic from Workstation B gets to VLAN 15 SVI gateway
- VLAN 15 SVI doesn't recognise the IP address and uses inter-VLAN routing to reach its next hop, which is the VLAN 300 SVI
- VLAN 300 SVI also doesn't recognise the IP address, tags the packet with PVID 300 and sends it to pfSense
- pfsense gets the packet on FIBRE1 port, doesn't recognise the IP address but uses its routing table to figure out where the packet should egress (that would be FIBRE0 port)
- packet reaches the destination on FIBRE0 net and the inverse process begins for the response
Not quite...

Question:
- traffic from Workstation B gets to VLAN 15 SVI gateway (egress)
- The switch realizes that you're trying to get to 192.168.1.x, which is not in VLAN15
- It checks it's routing table to see if it knows how to route to 192.168.1.x and....finds VLAN 1?? How is VLAN1 defined at the switch level? What's the SVI for VLAN1?
 

phil9878

Member
Apr 2, 2021
39
8
8
@kapone: as I understood it, he described the routing table to internet traffic, which is quiet right in his case. But yes, it can be he thinks it is also for the inter VLAN routing. I saw in his table that he has no internet traffic at all and cannot ping VLAN 300 interface on pfSense

@Pakna

For inter VLAN routing:
- traffic never reaches pfSense. If you are not using it as a DHCP (which is your case) or internal DNS server, you can turn off pfSense and it must work
- ALL HOSTS must point their gateway to their VLAN SVI on the switch: in your case, it means also Workstation A ! Drop the DHCP for now and manually set its IP in VLAN 1 AND the gateway to 192.168.1.2 if that is your VLAN 1 SVI on the switch
- If you really pointed Workstation A gateway to Fiber 0 on pfSense, you will not be able to route beteen the two VLANs because of asymmetric traffic

For internet Traffic:
your routing table understanding is nearly right, except:
- VLAN 300 sends the packet to its pfSense interface tagged with PVID 300, untagged since it is an access port
- pfsense gets the packet on FIBRE1 port, doesn't recognise the IP address but uses its routing table to figure out where the packet should egress (that would be FIBRE0 port) that is the WAN port to internet. pfSense keeps a state knowing that the traffic comes from Workstation B IP address
- packet reaches the destination on FIBRE0 net on internet and the inverse process begins for the response
- pfSense sends back the internet traffic from Fiber 1 to VLAN 300 SVI !!!
- the switch routes the internet traffic back to Workstation B on VLAN 15

So, in pfSense, to have internet working and to be able to ping Fiber 1 interface from VLANs 15 and 1, you must add specific ALLOW rules on the Fiber 1 interface (which is your VLAN 300 transit domain/route). The ALLOW rules must allow traffic from "any" to "any" to start with

Once it works, you can restrict access by specific Allow rules:
- Allow PING from Any to Fiber 1 address (172.16.0.1)
- Allow DNS port 53 from Any to Fiber 1 address (172.16.0.1) if you use it as DNS server
- Allow NTP port 123 from Any to Fiber 1 address (172.16.0.1) if you use it as NTP server
- Allow from 10.0.15.0/24 (aka VLAN 15) to NOT (invert match rule) an alias named Private_Networks_RFC1918 with 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16.
- Allow from 192.168.0.0/24 (aka VLAN 1) to NOT (invert match rule) the alias named Private_Networks_RFC1918

Any traffic not matching these rules will be dropped
If your network is properly configured, only internet traffic will reach pfSense Fiber 1.
If you use pfSense as DNS or NTP server, add the allow rules also from the VLAN networks to the pfSense corrsponding interface
 
Last edited:

coxhaus

Active Member
Jul 7, 2020
109
36
28
Thanks for the time looking at my setup in more details - appreciate it.

As for unrelated advice you had made, I agree with it completely and I'll apply the changes as my testing progresses but was trying to make simplest config work end-to-end. I can report back on that later.

I've ensured Workstation B has the VLAN 15 SVI as its gateway - Windows are really annoying with this but as you mention, ipconfig confirms this value is correctly set. Also confirmed VLAN 300 port is in Access mode and not member of any trunks.

As for the pfSense configuration, here is the part where I am confused - based on the previous posts on this threads (and similar blogs related to what I am trying to do here), my understanding was there shouldn't be _any_ VLAN definitions on the pfSense with this setup and didn't create any other VLAN-specific interface.
The current config is as it's on the diagram - for firewall end of transit net, I have defined just the FIBRE1 interface with static IP 172.16.0.1 (to be honest, I don't fully understand how that works with packets tagged with VLAN 300 or why I even needed to go through that trouble if I didn't have a VLAN-aware pfSense interface but.....I went with it anyway). The switch default IP route targets the IP of the FIBRE1 and I am allowing any and all traffic on that FIBRE1 port.

I suspect there is a problem the way this is configured, so suggest we skip the rest of your points to see if this is the reason why it's not working?

Here is how I thought traffic would be routed:
- traffic from Workstation B gets to VLAN 15 SVI gateway
- VLAN 15 SVI doesn't recognise the IP address and uses inter-VLAN routing to reach its next hop, which is the VLAN 300 SVI
- VLAN 300 SVI also doesn't recognise the IP address, tags the packet with PVID 300 and sends it to pfSense
- pfsense gets the packet on FIBRE1 port, doesn't recognise the IP address but uses its routing table to figure out where the packet should egress (that would be FIBRE0 port)
- packet reaches the destination on FIBRE0 net and the inverse process begins for the response
I build networks so workstations don't know anything about VLANs. The workstations just receive IP addresses from DHCP. Why would you want to involve a workstation in your network? I consider the workstation a user of the network not part of it.
 

Pakna

Member
May 7, 2019
50
3
8
Not quite...

Question:
- traffic from Workstation B gets to VLAN 15 SVI gateway (egress)
- The switch realizes that you're trying to get to 192.168.1.x, which is not in VLAN15
- It checks it's routing table to see if it knows how to route to 192.168.1.x and....finds VLAN 1?? How is VLAN1 defined at the switch level? What's the SVI for VLAN1?
I think phil9878 got the gist of my explanation right - but to be exact: VLAN1 is merely the default VLAN specified with the switch ("stock" switch configuration).
It does not have its SVI defined at all. My understanding is that it must be defined by default in all switches but should not be used unless required (I believe infrastructure traffic like VRRP/HA, switch stacking or OSPF discovery would be the kind of things that should be traversing VLAN 1. Correct me if I am wrong.)
AFAIK, one can't delete VLAN1 - it must be defined.
The only reason why I am using it is because I am trying to prototype this as simple as possible - as I progress, the intent is to, of course, not to use it.

I build networks so workstations don't know anything about VLANs. The workstations just receive IP addresses from DHCP. Why would you want to involve a workstation in your network? I consider the workstation a user of the network not part of it.
I have to admit that your questions confuse me.

Previously you stated I am using multiple transit network - I can't see where this is the case in my configuration.
You are now stating that I would involve a workstation in my network - but that was never my intent.
In my design, I see no reason to have leaf network nodes (i.e. workstations or servers) VLAN-aware - that is just a complication without any benefit, in my situation.
Hence I use Access not Trunk ports.

Perhaps my explanations are unclear - I apologise, if that is the case. I am a software, not a network engineer, so my lingo may not be exact.

If you still think that I am somehow configuring multiple transit nets, please do explain.
 

Pakna

Member
May 7, 2019
50
3
8
@kapone: as I understood it, he described the routing table to internet traffic, which is quiet right in his case. But yes, it can be he thinks it is also for the inter VLAN routing. I saw in his table that he has no internet traffic at all and cannot ping VLAN 300 interface on pfSense

@Pakna

For inter VLAN routing:
- traffic never reaches pfSense. If you are not using it as a DHCP (which is your case) or internal DNS server, you can turn off pfSense and it must work
- ALL HOSTS must point their gateway to their VLAN SVI on the switch: in your case, it means also Workstation A ! Drop the DHCP for now and manually set its IP in VLAN 1 AND the gateway to 192.168.1.2 if that is your VLAN 1 SVI on the switch
- If you really pointed Workstation A gateway to Fiber 0 on pfSense, you will not be able to route beteen the two VLANs because of asymmetric traffic

For internet Traffic:
your routing table understanding is nearly right, except:
- VLAN 300 sends the packet to its pfSense interface tagged with PVID 300, untagged since it is an access port
- pfsense gets the packet on FIBRE1 port, doesn't recognise the IP address but uses its routing table to figure out where the packet should egress (that would be FIBRE0 port) that is the WAN port to internet. pfSense keeps a state knowing that the traffic comes from Workstation B IP address
- packet reaches the destination on FIBRE0 net on internet and the inverse process begins for the response
- pfSense sends back the internet traffic from Fiber 1 to VLAN 300 SVI !!!
- the switch routes the internet traffic back to Workstation B on VLAN 15

So, in pfSense, to have internet working and to be able to ping Fiber 1 interface from VLANs 15 and 1, you must add specific ALLOW rules on the Fiber 1 interface (which is your VLAN 300 transit domain/route). The ALLOW rules must allow traffic from "any" to "any" to start with

Once it works, you can restrict access by specific Allow rules:
- Allow PING from Any to Fiber 1 address (172.16.0.1)
- Allow DNS port 53 from Any to Fiber 1 address (172.16.0.1) if you use it as DNS server
- Allow NTP port 123 from Any to Fiber 1 address (172.16.0.1) if you use it as NTP server
- Allow from 10.0.15.0/24 (aka VLAN 15) to NOT (invert match rule) an alias named Private_Networks_RFC1918 with 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16.
- Allow from 192.168.0.0/24 (aka VLAN 1) to NOT (invert match rule) the alias named Private_Networks_RFC1918

Any traffic not matching these rules will be dropped
If your network is properly configured, only internet traffic will reach pfSense Fiber 1.
If you use pfSense as DNS or NTP server, add the allow rules also from the VLAN networks to the pfSense corrsponding interface
Phil, I think it works - thanks a lot for these pointers!

It was a compound configuration error (the worst kind):
- the FIBRE1 allow-all rule had its Source set to.... FIBRE1 net, which I somehow misunderstood as "any traffic coming from FIBRE1 NIC" as opposed to "any traffic coming from FIBRE1 NET".
I've set the Source to "any" for the time being.

1623650865891.png

- it seems that with Dell 8024F, it is not enough to simply define a global default route and use it - the default route has to have its routing mode "Enabled".
My best guess is that it is defining the global default route as an SVI (or - "routing interface" in Dell lingo) that you have to explicitly enable.
I do not recall finding this detail in any of the Cisco manuals or articles I've gone through.
Haven't configured any other switch in this fashion but this seems a bit surprising to me. Here is the picture of this option I had to turn on manually.

1623650831195.png

After sorting out these two problems, I was able to ping the FIBRE1 IP (i.e. firewall end of transit network).

But I was still not able to ping internet addresses - despite having the FIBRE1 IP set as the DNS server.
Turns out pfSense's DNS resolver was refusing queries from VLAN 15, so I added an entry to its Access List.

dns.JPG

Another salient point - I have reduced the transit net subnet mask to /29 - in the future, I intent to explore running pfSense in active-active HA via Carp, so there might be a few more hosts coming to that VLAN 300).
Not sure if that subnet mask restriction did anything here - my understanding is that it just has to match on both ends of the transit net and that is it.

My next steps are:
- enable DHCP server on the switch and make sure that it assignes the correct gateway/DNS values to the NIC
- restrict the firewall rules on FIBRE1 as you suggest

I'll see how that goes and report back in.

Thanks again @phil9878 and to the rest of the people in this awesome thread. I've read through hundreds of pages of switch manual(s) and dozens of articles dealing with similar setup - without you, I would probably lose what little hair I have trying to make this work. It is surprisingly tricky, at least to a layman like myself.
 

coxhaus

Active Member
Jul 7, 2020
109
36
28
I have to admit that your questions confuse me.

Previously you stated I am using multiple transit network - I can't see where this is the case in my configuration.
You are now stating that I would involve a workstation in my network - but that was never my intent.
In my design, I see no reason to have leaf network nodes (i.e. workstations or servers) VLAN-aware - that is just a complication without any benefit, in my situation.
Hence I use Access not Trunk ports.

Perhaps my explanations are unclear - I apologise, if that is the case. I am a software, not a network engineer, so my lingo may not be exact.

If you still think that I am somehow configuring multiple transit nets, please do explain.
I never stated you were using multiple transit network. Maybe I am not understanding you. I was a network guy before I retired. If you are using an access port for your client then he is not involved in the network and your client is not aware of VLANs as he is just talking to the IP assigned to him I assume by DHCP.

All my talk is high level network talk. It has been a few years since I ran pfsense and I don't remember setting as I have used lots of routers. It took about 4 hours to get my pfsense working with an L3 switch. There is a thread on pfsense under my username same as here where I asked a few question on how pfsense handled things.

I suggest you open up the networks in pfsense in the firewall that will be handled by the L3 switch and not complicate things by defining VLANs in pfsense. Work at layer 3 in pfsense like it is a router.
 
Last edited:
  • Like
Reactions: Pakna

phil9878

Member
Apr 2, 2021
39
8
8
- the FIBRE1 allow-all rule had its Source set to.... FIBRE1 net, which I somehow misunderstood as "any traffic coming from FIBRE1 NIC" as opposed to "any traffic coming from FIBRE1 NET".
I've set the Source to "any" for the time being.
It is the best to start with
After that, just reduce it to the domains corresponding to your VLANs for source and only to non Local networks for the target
Remember to allow access to DNS server address, NTP server or any service you need on pfSense
If you add an explicit deny rule, it is advised to manually enable logging for debugging, at least at the beginning. Usually, the default deny rule implies that you should never need a lot of explicit deny rules if your rules are properly setup.

- it seems that with Dell 8024F, it is not enough to simply define a global default route and use it - the default route has to have its routing mode "Enabled"...
I do not recall finding this detail in any of the Cisco manuals or articles I've gone through.
In fact, on Cisco switches there is not an enable/disable for the Next Hop Router, so I wasn't aware of it in your case.

Turns out pfSense's DNS resolver was refusing queries from VLAN 15, so I added an entry to its Access List.
You are right. Since you define VLAN interfaces outside pfSense, it is not aware about those domains.
You can read it in the doc:

By default, IPv4 and IPv6 networks residing on internal interfaces of this firewall are permitted. Additional networks must be allowed manually.
Take time to read the help pages on pfSense. They are well done in my opinion. There are also a lot of tricks to restrict the DNS resolver and secure it with good examples.

Not sure if that subnet mask restriction did anything here - my understanding is that it just has to match on both ends of the transit net and that is it.
No issues. The transit route is just not supposed to have hosts / clients on it. So limit it to the bare minimum. It must be in access mode by the way. If you stick to the rule of never having a native VLAN on your trunks, you understand that ideally it should have its own physical interface. Except aggregation, I do not see the need to have it greater than /30

My next steps are:
- enable DHCP server on the switch and make sure that it assignes the correct gateway/DNS values to the NIC
- restrict the firewall rules on FIBRE1 as you suggest
Do not forget the above posts: your gateway must be the interface SVI on the switch if you need interVLAN routing. If you do not need it for the set VLAN, you can set it to pfSense interface. However, it is more complexity for no benefit. Just use teh ACLs rules on the switch to secure traffic

About your question on VLAN1: even if you delete it like in Cisco, it will still be used. So just keep it and if possible do not assign it any interface (Cisco allow it). If an interface is always assigned, just do not assign VLAN 1 on any port. Rename your management VLAN to something like VLAN 950 or what ever you like. Keep it only for management. I use a dedicated interface that is only activated if I need it. However, on a home / small biz setup, it is really easier and rather safe to allow management from the workstation VLAN too. Just secure your ACLs, restrict access to management interfaces from other VLANs/IPs and activate DHCP snooping for additional security
 
  • Like
Reactions: Pakna

seatrope

New Member
Oct 5, 2018
27
11
3
Maine
www.ychng.com
Thank you all for sharing your expertise, especially @kapone!

Following all these guides and posts I managed to get L3 inter-VLAN routing on my ICX-6610 working well, with pfSense only serving firewall duties. DHCP/DNS to all subnets handled by piHole (dnsmasq) which seems to work well while blocking ads too.
I have used a "TRANSIT" VLAN/connection between the 6610 and pfsense as @kapone detailed.

However, the final piece eludes me. Right now I still have 2 physical links between the 6610 and the pfsense box, the "TRANSIT" link and the former "LAN" link.

I can detach the LAN link and everything else works fine, including access to pfSense GUI. The LAN link is marked as being "DOWN". I can't seem to figure out a way to get rid of the "LAN" link and have just the one "TRANSIT" link.

1659012829060.png


Have others gotten rid of the original "LAN" link? Seems like this is tied to pfSense's management/GUI IP, so maybe this is not possible. Do others just have the "LAN" or whatever you name it interface remaining, but disconnected like I do above?
Have spent 2 days trying to figure this piece out without any luck! Functionally doesn't impact anything but it bugs my OCD lol.

Thanks so much!
 

seatrope

New Member
Oct 5, 2018
27
11
3
Maine
www.ychng.com
Never mind, I'm an idiot. just figured it out, just remembered that I could use the vif IPs to access the brocade switch on any of my VLANs. Same thing here, realized that i just needed to use the corresponding pfsense interface IP on the TRANSIT network as the pfsense GUI!

All fixed :)

1659013163576.png
 

tfran1990

New Member
Jul 19, 2019
3
1
1
I am using this guide for the transit link approach for my icx6450 with pfsense
i seem to be missing something for step 9 firewall rules

[Source] 192.168.10.0/24
[Port] *
[Destination] This Firewall
[Port] "your PfSense Web Port"


on my firewall log when i filter blocked i see


192.168.20.0 is my vlan 20 its like its not able to get out to the WAN

not seeing where to set
[Port] "your PfSense Web Port"


This is what i have
1666332704180.png1666332704180.png
 
  • Like
Reactions: shremi

thehedgefrog

New Member
Sep 14, 2022
5
1
3
I'm having a hard time finding my specific scenario anywhere. I currently have a segmented L2 network but it's overtaxing my pfSense firewall. I am switching my Cisco SG300 with an ICX-6610 and want to have my trusted high-speed VLANs routed via L3 on the switch while keeping some on the firewall.

DHCP/DNS is on Windows Server. I know I have to create router interfaces for my 4 trusted VLANs, change DHCP option 3 to the IP of that interface and then it will route on the switch. If I ping a device on VLAN 200-Storage from VLAN 20-Trusted, it works. However, what happens when a device on 20 wants to ping a device on VLAN 30-IoT? The rules on the firewall allow that, but how do I ensure what is NOT going to my L3 VLANs get routed to pfSense?