Layer 3 Switch w/ PFSense

coxhaus

Member
Jul 7, 2020
86
32
18
Yes it does.
I defined no VLANs on my pfsense. It may default to VLAN1 but I did not define it. And when I connect my VLAN 10 on my L3 switch it talks to pfsense fine. So I think there really is no VLAN defined. When I change from a 24 bit mask to a 30 bit mask I pulled DHCP on pfsense and used static IPs. I started with a full class C network and pfsense recommended I move over to a 30 bit mask.
 
Last edited:

kapone

Well-Known Member
May 23, 2015
1,036
603
113
I defined no VLANs on my pfsense. It may default to VLAN1 but I did not define it. And when I connect my VLAN 10 on my L3 switch it talks to pfsense fine. So I think there really is no VLAN defined.
"Talking" is...not what we're talking about here, are we? We're talking about pfSense being the DHCP server for a LAN/VLAN, for which it has NO defined interface. That's not going to happen (at present).
 
  • Like
Reactions: phil9878

coxhaus

Member
Jul 7, 2020
86
32
18
"Talking" is...not what we're talking about here, are we? We're talking about pfSense being the DHCP server for a LAN/VLAN, for which it has NO defined interface. That's not going to happen (at present).
There is a defined interface. It is the NIC on pfsense on the LAN side where you assign an IP address. Besides don't use DHCP on pfsense as I stated above.
 

kapone

Well-Known Member
May 23, 2015
1,036
603
113
pfsense does not need a VLAN for DHCP to work. In the past you could define DHCP for 1 network without a VLAN. I think the less information in your firewall device the better. I use static IPs in my router LAN network to my L3 switch.
Maybe I interpreted it wrong, but that's not quite what I understood from your statement.

Regardless, I think we're on the same page on that.
 

Vesalius

Active Member
Nov 25, 2019
121
83
28
pfsense does not need a VLAN for DHCP to work. In the past you could define DHCP for 1 network without a VLAN. I think the less information in your firewall device the better. I use static IPs in my router LAN network to my L3 switch.
Well I would be happy to be proven wrong if pfSense can indeed provide dhcp for multiple networks, such as all the vlans only defined on your L3 switch, while pfSense only has a single transit interface defined in it's settings?

I think we may be talking about slightly different scenarios. If your L3 switch has a well implemented dhcp server then this is obviously a viable route to take wITH only have a single TRANSIT connection defined in pfSense/opnsense and no vlan. That is not an option for many of us here, including the OP of this thread, that use brocade L3 icx switches as the Ruckus/brocade dhcp server implementation is not authoritative and won’t suffice as many connected devices fail to obtain a dhcp address from them.

Does the cisco sg350x that @phil9878 denotes in his network diagram have a dhcp server up to the task of his network plan?
 
Last edited:
  • Like
Reactions: phil9878

phil9878

Member
Apr 2, 2021
39
6
8
Does the cisco sg350x that @phil9878 denotes in his network diagram have a dhcp server up to the task of his network plan?
You are totally right: pfSence DHCP server won't handle the task except if I add the VLAN interfaces in pfSense or if I connect physical interfaces. I went with option 2.

Both the SG350X and SG350 can have a DHCP server. However, it is far from convenient like in pfSense. It doesn't support assigning a DNS by lease or by subnet if I am not wrong, at least not in the same easy way as pfSense. I tried it and I would rather go setting up a dedicated DHCP machine later instead of using the Cisco DHCP on the switch. I can use it however for light tasks like assigning DHCP to VLAN5, 40 and eventually 50 in my diagram as I don't care a lot about those devices right now.

...You also need to add ACLs for the gateways to allow internet access as pfsense blocks all traffic other than pfsense traffic.
...pfsense gateways will point to the IP address on the L3 switch which connects to pfsense.

To define pfsense and a L3 switch together use example, 192.168.10.1/30 pfsense and 192.168.10.2/30 L3 switch. The mask would be 255.255.255.252. On the L3 switch side create a VLAN 10 and assign an IP of 192.168.10.2 255.255.255.252. Then on the L3 switch assign the default route, some may have default gateway, 192.168.10.1 which is pfsense's LAN interface. Assign a port to VLAN 10 and connect a CAT5e cable from the assigned port to the LAN port on pfsense.

There is a thread on L3 switches on pfsense forums as I wrote all that I did to make pfsense work with a Cisco L3 switch.
Thank you again.

The quad NIC will arrive tomorrow. I will play with it the next days with all the suggestions in this thread.

The complexity is because of my 2 switch layers. I could simplify it and put the two switches on the same VLAN, but the network isolation would be less optimal I guess.

In my diagram, as I understand it:
- Switch SG350X doing all the interVLAN routing (10.0.10.254) would need for each VLAN a static route pointing to the SG350 (10.0.20.254)
- Switch SG350 (10.0.20.254) would need for each VLAN a static route pointing to pfSense (10.0.20.1)
- pfSense will need gateways pointing to the SG350 (10.0.20.254) for each interface + ACL/NAT rules to allow internet traffic for each of the subnets I guess

Internet Network Diagram Template 2.jpg

Is my understanding good or am I still missing some parts ?

Edit: I will also do the reading of your thread here: Installing pfSense with a layer 3 switch
 
Last edited:

coxhaus

Member
Jul 7, 2020
86
32
18
In my diagram, as I understand it:
- Switch SG350X doing all the interVLAN routing (10.0.10.254) would need for each VLAN a static route pointing to the SG350 (10.0.20.254)
- Switch SG350 (10.0.20.254) would need for each VLAN a static route pointing to pfSense (10.0.20.1)
- pfSense will need gateways pointing to the SG350 (10.0.20.254) for each interface + ACL/NAT rules to allow internet traffic for each of the subnets I guess
You do need a static route from pfsense to the L3 switch. There is no static route pointing to pfsense as default route or default gateway points to pfsense which takes care of the forwarding for unknown traffic. And by default, pfsense blocks all traffic except for pfsense traffic which would be directly connected networks so you need ACLs for L3 networks that are not directly connected to pfsense.
 
  • Like
Reactions: phil9878

phil9878

Member
Apr 2, 2021
39
6
8
I am a bit disppointed, since I asked also in this thread after you pointed me to Netgate forums:

As you see, the moderator there states that this is impossible because of the asymmetrical routing issue.

For him, what was always advised here, adding real physical NICs for each interface, is not true. pfSense would still route directly the downstream traffic like it would do with VLANs for each interface and a trunk, thus leading to asymmetrical routing.

Any one with advanced network knowledge can point to a good reference where this issue is clearly addressed ? I will be waiting before opening the new NIC packet so that I can RMA it if it doesn't answer my needs.

Thank you again for your help
 
Last edited:

coxhaus

Member
Jul 7, 2020
86
32
18
Sounds like pfsense only supports directly connected networks for DHCP. You need an L3 switch for DHCP or maybe Microsoft DHCP. I have used both of these. There are other options. There really is no need for DHCP on pfsense when you use a 30 bit mask connected to an L3 switch as it is a point-to-point connection. If you use a full class C IP address space then you can use DHCP as I did at first but pfsense does not recommend it. They told me to use a 30-bit mask instead.

Like I said pfsense does not want you really to use an L3 switch. They don't code for it. It will work. This is why I say pfsense is only for tiny networks as all large networks have L3 switches in them. This is why I switched back to a Cisco RV340 router instead of pfsense plus the new firmware does not have bugs like pfsense.

If you want a much better firewall for home use you can run Untangle for $50 a year. It runs off a PC like pfsense. It is not cheap for business use.

Your diagram looks like it is setup so pfsense does all the routing and you don't need layer 3 switching. It will not run as fast as L3 wire speed switching can. If your load is very light then you may not notice. There is no reason for a 4 port NIC card in pfsense when you use L3 switching.

I worked with John on the pfsense forums. He is a hard man to work with. Last I knew he used his Cisco small business L3 switch as layer 2 so pfsense could do all the routing. He does understand networking.
 
Last edited:
  • Like
Reactions: phil9878

itronin

Well-Known Member
Nov 24, 2018
646
372
63
Denver, Colorado
Sounds like pfsense only supports directly connected networks for DHCP. You need an L3 switch for DHCP or maybe Microsoft DHCP.
sigh :confused: There is no "Sounds like". It is a fact (covered previously in this thread #24 and many times elsewhere). If you want to use PFsense for DHCP in a specific vlan then PFsense MUST have an interface in that vlan. If you decide to use your L3 switch make darn sure that it really and correctly implements DHCP including all the options you may need (check bug reports, user experiences etc.) OR you may find yourself chasing down rabbit holes needlessly :eek:

Better and concur with using something else: Microsoft DHCP is great, very mature works well especially if you have a windows approach to the world. ISC DHCP if you have a *nix approach to the world "tastes great less filling" :p
 

coxhaus

Member
Jul 7, 2020
86
32
18
Well I would be happy to be proven wrong if pfSense can indeed provide dhcp for multiple networks, such as all the vlans only defined on your L3 switch, while pfSense only has a single transit interface defined in it's settings?

I think we may be talking about slightly different scenarios. If your L3 switch has a well implemented dhcp server then this is obviously a viable route to take wITH only have a single TRANSIT connection defined in pfSense/opnsense and no vlan. That is not an option for many of us here, including the OP of this thread, that use brocade L3 icx switches as the Ruckus/brocade dhcp server implementation is not authoritative and won’t suffice as many connected devices fail to obtain a dhcp address from them.

Does the cisco sg350x that @phil9878 denotes in his network diagram have a dhcp server up to the task of his network plan?
Yes, the Sg350x L3 switch does have a DHCP server. It is not as robust as Microsoft's DHCP server. It does have what I need for home. I like Microsoft's better but I have turned off my server rack.

I have used DHCP on Cisco's L3 switches SG300, SG500X, SG350, and SG350X. The SG350X I have I have only played with it. The fans were too noisy on the SG500X-24 for me. I set it up at my daughter's work.
 
Last edited:
  • Like
Reactions: phil9878

phil9878

Member
Apr 2, 2021
39
6
8
My main question is now: do my setup have any real asymmetric routing since no devices can take different routes for return traffic. So, as long as I am aware of it, I could use it until I can afford a dedicated DHCP server.

Is it really unsafe to run this way in my case because of asymmetric routing I am not aware of ?
 

kapone

Well-Known Member
May 23, 2015
1,036
603
113
My main question is now: do my setup have any real asymmetric routing since no devices can take different routes for return traffic. So, as long as I am aware of it, I could use it until I can afford a dedicated DHCP server.

Is it really unsafe to run this way in my case because of asymmetric routing I am not aware of ?
I have said this many many times in many threads like this.

Start SIMPLE.

- Question your need for two separate switches. Why do you need two? The answer may be yes, but make sure you understand WHY you're using two switches.
- Start with one switch, forget DHCP/DNS/pfSense. Configure any and all VLANs on it and test using IP addresses only. Is the switch doing inter-vlan routing correctly?
- Start with the second switch, COMPLETELY disconnected from the first switch. Repeat the test above.
- Now decide, how are these two switches going to connect together? Again, forget DHCP/DNS/pfSense. Can a device on one VLAN on switch 1 talk correctly to a device on a VLAN on switch 2?

If the answers/tests to all of these are correct, now we address the rest of the issues.

- Forget pfSense, let's address DHCP and DNS.
- How and what will be your DHCP/DNS server for both of these interconnected switches?
- If the answer is well...that's what I was hoping pfSense will do...then...you have absolutely NO choice. You MUST have individual interfaces in pfSense that represent each VLAN on BOTH switches AND you need to configure pfSense correctly so that it has static gateways and static routes to send traffic back to each switch. This is not easy or straightforward.

My advice? Think real hard as to why you're using two switches.
 

coxhaus

Member
Jul 7, 2020
86
32
18
My main question is now: do my setup have any real asymmetric routing since no devices can take different routes for return traffic. So, as long as I am aware of it, I could use it until I can afford a dedicated DHCP server.

Is it really unsafe to run this way in my case because of asymmetric routing I am not aware of ?
That asymmetric routing seems to be one of John's favorite terms. I would not worry about it. I ran with what he called asymmetric routing and I saw no issues. I don't think your diagram is right for L3 switching. So, I think you kind of need to decide if you are going to do L3 switching or L2 switching. I hang L2 switches off my L3 switch but I don't let my router do local routing. I save my router for internet traffic only.

For DNS I use QUAD9 as I think it is the best in times like now. It may not be the fastest but I think it is the safest but this is my opinion. When I used a local DNS, I used Microsoft.
 
Last edited:
  • Like
Reactions: phil9878

jjacobs

Member
Dec 25, 2020
48
21
8
NC
I have said this many many times in many threads like this.

Start SIMPLE.

- Question your need for two separate switches. Why do you need two? The answer may be yes, but make sure you understand WHY you're using two switches.
- Start with one switch, forget DHCP/DNS/pfSense. Configure any and all VLANs on it and test using IP addresses only. Is the switch doing inter-vlan routing correctly?
- Start with the second switch, COMPLETELY disconnected from the first switch. Repeat the test above.
- Now decide, how are these two switches going to connect together? Again, forget DHCP/DNS/pfSense. Can a device on one VLAN on switch 1 talk correctly to a device on a VLAN on switch 2?

If the answers/tests to all of these are correct, now we address the rest of the issues.

- Forget pfSense, let's address DHCP and DNS.
- How and what will be your DHCP/DNS server for both of these interconnected switches?
- If the answer is well...that's what I was hoping pfSense will do...then...you have absolutely NO choice. You MUST have individual interfaces in pfSense that represent each VLAN on BOTH switches AND you need to configure pfSense correctly so that it has static gateways and static routes to send traffic back to each switch. This is not easy or straightforward.

My advice? Think real hard as to why you're using two switches.
I agree with all of this.

I'll add that at the end of the process you may want to consider whether pfsense is the correct choice for your intended architecture? pfsense is a useful and capable all-in-one appliance but something like VyOS may serve you better as a simpler LAN<->WAN router/firewall.
 
  • Like
Reactions: phil9878

phil9878

Member
Apr 2, 2021
39
6
8
My advice? Think real hard as to why you're using two switches.
Yes, I have no choice. The locations in my diagram are hardwired and all cabling reaches the SG350 switch. I only have two hardwires reaching the SG350X server and the setup there needs to be 10 Gb. The desktop in reality is in the same location as the SG350 and uses a hard wired direct to the SG350X.

I will start step by step and be back when i have issues or to give feedback

Thank you all again
 
Last edited:
  • Like
Reactions: jjacobs

coxhaus

Member
Jul 7, 2020
86
32
18
So, looking at your diagram to change to layer 3 switching you would need to change pfsense to not use VLANs on pfsense and define DHCP on the SG350 switch or like a Microsoft DHCP server. You might be able to do DHCP with raspberry PI but I have experience with one. I know a lot of people are using them.

If you could use your SG350X as the core switch connected to pfsense that would be better as it has more bandwidth being 10 gig. You would still want your servers connected to 10 gig. That would require moving pfsense or your servers. But looking at your diagram I don't see where your local LAN can pull enough data to saturate your 1 gig trunks from the SG350X switch. You don't really have that many devices.

If you go with what you have then you will be doing layer 2 switching not L3 switching.
 
  • Like
Reactions: phil9878

coxhaus

Member
Jul 7, 2020
86
32
18
If he added some workstations off the SG350 switch then he would have potential to saturate his backbone trunk ports which would be an issue. But a couple of iPhones, iPads and some TVs are a pretty light load. And the wireless laptop is not a problem as the wireless AP will not pull a gig even with all the devices on.

Oops, found a workstation in the bottom right corner. He has the potential to saturate his backbone if it is fast enough.
 
  • Like
Reactions: phil9878