Move routing off pfSense to L3 switch?

IamSpartacus

Well-Known Member
Mar 14, 2016
2,229
532
113
Now that I've upgraded to a L3 switch (X1052) with 10Gb capabilities I've been contemplating whether not I should continue to have my pfSense firewall do all my inter-VLAN routing or should I move those duties over to my L3 switch. I'm mainly concerned about maximizing the performance of my VMware cluster VLANs (VM Networks, vSAN, vMotion, etc.).

Having pfSense manage all my VLANs is my preferred way because I can manage all the firewall rules easily that way but I also don't want to have an extra unnecessary hope for all the traffic on my cluster. Having to have all traffic hit my firewall over a 1Gb port doesn't seem like the most efficient setup.

Thoughts?
 

PigLover

Moderator
Jan 26, 2011
2,911
1,231
113
It all depends on the type and volume of traffic that crosses between VLANs.

If your VLANs are reasonably isolated and independent with limited traffic between them then leave it on the pfSense box. You have better control and reporting there than you do on the L3 switch.

If, however, there is a lot of traffic crossing between VLANS - high bandwidth, lots of PPS or latency sensitive traffic - then you'd be better off routing it on the L3 switch. For example, if you have a file server on one VLAN and the clients of that FS on another then the pfSense box becomes a bottleneck and you should do it locally.

Adding some esthetic comments and perhaps value judgements, if you find the pfSense box to be a bottleneck then you've likely done the VLAN design wrong. VLAN should be used to isolate things that are, in fact, isolated.

Sent from my SM-G925V using Tapatalk
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,229
532
113
It all depends on the type and volume of traffic that crosses between VLANs.

If your VLANs are reasonably isolated and independent with limited traffic between them then leave it on the pfSense box. You have better control and reporting there than you do on the L3 switch.

If, however, there is a lot of traffic crossing between VLANS - high bandwidth, lots of PPS or latency sensitive traffic - then you'd be better off routing it on the L3 switch. For example, if you have a file server on one VLAN and the clients of that FS on another then the pfSense box becomes a bottleneck and you should do it locally.

Adding some esthetic comments and perhaps value judgements, if you find the pfSense box to be a bottleneck then you've likely done the VLAN design wrong. VLAN should be used to isolate things that are, in fact, isolated.

Sent from my SM-G925V using Tapatalk
Well, things like vSAN, vMotion, FT/HA are indeed completely isolated so I don't need to worry about routing those VLANs. However you brought up file servers. Are you saying it's not good practice to have your clients on a different VLAN then your servers (in this case media server)?
 

PigLover

Moderator
Jan 26, 2011
2,911
1,231
113
I think, in general, your clients performance is highly dependent on performance of the network they use to access the servers. As always, the correct approach is depends on application requirements. But in general the clients should be on the same VLAN as the servers they depend upon. Even the L3 routing in your switch will add latency and could limit throughput. And the common file sharing protocols (NFS & SMB) are highly latency sensitive.

That does not prevent the severs from also being connected privately on a second interface/VLAN for performance and/or security isolation.

Sent from my SM-G925V using Tapatalk
 

whitey

Moderator
Jun 30, 2014
2,770
865
113
37
Now that I've upgraded to a L3 switch (X1052) with 10Gb capabilities I've been contemplating whether not I should continue to have my pfSense firewall do all my inter-VLAN routing or should I move those duties over to my L3 switch. I'm mainly concerned about maximizing the performance of my VMware cluster VLANs (VM Networks, vSAN, vMotion, etc.).

Having pfSense manage all my VLANs is my preferred way because I can manage all the firewall rules easily that way but I also don't want to have an extra unnecessary hope for all the traffic on my cluster. Having to have all traffic hit my firewall over a 1Gb port doesn't seem like the most efficient setup.

Thoughts?
Most often data plane service in a virtualization env such as IP based stg, vMotion, sVMotion, FT, vSAN, etc. would be done over L2 switching/vlan's...no need for L3, now your mgmt traffic may very well go over a routed L3 FW conn if you set it up that way but all the other traffic is typically contained on a L2 switch/stack on seperate vlan's providing blazing fast 10G conn's ideally for these typoes of virt data plane traffic that chew up throughput.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,229
532
113
I decided against moving things of pfSense. Instead I added a LAGG group between pfSense and my switch for fault tolerance and defined my VLANs on both pfSense (x.x.x.1) and my switch (x.x.x.2).