With a layer 3 switch, the general recommendation is to either let the switch do DHCP duties (it can), or have a dedicated DHCP server that can handle VLANs (pfSense cannot, at this time...).
With a layer 3 switch, I prefer to let pfSense be a pure firewall appliance. Have a dedicated "transit" VLAN in your layer 3 switch connected to pfSense. As an e.g.
- You've created a VLAN on your layer 3 switch called TRANSIT, added a single port to it, enabled the virtual interface on it, and the IP is set to 192.168.2.2/xx
- The physical port from above is connected to a port on your pfSense box on let's say the OPT1 interface.
- In pfSense this interface (OPT1) is set to a static IP = 192.168.2.1/xx
At this point, from your console on the switch, you should be able to ping 192.168.2.1 and get a response back. The switch is talking to your pfSense box, but your pfSense box only understands traffic on the 192.168.2.x network.
Now, in pfSense you create a Gateway (System-->Routing-->Gateways). In the settings for the gateway:
- Interface should be OPT1
- Address Family - whatever, v4/v6
- Name - Something meaningful like ICX6610-1 etc
- The Gateway IP address should be what the layer 3 switch TRANSIT IP is. In my example, that would be 192.168.2.2
Now, we have a gateway defined, but pfSense still doesn't know what type of traffic to expect/handle. So, go into System-->Routing-->Static Routes and define one or more static routes. As an e.g.
- Click Add
- The Destination Network should be one or more of the VLAN IP ranges on the layer 3 switch. For e.g. 192.168.20.0/24 (The zero is important...)
- The Gateway should be set to the gateway we just defined, as an e.g. ICX6610-1.
- Give it some meaningful description.
- Save.
At this point, the pfSense box, knows where the 192.168.20.x/xx traffic is coming from and how to talk back to it. But wait...things are probably not working just yet.
That's because your layer 3 switch is not configured with a "default" route that is pointing to the pfSense box. So, add a default route of 0.0.0.0/xx and point it to 192.168.2.1 which is the pfSense box.
At this point, all your VLANS on the layer 3 switch should be able to talk to the switch, the pfSense box, the internet (if it's set up in pfSense) and vice versa. What's not working is DHCP and DNS...To solve that, there's many different strategies, and I'm not going to go into the details of them. In my case, I use a dedicated Windows server for both DHCP/DNS and in the layer 3 switch, for EACH virtual interface, an "ip-helper" is setup to point to this Windows box (There's more configuration to be done for DNS to work efficiently/correctly, but that's a different topic).
What this does, is that all routing is local to the layer 3 switch, and only external networks are routed to the pfSense box. All DHCP is handled at the switch level/dedicated server and not by pfSense. If you've setup DNS correctly, all DNS queries by any clients on any of the VLANS, go to the local DNS server and if the entry is not in the DNS cache, only then does it go outside the network.
My .02