Bookmarked this thread. Very good info and very similar setup to mine:
2x Ruckus R710 w/ Unleashed
Dell R210ii w/ PFSense
Brocade ICX 6450-48P
I actually had two VLANS for a long time, just the default VLAN 1 for all client traffic and a VLAN 10 for all Servers.
I broke it out into VLANS for two reasons.
1.) Security
2.) Performance
3.) Ok 3 Reasons, because I am a Nerd
The Performance bit was about enabling Jumbo Frames so every server has a 10gb interface and I enabled Jumbo Frames.
I had it all working but actually just last week flattened my network back to a single LAN again.
The main reason was to simply management and fix some issues with communication. Some devices just need to be on the same Layer 2 domain as your server to work. Trying to stream content on VLC from a FireTV for example it couldnt' see my Freenas Server for some reason. Or my Behringer Mixer connected to the network the application to control it just didn't work with auto discovery because it coudln't see broadcast.
The reason I keyed in though is more of a comment/question.
I did not have routing being done by PFSense I had all my routing done by my switch because thats kind of the point of a Layer 3 switch. So my default gateway for each VLAN was a virtual interface on the switch. This is supposed to be preferred for performance, but then of course I lose all the ability to inspect and use firewall rules on that traffic on PFSense.
I had issues with PFSense crashing on me every once in a while so I didn't want to send even more traffic at it (Some kind of interface panic with Suricata that I have tried every tweak I can imagine to fix, including adding an Intel NIC to the PFSense box in case it was the crappy Realtek interfaces)
I am watching this Video now:
So all the routing is going thru PFSense and for me that means I would lose my 10gb connection between my computer and my NAS unless I add 10gb to the Firewall (Currently it does not because its only really used for WAN access and my internet is only 150mb/s)
So I wonder when/if I decide to mix things up again if I want to route everything in PFSense or still use the switch to do the routing.
I also probably might like to do physical interfaces for the main VLANS instead of a trunk, or perhaps a bonded pair of trunks. So I am curious what kind of configuration differences I would need to make for that to work.