Network design help.

[xhypno402]

Hi all,

I have been browsing for a while and posting an answer here and there, but I am to the point in my new lab/network build out that I need a bit of help.

I have a 2gbps fiber loop being installed at the end of January and I decided to start getting everything ready for it. Until this point I have been running unmanged switches in my home with a multitude of servers/hardware connected over 1000base-t.

I have experience with cisco gear but have not worked with it in almost 5 years but figured what the hell, lets jump into managed. I am an OpenStack developer and having better separation of my personal cloud and development gear will be a great addition.

Here is my current hardware:

Personal/family gear (4-5 active users in house at all times)
----------------------------------------------------------------------------
4 gaming desktops
4 laptops
4 tablets
1 directv internet connect reciever
1 directv go unit
1 laser printer

Development Lab
-----------------------
1 dual e5-2670 + 128gb ram + 48tb storage - NAS
3 dual e5-2670 + 256gb ram - OpenStack cluster (1 control/compute node and 2 pure compute nodes)
20 raspberry PI 3's in an arm based cluster for running distributed lint and python tests.

Network gear
-----------------
2 Netgear Prosafe 24 port unmanged switches
1 Netgear R8500 Wifi router
1 Netgear C3000dcr in bridge only mode

After reading the posts about the Quanta surplus gear and a few lucky OBO's on ebay, I now have the following in my possession.

1 Quanta LB6m
1 Quanta LB4m
7 Intel x520-da1
2 Intel x520-da2
1 supermicro 1U Server X10SLM+-LN4F with i3-4370 + 8gb ram + 120gb i3700 ssd + 2x 400gb i3700 ssd running pfsense


Here is my goal for the network:

Comcast bridge/mvr (once fiber installed) -> pfsense -> LB6m -> LAGG 2 x 10gb -> LB4m

Connected to the LB6m:
4 gaming systems each with a single fiber pair on an x520-da1
1 NAS using 2x 10gb on x520-da2 with DACs
3 Servers using 1x 10gb on x520-da1 with DACs

Connected to the LB4m:
20 raspberry PI 3's
1 directv recieveer
1 directv go
1 laser printer
1 Netgear r8500

I do not want to have the Netgear r8500 handle routing any more, I want all of that handled by either the pfsense box or one of the switches (looks like my LB6m and LB4m both have the layer 3 capable firmware).

I want to segment my network and have multiple vlans.

vlan1 - IPMI/console management
vlan5 - DMZ on Internet
vlan10 - OpenStack Cluster management
vlan11- OpenStack VM's
vlan20 - Storage
vlan30 - 10gb connected systems (non-storage/server)
vlan40 - 1gb connected systems
vlan50 - trusted wifi
vlan60 - untrusted wifi with captive portal
vlan70 - raspberry cluster

What suggestions do you all have? Does this setup seem completely out there?

Can I even get it all running with the LB6m being the core router with it trunked to the pfsense box?

Should I ignore in switch routing and push all routing to the pfsense box?

Is it possible to have 2 separate wifi networks on different vlans from 1 wifi router? (the r8500 has 3 active wifi networks allowed at once)

-Philip

 

[Jon Massey]

What suggestions do you all have? Does this setup seem completely out there?

Seems pretty sensible.

Can I even get it all running with the LB6m being the core router with it trunked to the pfsense box?

Depends on the firmware - Quanta LB6M (10GbE) -- Discussion

Should I ignore in switch routing and push all routing to the pfsense box?

That's what I'd be inclined to do

Is it possible to have 2 separate wifi networks on different vlans from 1 wifi router? (the r8500 has 3 active wifi networks allowed at once)

Depends on the R8500. My Linksys APs let me do this - I have private and guest SSIDs connected to their own VLANs with the routing handled by my draytek router.

HTH,
Jon

 

[Mike Bailey]

Just be aware that if you're running routing in pfSense, you're going to take a latency and throughput hit.

I don't know what set of hardware you'll be running the pfSense on, but the LB4M / LB6M are be line rate forwarding and routing.

With routing in the LB6M, you won't have an inter-VLAN routing if the pfSense VM goes down for any reason (reboot, config change, etc).

The only thing I'd keep on the pfSense then would be:

- IPMI (if you want to restrict access from specific VLANs in your core, if not, leave this on the LB6M)
- DMZ
- Untrusted Wi-Fi

That's basically what I have running in my lab rack. One core switch stack handles all the routing and my DMZ and Guest Wi-Fi run behind a firewall.

It works extremely well.

 

[fractal]

I agree with Mike Bailey. It looks to me that a bunch of your vlans are there just because you can. Many of them have the same security profile and similar routing rules. They are separate just because you can, and perhaps to reduce the broadcast domain. Go ahead and route them on the L3 switch. The worst that can happen if the switch were to "fail open" is ... well ... more broadcast traffic.

I like well defined security borders. Having a separate pfSense appliance gives me that. All the soft - squishy inside stuff is inside. All the semi protected but risky stuff is on that DMZ cable. All the wild world stuff is on the outside. I may be wrong but you are unlikely to run IDM rules between your raspberry pi network and your openstack vm network.

So, in summary, consider running the routing between soft-squishy trusted VLANs on your L3 switch and run intra-trust domain routing on your pfSense box. Keeping your soft-squishy network alive without the pfSense router might involve hosting DHCP/DNS on the L3 switch as well.

Or not. My thinking is work in progress as well

 

[xhypno402]

I haven't had much time to look into routing on my LB4M and LB6M but I guess that will be the next step for me.