Network design help.

Discussion in 'Networking' started by xhypno402, Dec 19, 2016.

  1. xhypno402

    xhypno402 New Member

    Dec 7, 2016
    Likes Received:
    Hi all,

    I have been browsing for a while and posting an answer here and there, but I am to the point in my new lab/network build out that I need a bit of help.

    I have a 2gbps fiber loop being installed at the end of January and I decided to start getting everything ready for it. Until this point I have been running unmanged switches in my home with a multitude of servers/hardware connected over 1000base-t.

    I have experience with cisco gear but have not worked with it in almost 5 years but figured what the hell, lets jump into managed. I am an OpenStack developer and having better separation of my personal cloud and development gear will be a great addition.

    Here is my current hardware:

    Personal/family gear (4-5 active users in house at all times)
    4 gaming desktops
    4 laptops
    4 tablets
    1 directv internet connect reciever
    1 directv go unit
    1 laser printer

    Development Lab
    1 dual e5-2670 + 128gb ram + 48tb storage - NAS
    3 dual e5-2670 + 256gb ram - OpenStack cluster (1 control/compute node and 2 pure compute nodes)
    20 raspberry PI 3's in an arm based cluster for running distributed lint and python tests.

    Network gear
    2 Netgear Prosafe 24 port unmanged switches
    1 Netgear R8500 Wifi router
    1 Netgear C3000dcr in bridge only mode

    After reading the posts about the Quanta surplus gear and a few lucky OBO's on ebay, I now have the following in my possession.

    1 Quanta LB6m
    1 Quanta LB4m
    7 Intel x520-da1
    2 Intel x520-da2
    1 supermicro 1U Server X10SLM+-LN4F with i3-4370 + 8gb ram + 120gb i3700 ssd + 2x 400gb i3700 ssd running pfsense

    Here is my goal for the network:

    Comcast bridge/mvr (once fiber installed) -> pfsense -> LB6m -> LAGG 2 x 10gb -> LB4m

    Connected to the LB6m:
    4 gaming systems each with a single fiber pair on an x520-da1
    1 NAS using 2x 10gb on x520-da2 with DACs
    3 Servers using 1x 10gb on x520-da1 with DACs

    Connected to the LB4m:
    20 raspberry PI 3's
    1 directv recieveer
    1 directv go
    1 laser printer
    1 Netgear r8500

    I do not want to have the Netgear r8500 handle routing any more, I want all of that handled by either the pfsense box or one of the switches (looks like my LB6m and LB4m both have the layer 3 capable firmware).

    I want to segment my network and have multiple vlans.

    vlan1 - IPMI/console management
    vlan5 - DMZ on Internet
    vlan10 - OpenStack Cluster management
    vlan11- OpenStack VM's
    vlan20 - Storage
    vlan30 - 10gb connected systems (non-storage/server)
    vlan40 - 1gb connected systems
    vlan50 - trusted wifi
    vlan60 - untrusted wifi with captive portal
    vlan70 - raspberry cluster

    What suggestions do you all have? Does this setup seem completely out there?

    Can I even get it all running with the LB6m being the core router with it trunked to the pfsense box?

    Should I ignore in switch routing and push all routing to the pfsense box?

    Is it possible to have 2 separate wifi networks on different vlans from 1 wifi router? (the r8500 has 3 active wifi networks allowed at once)

  2. Jon Massey

    Jon Massey Active Member

    Nov 11, 2015
    Likes Received:
    Seems pretty sensible.

    Depends on the firmware - Quanta LB6M (10GbE) -- Discussion
    That's what I'd be inclined to do
    Depends on the R8500. My Linksys APs let me do this - I have private and guest SSIDs connected to their own VLANs with the routing handled by my draytek router.

  3. Mike Bailey

    Mike Bailey Member

    Sep 24, 2015
    Likes Received:
    Just be aware that if you're running routing in pfSense, you're going to take a latency and throughput hit.

    I don't know what set of hardware you'll be running the pfSense on, but the LB4M / LB6M are be line rate forwarding and routing.

    With routing in the LB6M, you won't have an inter-VLAN routing if the pfSense VM goes down for any reason (reboot, config change, etc).

    The only thing I'd keep on the pfSense then would be:

    - IPMI (if you want to restrict access from specific VLANs in your core, if not, leave this on the LB6M)
    - DMZ
    - Untrusted Wi-Fi

    That's basically what I have running in my lab rack. One core switch stack handles all the routing and my DMZ and Guest Wi-Fi run behind a firewall.

    It works extremely well.
  4. fractal

    fractal Active Member

    Jun 7, 2016
    Likes Received:
    I agree with Mike Bailey. It looks to me that a bunch of your vlans are there just because you can. Many of them have the same security profile and similar routing rules. They are separate just because you can, and perhaps to reduce the broadcast domain. Go ahead and route them on the L3 switch. The worst that can happen if the switch were to "fail open" is ... well ... more broadcast traffic.

    I like well defined security borders. Having a separate pfSense appliance gives me that. All the soft - squishy inside stuff is inside. All the semi protected but risky stuff is on that DMZ cable. All the wild world stuff is on the outside. I may be wrong but you are unlikely to run IDM rules between your raspberry pi network and your openstack vm network.

    So, in summary, consider running the routing between soft-squishy trusted VLANs on your L3 switch and run intra-trust domain routing on your pfSense box. Keeping your soft-squishy network alive without the pfSense router might involve hosting DHCP/DNS on the L3 switch as well.

    Or not. My thinking is work in progress as well ;)
  5. xhypno402

    xhypno402 New Member

    Dec 7, 2016
    Likes Received:
    I haven't had much time to look into routing on my LB4M and LB6M but I guess that will be the next step for me.
Similar Threads: Network design
Forum Title Date
Networking Designing a house with networking in mind Jul 31, 2019
Networking Home Network Design & Set-Up Feb 10, 2018
Networking Help designing a new network Jun 13, 2017
Networking Render Farm Network Design sanity check Sep 21, 2015
Networking Advice on network design Jul 10, 2014

Share This Page