New home networking layout

[NB2020]

I want to start thinking about the layout of the home network before proceeding with wiring and would like to make sure I'm not making any mistakes.

My current plan is to have all of the CAT cables wired to where the rack will be, which should allow me to have whatever network layout I want. I've attached a high level network diagram that shows how I plan to connect all of the components together, but I am not sure if what I have is considered best practice.

1. I want to be able to semi-isolate parts of the network. For instance, home automation devices, and surveillance cameras, etc. However, I still want to be able to access some of these resources remotely via VPN, such as the surveillance recordings. Is it feasible to have the cameras or devices themselves be isolated, but the server managing those devices be accessible over the WAN? Another reason for isolating is to prevent the recording traffic from affecting the main network.

2. Other parts of the network would be isolated completely (IPMI, private file servers). Does the setup in the diagram make sense or am I over-complicating things? I'm not sure how this is normally done. Is VLAN or subnetting a more practical way to accomplish this?

3. Should I go virtual for everything (VLANs, VM's)? I am hesitant because most of this is new to me and I figured it might be easier to learn by starting off with everything physical and virtualize later.

I don't think the platform would affect the layout, but I'm listing it just in case:

- firewall/router: OPNSense
- Managed switches (POE-capable for Wireless AP and IP Cameras)
- PC NVR: Blue Iris
- Home automation: Home Assistant
- File server: TrueNas
- Stream server: Plex

FYI, these are just solutions I've chosen based on my research. I am not set on any of them so please let me know if there are better alternatives.

I haven't completely figured out the home automation yet, but it seems like most devices communicate through mesh network which would have a hub that connects to the home network via ethernet.

 

[sic0048]

I'm certainly no professional, but I would definitely suggest using VLANs and to think about your VLAN structure before setting everything up. VLANs are not complicated at all but it is easier to put devices on a VLAN originally vs trying to put everything on one network and then trying to sort it out later (by adding VLANs at a latter date).

Here is what I have set up for VLANs:
- Main VLAN - has trusted computers and mobile devices on it (access to internet and all other VLANs)
- IOT with internet - for smart TVs, streaming devices, etc that require internet to work (access to internet but no other VLANs)
- IOT w/o internet - for smart appliances, thermostats, smart plugs/switches (flashed to Tasmota), etc - basically all IOT device that don't need an internet connection to work (no internet, no VLAN access)
- Printers - all networked printers are on their own VLAN (no internet access, no VLAN access). Allows me to isolate them from other devices (no need to have them on the IOT VLAN and easily open it up to the guest VLAN if needed).
- Guest VLAN - (Internet access, no VLAN access) - I don't isolate the devices on that network, but you can if that is desired (but it breaks things like iOS sharing, etc)
- Gaming Consoles - these generally require a more open set of security rules to connect to online gaming servers, so I isolate them on their own VLAN (internet access, no VLAN access)
- CCTV network - all cameras are on their own VLAN (no internet, no VLAN access)
- Digital phone system - all phones are on their own VLAN (no internet, no VLAN access)

I'm not sure what "internet facing servers" means in your system but they should be in their own DMZ if they are public servers. If they are private servers that you want to access while away from home, then a VPN is the best way to accomplish this. Those private servers might be part of your main VLAN or they might be on their own VLAN.

Again, setting up something like this is easy if you do it in the beginning.

 

[NB2020]

Thanks for the info! I was initially going to connect the LANs physically since it thought it would make it easier to understand the logical layout, but I can see with multiple LANs everything can become too much to manage. I was also paranoid about having things I don’t want to have internet access physically wired to the router, even though it can be configured to ‘no access’

I assume the VLANs can be managed on the router or switch. Is there an advantage for one over the other? I plan on having 1 WAN and 1 LAN on the router and the rest of the network on a main switch. If it’s better to configure VLAN through router, I would add more LAN ports to the router.

By internet facing I meant devices with internet access. I want to have devices on some LANs with internet access backup to a file server with no internet access. From your explanation this can be easily done. Do the internet and VLAN access settings allow/block data in both directions or can the access be more granular (I.e., internet receive only, send only, access specific VLANs only). On a related note, does configuring a VLAN with no VLAN access make it inaccessible to other VLANs? I would understand if this is more dependent on the system being used or if this is typically done through other methods such as firewall rules.

 

[sic0048]

You are correct that you can manage the VLANs from either the router or switch (if it is a L3 switch). I think managing VLANs from the switch is faster (it cuts the router out of the equation), but it is harder to set up (at least for me - a non-professional). Personally I have my VLANs set up in the pfSense device because it was easier and I don't have any speed issues on my network.

As far as your other question, it's common for beginners to think that when you block a VLAN (VLAN-A) from accessing other VLANs (VLANs B & C & D) that you block ALL traffic in both directions. In fact you are just blocking the devices on the VLAN (VLAN-A) from initiating contact with devices on the blocked VLANs (VLAN C & D & E). However, if a device on VLAN B or C or D initiates the communication, then the device on the VLAN-A will respond.

That might be hard to grasp at first, so let me give you an example. My CCTV cameras and BlueIris server are on their own VLAN (CameraVLAN). The CameraVLAN does not have access to other VLANs or the internet. The devices I use to access the BI server however are on my main "trusted" VLAN (MainVLAN) which has access to all VLANs and the internet. What this means is that the devices on the CameraVLAN cannot initiate a connection with any computer or mobile device on the MainVLAN (or any other VLAN - or internet). The devices on CameraVLAN don't even "know" that a world exists outside of the devices on the CameraVLAN. However, computers and mobile devices on the MainVLAN can access BI and the cameras directly and those devices on the CameraVLAN will respond normally. But once that communication chain ends, the CameraVLAN device still doesn't "know" those other devices exist.

What this means is that the devices on MainVLAN can display the camera feeds or I can change settings on the cameras themselves by accessing the camera GUI using my regular computers or mobile devices. But the devices on the CameraVLAN can't communicate with my MainVLAN devices directly themselves.

When I am away from the house, I use a VPN connection to access my MainVLAN and can therefore pull up the BI feeds or access the cameras directly just as if I was sitting in my house.

Hopefully that helps and doesn't muddy the water or make things harder to understand!

 

[ArmedAviator]

If only a single 1Gbit link is used between a switch and pfSense, there can be significant performance ramifications since any VLAN boundaries being crossed must go to the router and back. In a home network, this may not be an issue. If you have alot of file transfers across VLANs on multiple PCs and servers, performance would be best doing your routing on the L3 switch.

As far as your other question, it's common for beginners to think that when you block a VLAN (VLAN-A) from accessing other VLANs (VLANs B & C & D) that you block ALL traffic in both directions. In fact you are just blocking the devices on the VLAN (VLAN-A) from initiating contact with devices on the blocked VLANs (VLAN C & D & E). However, if a device on VLAN B or C or D initiates the communication, then the device on the VLAN-A will respond.

This is true if you use a stateful firewall (which happens to also be a router), such as OPNSense or pfSense. If one were to use ACLs in an L3 switch or router, this is unlikely to work. The popular Brocade ICX line does support "Established TCP" connections as an ACL option (stateful), but not for UDP so the ACLs need to be written for both sides of the VLAN ACLs if both have restrictions.

 

[sic0048]

This is true if you use a stateful firewall (which happens to also be a router), such as OPNSense or pfSense. If one were to use ACLs in an L3 switch or router, this is unlikely to work. The popular Brocade ICX line does support "Established TCP" connections as an ACL option (stateful), but not for UDP so the ACLs need to be written for both sides of the VLAN ACLs if both have restrictions.

Thanks for the clarification!

As noted, I'm a non-professional that has some experience with pfSense, but not much else so my knowledge level is limited. I normally try not to interject a lot on the forum because my knowledge is so limited. But I also hate seeing questions go unanswered for a few days when it is something I have done myself and can talk about my experiences.

 

[clcorbin]

You are correct that you can manage the VLANs from either the router or switch (if it is a L3 switch). I think managing VLANs from the switch is faster (it cuts the router out of the equation), but it is harder to set up (at least for me - a non-professional).

I was in this exact same position a few years ago when I decided to buy a pair of Aruba S3500 switches (needed one on opposite sides of the house). Besides TONS of reading and watching videos, what it REALLY took was going into the switches, stacking them, then configuring all the vlans, ports, lags, etc. and testing them (my raspberry pi with WiFi and ethernet port was invaluable for this for me!) over an over again until I understood exactly what each change was doing and why.

Early on, I ended up factory defaulting the stack a few times to wipe everything clean so I could start over. But, after several stabs at it, I got it properly configured the way I wanted and could test and prove it was routing the way I wanted. All those "wipe and start over" efforts really did pay off in my understanding what was going on. And it let me develop a good set of documentation so I could quickly find info on the current setup.

As I am moving over to the ICX6610s, getting them configured was MUCH easier. While there are definitely a lot of specific differences between the Aruba's and the Brocades', the fundamental concepts were pretty much identical. So, all I really had to learn was the differences in commands.

Would I do it again? Absolutely.

 

[sic0048]

I was in this exact same position a few years ago when I decided to buy a pair of Aruba S3500 switches (needed one on opposite sides of the house). Besides TONS of reading and watching videos, what it REALLY took was going into the switches, stacking them, then configuring all the vlans, ports, lags, etc. and testing them (my raspberry pi with WiFi and ethernet port was invaluable for this for me!) over an over again until I understood exactly what each change was doing and why.

Any videos that stood out to you? I have an Aruba S2500. Other than setting up the VLAN ports on it, I really haven't done much in the way of configuration. I'm certainly willing to learn and try new things, but haven't really looked into the whole L3 setup much.

 

[clcorbin]

Any videos that stood out to you? I have an Aruba S2500. Other than setting up the VLAN ports on it, I really haven't done much in the way of configuration. I'm certainly willing to learn and try new things, but haven't really looked into the whole L3 setup much.

"Vicious Computers" (A STH member if I recall correctly) had a couple of good videos on initial setup and configuring that I used to get started. After that, I tended to search for the topic at hand with "Aruba S2500" or "Aruba S3500" tagged on to see what I could find.

One thing to keep in mind is the Aruba's have a little bit different way of doing things with their assorted groups, so sometimes you have to do a bit of digging to figure out exactly how to do it the Aruba way.

Oh, and start a good cheat sheet! Once you have figured out the right way to do it, write it down for future reference. Of course, your memory may not be as BAD as mine is, so this might not be necessary.

 

[whiskytangofoxconn]

I'm a little late to the party but...

Running smurf tubes was the best thing I ever did. (I blew out our walls to studs room-by-room in whole house.) It's awesome: The chaser lines are the best part because now I don't need to worry about connectivity... Wire & fiber is cheap --actually the drywall and mud isn't expensive either, it just not fun.

I'm redoing my pfsense/home network. In doing that I'm also watching videos to dust the cobwebs from ccna stuff I studied and have long since forgotten. Stuff like: Don't mix tagged and untagged vlans and I remember someone saying at a study class: jails are jails are jails. There is a reason why we put dangerous people in solitary. (There is no security like physical separation/isolation.)

My old pfsenes network was 2x 1gbe rj45 ports with a LACP to a L3 switch. My new hardware is basically the same, a tiny mini micro Lenovo m720q with a dual SFP+ NIC. (new hardware has one extra interface, 1x rj45 gig port) Hindsight being 20/20, maybe I would have gone with slightly different hardware. I don't really need the full 10gbe speed/throughput and I could use a couple gigabit ports on pfsense. Breaking out VMs, or... Breaking out ESXi hosts (either move VMs to bare metal or... ??? Build dedicated dmz esxi hosts? Neither are cost effective in a home)

What NVR software are you running? I'm running zoneminder on a xeon e3-1245 v5 with 6-ish cameras and a YOLO object ID overlay that runs on a used Jetson Xavier I picked up cheap. I have cheap Chinese cameras that work great except they are brutal with things like "et phone-home" and streaming multicast on the network. I used to have them on an old Cisco Catalyst 3560 10/100 24p poe switch. That was terrible: I don't know if the switch was old/tired or what. Eventually I put everything on a dedicated switch, a Cisco SG300-10mpp and it made a huge, huge, huge difference. Depending on what you are running for cameras, etc. I would highly recommend spending some time with wireshark and figuring out what you need to do to isolate your cameras/nvr system. For me, breaking that out of the rest of our network made a huge difference. (the Jetson runs on a different VLAN in a different network, just zoneminder and the cameras are on the same switch)