Happy Wife ->->-> Happy Home Labbing

[Scoobey]

Brief History

I run a number of Ubiquiti network switches and access points all managed by a self hosted manager running in a VM. Behind an OPNsense router with multiple physical NICs and physical networks.

Original setup:
Internet-->ISPModem_10.0.0.x-->OPNsense-->Mutiple Home/LAB networks (all RFC1918 networks)

This setup worked great as I was able to setup firewall rules to allow access between Home/LAN Networks as well as any devices in the ISP Network (10.0.0.x). The only problem with this was my Home/Family network would go down if/WHEN I broke it and reprograming all the IOT devices and computers was very annoying. To resolve this I purchased a Ubiquiti Cloud Gateway Ultra (UCGU) to manage the Home/Family Network as well as manage all the Ubiquiti devices on the Home/Family network. I installed this along side the OPNsense router that now only serves to create my LAB Networks.

ISPModem_10.0.0.x/24-->OPNsense-->LAB networks (all RFC1918 networks)
ISPModem_10.0.0.x/24-->UCGU-->Home/Family Network(RFC1918 network)

I got really busy recently and am wondering if chasing an actual workable solution: Would setting up OSPF on the Gateway Ultra and OPNSense allow each system to talk with the other without the need for NAT port forwarding? I would like to use my Home/Family network for example to access one or all of my LAB Networks or vice versa. I could then utilize firewall rules on OPNsense or UCGU to further harden the networks.

 

[is39]

It seems your ISPModem is already doing NAT (if it's indeed giving you 10.0.0.x/24 IPs);
ideally it would be better if you would just get two public IPs for each of your firewalls.

That 10.0.0.x/24 range is most likely connected as WAN in your OPNsense and UCGU, which means you would NAT again.
If you would interconnect your Home and Lab networks via that 10.0.0.x segment it would be ugly and require port forwardings you're trying to avoid.

Instead all you need is "behind the firewall" interconnect, which is how two branches of a business would do; create another network for interconnect (use VLAN or port if you've extra), and route your private network(s) behind each of your firewalls to each other. I believe static routing would be enough, there is little reason to involve OSPF.
You would not have any NAT between Home and Lab and would be able to make rules protecting those two network from each other (usually on the firewall receiving packets).

You could also run a VPN between your two firewalls, using your 10.0.0.x interfaces, and route private networks over the VPN.
While this would likely slow things down a bit, in some situations it's simpler to setup.
Ensure that your 10.0.0.x addresses assigned to your OPNsense and UCGU are static (or static DHCP); it's good for routers anyway, but it would be required for the VPN route.

 

[UnbentTulip]

I'm not sure I'm 100% understanding the question, but a general suggestion.

I just set up VLAN's in my lab/home network. And they're all "separate" except for obvious trunk ports for router/firewall, and Access Points. For Wifi, my access points tag the VLAN based on the SSID it joins, so then it gets separated there.

And then I opened things up as "needed" in the firewall (pfsense). For instance, IOT VLAN isn't allowed to access anything except the IP address for home assistant.