networking help

[m_b]

Ah, that makes sense - also sucks that AT&T doesn't make things easy...

In that case, one change I might make to your design is to move the Dev VMs (the 1.1/24 segment) under the UTM, so pfsense doesn't have to run a mix of static IPs and NAT (assuming you want the dev VMs to access the internet). That way the LAN and Dev VMs can just use the UTM as a default gateway to talk to each other (subject to whatever firewall rules you want to set up), while you have the choice of relying on NAT to access the internet accessible servers from the LAN or setting static routes on the pfsense and NAT bypass on the UTM.

 

[marcoi]

i have a dev pfsense - it just does dhcp and is attached to sophos. I do it this way to minimize the 50 device limit. I got my dev pfsense working off sophos now and feeding my dev vm ips. So now i need to figure out how to get FW1 pfsense serving my internet to pass IP .65 to sophos.

 

[m_b]

How many interfaces do you have on FW1, and what IP addresses are you running on each one?

 

[marcoi]

How many interfaces do you have on FW1, and what IP addresses are you running on each one?

4 interfaces on pfsense ATT FW1
igb0 - ont in
igb1 - lan with dhcp 1.1/24
igb2 - Wifi on vlan 10 for IOT devices
igb3 - Static IP testing

I can chance either igb1 or igb3

 

[m_b]

OK. I guess I'm a little confused by the interface layout. What I thought you'd done (based on my read of your posts):

ATT --- FW1 --- Static IP servers
           |
            --- FW2 (UTM) --- LAN (0.1/24)
                         |
                          --- Dev pfSense (FW3?) --- Dev VMs (1.1/24)

Three questions:

1. Is the diagram above correct? If so, where does igb1 connect?
2. Which device is NATing the IOT devices?
3. What are the IP addresses on FW1 igb0 and igb3? (masked to just show last octet and subnet mask is fine)

 

[marcoi]

OK. I guess I'm a little confused by the interface layout. What I thought you'd done (based on my read of your posts):

ATT --- FW1 --- Static IP servers
|
--- FW2 (UTM) --- LAN (0.1/24)
|
--- Dev pfSense (FW3?) --- Dev VMs (1.1/24)


Three questions:

1. Is the diagram above correct? If so, where does igb1 connect?
2. Which device is NATing the IOT devices?
3. What are the IP addresses on FW1 igb0 and igb3? (masked to just show last octet and subnet mask is fine)

1. As of right now the diagram is correct. I do not have the static ip server part working.
igb1 - Lan 1.1/24 is acting as wan interface to sophos UTM
2. I am using FW1 for IOT devices. Currently setup vlan 10 in pfsense on igb2 and allowing basic access to internet, no access to other interfaces. My WFI EAP devices are on network running sophos and have two WIFI networks. Wifi home is working with sophos utm for dhcp/blocking. Wifi IOT is setup to use vlan 10 so it going to FW1. I have certain ports on my switch set to general access with allowing vlan 1 default on switch and vlan 10 traffic for FW1.
3. igbo gets dhcp IP from ATT. for igb3 i am still trying to get that working but for now i have it setup with static ip of .46.70/29 which is my assigned gateway. My IP block of usable IPS are.65 to .69. I havent done alot of testing here, but if i can get this to work, i can then change my sophos wan interface from pfsense lan over to pfsense static and assign sophos wan an ip of .65.

Hope all that make sense.

 

[m_b]

Yes, that's helpful!

My advice would be to move the IOT devices to another segment under the UTM (if you have space on the license) or potentially move them under the pfsense VM you're using for the Dev VMs. Then on FW1, you can bridge igb1 and igb3 and set .70/29 as the IP address of that bridge interface. On the UTM, set the 'WAN' IP address as .65/29 with a gateway of .70 with a similar set up applied to the static IP servers, assuming you aren't trying to limit access to those servers from the UTM. If you want to be super security minded, you could also block traffic to .70 that appears on igb0.

Does that make sense?

 

[marcoi]

The UTM will hit the device limit with the IOT devices. Plus i want the iot not to be blocked by some of the UTM rules. For now i plan to keep it as is since it working.

 

[marcoi]

I switch the ports on my wan side of sophos to igb3 and assigned it my .65 IP and i can get to the internet from the sophos utm network side. I also opened a FW rule on WAN to allow all traffic to my VPN port on Sophos. When i check on my ip under Sophos UTM it still showing as dhcp assigned IP on ATT. Not sure how i get it to show the .65 IP in pfsense.

 

[marcoi]

I got my setup working for the most part. Pfsense replaced the ATT Router. I have my .65 going to Sophos UTM feeding home network. I have separate vlan for IOT Wifi and I got a second Pfsense running for External .66 to feed hosted services.

 

[Terry Wallace]

Did you fix your issue with the speed of throughput on the second pfsense firewall ?

 

[marcoi]

Did you fix your issue with the speed of throughput on the second pfsense firewall ?

yeah it wasnt the pfsense - it was the windows server 2012 r2 i was using as access point to test network. Its somehow limiting the speed. I added another VM to the lan side of the pfsense and it was fine.

Im happy i finally got this working. I have my first hosted service up and running and i can access it from home network as well now.

Spoiler: shameless plug lol

My Minecraft Server: minecraft.incorvm.com:25565
Dynamap

 

[marcoi]

I think my next step is build out a small fast CPU system.
I want to run on it:
1. ATT PFsense VM with passthrough nic
2. Sophos UTM VM
3. 1-4 External PFSenses VM
4. Windows Server 2k r2 Essentials VM
5. TP Link controller VM
6. Tester W10 VM for accessing all the PFsense VMs.

This way i can separate my home prod vms, from External, storage and dev VMs running on other servers.

Any suggestion on CPU that will accelerate Sophos and Pfsense vms? I think i want 3-4Ghz range.