Suggestion pfsense virtualization or not?

CosHiM

New Member
Apr 21, 2017
4
0
1
29
Hi,

Recently, I move my 2u server(computing node) to the Datacenter, and already bought some new hardware planning to use it as pfsense router, to run the ipsec between home and DC.

Hardware List:
  • J3355M
  • 4G *2 DDR3L
  • 32G SSD
  • 2T HDD
  • i340t 4 port NIC card.
  • Tdm400p (from 2u server)
Today, I finish assembly the hardware, test for the pfsense, the performance really good. (150Mbps over ipsec ase-cbc-128)

However, by the basic practice the pfsense do not work as my expected that can let me install lots of FreeBSD package. And due to I lost my server, I need this new server to run the Asterisk w/ dahdi and ZoneMinder for Video Surveillance. I not sure is a good idea to install these software under Pfsense(pfSense-install-FreeBSD-Package.sh - Install FeeBSD package at pfSense 2.3.x)

Otherwise, run Pfsense and Linux virtual machine over Proxmox/Esxi or Pfsense virtual machine over Linux(KVM/VMware).

Is there any recommendations for my software architecture?

- Pfsense
- Asterisk
- ZoneMinder
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,906
4,868
113
If you need to run many different services, then virtualized is probably the way to go. There is a ESXi pfSense image available to pfSense gold subscribers.
 

RTM

Active Member
Jan 26, 2014
554
194
43
One of the downsides to virtualizing your primary firewall, is that you may run into a chicken or the egg situation when updating the hypervisor.
It has been my experience that with some setups, where mgmt traffic goes through the firewall, you have to put the host in maintenance mode before updating it, requiring stopping VMs resulting in dropped connection to mgmt interface.

There are probably ways to deal with/minimize this issue, but you should be aware of what might happen.

I will give you another recommendation, while I am at it ;):
If you chose to use virtualization, find something where the impact of the platform is relatively small, think of it this way, why should it have a large assortment of web services etc.? Your machine does not have loads of RAM, CPU or disk resources, so what it has should be used for the VMs.
One way to help save some of your resources, might be to look into using containers for some of your apps rather than full blown VMs (though this can have an impact on security, as containers are not as isolated from the host environment as VMs).
 
  • Like
Reactions: PigLover

CosHiM

New Member
Apr 21, 2017
4
0
1
29
In the first place, I think to install few software in the pfsense. However, it is really difficult to install extra software in pfsense, when I test to install those software in my pfsense VM, whole pfsense stop to work.

The main problem is that many 3party software would broke the pfsense, I have to go virtualization to resolve the 3 party software problems.

Do you know other *unix router/firewall project which could install many 3rd party software packages?


One of the downsides to virtualizing your primary firewall, is that you may run into a chicken or the egg situation when updating the hypervisor.
It has been my experience that with some setups, where mgmt traffic goes through the firewall, you have to put the host in maintenance mode before updating it, requiring stopping VMs resulting in dropped connection to mgmt interface.

There are probably ways to deal with/minimize this issue, but you should be aware of what might happen.

I will give you another recommendation, while I am at it ;):
If you chose to use virtualization, find something where the impact of the platform is relatively small, think of it this way, why should it have a large assortment of web services etc.? Your machine does not have loads of RAM, CPU or disk resources, so what it has should be used for the VMs.
One way to help save some of your resources, might be to look into using containers for some of your apps rather than full blown VMs (though this can have an impact on security, as containers are not as isolated from the host environment as VMs).
 

poutnik

Member
Apr 3, 2013
119
13
18
I would ask, are you cash-constrained? (not meaning anything bad). Because I think that in a home environment, it's good to have a separate pfSense box. You could either use some old hardware you have (saving on cost, but paying more for energy), or install pfSense on something like the Alix APU boards. I know, they are not free (or cheap), but they get the job done very well, use very low energy and last years. I have 3 pfSense boxes like these and the youngest of them (HW-wise) is 3years old. The oldest is I think something between 7 and 10 years old. And they just run, no reboots, no maintenance (apart from occasional checking for update)...
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,007
1,569
113
CA
I believe in keeping your primary hardware to the "internet" not virtualized for reasons mostly mentioned by others.

It's VERY cheap relative to the servers most of us run to have 100% dedicated hardware for pfsense / routing to the internet.
The time saved from actually being able to get online to fix problems during any issue vs. having to use your phone or a hot spot that's slow, etc, alone is enough for me to run it dedicated :) non-virtualized. I'm running Cisco router now and even with that I purchased a 2nd identical unit loaded my config, and set it ready to swap out! I can go a day or two with no internet and no issues but I'm sure not going to wait a week+ for a new unit to arrive, reconfigure/test/etc and then get online again!!! I actually need to sell my old router I kept it JUST IN CASE even, ha haha!! (after cisco).

My vote = NO virtualized router
 

RobertFontaine

Active Member
Dec 17, 2015
666
148
43
53
Winterpeg, Canuckistan
I have more ram and processing available to a vm then I have on my pfsense box. It would be easy to dedicated a couple of ethernet ports to pfsense for bandwidth to the switch.

BUT I'm old fashioned and the idea of a firewall on its own hardware makes me feel better.
 

ttabbal

Active Member
Mar 10, 2016
743
199
43
43
I have a pfsense under proxmox on a dedicated server. The idea is to have some basic firewall/router/VPN stuff for everything else. It works, mostly. There's a nasty bug in the BSD virtio network drivers that kills throughput. I'm still trying various options to get it working well. Annoyingly, it only shows when using pf, which is sort of the point of the VM. It's been there for years, the BSD guys don't seem all that bothered by it.

At home I run it on its own hardware and it works great there.
 

CosHiM

New Member
Apr 21, 2017
4
0
1
29
I would ask, are you cash-constrained? (not meaning anything bad). Because I think that in a home environment, it's good to have a separate pfSense box. You could either use some old hardware you have (saving on cost, but paying more for energy), or install pfSense on something like the Alix APU boards. I know, they are not free (or cheap), but they get the job done very well, use very low energy and last years. I have 3 pfSense boxes like these and the youngest of them (HW-wise) is 3years old. The oldest is I think something between 7 and 10 years old. And they just run, no reboots, no maintenance (apart from occasional checking for update)...
No. In the first place I design this hardware, I only considered low TDP and high ipsec throughput. In my mind, I thought the pfsense would allow user to run some light process (such as asterisk, python script and Zoneminder). However, it really difficult to do that with pfsense.

I planning to add more hardware in the future after the basement finished(server noise problem).
 

cheezehead

Active Member
Sep 23, 2012
717
174
43
WI
Use to run ESXi + pfSense VM for a few years...no perf issues for home/homelab use. I wouldn't want to have it handle 10GB wan links as a VM but light duty it works just fine.

"use to..." , I've since split it back out to a separate physical box due to the chicken and egg issue. Anytime the hypervisor needs patching your internet goes down. Also provides some electrical isolation in the sad event of a power surge from the cable company....copper on the wan side, glass internally just in case.
 

spazoid

Member
Apr 26, 2011
91
10
8
Copenhagen, Denmark
Same for me as for some of the others; running virtualized at the moment, but intend to switch to a dedicated setup when I find something that'll handle 1000/1000 mbit relatively cheap (low TCO).

Virtualized pfsense
Pros:
Cheap (as in free since you have the hardware)
Low power (you're already running the server)
Easy to snapshot
Scaleable (just give it more resources if required)
Flexible (easy to move to new hardware, but pfsense generally is with the backup/restore feature)

Cons:
Chicken and egg when updating the hypervisor
 

K D

Well-Known Member
Dec 24, 2016
1,426
305
83
30041
+1 for dedicated device. Keep home and Lab separate. That way your tinkering will not interfere with anything else.

Also if I am reading it right, it looks like you need pfsense only to establish an IPSec tunnel to your Colo and are not bound to using it as your edge device. Can't you have a dedicated edge device and have your IPSec software run in your new local server? Ubiquiti er-x is small, low power and has a decent CLI(not that I know how to use it but I've always been able to find the required scripts in their forum)