Hardware to run pfsense ?

BlueFox

Well-Known Member
Oct 26, 2015
1,412
795
113
Well, it seems someone has been removed from the forum, so, back to more appropriate recommendations? I still think the Xeon-D motherboard I linked is the best value if one needs 10GbE connectivity.
 

RTM

Well-Known Member
Jan 26, 2014
763
279
63
So given the fact that OP is in the UK, it got me thinking what good options there are for buying gear.

So here are some stray thoughts:
  1. Consider looking at mini-itx.com, they have loads of mini PC / appliance style devices, like:
    1. This Core i5 mini PC from gigabyte with 1x 2.5G LAN and 1x 1G for approx 400 GBP (they have i3 and i7 versions too)
    2. This fanless quad 1G LAN(Intel) with a N3160 (quad core, I believe it is essentially an "Atom" core) for 330 GBP
      1. It should be plenty fast for 1G (>1G when aggregating bandwidth on all ports simultaneously), but keep in mind that the N3160 is a bit old (IMO the price offsets this)
  2. mini-itx also sells various Asrock rack boards, like this C3558 for ~360 GBP, add in a NIC like this one, some cheap 8GB sticks and your case of choice that supports a PCIe slot, and you should still have some change (could be used to upgrade to the C3758 version) ;)
  3. You may also want to consider some of the UK sellers on eBay.
    1. Before Brexit I bought a nice board from fixedasset-sales, it may be worth it to keep an eye out for what they have (I believe they have had more than one listing posted on the best deals section)
    2. Similar to the Asrock C3558, you could also consider this board, while it is only a 2core board, it has a decent amount of ports, ability to take an add-in card, and probably enough CPU performance to route >1G (but probably not 10G)
To be honest my personal favorite is the fanless quad core from Jetway (sold by mini-itx), obviously it will not do >1G on one interface, but the price is really nice and as I mentioned before, internal traffic really ought to be handled by a L3 switch.
 

newabc

Active Member
Jan 20, 2019
171
44
28
If the aliexpress.com can ship to UK, the 6-port or 8-port mini pc in Qotom official store may be a good choice.(link)

But since it is using 1Gbps intel NICs, so bonding or LACP with router and switch are needed if involving 1.2Gbps and up.
If only 1Gbps and less, 4-port ones should be good.
 

newabc

Active Member
Jan 20, 2019
171
44
28
By the way, I don't know whether there is a traffic cap for the 1Gbps and up bandwidth on UK, but as I know the Comcast in U.S.(a Cable TV/phone/Internet service provider), only the 2Gbps and up have no traffic cap (which is $299/month), and the 1.2Gbps and less have a traffic cap as 1.2Tbytes/month (which are less than $100/month or even $70/month if anyone can get a deal).

So if there is a traffic cap, the 1Gbps and up are not worth the design of LACP, bonding or 2.5Gbps/10Gbps ports.
 

Frank Bello

Member
Nov 14, 2018
34
8
8
Maybe I'm missing something...

I can do any pfSense package you can think of with symmetric gigabit routing, and a tiny system at that, with < $60 worth of hardware. Is it fanless? Nope. So what? It (among other things in my rack) sits in a corner of my basement, away from anything else. Why are we spending 100s if not thousands !! for a router??
Erm, it's not just a router... the more advanced firewalls are doing SSL/TLS inspection - decrypt/packet inspection/re-encrypt, so the CPU load is way more than just L3 forwarding.

What $60 hardware are you using that can do 1G symmetric with firewalling, IDS and IPS enabled ? I'm genuinely interested.
 

Frank Bello

Member
Nov 14, 2018
34
8
8
If the aliexpress.com can ship to UK, the 6-port or 8-port mini pc in Qotom official store may be a good choice.(link)

But since it is using 1Gbps intel NICs, so bonding or LACP with router and switch are needed if involving 1.2Gbps and up.
If only 1Gbps and less, 4-port ones should be good.
Thanks - the Qotom offerings are similar to the Kettop box that I mentioned at the top of the thread. There are a few of these vendors doing cheap 4x1G or 6x1G boxes, specifically for pfsense and similar uses.

I wouldn't have a bonding issue on the upstream side, as my ISP delivers their service through a 1G port. I'm on 300Mbit/s at the moment which costs £56/mo and I can upgrade to 1G for another £8/mo. Very tempting, but my current firewall maxes out at 500M. The offering is non-symmetric, because it's a cable service.

I'm guessing that within another 2-3 years, we'll see a >1G offering from ISPs here. (I know that's already possible if I want to pay $$$ as a commercial customer, I'm talking consumer-grade offerings).
 

Frank Bello

Member
Nov 14, 2018
34
8
8
The E302-9D is a comedy option as spending $1500 (Europe tax + other components) for a home router is a bit silly. The Xeon-D motherboard I linked, once you add in an inexpensive chassis, RAM, and SSD is going to be 1/5th the price and even then, it's likely more than you need.
Well, $1500 is certainly above my budget.

I'm just looking at pricing for that SuperMicro X10SDV-4C-TLN2F. A US seller has one on offer 2nd hand and will ship to the UK for $229, $19 shipping and $57 import; total $305.

Case £50, nano 12V PSU and AC/DC adapter block, £79, 2x8GB unbuffered ECC £90, SSD 128GB £25. Total £244 or $344 for the parts, added to $305, call it $650 in total. I haven't added a case fan - this particular ebay seller is including a CPU fan.

In summary - this hardware totals about $650 plus possibly the cost of a case fan. It seems to me that each time I run the numbers for the roll-your-own offering, they end up in the same ballpark as the Netgate offering ($699+shipping+import, probably about $780). Either of them is in (IMHO) the "affordable" bracket for a home/hobby firewall.

As computer hardware only depreciates, you might just be better off getting the cheapest option that will fit your needs today and then upgrade further down the line instead of paying a premium for it now.
Yeah, the eternal dilemma of IT... buy now or wait for something better. Assuming I buy the 1Gbit/s package from my ISP then I could certainly live with a 1G solution today. That would put me in the Qotom/Kettop price bracket. But the savings is the difference between $550 for the Kettop box and $650 for the X10SDV-4C-TLN2F - it's not really that much more for a Passmark score of 6017 for the Xeon versus 3403 for the Kettop box and 10G interfaces instead of 1G. Even if the performance of the Xeon is only 2Gbit/s, the additional cost is then about 15% for 2x the performance. I'd certainly spend $100 more now to avoid spending $650 in 2 years' time.
 

Frank Bello

Member
Nov 14, 2018
34
8
8
So given the fact that OP is in the UK, it got me thinking what good options there are for buying gear.

So here are some stray thoughts:
  1. Consider looking at mini-itx.com, they have loads of mini PC / appliance style devices, like:
    1. This Core i5 mini PC from gigabyte with 1x 2.5G LAN and 1x 1G for approx 400 GBP (they have i3 and i7 versions too)
    2. This fanless quad 1G LAN(Intel) with a N3160 (quad core, I believe it is essentially an "Atom" core) for 330 GBP
      1. It should be plenty fast for 1G (>1G when aggregating bandwidth on all ports simultaneously), but keep in mind that the N3160 is a bit old (IMO the price offsets this)
  2. mini-itx also sells various Asrock rack boards, like this C3558 for ~360 GBP, add in a NIC like this one, some cheap 8GB sticks and your case of choice that supports a PCIe slot, and you should still have some change (could be used to upgrade to the C3758 version) ;)
  3. You may also want to consider some of the UK sellers on eBay.
    1. Before Brexit I bought a nice board from fixedasset-sales, it may be worth it to keep an eye out for what they have (I believe they have had more than one listing posted on the best deals section)
    2. Similar to the Asrock C3558, you could also consider this board, while it is only a 2core board, it has a decent amount of ports, ability to take an add-in card, and probably enough CPU performance to route >1G (but probably not 10G)
To be honest my personal favorite is the fanless quad core from Jetway (sold by mini-itx), obviously it will not do >1G on one interface, but the price is really nice and as I mentioned before, internal traffic really ought to be handled by a L3 switch.
Thanks! - will take a look. There are loads of options to consider...

I gave some more thought to your previous post as well. Actually, I only really need separate hardware for the public internet. Internal VLAN-to-VLAN firewalling (if required) probably can run on pfsense on ESXi... if I cut myself off from the management VLAN, it's the work of seconds to plug my laptop into a management VLAN port on my switch. If it really comes to the worst, I can hook up a keyboard and monitor to the back of the server. That approach would give me two firewalls - one internal-only and one internal-to-external, but I don't think that's necessarily a bad thing.
 

kapone

Well-Known Member
May 23, 2015
1,046
620
113
Erm, it's not just a router... the more advanced firewalls are doing SSL/TLS inspection - decrypt/packet inspection/re-encrypt, so the CPU load is way more than just L3 forwarding.

What $60 hardware are you using that can do 1G symmetric with firewalling, IDS and IPS enabled ? I'm genuinely interested.
Your original post says "pfSense hardware". Are you doing "SSL/TLS inspection - decrypt/packet inspection/re-encrypt" with pfSense?

p.s. The CPU in my $60 hardware is an i5-3570s, a 3.1GHz 4C/4T CPU. Way more than enough. I've yet to see it above 30% usage with a number of things (including 1gb routing) at full tilt.
 
  • Like
Reactions: cesmith9999

RTM

Well-Known Member
Jan 26, 2014
763
279
63
Well, $1500 is certainly above my budget.

I'm just looking at pricing for that SuperMicro X10SDV-4C-TLN2F. A US seller has one on offer 2nd hand and will ship to the UK for $229, $19 shipping and $57 import; total $305.

Case £50, nano 12V PSU and AC/DC adapter block, £79, 2x8GB unbuffered ECC £90, SSD 128GB £25. Total £244 or $344 for the parts, added to $305, call it $650 in total. I haven't added a case fan - this particular ebay seller is including a CPU fan.
Technically that board does not require a PicoPSU, it accepts 12V in directly (though you will need to buy the proper converter from 12V barrel to the P4 plug)

You also do not have to buy unbuffered memory, it supports RDIMMs, something like a pair of these should be cheaper (20GBP / each + shipping).
 
  • Like
Reactions: Frank Bello

Frank Bello

Member
Nov 14, 2018
34
8
8
Your original post says "pfSense hardware". Are you doing "SSL/TLS inspection - decrypt/packet inspection/re-encrypt" with pfSense?
Suricata is a nice-to-have. I don't know whether it does the re-encrypt part.

p.s. The CPU in my $60 hardware is an i5-3570s, a 3.1GHz 4C/4T CPU. Way more than enough. I've yet to see it above 30% usage with a number of things (including 1gb routing) at full tilt.
Thank you. That's exactly the sort of information I was hoping for. Not sure if Passmark is a reliable indicator here, but that CPU gets a Passmark score about 1.3x higher than the Core I5-7200U I was originally looking at. So, given your 70% headroom, the Core I5-7200U would be fine and would see a load of maybe around 50% or so. It also confirms my original suspicion that the Kettop/Qotom/etc offerings based on the Celeron 3865U and similar are underpowered for this task.
 

Frank Bello

Member
Nov 14, 2018
34
8
8
Technically that board does not require a PicoPSU, it accepts 12V in directly (though you will need to buy the proper converter from 12V barrel to the P4 plug)

You also do not have to buy unbuffered memory, it supports RDIMMs, something like a pair of these should be cheaper (20GBP / each + shipping).
Thanks! I had a quick look at the Jetway website as well and it's well organised, with drivers, manuals, etc.:)
 

whiskytangofoxconn

New Member
Aug 17, 2021
27
5
3
Do you remember the ESXi learning curve? Remember trying to figure out how 4x vCPU cores compared to a bare-metal quad-core or if 4gb of vRAM was the same as 4gb of bare-metal RAM? Pfsense is probably going to be a little bit like that. It all depends on what packages & services you run and how you have things configured. I think your budget is too high. I think you can probably get what you want in the 400-500-ish (pounds/sterling) range. I also agree with BlueFox: You don't need much to route 1Gbit.

My suggestion:
Go find an old, cheap/free PC, throw pfsense on it and break it a few times. Put it between your current FW and LAN, and then practice double-nat a bunch. Seriously, you have a budget pushing upwards of $1,000. The last thing you want is to over-spend and have hardware sitting idle. Or worse: How much would it stink to spend $400-$500 only to learn that you regret it and wish you had spent $600 (but now it's too late).

Old appliances (without AES-NI) are selling for stupid cheap online/used. Here in the US there are a couple appliances in the $50-$60 range (free shipping). Just buy something old and slow that you can learn on, break a few times and that you won't be worried to bin it if something happens. Installing and configuring the packages you'll want should give you a really good idea of the hardware resources you'll need. And, if you can get it sorted on old, slow hardware then upgrading to the same specs on current hardware should buy you a bit of future-proofing.

Just my 02 pence.

Last thought: I've been using pfsense since 1.2.3-release and esxi & xen for about as long. Ever hear the phrase "happy wife, happy life"? Every time I have virtualized pfsense I regret it. When you have to cancel plans or work on an esxi host for HOURS longer than it was supposed to take, there is something to be said about the internet not being down at the same time.
 

Frank Bello

Member
Nov 14, 2018
34
8
8
Ever hear the phrase "happy wife, happy life"? Every time I have virtualized pfsense I regret it. When you have to cancel plans or work on an esxi host for HOURS longer than it was supposed to take, there is something to be said about the internet not being down at the same time.
You make some very good points... I've pretty much decided to buy some dedicated hardware, plus I'm fairly unhappy about the thought of connecting ESXi directly to the internet.
 

kapone

Well-Known Member
May 23, 2015
1,046
620
113
You make some very good points... I've pretty much decided to buy some dedicated hardware, plus I'm fairly unhappy about the thought of connecting ESXi directly to the internet.
1. If I had to let any OS connect directly to the internet, ESXI would be one of the few I'll allow. It is hardened, secure, very low attack footprint, and by default, does almost nothing until you tell it to.

2. This is the kind of situation where a managed L3 switch comes in handy. You terminate your WAN on your core switch, in a dedicated VLAN. From there nothing can access it, unless they are on that VLAN, which is stupid easy to control. Then it doesn't matter if your router/FW is bare metal or virtual.
 
  • Like
Reactions: clcorbin

Scarlet

Member
Jul 29, 2019
63
22
8
1. If I had to let any OS connect directly to the internet, ESXI would be one of the few I'll allow. It is hardened, secure, very low attack footprint, and by default, does almost nothing until you tell it to.
And ESXi is so widely used that it is virtually guaranteed to have a backdoor for some agencies.

2. This is the kind of situation where a managed L3 switch comes in handy. You terminate your WAN on your core switch, in a dedicated VLAN. From there nothing can access it, unless they are on that VLAN, which is stupid easy to control. Then it doesn't matter if your router/FW is bare metal or virtual.
Using a switch to terminate the internet connection is a viable solution, a L2 switch with VLANs is sufficient, does not have to be L3.

I strongly disagree with a virtualized firewall. First I would not virtualize anything security related and second nothing that everything else (including your family) depends on should go out of production with the virtualization host:
Ever hear the phrase "happy wife, happy life"?
 

kapone

Well-Known Member
May 23, 2015
1,046
620
113
guaranteed to have a backdoor for some agencies.
I didn't realize we were trying to prevent State level actors from getting into your network. If that's the case, ESXi or pfSense is the least of your worries.

a L2 switch with VLANs is sufficient
My apologies, I was gonna write "managed switch" without the L3 part, but somehow ended up writing that. Probably because L3 routing at the switch makes a lot of sense, is how I run my network, and is quite performant. But yes, L2 would be just fine.

I strongly disagree with a virtualized firewall. First I would not virtualize anything security related and second nothing that everything else (including your family) depends on should go out of production with the virtualization host:
While agreeing generally with this, it's not black and white. There are commercial FW products that offer VM images for their products and have produced benchmarks validating the design, compared to bare metal, and virtualized works just fine.

That said, "happy wife...family..." IS important, and to that, I think you're making somewhat of an assumption that the user has only one server.

I'm running bare metal on my FW now (for many other reasons, including running a business), but when I had it virtualized (on esxi/pfsense), I had a tiny server that ran just pfSense and my Windows DC. Just that. And the "main" server(s), which are a lot more powerful, can easily be used to move those VMs there, temporarily. No loss of the network at all.

Different strokes for different folks....and all that.
 

Frank Bello

Member
Nov 14, 2018
34
8
8
I have one server for ESXi and an Asrock Deskmini which serves DNS and acts as a repository for ESXi backups (I cannot store those on my FreeNAS VM, since that runs on ESXi, so it's a vicious circle type of problem). But I digress...

Putting ESXi interfaces on the internet doesn't pass the defence in depth test. At the moment, if someone was going to get to ESXi they'd have to break into my firewall first, and at the moment it's not possible for me to fire up a VM (e.g. vyos) that somehow (maybe due to a bug, or just finger trouble) forwards between the internet and internal networks, because that external connectivity isn't present. So it guards against human error to some extent.

I also like to plug my internet-facing cable directly into a physical port on the firewall, because I know how easy it is to mess up VLAN configs at 2AM (insomnia...). If the cable sits in a port marked "WAN" and has a big red label on it saying "internet", that's one less thing to go wrong.
 

Stephan

Active Member
Apr 21, 2017
248
123
43
Germany
Let me suggest not virtualizing firewalls, because a multitude of extra things can go wrong on cold-startup. Like VMs not starting because storage took a while longer and you didn't bother to automate recovery in some VMware solution. Wrong defaults on boot that no longer let pass PPPoE frames out an interface. Booting firewall VM on a cluster host that does not have the DSL modem or fiber link connected. Misconfigured switch topologies, etc. pp. Also think of lightning. Virtualized firewall means something going to your house like two copper wires will need to be awefully close to your server electrically. If some cheap firewall hardware gets a port fried, oh well, use the cold standby. Of course you could always build a 4G/5G back-entry with a separate router, but who does. Once power or the link comes back, the firewall has to come back up and forward packets. Overly complicated setups here are the enemy of reliability.

With regard to state level actors, the trick in 99.999% of cases is to not be a target in the first place. I.e. if Snowden were to drop another cache, don't download it or participate in a torrent. Just don't. Same with some 1.9M entry no-fly list that someone downloaded from the FBI. Others will, and report on findings. Because didn't you suspect since at least 2013 anyway, that every packet of data from your premises to the outside is fair game to really anybody who cares? Maybe you're a journalist for hot topics that aggravates people in power? I suggest to learn a different profession, because your employer can't and won't protect you, their duty is to sale of ads and make payroll. Same with arms or drug trafficking, just get a job at the CIA and do it legally. /sarcasm In the very unlikely event that you are on some kind of jihad, well can't help you with that, son. But those are the groups of people that should concern themselves with "state level actors". Are you among them? (I guess the last one is a trick question)

Anyhow. Protecting against well-funded state level actors is pretty much impossible, as long as you are connected to the outside world. Because anything and everything has flaws that could be exploited, if you are a high enough value target. The Empire has deep enough pockets to just buy any exploit for whatever device you have. Or let somebody else buy it and use it for them, see Pegasus in recent news. From the Qualcomm baseband in your phone, to your Chromium-based browser, to that Windows app which keeps downloading and installing unsigned binaries, to that device you have which keeps installing signed drivers but which have a gaping hole allowing anybody to be SYSTEM.

Btw ESXi is used by state-level actors in some "diode" aka malicious packet injection traffic redirection setups, going by reports in The Guardian from back 2013 or 2014. Edward told them. All in all seems pretty tight, if you patch known flaws regularly I guess. Really the least of your worries, they sell to governments alot and malicious backdoor would put them out of business. They are still in business and even Dell couldn't change that.

My personal opinion on firewalls is, that everything with a web GUI is doomed. The only externally reachable port open should be SSH on a non-standard port and from a whitelisted source ip range, and/or a single UDP port for OpenVPN or Wireguard. To fight falling victim to security theater monocultures, you should also roll your own firewall setup script on boot with Before=network-pre.target and DefaultDependencies=no if you are into systemd, and a stop command on the service that just does nothing. If you choose iptables, you could block on INPUT any new connections from the ISC SANS research list (hxxps://isc.sans.edu/api/threatcategory/research/?text) so your box almost never shows up on Shodan et.al. You should also block the most aggressive comrades from the dshield list (hxxps://isc.sans.edu/block.txt). Insert into ipsets and reload once a day after aggregate6.py treatment automatically. The solution you picked should also be able to block and log any device you have based on its MAC address never mind what protocol it uses, because what business does your wifi enabled light bulb or printer have reporting back to its true owner how you are using them? Many devices have no business talking with the Internet at all. If the product you choose has all that, very good, probably a keeper.

Did you also consider that most exploits these days are written for x86 (PCs, Macs) and ARM (Android, Apple)? These won't work on SPARC or POWER, providing extra protection through non-monocultural obscurity. A SPARC T2 server is only 500 fiat on ebay these days and nobody sane would write exploits for such a system these days. OpenBSD has a nice list: hxxps://www.openbsd.org/sparc64.html, Gentoo also has a nice selection of supported architectures. My ranking from vulnerable to more secure would be: x86 < ARM < MIPS64 (Octeon) < POWER8/9 (IBM or Raptor) < SPARC (Sun/Oracle).
 

zer0sum

Active Member
Mar 8, 2013
520
216
43
Good lord this thread is full of weird advice! :p

I'm a 20+ year Security nerd, and specialize in firewalls, advanced threats and malware etc.
You either "need" very moderate security for your home internet, or you know you're doing really shady stuff and should take whatever precautions you can dream up.
You would be an absolute idiot if you were doing things from your house that would get the attention of nation states

This is my setup, and I'll hand out my public IP for anyone that thinks they can hack it :D

Lenovo M920Q
Onboard Intel i219
Mellanox CX3
USB3 network adapter for mgmt

ESXi with hardware passthrough of the CX3 and onboard Intel i219 network cards to my firewall virtual machine.
This changes all the time as I test stuff, but could be pfsense, opnsense, Palo Alto, Juniper, Fortinet etc.

The Intel i219 is just a 1G card, and is connected to my cable modem and gets my public facing IP address.
You can't reach ESXi through this port as it's hardware passthrough, and my firewall has no open internet facing mgmt ports.

The CX3 card is 10/40/56G and connects internally to my switches.
This is where you can connect to the firewall through ssh or web management

The USB network adapter is out of band and is just used for ESXi management.

Could it be more secure? Absolutely!
Is it secure enough for a home internet connection?
This setup is dead silent, tiny, and easily does 1G symmetric :D
 
Last edited:
  • Like
Reactions: Vesalius