Hardware to run pfsense ?

Frank Bello

Member
Nov 14, 2018
34
8
8
Hello, I'm thinking of buying some hardware to run pfsense for home use and would welcome opinions on the following options. The main requirements are: it must be quiet (or nearly quiet) and it should be capable of forwarding at >1Gbit/s, since I don't want to buy into a solution that will be obsolete in a couple of years. A rack-based solution isn't going to fly...I don't have the space. I need to forward between internal VLANs and internal/internet but I don't need IPSEC. Suricata would be a nice-to-have, so it would be good to have the option of extending RAM above 8GB.

1. Kettop Home Router I5 Mi7200L6 Core I5-7200U (16Gb Ddr4 Ram 256Gb Ssd Wifi) Aes-Ni,2.5Ghz Dual Core Fanless,6 Intel Gigabit Ethernet. It's the cheap option at $550 (£395 here in the UK). Fails the >1G requirement so it's probably a stopgap solution at best. On the plus side, it has a 60W PSU and is fanless. Other downside: Chinese manufacturer website has nothing in the way of documentation and presumably no support will be forthcoming if it ever has a fault. There are a number of very similar offerings from other vendors (protectli, andaqi and more).

2. Netgate 6100. $699, but many netizens say it is underpowered. Passmark gives it a score of 2360 which is actually worse than the CPU score of the cheaper Kettop box which scores 3403. On the plus side, it does 10G of firewall-only forwarding according to Netgate ("10.1 Gbps firewall performance, 7.9 Gbps NAT handling, or 2.1 Gbps of AES-GCM IPsec VPN"), which is pretty much my use case. Also on the plus side, it will run pfsense+... not sure if that makes much of a difference. Support is available. The 8GB of memory might be a bit too small to run Suricata ?

3. Roll-your own based on the EPYC 3101 (or a similar solution in the 30W TDP range). I like the look of the Asrock EPYC3101D4I-2T:


"Supports 4x DDR4 ECC UDIMM, RDIMM, LRDIMM up to 2666MT/s (2DPC)
Supports up to 1 x OCulinks (PCIe3.0x4 or 4x SATAIII 6Gb/s)
1 x PCIe x 16 slot
Integrated IPMI 2.0 with KVM and Dedicated LAN (RTL8211E)
2 x RJ45 10GLAN by Intel® X550-AT2"

but there are only two 10G ports on board, apart from the IPMI port, so I'd need to buy a separate managed 10G switch to break out some VLANs (or, as a compromise, there are 4-port 2.5G PCIe cards available, but they seem pricey). The motherboard is $700 (£500 here) so by the time I've added memory, SSD, power supply and case, I won't have much change from $1000. Passmark doesn't seem to have a score for the 3101, but it got good reviews here on STH.

Another roll-your-own would be the Supermicro M11SDV-4CT-LN4F. It has 4 x 1G built in but could be upgraded with a 10G NIC. Cost $577 (£417) plus case, RAM, SSD, 10G NIC, etc. Unfortunately, the Supermicro ready-built version (AS-E301-9D-8CN4) is too loud for a desk, according to the STH review, which is a shame.

4. OpnSense DEC840. "8GB DDR4 RAM, 256GB M.2. Solid State Flash and can handle upto 14.6Gbps Firewall & 2.3Gbps IPsec". It's an Epyc 3101 board but at $1176 ( €999.00 ), it's the most expensive option so far - the performance is in the same ballpark as the Netgate 6100 but it's twice the price. Having said which, I don't think the roll-your-own option #3 would end up being as small and quiet as this option - it is fanless. Will the 8GB of RAM be too small for Suricata ? If so, then #3 wins. Is OpnSense worth the extra cost, compared to pfsense+ on the Netgate 6100 ?

5. I could run pfsense CE on a VM on my home server (ESXi). Upside: cost $0. Downside: I'm very reluctant to connect the public internet directly to my server. It completely breaks the concept of defence in depth by putting my "core" compute asset directly on the public internet. Plus, this is an all-eggs-in-one-basket solution - if I have to rebuild my server for any reason, internet access will be down until I get it going again. So, this doesn't feel like a safe option.

6. Xeon-D boards like the X10SDV-7TP8F run to $3500 here (£2537). That's the bare board, not a server. Not happening...


Any thoughts, please let me know below.
 

BlueFox

Well-Known Member
Oct 26, 2015
1,412
795
113
Routing 1GBit does not take much and can be easily performed on 5+ year old low power hardware. Depends on how much above that you need? 10Gbit is a different game.
 
  • Like
Reactions: ipocnveg

RTM

Well-Known Member
Jan 26, 2014
763
279
63
Okay, so you certainly have found a lot of options, I think it is quite confusing (too many options), so let me start with some recommendations.

Figure out what your requirements are, what your budget is, and categorize the potential paths you can take (embedded under £500, embedded under £1000, built from standard components under £500 and so on).

I do not recommend going the route of virtualizing the firewall, if you depend on it to access the ESXi mgmt, you will not be able to access the ESXi when you put the ESXI machine into maintenance mode for updates (you do update right? ;) ).

You may want to consider buying used, there are plenty of interesting options that will easily route >1G if needed.

With the recommendations out of the way, let's try to break down some of the things you wrote, you specify:
  • >1G
  • £2500 is too expensive
  • If you buy the Asrock epyc board w. accessories you will not have much change from £1000
  • If you buy that Asrock epyc board you would have to buy a 10G switch.
All this tells me that you do have a upper limit on your budget (is it £1000 give or take at the max?) and that you may not already have a switch.

To help you select the best path and the best options from there, I suggest you give us some more information, like what do you have already in terms of network equipment and how many devices do you need to connect (and how).

Generally speaking, if you do not already have a VLAN capable switch, I would strongly suggest getting one while you are it.
There are plenty of options for cheap switches that will give you 10G ports (given an overall budget of say £1000, 500 may be a little tougher), though they are usually based on SFP+ rather than 10G on RJ45 like that Asrock board.
Getting a switch that will support L3 "switching" (routing) will allow you to "offload" the internal traffic to it, meaning that you firewall does not have to be as beefy (if you are to forward 10G and up internally, it will have to be quite beefy, compared to 1-2G).

Other comments:

  1. While that specific X10SDV board is quite expensive, it is also probably top of the range, there are often decent deals used on the less versions with fewer cores, 4 cores (probably 2) should be plenty for your use case.
    1. X10SDV-TP8F looks interesting (the whole X10SDV series is getting old, so you may want to get the newer X11SDV-4C-TP8F)
  2. The Netgate 6100 is quite expensive, keep in mind that you can get boards with the C3558 SOC for cheap from Supermicro and Asrock for cheap, you will have to add a 10G nice and other stuff, but it may well be cheaper. As far as I recall the main benefit of pfSense+ is QAT acceleration for IPSEC VPN, since you don't need this....
    1. If you buy a board and build it yourself, you could consider getting a C3758 or better board to double the amount of cores, for not that much extra. (the A2SDI-TP8F (or the 16 core model) have a lot of NICs but may be beyond your budget)
  3. The Opnsense device is still quite new, in general I have not heard of their devices until recently, so perhaps it would be wise to let someone else be the test subject to figure out if they are nice or not.
    1. I suspect that you can install something else like pfSense on it, should you so chose.
  4. While I have no experience with Kettop, I personally would not trust the device meant to control access at the perimeter of your network, to a manufacturer like that.
 
  • Like
Reactions: ipocnveg

cesmith9999

Well-Known Member
Mar 26, 2013
1,292
407
83
I second @BlueFox suggestion. SFF boxes can be had for cheap. You can add whatever expansion card if you need to.

@RTM has a the good suggestion of getting your list of requirements.
throughput - both physical and service wise - as adding services may slow down your throughput as all of the packets need to be scanned
services - IPS/IDS/virus checking/etc
VPN?
...

and virtualizing the firewall may cause WAF (Wife Acceptance Factor) issues... they are usually less tolerant of downtime than you or I.

Chris
 
  • Like
Reactions: ipocnveg

Frank Bello

Member
Nov 14, 2018
34
8
8
Figure out what your requirements are, what your budget is, and categorize the potential paths you can take (embedded under £500, embedded under £1000, built from standard components under £500 and so on).
Hi RTM, thanks for the detailed reply. I don't really want to buy anything more expensive than the OpnSense DEC840. My internet feed is 300Mbit/s at the moment, but it could easily exceed 1Gbit/s in the next year or two, and if it does then I don't want to have to start over. That's probably the #1 requirement right now.

Fortunately, I do have a switch that can break out VLANs from a 10G port; the bad news is, it's layer 2 only, so I can't offload the routing to it. My inter-VLAN traffic is minimal right now, but that's a reflection of the fact that I'm not offloading the routing, so most of the network devices are sitting in one VLAN.

And I definitely agree with your point about not virtualizing the firewall and then not being able to reach the ESXi managment address - that's exactly how my network is set up.

That A2SDI-TP8F board, with 4x10G and 4x1G ports, looks ideal for a firewall. Thanks for the recommendation. It's a bit more expensive than the Netgate, but not by much, considering I'd probably end up paying for shipping from the US and import duties for the Netgate device.
 
  • Like
Reactions: ipocnveg

Frank Bello

Member
Nov 14, 2018
34
8
8
I recently rebuilt my pfSense with a X11SDV-4C-TP8F and its working great.
Thanks! That's about £700 locally so the cost is in the same ballpark as the C3758. 60W versus 25W though... might be worth it for better performance however. What kind of throughput do you get from the Xeon D ?
 

Frank Bello

Member
Nov 14, 2018
34
8
8
Routing 1GBit does not take much and can be easily performed on 5+ year old low power hardware. Depends on how much above that you need? 10Gbit is a different game.
Thanks! Both the Netgate and the OpnSense box claim to support (at least) 10Gbit/s for rules-based firewalling (not IPSEC) so I'm tempted to say I'll future-proof to that level - given that the 1G solution (option #1) is $550 and 10G from Netgate is $699, I'd take the 10G option as a no-brainer. Or to put it another way, if 1G and 10G are now nearly at level pricing, there's no incentive to buy the 1G option, as it won't have a useful lifespan.
 

newabc

Active Member
Jan 20, 2019
171
44
28
I knew in Europe it is hard to get 2nd hand stuffs.
If you were in US, I will recommend HP T730 or Wyse 5070 extended. Both have passmark on 3000 or little bit more which can handle 1 gigabit with IDS which fully loaded the current Snort rulesets.

I think other CPUs with 4-core and more and the passmark on 3000 and above can do so.
 

firworks

New Member
May 7, 2021
25
14
3
Wow that thing is pure sex. AND it's running an Epyc CPU. I can see why it's so expensive. Definitely stands out against Netgate's offerings.
 
  • Like
Reactions: ipocnveg

newabc

Active Member
Jan 20, 2019
171
44
28
This form factor let me think about this one, both of them are pretty small and sexy. The DEC840 is smaller and sexier.
 
  • Like
Reactions: ipocnveg

Frank Bello

Member
Nov 14, 2018
34
8
8
Wow that thing is pure sex. AND it's running an Epyc CPU. I can see why it's so expensive. Definitely stands out against Netgate's offerings.
... and none of the roll-your-own options get close to it in terms of compactness and passive cooling (both major WAFs). It's very tempting, despite the price. On the other hand, OpnSense has not been around for as long as Netgate and has 3 employees versus Netgate's 75. Decisions, decisions...
 
  • Like
Reactions: ipocnveg

Frank Bello

Member
Nov 14, 2018
34
8
8
The bad thing about buying a box in opnsense or pfsense (apart from the price) is that you will not be able to try another firewall for example openwrt (I like this one because you can install the packages you want using this page to download the firmware: OpenWrt Firmware Selector), untangle, sophos, opnsense if you buy box in pfsense or pfsense if you buy box in opnsense.

I think the wisest decision is to build a router yourself with parts that are not old and especially from intel because intel is stability and low consumption in idle (because this is how the box will spend most of the time).
Thanks! I get the point about vendor lock-in with proprietary hardware. But there is another side to the decision which is that the self-build option will probably not be compact and fanless like the Netgate and OpnSense solutions (WAF problem), and by the time I've added in a case, fans, PSU, etc., it's almost certainly more expensive. It's also the case that we pay top dollar for Supermicro here in the UK (M11SDV-4CT-LN4F - $399 at Newegg in the US and $573 in the UK - and the second-hand market for the likes of Supermicro and Asrock Rack is nearly non-existent. Also, all the UK suppliers quote 21-day delivery for Supermicro (that's 21 working days - so actually a month) - which is a real issue, because if the motherboard fails, I have to wait a month for a replcement. There is no expedited shipping, at least not for retail customers. One potential benefit of Opnsense is 2-4 days shipping and the option to expedite it if required.

I guess part of my question was to find out if there have been any really bad experiences with any of the vendors mentioned. Fortunately, no-one has mentioned any so far.

I don't think the box will run in idle though - I'd expect Suricata to be busy 24x7, for example. And my devices are always backing up to the cloud, even overnight.
 

StevenDTX

Active Member
Aug 17, 2016
471
162
43
Thanks! That's about £700 locally so the cost is in the same ballpark as the C3758. 60W versus 25W though... might be worth it for better performance however. What kind of throughput do you get from the Xeon D ?
Unfortunately, I only have 300/300 internet connection, but I can get full speeds. I have never really watched the CPU load as I was maxing it out. I had a C2000 board that needed replacing, so I wanted to future proof a bit.
 
  • Like
Reactions: Frank Bello

Frank Bello

Member
Nov 14, 2018
34
8
8
Since you don't want to build yours, you can see and buy this box fanless 10G, it has intel processor D-2123IT, intel nics (Quad 1GbE with Intel I350-AM4, Dual 10GBase-T with Intel X557 and Dual 10G SFP+ via SoC), price $1200.




Guaranteed that work without problems, because it has Intel.
Thanks - was looking at that unit earlier today. Anandtech seemed happy with it, which is encouraging.
 
  • Like
Reactions: ipocnveg

BlueFox

Well-Known Member
Oct 26, 2015
1,412
795
113
The E302-9D is a comedy option as spending $1500 (Europe tax + other components) for a home router is a bit silly. The Xeon-D motherboard I linked, once you add in an inexpensive chassis, RAM, and SSD is going to be 1/5th the price and even then, it's likely more than you need. As computer hardware only depreciates, you might just be better off getting the cheapest option that will fit your needs today and then upgrade further down the line instead of paying a premium for it now.
 
  • Like
Reactions: ipocnveg

BlueFox

Well-Known Member
Oct 26, 2015
1,412
795
113
I think you're likely the only one that believes that is a cheap option for home use. Fanless does not require spending anywhere near that much. One can just get an Akasa case or the likes.
 
  • Like
Reactions: ipocnveg

kapone

Well-Known Member
May 23, 2015
1,046
620
113
Maybe I'm missing something...

I can do any pfSense package you can think of with symmetric gigabit routing, and a tiny system at that, with < $60 worth of hardware. Is it fanless? Nope. So what? It (among other things in my rack) sits in a corner of my basement, away from anything else. Why are we spending 100s if not thousands !! for a router??

The top left board in this system is my pfSense box. Other boards do other things.