First pfSense Build

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

lunadesign

Active Member
Aug 7, 2013
256
34
28
I'm looking for a fairly robust firewall/VLAN router for my home office/lab to replace my ancient/overtaxed Cisco ASA 5505.

I've got a bunch of physical and virtualized servers and a few WAPs all running on a handful of subnets. My current WAN is 100 Mbit fiber but I'll likely upgrade to 500 Mbit or 1 Gbit in the near future.

I plan to run Snort/Suricata and a DNS server on this box. No current plans for VPN but I may need it down the road. Onboard SPF+ is important so I can feed my switches with 10G goodness.

Here's what I'm thinking:

Supermicro X10SDV-TP8F (Xeon D-1518 @ 35W, 2 SPF+, 6 GbE)
2 Hynix HMA81GR7CJR8N-VK (8GB DDR4 1.2V 2666MHz Registered ECC)
2 Intel D3-S4610 240GB SSDs in ZFS mirror
Pico PSU TBD (any suggestions?)
Case TBD

This board is a "Flex ATX" motherboard which gives it enough room to have regular DIMM slots (not SODIMMS) and 2 PCI-E 3.0 x8 slots (great for future expansion).

The case is going to be tricky because I need this box to be quiet. Ideally it would be a small box just large enough to hold the the board (including expansion slots), SSDs and a 120mm fan pushing air front-to-back over the CPU heatsink. Something like the Supermicro E300 but tall enough to fit larger (quieter) fans would be ideal. I've scoured the Internet but haven't found a good fit yet.

I was originally looking at the C3000 (Denverton) motherboards but the ones with onboard SFP+ are pretty pricey compared to this one. I couldn't find *any* D-1600 boards.

Using the Netgate appliances as a comparison, this box would sit somewhere between an XG-7100 and an XG-1537. Probably a bit over-powered for my needs but having headroom for future growth is good as long as it's not sucking too much power.

Thoughts? Suggestions?
 

PigLover

Moderator
Jan 26, 2011
3,184
1,545
113
You’ve made some decent choices. The disk config is massive overkill for pfSense but they are cheap enough these days so why not.

With that MB you don’t need the pico-PSU. It can run on 12v power from the 4-pin connector only (half of the 8-pin cpu power connector). You just need a cable to plug your power brick’s barrel connector into the 4 pin. The there is a regular 4-pin Molex connector for power to your SSDs.
 

lunadesign

Active Member
Aug 7, 2013
256
34
28
You’ve made some decent choices. The disk config is massive overkill for pfSense but they are cheap enough these days so why not.
Thanks!

On the disk config, yeah, I kinda figured that was the case. However, I saw a bunch of forum posts about UFS corruption after power failures. While I've got a UPS, I've had a few extended outages that outlasted my UPS. So, apparently ZFS is the recommendation nowadays. And since I'm already at ZFS, might as well throw another cheap SSD in there and mirror it.

BTW, I picked the Intel drives for their power loss protection. Otherwise, I would have gone with some Samsungs I already have.
With that MB you don’t need the pico-PSU. It can run on 12v power from the 4-pin connector only (half of the 8-pin cpu power connector). You just need a cable to plug your power brick’s barrel connector into the 4 pin. The there is a regular 4-pin Molex connector for power to your SSDs.
Ah....I didn't realize that. I'll do a bit more research on that but if you have any power brick part numbers handy, please let me know.

Thanks again!
 

BeTeP

Well-Known Member
Mar 23, 2019
653
429
63
I do not think I understand the reason why you think that you need any SFP+ ports in the router.
Since your uplink is not even a gigabit - you don't have any "10G goodness to feed to your switches" to begin with.
 
  • Like
Reactions: gb00s

lunadesign

Active Member
Aug 7, 2013
256
34
28
I do not think I understand the reason why you think that you need any SFP+ ports in the router.
Since your uplink is not even a gigabit - you don't have any "10G goodness to feed to your switches" to begin with.
Good question. I probably should have explained that in the original post.

It's basically to avoid cases where a 1G connection between the pfSense box and the 1st switch would be saturated by a mixture of inter-VLAN routing (i.e., copying a large file from one system to another on a different subnet) and WAN-to-LAN traffic. It's certainly not going to be at the 10G level but maybe 2-4G level at times.

I'm familiar enough with the limitations of LAGGs and their hash algorithms so I didn't want to bother with that. And none of my switches support 2.5 or 5G Ethernet. So 10G was the easiest way to go.
 

zack$

Well-Known Member
Aug 16, 2018
701
315
63
If you want 10G goodness may I suggest this beaut with your choice of SFP+ to RJ45 transceiver for peanuts: Supermicro X10SLH-LN6TF Motherboard w/ onboard 6x 10G 3x X540-T2 Nics &TPM &HSU | eBay

(No afflitation with the seller.)

It will likely be cheaper than your original choice and with an e3-12XXL v3/4 chip, you could still maintain a low tdp with still more power.

At that rate you could run pfsense as a VM as well as others.
 
  • Like
Reactions: itronin

Terry Wallace

PsyOps SysOp
Aug 13, 2018
197
118
43
Central Time Zone
Your intervlan routing should happen in your switch ideally... leave the firewall to just firewalling :)

Also unless I am recalling wrong.. the pfsense install is its own os and doesn't run zfs under it.. but you can pick mirrored install slices on 2 devices if I recall correctly.
 

lunadesign

Active Member
Aug 7, 2013
256
34
28
Your intervlan routing should happen in your switch ideally... leave the firewall to just firewalling :)
My switches are pretty simple L2 switches (Mikrotik CSS326 and Cisco SG220) so I sized up the pfSense box to handle the inter-VLAN routing. Plus it allows me to run some filtering (ie, home subnet can't get to any of the work subnets).
Also unless I am recalling wrong.. the pfsense install is its own os and doesn't run zfs under it.. but you can pick mirrored install slices on 2 devices if I recall correctly.
pfSense is it's own OS (actually a FreeBSD fork) but as part of the OS install you have to pick which filesystem to use. ZFS is apparently a fairly recent addition.
 

lunadesign

Active Member
Aug 7, 2013
256
34
28
If you want 10G goodness may I suggest this beaut with your choice of SFP+ to RJ45 transceiver for peanuts: Supermicro X10SLH-LN6TF Motherboard w/ onboard 6x 10G 3x X540-T2 Nics &TPM &HSU | eBay

(No afflitation with the seller.)

It will likely be cheaper than your original choice and with an e3-12XXL v3/4 chip, you could still maintain a low tdp with still more power.

At that rate you could run pfsense as a VM as well as others.
That is mighty interesting! For some reason, I can't find that board on the Supermicro site. It looks like a proprietary motherboard size. I'll have to ponder this. Thanks!
 

kapone

Well-Known Member
May 23, 2015
1,095
642
113
That is mighty interesting! For some reason, I can't find that board on the Supermicro site. It looks like a proprietary motherboard size. I'll have to ponder this. Thanks!
It's a relatively standard uATX sized board. I think one or two of the holes don't line up, but it'll be fine overall.
 

zack$

Well-Known Member
Aug 16, 2018
701
315
63
It's actually an ATX board with the IO ports on the opposite side. Because of this, it will be closer to an E-ATX form factor.

It will fit in an E-ATX chassis like the CSE-815TQ for example.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
That board would be super attractive if it could fit in a case that was more suited for a networking closet (less than 15" deep).
 

ReturnedSword

Active Member
Jun 15, 2018
526
235
43
Santa Monica, CA
For your use case it seems like Xeon D is a bit overkill. If you do the VLAN routing on your switch, the router itself only needs to be powerful enough to run the services.

Correct me if I'm wrong as my networking knowledge is rusty - I mostly do design for years now so I focus on more high level. This is a simplified explanation. The router is only needed for the initial connection. When new devices are plugged into a switch, the packets are broadcast to all ports (like a hub). Over time (pretty quickly), the switch, even if it's a dumb one, will learn which MAC address is connected to each port via its switching table, and thus transfers will happen on the switch itself (i.e. if you are initiating a 10 Gbps transfer between two devices, it would be routed directly by the switch, assuming the device and switch port all support 10 Gbps). Thus you do not need 10 Gbps on the router itself, not that any reasonably priced hardware (CPUs) can support 10 Gbps anyway. This is mostly the domain of dedicated networking ASICs and FPGAs.
 

lunadesign

Active Member
Aug 7, 2013
256
34
28
For your use case it seems like Xeon D is a bit overkill. If you do the VLAN routing on your switch, the router itself only needs to be powerful enough to run the services.

Correct me if I'm wrong as my networking knowledge is rusty - I mostly do design for years now so I focus on more high level. This is a simplified explanation. The router is only needed for the initial connection. When new devices are plugged into a switch, the packets are broadcast to all ports (like a hub). Over time (pretty quickly), the switch, even if it's a dumb one, will learn which MAC address is connected to each port via its switching table, and thus transfers will happen on the switch itself (i.e. if you are initiating a 10 Gbps transfer between two devices, it would be routed directly by the switch, assuming the device and switch port all support 10 Gbps). Thus you do not need 10 Gbps on the router itself, not that any reasonably priced hardware (CPUs) can support 10 Gbps anyway. This is mostly the domain of dedicated networking ASICs and FPGAs.
I agree the Xeon D is probably overkill but nice to have the headroom (as long as it's not ridiculous). I tend to keep my gear in service for 5+ years and its hard to know where things will go 5 years from now. It wouldn't surprise me if IPS/IDS functionality gets more complicated and requires more CPU resources.

With regards to the VLAN routing on the switch, I'm not sure that's accurate. At a minimum, I can't see a switch figuring out on its own the various firewall rules between the VLANs. For example, with a firewall doing the routing, systems on the work VLAN can ping those on the home VLAN and the response is allowed due to the firewall state table. But the home VLAN system can't ping the work VLAN systems due to the firewall rule. I'm not sure how a switch would be able to mimic that.

With regards to routing performance, I was thinking the same thing as you. However, I did a quick test the other day with a temporary install of pfSense on an E5-1650V2. I first had two other systems with 10G ports connected directly with a DAC and saw iPerf hit a sustained 9.85 Gbits/s. Then I put those ports on different subnets and put the pfSense system in between (each connected via DAC cable) and saw 8.8 - 9.6 Gbits/s. That was without any tuning or tweaking and the CPU wasn't taxed at all. I'm not sure if this is a fair test but it was better than I expected for a general purpose CPU.
 

lunadesign

Active Member
Aug 7, 2013
256
34
28
With that MB you don’t need the pico-PSU. It can run on 12v power from the 4-pin connector only (half of the 8-pin cpu power connector). You just need a cable to plug your power brick’s barrel connector into the 4 pin. The there is a regular 4-pin Molex connector for power to your SSDs.
Are you sure I only need to use 4 pins? The manual refers to it as the "8-pin DC power connector" but doesn't say anything about only using half of it. :)

Also, where do I find a cable that converts from the usual power brick barrel connector to the 4/8 pin block?

Thanks!
 

PigLover

Moderator
Jan 26, 2011
3,184
1,545
113
Absolutely sure that you just need the 4-pins (or the two systems I have running are an anomaly).

I have them in SM CSE-e300 chassis and the connector was included with the chassis. Note that newer e300 chassis are delivered with an 8-pin connector. Unfortunately I don’t have a link to the cable.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
Absolutely sure that you just need the 4-pins (or the two systems I have running are an anomaly).

I have them in SM CSE-e300 chassis and the connector was included with the chassis. Note that newer e300 chassis are delivered with an 8-pin connector. Unfortunately I don’t have a link to the cable.
May I ask how the noise is on that case and what CPU you're running?
 

PigLover

Moderator
Jan 26, 2011
3,184
1,545
113
May I ask how the noise is on that case and what CPU you're running?
I have three of them. Two were originally x10sdv-tp8f (D1518), but have been replaced with x11sdv-tp8f (D2166NT). The other is A2sdi-h-tp8f (c3958).

All three chassis have 3 standard supermicro fans.

I don’t have dB measurements on them, but during normal operation the fans slow down and a reasonably quiet. I wouldn’t want them in my living room but if they don’t need to be in a soundproofed closet either. At when the fans are running full bore it is reasonably loud. If you get the 12 core “D’s” running high load for a while they will ramp up to something you’d definitely notice.
 
  • Like
Reactions: nikalai

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
I have three of them. Two were originally x10sdv-tp8f (D1518), but have been replaced with x11sdv-tp8f (D2166NT). The other is A2sdi-h-tp8f (c3958).

All three chassis have 3 standard supermicro fans.

I don’t have dB measurements on them, but during normal operation the fans slow down and a reasonably quiet. I wouldn’t want them in my living room but if they don’t need to be in a soundproofed closet either. At when the fans are running full bore it is reasonably loud. If you get the 12 core “D’s” running high load for a while they will ramp up to something you’d definitely notice.
Thanks for the feedback.