First pfSense Build

Discussion in 'DIY Server and Workstation Builds' started by lunadesign, Feb 12, 2020.

  1. lunadesign

    lunadesign Member

    Joined:
    Aug 7, 2013
    Messages:
    106
    Likes Received:
    7
    I'm looking for a fairly robust firewall/VLAN router for my home office/lab to replace my ancient/overtaxed Cisco ASA 5505.

    I've got a bunch of physical and virtualized servers and a few WAPs all running on a handful of subnets. My current WAN is 100 Mbit fiber but I'll likely upgrade to 500 Mbit or 1 Gbit in the near future.

    I plan to run Snort/Suricata and a DNS server on this box. No current plans for VPN but I may need it down the road. Onboard SPF+ is important so I can feed my switches with 10G goodness.

    Here's what I'm thinking:

    Supermicro X10SDV-TP8F (Xeon D-1518 @ 35W, 2 SPF+, 6 GbE)
    2 Hynix HMA81GR7CJR8N-VK (8GB DDR4 1.2V 2666MHz Registered ECC)
    2 Intel D3-S4610 240GB SSDs in ZFS mirror
    Pico PSU TBD (any suggestions?)
    Case TBD

    This board is a "Flex ATX" motherboard which gives it enough room to have regular DIMM slots (not SODIMMS) and 2 PCI-E 3.0 x8 slots (great for future expansion).

    The case is going to be tricky because I need this box to be quiet. Ideally it would be a small box just large enough to hold the the board (including expansion slots), SSDs and a 120mm fan pushing air front-to-back over the CPU heatsink. Something like the Supermicro E300 but tall enough to fit larger (quieter) fans would be ideal. I've scoured the Internet but haven't found a good fit yet.

    I was originally looking at the C3000 (Denverton) motherboards but the ones with onboard SFP+ are pretty pricey compared to this one. I couldn't find *any* D-1600 boards.

    Using the Netgate appliances as a comparison, this box would sit somewhere between an XG-7100 and an XG-1537. Probably a bit over-powered for my needs but having headroom for future growth is good as long as it's not sucking too much power.

    Thoughts? Suggestions?
     
    #1
  2. PigLover

    PigLover Moderator

    Joined:
    Jan 26, 2011
    Messages:
    2,824
    Likes Received:
    1,153
    You’ve made some decent choices. The disk config is massive overkill for pfSense but they are cheap enough these days so why not.

    With that MB you don’t need the pico-PSU. It can run on 12v power from the 4-pin connector only (half of the 8-pin cpu power connector). You just need a cable to plug your power brick’s barrel connector into the 4 pin. The there is a regular 4-pin Molex connector for power to your SSDs.
     
    #2
  3. lunadesign

    lunadesign Member

    Joined:
    Aug 7, 2013
    Messages:
    106
    Likes Received:
    7
    Thanks!

    On the disk config, yeah, I kinda figured that was the case. However, I saw a bunch of forum posts about UFS corruption after power failures. While I've got a UPS, I've had a few extended outages that outlasted my UPS. So, apparently ZFS is the recommendation nowadays. And since I'm already at ZFS, might as well throw another cheap SSD in there and mirror it.

    BTW, I picked the Intel drives for their power loss protection. Otherwise, I would have gone with some Samsungs I already have.
    Ah....I didn't realize that. I'll do a bit more research on that but if you have any power brick part numbers handy, please let me know.

    Thanks again!
     
    #3
  4. BeTeP

    BeTeP Active Member

    Joined:
    Mar 23, 2019
    Messages:
    370
    Likes Received:
    188
    I do not think I understand the reason why you think that you need any SFP+ ports in the router.
    Since your uplink is not even a gigabit - you don't have any "10G goodness to feed to your switches" to begin with.
     
    #4
    gb00s likes this.
  5. lunadesign

    lunadesign Member

    Joined:
    Aug 7, 2013
    Messages:
    106
    Likes Received:
    7
    Good question. I probably should have explained that in the original post.

    It's basically to avoid cases where a 1G connection between the pfSense box and the 1st switch would be saturated by a mixture of inter-VLAN routing (i.e., copying a large file from one system to another on a different subnet) and WAN-to-LAN traffic. It's certainly not going to be at the 10G level but maybe 2-4G level at times.

    I'm familiar enough with the limitations of LAGGs and their hash algorithms so I didn't want to bother with that. And none of my switches support 2.5 or 5G Ethernet. So 10G was the easiest way to go.
     
    #5
  6. zack$

    zack$ Active Member

    Joined:
    Aug 16, 2018
    Messages:
    270
    Likes Received:
    97
    If you want 10G goodness may I suggest this beaut with your choice of SFP+ to RJ45 transceiver for peanuts: Supermicro X10SLH-LN6TF Motherboard w/ onboard 6x 10G 3x X540-T2 Nics &TPM &HSU | eBay

    (No afflitation with the seller.)

    It will likely be cheaper than your original choice and with an e3-12XXL v3/4 chip, you could still maintain a low tdp with still more power.

    At that rate you could run pfsense as a VM as well as others.
     
    #6
    itronin likes this.
  7. Terry Wallace

    Terry Wallace PsyOps SysOp

    Joined:
    Aug 13, 2018
    Messages:
    134
    Likes Received:
    75
    Your intervlan routing should happen in your switch ideally... leave the firewall to just firewalling :)

    Also unless I am recalling wrong.. the pfsense install is its own os and doesn't run zfs under it.. but you can pick mirrored install slices on 2 devices if I recall correctly.
     
    #7
  8. lunadesign

    lunadesign Member

    Joined:
    Aug 7, 2013
    Messages:
    106
    Likes Received:
    7
    My switches are pretty simple L2 switches (Mikrotik CSS326 and Cisco SG220) so I sized up the pfSense box to handle the inter-VLAN routing. Plus it allows me to run some filtering (ie, home subnet can't get to any of the work subnets).
    pfSense is it's own OS (actually a FreeBSD fork) but as part of the OS install you have to pick which filesystem to use. ZFS is apparently a fairly recent addition.
     
    #8
  9. itronin

    itronin Active Member

    Joined:
    Nov 24, 2018
    Messages:
    290
    Likes Received:
    177
    #9
  10. lunadesign

    lunadesign Member

    Joined:
    Aug 7, 2013
    Messages:
    106
    Likes Received:
    7
    That is mighty interesting! For some reason, I can't find that board on the Supermicro site. It looks like a proprietary motherboard size. I'll have to ponder this. Thanks!
     
    #10
  11. kapone

    kapone Well-Known Member

    Joined:
    May 23, 2015
    Messages:
    700
    Likes Received:
    313
    It's a relatively standard uATX sized board. I think one or two of the holes don't line up, but it'll be fine overall.
     
    #11
  12. zack$

    zack$ Active Member

    Joined:
    Aug 16, 2018
    Messages:
    270
    Likes Received:
    97
    It's actually an ATX board with the IO ports on the opposite side. Because of this, it will be closer to an E-ATX form factor.

    It will fit in an E-ATX chassis like the CSE-815TQ for example.
     
    #12
  13. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    2,113
    Likes Received:
    491
    That board would be super attractive if it could fit in a case that was more suited for a networking closet (less than 15" deep).
     
    #13
  14. ReturnedSword

    ReturnedSword Active Member

    Joined:
    Jun 15, 2018
    Messages:
    170
    Likes Received:
    36
    For your use case it seems like Xeon D is a bit overkill. If you do the VLAN routing on your switch, the router itself only needs to be powerful enough to run the services.

    Correct me if I'm wrong as my networking knowledge is rusty - I mostly do design for years now so I focus on more high level. This is a simplified explanation. The router is only needed for the initial connection. When new devices are plugged into a switch, the packets are broadcast to all ports (like a hub). Over time (pretty quickly), the switch, even if it's a dumb one, will learn which MAC address is connected to each port via its switching table, and thus transfers will happen on the switch itself (i.e. if you are initiating a 10 Gbps transfer between two devices, it would be routed directly by the switch, assuming the device and switch port all support 10 Gbps). Thus you do not need 10 Gbps on the router itself, not that any reasonably priced hardware (CPUs) can support 10 Gbps anyway. This is mostly the domain of dedicated networking ASICs and FPGAs.
     
    #14
  15. lunadesign

    lunadesign Member

    Joined:
    Aug 7, 2013
    Messages:
    106
    Likes Received:
    7
    I agree the Xeon D is probably overkill but nice to have the headroom (as long as it's not ridiculous). I tend to keep my gear in service for 5+ years and its hard to know where things will go 5 years from now. It wouldn't surprise me if IPS/IDS functionality gets more complicated and requires more CPU resources.

    With regards to the VLAN routing on the switch, I'm not sure that's accurate. At a minimum, I can't see a switch figuring out on its own the various firewall rules between the VLANs. For example, with a firewall doing the routing, systems on the work VLAN can ping those on the home VLAN and the response is allowed due to the firewall state table. But the home VLAN system can't ping the work VLAN systems due to the firewall rule. I'm not sure how a switch would be able to mimic that.

    With regards to routing performance, I was thinking the same thing as you. However, I did a quick test the other day with a temporary install of pfSense on an E5-1650V2. I first had two other systems with 10G ports connected directly with a DAC and saw iPerf hit a sustained 9.85 Gbits/s. Then I put those ports on different subnets and put the pfSense system in between (each connected via DAC cable) and saw 8.8 - 9.6 Gbits/s. That was without any tuning or tweaking and the CPU wasn't taxed at all. I'm not sure if this is a fair test but it was better than I expected for a general purpose CPU.
     
    #15
  16. lunadesign

    lunadesign Member

    Joined:
    Aug 7, 2013
    Messages:
    106
    Likes Received:
    7
    Are you sure I only need to use 4 pins? The manual refers to it as the "8-pin DC power connector" but doesn't say anything about only using half of it. :)

    Also, where do I find a cable that converts from the usual power brick barrel connector to the 4/8 pin block?

    Thanks!
     
    #16
  17. PigLover

    PigLover Moderator

    Joined:
    Jan 26, 2011
    Messages:
    2,824
    Likes Received:
    1,153
    Absolutely sure that you just need the 4-pins (or the two systems I have running are an anomaly).

    I have them in SM CSE-e300 chassis and the connector was included with the chassis. Note that newer e300 chassis are delivered with an 8-pin connector. Unfortunately I don’t have a link to the cable.
     
    #17
  18. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    2,113
    Likes Received:
    491
    May I ask how the noise is on that case and what CPU you're running?
     
    #18
  19. PigLover

    PigLover Moderator

    Joined:
    Jan 26, 2011
    Messages:
    2,824
    Likes Received:
    1,153
    I have three of them. Two were originally x10sdv-tp8f (D1518), but have been replaced with x11sdv-tp8f (D2166NT). The other is A2sdi-h-tp8f (c3958).

    All three chassis have 3 standard supermicro fans.

    I don’t have dB measurements on them, but during normal operation the fans slow down and a reasonably quiet. I wouldn’t want them in my living room but if they don’t need to be in a soundproofed closet either. At when the fans are running full bore it is reasonably loud. If you get the 12 core “D’s” running high load for a while they will ramp up to something you’d definitely notice.
     
    #19
    nikalai likes this.
  20. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    2,113
    Likes Received:
    491
    Thanks for the feedback.
     
    #20
Similar Threads: pfSense Build
Forum Title Date
DIY Server and Workstation Builds New 1U pfSense build - Can't decide on hardware Aug 18, 2018
DIY Server and Workstation Builds Pfsense Build - 1U, Quiet, Short Depth Aug 2, 2018
DIY Server and Workstation Builds Open Compute PFSense Build, Maybe a little overkill ? Jun 28, 2018
DIY Server and Workstation Builds Low Power pfsense build ... Denverton? Aug 21, 2017
DIY Server and Workstation Builds Small and Silent PFSENSE build recommendations Jul 30, 2017

Share This Page