pfsense Multigigabit

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

nickf1227

Active Member
Sep 23, 2015
198
129
43
33
Hi All,

Posting to see if others have any experience to speak of when it comes to pfsense with multigigabit WAN connections.

Recently an ISP in my area began offering FTTH with 3 offerings, 500/500 1000/1000 and 2000/2000.
I just booked installation for next weekend for the gigabit tier for $60/mo, but I am considering calling them back and asking for 2 gigabit. I am already paying $200 a month for internet with 2 different ISPs (cable and Starlink) and so the 2 gigabit offering at $149/mo will actually still save me money.

Currently, I terminate one of my ISPs to an SG-5100 and the other terminates to a VM using VMXNET3 adapters. I am considering replacing both of those routers with a new virtual router with a direct PCI-E passthru.

The handoff for the XGS-PON Frontier ONT would be 10 Gigabit.
I am considering picking up an Intel X710-DA4 or a Chelsio T540-LP-CR or another NIC if there is a specifc suggestion. What is the best option today for high-speed pfSense NICs? Historically Chelsio was the go-to but the X710 uses newer silicon and seems well supported. 4 ports are a preference, but not a requirement. Ideally, I would like the best performance in terms of latency and CPU offloading. T6225-SO-CR is also on my radar if its worth it?

Mellanox, from my understanding, hasn't had the best FreeBSD support historically which is why I hadn't mentioned Mellanox. Are ConnectX4s or 5s any good from a driver perspective in FreeBSD?
 
Last edited:

dragonian

Member
Jan 3, 2020
47
30
18
I have a Mellanox ConnectX3 in my opnsense box. I haven't had any issues with FreeBSD. (or previously HardenedBSD)
 

Bjorn Smith

Well-Known Member
Sep 3, 2019
876
482
63
49
r00t.dk
X710 has low power consumption, which is nice for something that runs 24/7.

Have you considered if the ISP will deliver the internet with a SFP plug - and not RJ45? - In Denmark - if you go above 1Gbit/s you get SFP - and if you go higher than 10Gbps probably something else.

So before you buy anything - confirm with the ISP what kind of connectivity you would require.
 

oneplane

Well-Known Member
Jul 23, 2021
844
484
63
Can confirm that ConnectX3 works fine in OpnSense here as well. We also have a bunch of Aquantia based NICs in use in OpnSense, works well too but some models need to explicitly have a kernel driver loaded on boot for proper detection but that is one-time setup you have to do that persists across upgrades just fine. A few non-production boxes with ConnectX5 cards work too but we haven't had them in a WAN or production setup yet, mainly because the combination of high-end WAN and all-in-one OpnSense box hasn't been a requirement yet. Often, when we have a higher speed configuration we also have SDN and the likes where OpnSense doesn't really fit in anywhere.
 

llowrey

Active Member
Feb 26, 2018
167
140
43
I'm running pfsense as a VM with passthrough of two Intel X550-AT2 ports (RJ45) and ConnectX-4 SR-IOV virtual function (VF) port. I have a 2.5GbE modem attached to the X550. The port links just fine but the driver shows the port speed as "Unknown". It easily hits the 1.4Gbps of my connection. The CX4 was well supported out-of-the-box.

1652461691217.png
 

zer0sum

Well-Known Member
Mar 8, 2013
849
474
63
Is your handoff from the ISP device copper or fiber?

If you want to go budget I've found the Supermicro AOC-STGN-I2S is much better than the connectX-3 cards as SR-IOV works a lot better.
The CX3 card can only present both ports using SR-IOV, whereas the 82599 based cards can present ports individually. For me, that is huge.

If you need a copper transceiver I've also found the Ipolex asf-10g-t does not play nicely with Mikrotik or my AT&T BGW320.
Download speeds are great and I can hit 2.3Gbps, but uploads are in the 1-100Mbps range, so clearly there is a big issue.

My MikroTik S+RJ10 are flawless, and the only ones I'll be buying moving forward :)
 

oneplane

Well-Known Member
Jul 23, 2021
844
484
63
For the transceivers, I've found that the generic FS.com ones work pretty well. Copper, fiber, DAC, it's all good. Mostly useful if they have a warehouse nearby, otherwise you might be waiting a while.
 

nickf1227

Active Member
Sep 23, 2015
198
129
43
33
I picked up one of these. I'm not sure why FS is so expensive these days
10G SFP+ to RJ45 for Cisco SFP-10G-T-80 Compatible, 10GBASE-T SFP+ Copper RJ-45 80m Transceiver Module : Electronics (amazon.com)

Handoff is copper 10G not 2.5G.

Found a quad X710 for $199
Cisco UCSC-PCIE-IQ10GF 10GB SFP+ Nic Network Server Adapter | eBay

I also have a T520-CR. Curious what is better xD

EDIT: Decided to do this bare metal, for no particular reason other than reusing a chassis I have.
I got this to be the router
Supermicro X11SSQ-L-DE05B Embedded Intel G4600 Chip plus 8GB RAM 974575260957 | eBay
 
Last edited: