What SFP28 25G for pfsense Homelab Firewall?

[Yves]

Hi all,

I once again need the advice of the pro's. You all

I am currently running pfsense 2.6 on a Supermicro M11SDV-8C-LN4F. But I need to upgrade to 25G. So I thought I asked you on advice and possible caveats on upgrading.

Possible options are:

- Dell 540-BCNI Card (hopefully just a branded Mellanox MCX512A-ACAT) about 400$
- Broadcom BCM957414A4142CC (based on the Broadcom BCM57414 Chip) about 200$
- Intel XXV710AM2-F2 about 400$

I wanted to get the Mellanox MCX512A-ACAT but I don't seam to find it anywhere. Now the big question is: are the Intel or Mellanox worth 200$ on top of the Broadcom? Or should I just take the Broadcom since the difference will never be experienced in a homelab anyway?

Thanks for your input

Best,
Yves

 

[RTM]

I suppose you may want to ask on the pfSense forum, probably the Intel NIC will work find, the other ones I have no idea about.
In the olden days it used to be Chelsio first, then Intel, then others (Mellanox support is relatively new).

Another point I feel is worth making, is that I have a feeling that it may be difficult to achieve routing 25G with pfSense.

 

[Yves]

I agree with you. I might have done this wrong since I could have asked the same question directly in the pfSense forum.

Do you think its really that hard to get routing at 25G?

 

[RTM]

I agree with you. I might have done this wrong since I could have asked the same question directly in the pfSense forum.

Do you think its really that hard to get routing at 25G?

It is always tough to say exactly what kind of performance you can get with a specific piece of hardware.
But you can use the numbers Netgate publish for their appliances to get an idea of the performance you can achieve.

As I see it, you may be able to achieve around 20Gbps of bandwidth, but that is best case scenario, the mixed traffic bandwidth will be much less.

Also I suppose I have to ask this: are you looking for 25Gbe NICs because of your internet connection? or is it just to move data between systems on your local network?

If it is the latter, you could probably benefit from letting a L3 switch route the internal traffic, that way you just have to scale your firewall to your internet connection (and other features like IDS/IPS, proxy etc).

 

[Yves]

Okay, I check there quickly. It seams that they mainly use Intel. I went AMD since I am an AMD fanboy since Ryzen / Epyc etc. started. I am not sure if my AMD EPYC 3251 (8 Core / 16 Threads / base 2.5GHz / boost 3.1GHz) is better or faster. But it for sure can not do Intel QAT. But its a pretty good little server and has a lot of power per wattage.

I guess the 20Gbps is about what I think it will be able to do as well. But we will see maybe I need to upgrade to a bigger Epyc Embedded.

The 25Gbps is mainly coming from my new ISP. I will be able to switch from 1Gbps / 100Mbps Coax to 25/25Gbps Fiber within the next month. I am totally aware that having a 25Gbps at home is insanly pointless. Never the less I am a nerd! I love insane stuff and since it is the same monthly fee for 10/10Gbps Fiber as for the 25/25Gbps Fiber so I thought it might be fun

Local network is currently completly thru a noisy Mikrotik CRS354-48P-4S+2Q+RM. Which is the next Issue no SFP28 Port on that thing. So I already have an Ubiquiti Switch Pro Aggregation ready. So I can connect the ISP to my Supermicro Server (probably thru the Broadcom Dual 25G Card), my Supermicro to the Ubiquiti Switch Pro Aggregation -> Ubiquiti Switch Pro Aggregation with dual 10Gbps DAC LAGG to the existing CRS354-48-4S+2Q+RM. Like this I have 3 Additional SFP28 25G Ports where I can use one for my TrueNAS Scale Supermicro Server (also with a Broadcom Dual 25G Card), one for a Connection to my Workstation and have one Spare.

I don't need IDS/IPS, PROXY I only use NAT / Firewallung Features on the pfSense. Some VLANs/DNS/DHCP pretty basic stuff.

 

[bleomycin]

Okay, I check there quickly. It seams that they mainly use Intel. I went AMD since I am an AMD fanboy since Ryzen / Epyc etc. started. I am not sure if my AMD EPYC 3251 (8 Core / 16 Threads / base 2.5GHz / boost 3.1GHz) is better or faster. But it for sure can not do Intel QAT. But its a pretty good little server and has a lot of power per wattage.

I guess the 20Gbps is about what I think it will be able to do as well. But we will see maybe I need to upgrade to a bigger Epyc Embedded.

The 25Gbps is mainly coming from my new ISP. I will be able to switch from 1Gbps / 100Mbps Coax to 25/25Gbps Fiber within the next month. I am totally aware that having a 25Gbps at home is insanly pointless. Never the less I am a nerd! I love insane stuff and since it is the same monthly fee for 10/10Gbps Fiber as for the 25/25Gbps Fiber so I thought it might be fun

Local network is currently completly thru a noisy Mikrotik CRS354-48P-4S+2Q+RM. Which is the next Issue no SFP28 Port on that thing. So I already have an Ubiquiti Switch Pro Aggregation ready. So I can connect the ISP to my Supermicro Server (probably thru the Broadcom Dual 25G Card), my Supermicro to the Ubiquiti Switch Pro Aggregation -> Ubiquiti Switch Pro Aggregation with dual 10Gbps DAC LAGG to the existing CRS354-48-4S+2Q+RM. Like this I have 3 Additional SFP28 25G Ports where I can use one for my TrueNAS Scale Supermicro Server (also with a Broadcom Dual 25G Card), one for a Connection to my Workstation and have one Spare.

I don't need IDS/IPS, PROXY I only use NAT / Firewallung Features on the pfSense. Some VLANs/DNS/DHCP pretty basic stuff.

I know this is an old post but I'm curious to hear what you wound up doing and how the performance worked out? Very little out there on this subject especially any kind of routing beyond 10Gbps in pfsense.