Intel N305 and Intel 82599ES 10 Gbps SFP+ NICs: PCIe Bandwidth Question

[John T Davis]

I bought one of those Topton N305 OPNSense boxes with 3x Intel 2.5 Gbps NICs (I226-V) and 2x SFP+ NICs.
The SFP+ NICs are 82599ES, PCIe 2.0 (5 GT/s, downgraded to x4).

So, that's 4GB/s shared between them, I think (500 Mbps * 4).

If I just use the SFP+ ports for OPNSense, with one being LAN and the other being WAN, and don't do inter-VLAN (Level 3) switching/routing, is this a signficant bottleneck for a small home/home office environment?
I suspect those two SFP+ ports are sharing PCIe lanes based on their PCIe addresses in Linux Mint (e.g., 02.00.0 and 02.00.1), so using them both could be a problem.

If that is a problem, would it be improved by using one of the 2.5 Gbps RJ-45 NICs for WAN uplink?
I have no real need for 10 GbE on my WAN. My WAN speed will never ever be faster than 2 Gbps. It's 1.3 Gbps max now on bursty connections, and otherwise usually around 1 Gbps.

I don't mind being told I've made a poor purchasing decision, but since I'm not trying to do L3 switching between the two SFP+ ports, hopefully this will work out.

Mostly I was going for low noise and small physical footprint, at lower cost (I wanted a Deciso box, but their prices are bonkers for my budget), but I do regret not paying closer attention to the 10Gbps NICs before I bought it.

 

[nexox]

Your PCIe bandwidth should be alright - 2.0x4 is 16Gbps full duplex, so you can (theoretically) do a 10Gbps download while uploading at 6Gbps (minus some overhead.) Whether the rest of the system can keep up depends on the software/OS and the packet sizes your workloads use (fortunately most high bandwidth applications at home use large packets, which are easier for the router CPU to process.)

 

[John T Davis]

Thanks!

That's good to hear. Though, now I'm a bit confused about why I was running into so much static online about the quality of this board for 10 Gbps.

Maybe those versions had more SFP+ ports and were bottlenecking from that.

 

[kapone]

@John T Davis - You'll be fine unless you're pushing the limits of 10g. Which is somewhat doubtful, if you're gonna use this for OPNsense, and your WAN speed is not 10g.

The only traffic hitting that Nic will be internet traffic, if you're doing L3 routing on a switch.

 

[John T Davis]

@John T Davis - You'll be fine unless you're pushing the limits of 10g. Which is somewhat doubtful, if you're gonna use this for OPNsense, and your WAN speed is not 10g.

The only traffic hitting that Nic will be internet traffic, if you're doing L3 routing on a switch.

Thanks for the reassurance.

My core switch isn't even an L3 switch. I've structured my network to avoid inter-VLAN switching.
So, WAN comes in through OPNSense and goes to the core switch, and all switching is handled there.
My OPNSense is relatively idle most of the time.

 

[nexox]

Though, now I'm a bit confused about why I was running into so much static online about the quality of this board for 10 Gbps.

Well it can't do 10G symmetric at full speed, and quite a few people will insist that's a huge flaw, even though it's extremely uncommon to actually do anything like that kind of bandwidth if you do have 10G WAN at home.

 

[John T Davis]

Well it can't do 10G symmetric at full speed, and quite a few people will insist that's a huge flaw, even though it's extremely uncommon to actually do anything like that kind of bandwidth if you do have 10G WAN at home.

Ah. That makes sense.

That's such an unrealistic use case that I hadn't even considered it.
I wish I was interesting enough to be doing 10 Gbps full duplex workloads at home.