R86S Series boxes
- Thread starter SDLeary
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
I have the 16gb N6005 version. Been running great for several months, pfSense virtualized on Proxmox. WAN port of pfSense is on one of the 2.5bge ports passed through and the LAN port is running to one of the 10gbe ports through VirtIO. Some other small tasks running on the box too but at 16gb it is a bit limited.
Getting full rate on a Comcast 1.25gbit cable modem service. I’m sure it could more. Running Suricate but if you activate more than just the basic rules it gets a bit CPU bound. The N305 version would likely be much better for IPS.
Getting full rate on a Comcast 1.25gbit cable modem service. I’m sure it could more. Running Suricate but if you activate more than just the basic rules it gets a bit CPU bound. The N305 version would likely be much better for IPS.
Yeah, I was remembering that, but a site search for R86S didn't return anything. Thanks for the link!
SDLeary
Have you been able to utilize the eMMC, or did you have to resort to the M.2?
SDLeary
If you want maximum routing and forwarding performance out of it, you should definitely be looking at TNSR, not pfSense. FD.io and VPP is significantly faster than kernel based networking on both FreeBSD and Linux. I'm talking a massive difference.
I'm actually also looking for TNSR/VPP performance benchmarks from a device like the one you describe.
I'm actually also looking for TNSR/VPP performance benchmarks from a device like the one you describe.
I tested booting Proxmox from the eMMC and it works great (instructions here). But since I had a spare m.2 NVMe card i decided against running it that way long term.
No question that vpp would lower required hardware resources, but is it necessary for networking limited to 10g? OPnsense/pfsense, openwrt and vyos should all handle close to 10g routing with a firewall thrown in as well (if that matters to the OP).
I don't know to be honest. The benchmarks I've seen haven't been detailed enough to make that clear.
Most people only post straight iperf3 tests without NAT, routing and firewall rules. Also, people tend to forget posting bidirectional tests. Ideally, I'd also like to see packets per second benchmarks.
I would imagine that VPP would be quite a bit faster for small packets, even at 10 Gbit/s.
The problem with OPNSense and pfsense seems to be low performance if used virtualized in Proxmox/KVM.
You can get 10Gbps routing on a decent CPU with Linux, for example with VyOS or even just OpenWrt. I'm trying out OpenWrt at home and am seeing <10% CPU usage when routing 8Gbps of Ookla Speedtest traffic, with an out-of-the-box configuration. Speedtest.net is IPv4-only at the moment, so that includes NAT overhead. I can get ~1.6Gbps with 150 byte packets using iperf3, but I didn't check CPU usage for that test. That's on a Core i5-9500 as I don't yet have an i3-N305 system to test with, but the N305 shouldn't be significantly slower than the 9500.
Yes, you can, at least with 1500 byte packets. VPP will still be much faster in terms of smaller packet sizes though and is much more likely to handle 10Gbps bidirectionally.
VyOS is in the process of adding VPP support and even a GUI, so that's looking really promising.
Thank you for providing the results of your testing by the way.
I briefly researched a little bit more into VPP/DPDK and realised running it will peg a core to 100% since the CPU has to poll for packets.Yes, you can, at least with 1500 byte packets. VPP will still be much faster in terms of smaller packet sizes though and is much more likely to handle 10Gbps bidirectionally.
VyOS is in the process of adding VPP support and even a GUI, so that's looking really promising.
Thank you for providing the results of your testing by the way.
I haven't run any power meter measurements with VPP/DPDK, but I'd imagine a single core pegged to 100% all the time might not result in favourable idle power consumption, which imo is an important point for home use.
But to hit line rate routing for 10G>, VPP is the way forward. Kernel based processing is too slow.
In polling mode, yes. There is also interrupt mode and adaptive mode available for NICs with driver support for those modes which should help reduce power consumption.
Even with VPP, I think my comment about Linux vs BSD still stands, since as far as I know BSD doesn't support DPDK/VPP very well yet. I remember seeing some work to integrate DPDK into OpenWRT but I'm not sure of the status of that work.
My use case is a pretty standard home use case. The only time I care about high download speeds is for downloads of large files, which will be using large packet sizes. I don't have a huge amount of traffic other than when downloads are happening.
What was your comment about Linux vs. BSD? I must've missed it.
Yes, VPP support for Linux is definitely better than BSD.Fair enough. It would be the same for me, but I prefer to avoid bottlenecks if I can at reasonable cost.