I've been playing with this a bit more. I bought this ethernet adapter addon (New OEM Dell WYSE 5070 RJ 45 Port Daughter board w/ cable - 442XR 565JP 9DJC4 | eBay
) it's a DELL OEM for the Wyse 5070. You can definitely get cheaper ones on Aliexpress and 3d print a mount for it but it's not that much cheaper. It's a Realtek 8111 chipset just like the onboard NIC.
I've tested PfSense and OPNsense on it. Pfsense requires you to install the Realtrek drivers while OPNsense has that prepackaged. OPNSense seems to provide better performance when using the DNS Resolver (uses Unbound). Performance on the Realtek NIC is about 5-10% less than that of an Intel NIC (I do not think CPU is a problem and Realtek NICs have been known to have slightly lower performance). I tested performance using iperf vs an old Xeon network appliance with Intel NICs.
The real gem happens when I tried out Mikrotik's RouterOS v7.5 x86 (installed bare metal and not CHR in a VM) with a demo license. I haven't tested the NIC performance yet, but it can't be worse than OPNsense. The real beauty is that RouterOS v.7.5 supports docker (Mikrotik has an abstraction layer on top for management). With the free demo license, I am able to run Unbound and PiHole and the DNS performance of that is far better than that on OPNsense or PfSense (we're talking about ms so you won't notice, but the numbers are there). I ran benchmarks using DNS Bench.
I think this is going to be my new router/firewall setup. At the end of the day, the firewall in Mikrotik's RouterOS is pretty damn good and the OS is pretty fast. The system boots and comes back online in about 30 seconds vs PFsense around a minute. Plus PFsense and OPNsense seem to have issues with their DNS resolver setup where saving changes will cause Unbound service to crash and restart.
A demo license for Mikrotik's RouterOS seems to fit the bill for me since it doesn't seem to have any bandwidth restrictions, and I don't use most of the features with restrictions. Although docker does take up 1 of the VETH interface afforded by the license.
Lastly, USB tethering with Android hotspots works as well. The device shows up as an LTE interface on RouterOS. I'll have to test this further. I'm currently using a T-Mobile Franklin T-9 with Google Fi as my failover. The issue I have with the device is that the DHCP lease seems to drop from time to time with PFSense and OPNsense and they do not automatically renew. I can write a script to bring it back up but I'll have to see if the same problem occurs with RouterOS. You'll have to tether using one of the USB-A ports instead of USB-C. The USB-C port doesn't seem to work on RouterOS. Could be chipset support related since the USB-C also doubles as a display out.
Things I've noticed:
- Hardware monitoring doesn't work, so no temps.
- The USB-C port doesn't work. You can probably get around this by using ESXI and the CHR version of RouterOS and doing a passthrough for the USB-C.
- Onboard EMMC storage is not recognized by the OS installer. You'll need to connect a SATA M.2 (probably better in the long run anyway).
- Mikrotik community support sucks! This is just a personal experience as a networking and Mikrotik RouterOS novice.
- Docker (Containers) does not behave the way that you are used to with traditional installs. For example, no port mapping support. It's also designed so that you'll have to create 1 VETH per container. For me running both Unbound and PiHole required that I create and mount a custom configuration for the unbound container to use port 5053 instead of 53 so that it doesn't conflict with PiHole since both are on the same VETH. Containers do not autostart on bootup and you have no control over container restart policies. I use netwatch (built in feature of RouterOS) start the containers on bootup using an http-get on Pihole for a 200 response. There's also a netwatch config to failover DNS to cloudflare's 126.96.36.199 if PiHole doesn't respond. The better way would be to poll PiHole's response on port 53 instead, I'll have to figure out how to do this, maybe a ping test with netwatch.