R86s Router

TFAN

New Member
Aug 20, 2022
2
1
3
I recently stumbled upon some offerings for a Router called "R86s" on aliexpress.
(They seem to even have an official product page at R86S)
It looks like it would be great for a small homelab.

It's a small Firewall Appliance featuring:
* Intel N5105 or N6005 Processor
* 8 or 16gb Ram
* 128gb EMMC Storage
* 2-3 (depending on the source) 2.5Gbe Ports
* optional 2x SFP+ Ports.

I'm just not sure if that small of a processor can actually do somewhat near wire-speed on dual 10gbe.
 

newabc

Active Member
Jan 20, 2019
300
140
43
Here is a review video in Chinese: link
Its RAM is soldered on the board, no RAM slot.

Below is another video(also in Chinese), but seems the video upper usually promotes similar products a lot, so you can see it as an Ad: link
 

prdtabim

Active Member
Jan 29, 2022
140
39
28
It is pretty interesting that a review video opened its case and found the SFP+ card is an OCP one.
It's interesting in a positive way since ocp cards are mostly spec standard and used as addon cards in servers. So isn't a random unknown implementation of 2 SFP+ ports, its a OCP based Mellanox Connectx-3.
 

TFAN

New Member
Aug 20, 2022
2
1
3
Here is a review video in Chinese: link
Its RAM is soldered on the board, no RAM slot.

Below is another video(also in Chinese), but seems the video upper usually promotes similar products a lot, so you can see it as an Ad: link
Thanks
The RAM is not an issue for me - The connectibility alone would make it a nice fit for my homelab; together with a MikroTik CRS305, a few TinyMiniMicro's and a 10G Nas...
Finding something that does 10Gbe routing and does not sip 50+ watts is somewhat hard in Europe...

Allegedly we will be getting one for review.
Hope it works out
I would really enjoy seeing like an iperf3 for doing layer3 across those sfp+
 
  • Like
Reactions: Amrhn

newabc

Active Member
Jan 20, 2019
300
140
43
Finding something that does 10Gbe routing and does not sip 50+ watts is somewhat hard in Europe...
Hope it works out
I would really enjoy seeing like an iperf3 for doing layer3 across those sfp+
You are right. I looked at the 1st review video again. From around 6:14, it said when the G3 version is using OpenWRT and set the dual SFP+ ports as a bridge, the N5105 can do full speed (around 9.3-9.4 Gbps on the video) between the 2 computers on its 2 SFP+ ports.

But I think if it needs IDS function(for example, Suricata), it won't reach such high speed even the Suricata is triggered by only 1 pattern(for example, iperf).
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,382
5,533
113
Was the allegation true? :) I find this device and its form factor so appealing.
It was. Genoa is sucking time this week, Supercomputing next. The 4x i226 Ryzen box video is with Alex for editing. When I get back from the Genoa launch I will be working on finishing the R86s review. It is super funky/cool.
 

Occamsrazor

Member
Feb 23, 2018
44
12
8
Thanks @Patrick it sounds great, I look forward to the review. Anyone else on here actually bought one? Just wondering if there are any massive red flags not to go for it...
 
  • Like
Reactions: Patrick

SimplyComplicated

New Member
Nov 13, 2022
2
1
3
On the surface the small form factor, 2x SFP+ and 2+ 2.5Gbe Ports are EXACTLY what I am looking for..

Fibre in my area now goes up to 2.5Gbit so I have been hunting around for a small form factor / reasonably priced solution to start upgrading my network.

I do however also want to run some traffic over a OpenVPN or Wiregard tunnel so processor speed still matters.

Would love to learn what the real world throughput is..I don't need 10Gbe speeds, but solid 2.5Gbe with SOME features running is my goal.. Most solutions I have looked at come out to close to over over $1000 (all said and done)

The SFP+ is important to me as I want to trunk to a switch or there are some direct termination options for fibre from my ISP where their fibre ONT is SFP+ and could in theory plug directly in skipping a separate fibre to Ethernet converter device.
 
Last edited:
  • Like
Reactions: Occamsrazor
Greetings!

First post in this forum. Straight-forward question:

How many Gbps can the R86S move through its SFP+ NIC that are IPSec AES-256 encrypted?

I am looking for a box on which to run OpenMPTCProuter (*) to aggregate a variety of WAN uplinks, from a VDSL, to multiple 5G modems using different MNOs, to Starlink. I am expecting the aggregate WAN bandwidth across some 4-5 uplinks to be about 2Gbps. I am planning to IPSec AES-256 (**) encrypt each of the WAN uplinks (simply different VLANs as far as the router is concerned) from the router to a host under my control that does have a proper 10Gbps WAN uplink, which will strip off the IPSec encryption and stitch the various uplinks back together into one TCP connection.

Another way of looking at this question is: can the R86S move 2Gbps of AES-256 encrypted traffic of WAN traffic and 2Gbps of associated LAN traffic out of the device's SFP+ interfaces in the form of a bunch of VLANs?

And if the device can't do that, how much can it push and does anyone here know what the the least power-hungry box is that can?

(*) I am aware of the messiness of OpenMPTCProuter, which is essentially gluing three different technologies together to achieve actual bonding. I am also aware that the various commercial SDWAN solutions that achieve the same objective far more cleanly run in the double-digit thousands of dollars. Which I can't justify in this case, since the use case here is the uplink to my own home.
(**) I am aware of WireGuard.

It would be awesome if @Patrick were to cover the encrypted throughput capabilities of that box in the upcoming review video. :cool:

Thanks!
-- Lucky
 

Occamsrazor

Member
Feb 23, 2018
44
12
8
I have no experience in this myself... but Zerotier SDWAN latest development builds are supposed to be able to do multipath/bonding:
Which is to say I have used Zerotier, but I've never tried bonding.
 
I have no experience in this myself... but Zerotier SDWAN latest development builds are supposed to be able to do multipath/bonding:
Which is to say I have used Zerotier, but I've never tried bonding.
[I don't wish to hijack this hardware thread with a discussion about multipath TCP. However, I also wish to save the reader some time walking down dead alleys].

Unfortunately, ZeroTier's new multipath features do not provide the capabilities that I mentioned above and that I am seeking: the ability to have one TCP connection with a throughput greater than that of any one WAN uplink. Of the multipath modes offered in the dev branch of ZeroTier, the "balance-rr" mode comes the closest, for which the ZeroTier Docs state:
Traffic is striped across multiple links. [...] This mode is not suitable for traffic that is sensitive to re-ordering such as TCP.
As such, OpenMPTCProuter remains the only non-crazy expensive solution that I am aware of that provides for striping of one TCP connection across multiple WAN connections. If anyone here knows of another solution providing such capability that doesn't cost many of thousands of dollars, please share.

At any rate, I am very much looking forward to @Patrick's review of the R86S in the hope that will at last be the hardware capable of handling multiple Gbps of aggregate WAN traffic.

Best,
-- Lucky
 

PigLover

Moderator
Jan 26, 2011
3,086
1,403
113
@Patrick Really looking forward to seeing your full review on this box. I'd love to see your impressions. I know you've got a lot in the queue right now but this thing is potentially really interesting.

Being impatient, of course, I went ahead and bought one. Initial impressions after playing with it for less than 24 hours:
  • I bought it on Ali from the Mii store. Used standard shipping and it arrived in 18 days (ordered Nov 11, arrived Nov 29). I don't think I've ever gotten anything through Ali any faster, even using DHL shipping options. No idea how that happened!

  • I got the "g3" version, N6005 with the 2x 10Gbe NICs.

  • Its packaged really professionally. Not that packaging is important to its operation, but it does show some care regarding the product. Double boxed, with the inside box being a fairly well done "store shelf ready" look. Actually thought it compared well with Apple's packaging.

  • The r86s itself looks and feels well engineered. The design is much more thoroughly thought through and implemented better than most of the other small 2.5g router systems people here have been looking at. The power block appears to be high quality, though I lack the tools you have to really test it.

  • It came with OpenWRT loaded on the embedded eMMC. I didn't do more than just boot it. It did boot and if OpenWRT is what you are after it looks like it should work right out of the box.

  • I popped in a random NVMe drive and installed Proxmox 7.3. All three 2.5g nics (i225) and both 10g nics recognized right out of the box. The 10g nics identify as Mellanox ConnectX-4 (nice!) and the dmesg logs suggest they have Infiniband capability enabled (interesting, though there really isn't much of a use case for 10g Infiniband).

  • Base stress testing suggests their cooling solution is really good. Running Prime95 for over 2 hours the CPU temps settled around 56c and after an initial startup jump never got over 61c. With all four cores loaded they settled in to an all-core turbo of 2.4ghz and were stable at that speed for the entire test run. There was some interesting behavior when the test was launched - for about the first 30 seconds all four cores jumped to between 3.2-3.6ghz and temps spiked to ~75c, but then everything settled in and speeds/temps quickly stabilized below 60c.

  • Also noticed that a single core stress test turbo'd that core to 3.6ghz and held it there consistently as long as not much was going on with the other cores. I found that interesting since the specs on ARC show max burst speed of 3.3ghz for the N6005.

  • I did some rudimentary network speed tests (just using iperf3). The 2.5g ports delivered as expected (~2.47g). The 10g ports also delivered expected speeds (~9.87g). I did test iperf3 to both 10g ports simultaneously with no degradation - both delivering 9.8+gbps. Note that I did not have to use multi-threaded tests to get these speeds - single thread iperf was fine (though I do have 9000 byte jumbo frames enabled). That was the limit of the network testing I could do without setting up some special configs. I'd love to see what happens if you try simultaneous 10g full duplex tests - at that point I think the PCIe 3.0 x4 interconnect to the Mellanox daughter card would limit you to something around 30gps net transfer speed rather than the 40ish that 2 10gbe full-duplex links would require.

  • Regarding the fans (yes - plural): there is a second fan on the bottom cooling the 10gbe board. That wasn't obvious from the stuff online. Even when stress testing the fans were very quiet. Not completely un-noticeable but nothing that we bother me even if it was sitting right next to me on a desk.

  • Last note: the G3 comes with WiFi+BT too. That's not too useful for me so I really didn't test it. It uses an Intel AX201 which is a pretty high-end laptop card, dual band with WiFi 6 / Bluetooth 5.2 and supports 2x2 MIMO. They also included two small "stubby" antenna. If you do need WiFi and/or BT this seems to be pretty well done too. The only thing better would be for it to be WiFi 6E, but that is a tall ask.
Overall, first impressions only, this thing is pretty awesome! Only two drawbacks for me are the small(ish) 16gb soldered RAM and, of course, the price is at the top end of the class for this type of box. But first impressions suggest to me that it is worth the price. And nothing else in this price class even offers the option of 10gbe.

All that a rather long post to let you know how much I am looking forward to your full review.
 

mach3.2

Member
Feb 7, 2022
36
25
18
@Patrick Really looking forward to seeing your full review on this box. I'd love to see your impressions. I know you've got a lot in the queue right now but this thing is potentially really interesting.

Being impatient, of course, I went ahead and bought one. Initial impressions after playing with it for less than 24 hours:
  • I bought it on Ali from the Mii store. Used standard shipping and it arrived in 18 days (ordered Nov 11, arrived Nov 29). I don't think I've ever gotten anything through Ali any faster, even using DHL shipping options. No idea how that happened!

  • I got the "g3" version, N6005 with the 2x 10Gbe NICs.

  • Its packaged really professionally. Not that packaging is important to its operation, but it does show some care regarding the product. Double boxed, with the inside box being a fairly well done "store shelf ready" look. Actually thought it compared well with Apple's packaging.

  • The r86s itself looks and feels well engineered. The design is much more thoroughly thought through and implemented better than most of the other small 2.5g router systems people here have been looking at. The power block appears to be high quality, though I lack the tools you have to really test it.

  • It came with OpenWRT loaded on the embedded eMMC. I didn't do more than just boot it. It did boot and if OpenWRT is what you are after it looks like it should work right out of the box.

  • I popped in a random NVMe drive and installed Proxmox 7.3. All three 2.5g nics (i225) and both 10g nics recognized right out of the box. The 10g nics identify as Mellanox ConnectX-4 (nice!) and the dmesg logs suggest they have Infiniband capability enabled (interesting, though there really isn't much of a use case for 10g Infiniband).

  • Base stress testing suggests their cooling solution is really good. Running Prime95 for over 2 hours the CPU temps settled around 56c and after an initial startup jump never got over 61c. With all four cores loaded they settled in to an all-core turbo of 2.4ghz and were stable at that speed for the entire test run. There was some interesting behavior when the test was launched - for about the first 30 seconds all four cores jumped to between 3.2-3.6ghz and temps spiked to ~75c, but then everything settled in and speeds/temps quickly stabilized below 60c.

  • Also noticed that a single core stress test turbo'd that core to 3.6ghz and held it there consistently as long as not much was going on with the other cores. I found that interesting since the specs on ARC show max burst speed of 3.3ghz for the N6005.

  • I did some rudimentary network speed tests (just using iperf3). The 2.5g ports delivered as expected (~2.47g). The 10g ports also delivered expected speeds (~9.87g). I did test iperf3 to both 10g ports simultaneously with no degradation - both delivering 9.8+gbps. Note that I did not have to use multi-threaded tests to get these speeds - single thread iperf was fine (though I do have 9000 byte jumbo frames enabled). That was the limit of the network testing I could do without setting up some special configs. I'd love to see what happens if you try simultaneous 10g full duplex tests - at that point I think the PCIe 3.0 x4 interconnect to the Mellanox daughter card would limit you to something around 30gps net transfer speed rather than the 40ish that 2 10gbe full-duplex links would require.

  • Regarding the fans (yes - plural): there is a second fan on the bottom cooling the 10gbe board. That wasn't obvious from the stuff online. Even when stress testing the fans were very quiet. Not completely un-noticeable but nothing that we bother me even if it was sitting right next to me on a desk.

  • Last note: the G3 comes with WiFi+BT too. That's not too useful for me so I really didn't test it. It uses an Intel AX201 which is a pretty high-end laptop card, dual band with WiFi 6 / Bluetooth 5.2 and supports 2x2 MIMO. They also included two small "stubby" antenna. If you do need WiFi and/or BT this seems to be pretty well done too. The only thing better would be for it to be WiFi 6E, but that is a tall ask.
Overall, first impressions only, this thing is pretty awesome! Only two drawbacks for me are the small(ish) 16gb soldered RAM and, of course, the price is at the top end of the class for this type of box. But first impressions suggest to me that it is worth the price. And nothing else in this price class even offers the option of 10gbe.

All that a rather long post to let you know how much I am looking forward to your full review.
Are you planning to try 10 Gig routing on pfsense/opnsense?
 

PigLover

Moderator
Jan 26, 2011
3,086
1,403
113
Are you planning to try 10 Gig routing on pfsense/opnsense?
Honestly I'm not exactly sure how I am going to use it. I had a rare confluence of spare time & spare cash at the start of November and kinda impulse purchased. I have a number of thoughts about it so we'll see. But virtualized pfSense is one possibility. A pretty likely possibility, actually.
 
  • Like
Reactions: mach3.2