10G Router CWWK/Topton/Qotom and further

[Mike9474593]

The Q10900U6 S13 (N305 + X710 + X540) appears to be available on AliExpress. Has anyone rolled the dice on one of these yet?

It sure is tempting for a firewall appliance. The RJ45 10g ports are nice for connecting to >1G ONT that seems to be on the rise for home fiber internet service, along with SFP+ for everything else 10g.

I am skeptical of the cooling capabilities of the enclosure. Imagine maxing out all four 10G ports and the CPU, I don't think there's enough aluminum there to safely keep the electronics happy. But a couple low-speed 120mm fans resting on top might be enough to keep things safe. Thoughts?

The device seems to be a pretty good choice for a router/firewall, at least on paper. As you say, N305 might run a bit hot, but N100/N150 might be better. Should be enough for 10G routing if no IDS/IPS and only DHCP is involved. I do not know how much CPU PPPoE is eating on 10G. In any case Linux is better than BSD/*sense for PPPoe performance. If anyone is willing to pay the fiber pull to my home, I'm pulling the trigger and will gladly test the device for you all

 

[blunden]

The Q10900U6 S13 (N305 + X710 + X540) appears to be available on AliExpress. Has anyone rolled the dice on one of these yet?

It sure is tempting for a firewall appliance. The RJ45 10g ports are nice for connecting to >1G ONT that seems to be on the rise for home fiber internet service, along with SFP+ for everything else 10g.

I am skeptical of the cooling capabilities of the enclosure. Imagine maxing out all four 10G ports and the CPU, I don't think there's enough aluminum there to safely keep the electronics happy. But a couple low-speed 120mm fans resting on top might be enough to keep things safe. Thoughts?

I have the Q10900H6 S13 with an N100 and Aquantia 2 x AQC113C NICs, similar to the one STH reviewed except that mine is fanless.

It has been solid so far (I replaced the thermal pad for the NICs as mentioned in the separate thread for that model), but I use it as a small server so I only use one NIC. The Aquantia NIC is more power efficient than the X540 though if I'm not mistaken, so it presumably generates less heat.

I don't think it's designed for you to use all NICs at once but instead exist to give you options. You would also have to make sure that all 8 PCI-E lanes needed have been connected to the NICs. I found Qotom easy to talk to, so you should be able to ask them about it if the information hasn't already been posted.

What kind of routing performance are you looking for? With or without PPPoE?

 

[matt_garman]

I don't think it's designed for you to use all NICs at once but instead exist to give you options. You would also have to make sure that all 8 PCI-E lanes needed have been connected to the NICs. I found Qotom easy to talk to, so you should be able to ask them about it if the information hasn't already been posted.

Yeah, I thought about this some more. Check my math on this, but:

• 10gigaBITS/sec = 1.25gigaBYTES/sec (GB/s)
• 1 lane PCIe 3.0 is roughly 1 GB/s

So that means you really need a minimum of two PCIe 3.0 lanes per 10gbps interface, otherwise the max bandwidth of the interface will be artificially limited - not dramatically so, and maybe not enough to make a difference in typical workloads/situations. But a single PCIe 3.0 lane is technically oversubscribed for a 10gbps NIC.

The Qotom Q10900H6 S13 device has four 10gpbs interfaces and two 2.5gbps interfaces. If we want two PCIe 3.0 lanes per 10gbps iface, that's eight total lanes; and then use a ninth lane for the two 2.5gbps interfaces. The N100/N305 CPUs only have nine total PCIe lanes.

But this device also has an M.2 PCIe 3.0 1x interface - so at least one of the network interfaces is getting sub-optimal PCIe bandwidth.

So clearly it's impossible to truly max out all network interfaces simultaneously, there just isn't enough PCIe bandwidth to do that.

Qotom really needs to provide a block diagram for how the PCIe lanes are used.

What kind of routing performance are you looking for? With or without PPPoE?

I dunno. I'm actually doing just fine with my humble N5105 with four 2.5gbps interfaces. I am getting an internet upgrade at some point in the future, to 5gbps. I'm not sure if it's PPPoE or not, but I suspect it's not. I don't really need the added bandwidth, but the service is a good deal. So now it's triggered a case of upgradeitis for my firewall. Having spent too much time looking at devices like this, I'm actually leaning towards just throwing together a "traditional" system with some parts I have on hand.

 

[blunden]

Yeah, I thought about this some more. Check my math on this, but:

• 10gigaBITS/sec = 1.25gigaBYTES/sec (GB/s)
• 1 lane PCIe 3.0 is roughly 1 GB/s

So that means you really need a minimum of two PCIe 3.0 lanes per 10gbps interface, otherwise the max bandwidth of the interface will be artificially limited - not dramatically so, and maybe not enough to make a difference in typical workloads/situations. But a single PCIe 3.0 lane is technically oversubscribed for a 10gbps NIC.

I believe the 10 Gbit/s NICs need 2 PCI-E 3.0 lanes each, yes. That's why I mentioned the need for 8 lanes just for the four 10 Gbit/s NICs.

The Qotom Q10900H6 S13 device has four 10gpbs interfaces and two 2.5gbps interfaces. If we want two PCIe 3.0 lanes per 10gbps iface, that's eight total lanes; and then use a ninth lane for the two 2.5gbps interfaces. The N100/N305 CPUs only have nine total PCIe lanes.

But this device also has an M.2 PCIe 3.0 1x interface - so at least one of the network interfaces is getting sub-optimal PCIe bandwidth.

Ideally they would've used a PCI-E switch so that the number of lanes could be slightly oversubscribed without hurting performance as long as you don't push all lanes at the same time. After all, most people using it as a router would only use two of the 10 Gbit/s interfaces, not all four. The downside is that building it like that increases the cost.

So clearly it's impossible to truly max out all network interfaces simultaneously, there just isn't enough PCIe bandwidth to do that.

Qotom really needs to provide a block diagram for how the PCIe lanes are used.

Not all of them, no.

I would try asking Qotom. They responded to my question about PCI-E lane allocation back when I bought my unit so it's certainly worth a shot.

I dunno. I'm actually doing just fine with my humble N5105 with four 2.5gbps interfaces. I am getting an internet upgrade at some point in the future, to 5gbps. I'm not sure if it's PPPoE or not, but I suspect it's not. I don't really need the added bandwidth, but the service is a good deal. So now it's triggered a case of upgradeitis for my firewall. Having spent too much time looking at devices like this, I'm actually leaning towards just throwing together a "traditional" system with some parts I have on hand.

When using an entirely software based router without any hardware offloading, PPPoE tends to hurt performance a lot for what I understand, at least without something like VPP. That's why it's important to know.