Qotom Denverton fanless system with 4 SFP+

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Ctullu

New Member
Aug 11, 2024
10
2
3
Technical issues aside, is anyone getting anything close to the 18gbps routing/firewall performance of a Netgate 8200?
It sounds like people aren't seeing more than 7gbs?
Unfortunately I don’t have an answer for you on this. I don’t have a setup to run this at 18+ Gbps. I do have the hardware to do it, if I get some time in the coming months maybe I will test this out.


As a side note, I have a 2.5Gbe network across my entire house, with a 10Gbe connection between my router (Proxmox/OPNSense) and my distribution switch, and from there to my office where I am setting up my HomeLab. The idea is to connect the NAS/File server via a 10Gbe connection. I would like to see something like this device with more storage options for running TrueNAS or Unraid as a NAS/Storage server with high speed connectivity.
 

sko

Active Member
Jun 11, 2021
380
234
43
I seem to recall people here claiming that the SFP+ vendor lock was disabled by default on this unit, but that doesn't appear to be the case for me except for DAC cables (which Intel never vendor locks). :confused: It refuses to link with transceivers that don't have Intel's vendor OUI, even if I unload the driver and then reload it again with allow_unsupported_sfp=1,1,1,1.

FlexOptix was nice enough to release a new vendor configuration for me within a few hours so it's all good, but it's still irritating that the lock is still there. My Intel coded Ipolex fiber transceivers also work, but I had hoped such things wouldn't be necessary. Unfortunately there doesn't seem to be any X553 unlocking tools available either.

EDIT: Actually, I got a non-Intel transceiver working now. Perhaps it's just extremely picky about other parts of the EEPROM contents.
I've been using cisco-branded FS.com transcievers in my unit from the beginning without any problems. I also had 2 huawei transceivers in it as well as some avago ones during testing...

Are you talking about copper transceivers? Those are *completely* out of spec for SPF+, so *especially* on a low-power SoC/embedded systems with very high certainty those won't work.


Technical issues aside, is anyone getting anything close to the 18gbps routing/firewall performance of a Netgate 8200?
It sounds like people aren't seeing more than 7gbs?
I'm pushing near line speed (~9.8Gbps) through my unit running FreeBSD 13.3-RELEASE (inter-vlan routing).
Don't use iperf directly on such a unit for any kind benchmarks - this tool has long surpassed its use as it's heavily CPU-bound especially for traffic generation. If at all, use it on 2 distinct hosts to push traffic *through* the system you are testing, not directly on a low-core, low-power unit you are trying to benchmark for routing capabilities.
Also, single-stream high-bandwidth is absolutely trivial to achieve at line speed nowadays (for sub-100Gbit speeds...) - what actually matters for real-world applications is actual pps with several dozen/hundred firewall rules and NAT.
 
Last edited:

blunden

Well-Known Member
Nov 29, 2019
883
293
63
I've been using cisco-branded FS.com transcievers in my unit from the beginning without any problems. I also had 2 huawei transceivers in it as well as some avago ones during testing...

Are you talking about copper transceivers? Those are *completely* out of spec for SPF+, so *especially* on a low-power SoC/embedded systems with very high certainty those won't work.
Like I said, I did eventually find an EEPROM config that worked without the vendor name or OUI being Intel. The spare fiber transceivers I have were already vendor coded to Intel so I didn't have anything else to test it with.

Yes, it was a 10GBASE-T transceiver, but one of those more expensive 100 meter ones with slightly lower power consumption. It worked fine when programmed with some EEPROM configurations where 2 out of 3 happened to have the Intel vendor OUI.

I know they are out of spec, but I don't really have a choice unfortunately. Once it reaches my apartment, everything else will be either DACs or fiber transceivers though. :)
 

rukiri

New Member
Aug 15, 2024
12
5
3
Also, single-stream high-bandwidth is absolutely trivial to achieve at line speed nowadays (for sub-100Gbit speeds...) - what actually matters for real-world applications is actual pps with several dozen/hundred firewall rules and NAT.
Yeah, that's exactly my concern. I'm looking for something that can take a 10gbe internet connection, and split it into 3 subnets, with light firewall and NAT routing. Something like this would let me avoid using vlans and having to buy a separate managed switch. Assuming it's powerful enough to do the routing and switching.

I see from the review that the c3758 cpu is used in appliance routers that can route 18+gbps, but my understanding is switching is a different ballgame?
 

VivienM

New Member
Jul 7, 2024
22
6
3
Toronto, ON
Yeah, that's exactly my concern. I'm looking for something that can take a 10gbe internet connection, and split it into 3 subnets, with light firewall and NAT routing. Something like this would let me avoid using vlans and having to buy a separate managed switch. Assuming it's powerful enough to do the routing and switching.

I see from the review that the c3758 cpu is used in appliance routers that can route 18+gbps, but my understanding is switching is a different ballgame?
But what you're discussing is routing, not switching, isn't it? Three interfaces (plus a WAN one) on three subnets and the Qotom box routes packets between the three subnets...
 

blunden

Well-Known Member
Nov 29, 2019
883
293
63
But what you're discussing is routing, not switching, isn't it? Three interfaces (plus a WAN one) on three subnets and the Qotom box routes packets between the three subnets...
Yes, from what I can tell. :)
 

sko

Active Member
Jun 11, 2021
380
234
43
I'm looking for something that can take a 10gbe internet connection, and split it into 3 subnets, with light firewall and NAT routing. Something like this would let me avoid using vlans and having to buy a separate managed switch. Assuming it's powerful enough to do the routing and switching.
Why would you want to "avoid" VLANs? If any you *want* to separate subnets into different VLANs as well for security reasons...
Also you don't want NAT for your local routing, only for egress traffic.

And in general, using NICs as a switch is a bad idea and a dirty hack at best. Just use a switch for switching.
Additionally, any halfway decent "L2+" switch nowadays can do basic routing, most even with policies - this way you can offload some routing (e.g. from your private VLAN to the NFS server in the DMZ) directly on the switch and hand off everything else to the router.
 
  • Like
Reactions: blunden

r3dux

New Member
Jul 12, 2024
13
13
3
Anyone tried updating AST2400 Advanced PCIe Graphics & Remote Management Processor(IPMI)? (stock firmware is 1.09.00 dated 29/01/19)


aspeedtech provided me no info but contact manufacturer and yook down firmwaree 1.13 for AST2400 (mirror provided)

qotom sent me an email will contact you two months ago, I hope that will contact me back this year

firmware 1.12.0 08/12/21
firmware 1.13.0 07/06/22 that aspeed took down from their side after my conversation with them

 
Last edited:
  • Like
Reactions: Pheckphul

rukiri

New Member
Aug 15, 2024
12
5
3
Why would you want to "avoid" VLANs? If any you *want* to separate subnets into different VLANs as well for security reasons...
Also you don't want NAT for your local routing, only for egress traffic.

And in general, using NICs as a switch is a bad idea and a dirty hack at best. Just use a switch for switching.
Additionally, any halfway decent "L2+" switch nowadays can do basic routing, most even with policies - this way you can offload some routing (e.g. from your private VLAN to the NFS server in the DMZ) directly on the switch and hand off everything else to the router.
Hmm, I may be misunderstanding the terms. I understood routing to refer more to managing the flow of data in and out of my network, like firewall rules and port forwarding. While switching was managing my internal network, like splitting traffic up into different subnets or vlans and actually moving all the data. Is that incorrect?

In any case, is there a difference between setting up different subnets on different ports vs setting up vlans off a single port? Can't you isolate them in either case? If I understand you correctly, you're saying it's better to use vlans and let a dedicated switch do the switching? Presumably because using something like this box for switching would have poor performance?

I think I understand. Though I'm not sure what all the ports are for then. On this box, or say the Netgate 8200 that also has lots of ports, you can't really use them? You'll always want a dedicated switch to handle the switching?
 

VivienM

New Member
Jul 7, 2024
22
6
3
Toronto, ON
Hmm, I may be misunderstanding the terms. I understood routing to refer more to managing the flow of data in and out of my network, like firewall rules and port forwarding. While switching was managing my internal network, like splitting traffic up into different subnets or vlans and actually moving all the data. Is that incorrect?

In any case, is there a difference between setting up different subnets on different ports vs setting up vlans off a single port? Can't you isolate them in either case? If I understand you correctly, you're saying it's better to use vlans and let a dedicated switch do the switching? Presumably because using something like this box for switching would have poor performance?

I think I understand. Though I'm not sure what all the ports are for then. On this box, or say the Netgate 8200 that also has lots of ports, you can't really use them? You'll always want a dedicated switch to handle the switching?
The traditional definition is that routing is layer 3, i.e. making decisions based on IP addresses, and switching is layer 2, i.e. making decisions based on MAC addresses and Ethernet stuff.

Part of what's created more confusion is that consumer "routers" also perform NAT functionality. Which is not technically routing - it's an additional thing... but somehow in consumerland the NAT part of the router seems to be what is seen as 'routing'

If you have multiple subnets, traffic needs to be routed between them. If your computer is 192.168.54.12 and it wants to send traffic to 192.168.88.5, it knows that i) it can talk to 192.168.54.* because it's directly connected to 192.168.54.*, but ii) it has no idea how to get traffic to any other subnet. So, if you want to send traffic to any other subnet, your computer will send it to the "default gateway" that you've either set manually or received via DHCP - this is an IP on your subnet that is presumed to contain a 'router' that is able to take packets to other subnets and send them somewhere useful.
(Nitpicker's corner: I am assuming a /24 "class C" 255.255.255.0 network mask)

Not sure you are correctly understanding VLANs, either - VLANs are a way to have multiple logically-separate layer 2 networks on the same infrastructure. So, for example, you could do the following:
- 192.168.10.* - marketing department
- 192.168.11.* - finance department
Have one switch for the marketing department, one switch for the finance department, run a cable from the marketing switch to the main router (with an interface 192.168.10.1), run a cable from the finance switch to the main router (to an interface 192.168.11.1).

And if someone in marketing wants to access a file server on the finance network, then they will be sending layer 3 packets to 192.168.11.5, say, their TCP/IP stack knows that that's not on their network, it will send those packets to the default gateway 192.168.10.1, and it is 192.168.10.1's job to get those packets to 192.168.11.*, i.e. by sending the packets out the interface that is connected to 192.168.11.*.

Or you can do the same thing with one switch and VLANs. If marketing is ports 1-10 and finance is port 11-20, you can set ports 1-10 to VLAN 10, ports 11-20 to VLAN 11, and have exactly the same thing with one switch instead of two.

Now, because switches are operating at level 2 and those VLANs are separate layer 2 (and 3) networks, those VLANs can't talk to each other unless i) you run a cable from a port on each VLAN to a router, ii) you run a single cable from the switch to the router and 'tag' both VLANs, or iii) you have a fancy switch with layer 3 functionality (in which case the router is essentially built-in to the switch).

A further note on VLAN tagging - this is where the switch sends traffic for multiple VLANs on the same port and 'tags' the frames to indicate what VLAN they're for. Then the device plugged into the switch basically has a separate logical 'subinterface' for each VLAN. So, to pick my example above, instead of having one port on the router for 192.168.10.1 and one port for 192.168.11.1, you could have one interface with tagged VLANs and VLAN 10 is 192.168.10.1 and VLAN 11 is 192.168.11.1.

VLANs are handy for home networks because, if you want to put different things on different subnets, they make wiring much easier. So for example, if you wanted to have some devices in your family room on two different subnets, without VLANs, you have two switches in the family room, two cables from the family room to wherever your network core is, etc. With VLANs, just put one switch in the family room, set each port to the proper VLAN based on what you want for the device plugged into it, and run one uplink with tagged VLANs back to your network core.

As an aside, for a home network, I would say that multiple subnets is... on the very sophisticated end of the spectrum.

In terms of why boxes like these Qotoms have many interfaces, there are lots of things you can do:
- the basic one-port-per-subnet interface in my first example - you need a port on the router for each subnet
- link aggregation - two or more ports to the same switch, possibly with tagged VLANs
- exotic WAN setups, e.g. having two ports going to two different ISPs, one port going to a private circuit somewhere, etc.

What you probably do NOT want to do is use two of those ports as a bridge, i.e. have two halves of one subnet plugged into separate interfaces, then have the Qotom box send frames from one port to the other. Switches are much better at doing these kinds of things than x86 boxes running a full operating system.

And I would note that the 9 interfaces on the Qotom isn't actually that many in the grand scheme of things. Boxes from people like SonicWall have a lot more interfaces...
 
  • Like
Reactions: hotgeek and blunden

rukiri

New Member
Aug 15, 2024
12
5
3
Thanks for the detailed response! That helps a lot. As you can tell, I'm just a guy w/a home network so I have a pretty layman / consumer understanding of all this stuff. The detailed description of how traffic flows was very helpful.

I think what you're saying about VLANs matches what I thought, but then I don't understand what is wrong -if anything is wrong?- about using separate interfaces for each subnet. As I understand it, the main benefit of VLANs compared to physically separate subnets is that you get to have different subnets on the same infrastructure. I don't have that need, the different subnets go to physically different areas. Someone was saying it was bad to "avoid" VLANs and to keep the subnets seperate with VLANs for security reasons, but can't you do that just as well w/subnets on different interfaces?


Currently I'm using a $50 Edgerouter X to handle a 1gbe internet connection. It's worked pretty well over the years, especially considering the price. It lets me setup different subnets on each port, though I do bridge two of the ports for one of the subnets. The setup seems to do what I want, but if I'm doing something horribly wrong I'd certainly want to know.

I'm looking for something to fulfill that same role but for a 10gbe internet connection. Most of the options around ~3-400, like a Ubiquity Dream Machine or TP-Link ER8411, have a 10g SFP+ uplink and maybe one more for downlink, then just 1gbe ethernet for the rest. I suppose I could buy one of those and then a managed switch with 10g SFP+ and 2.5g ethernet ports. However, if something like this Qotom can do what I want, then that saves me a switch?

Conversely, if it's best practice to use a separate managed switch anyway, then I'd probably just buy a Dream Machine for peace of mind.
 

VivienM

New Member
Jul 7, 2024
22
6
3
Toronto, ON
Thanks for the detailed response! That helps a lot. As you can tell, I'm just a guy w/a home network so I have a pretty layman / consumer understanding of all this stuff. The detailed description of how traffic flows was very helpful.

I think what you're saying about VLANs matches what I thought, but then I don't understand what is wrong -if anything is wrong?- about using separate interfaces for each subnet. As I understand it, the main benefit of VLANs compared to physically separate subnets is that you get to have different subnets on the same infrastructure. I don't have that need, the different subnets go to physically different areas. Someone was saying it was bad to "avoid" VLANs and to keep the subnets seperate with VLANs for security reasons, but can't you do that just as well w/subnets on different interfaces?


Currently I'm using a $50 Edgerouter X to handle a 1gbe internet connection. It's worked pretty well over the years, especially considering the price. It lets me setup different subnets on each port, though I do bridge two of the ports for one of the subnets. The setup seems to do what I want, but if I'm doing something horribly wrong I'd certainly want to know.

I'm looking for something to fulfill that same role but for a 10gbe internet connection. Most of the options around ~3-400, like a Ubiquity Dream Machine or TP-Link ER8411, have a 10g SFP+ uplink and maybe one more for downlink, then just 1gbe ethernet for the rest. I suppose I could buy one of those and then a managed switch with 10g SFP+ and 2.5g ethernet ports. However, if something like this Qotom can do what I want, then that saves me a switch?

Conversely, if it's best practice to use a separate managed switch anyway, then I'd probably just buy a Dream Machine for peace of mind.
I don't think there's anything 'wrong' per se with being a bit old-fashioned - subnet 1 goes to port 1 and is on switch 1, subnet 2 goes to port 2 and is on switch 2, etc. But there's also no reason not to add some VLANs in the picture - configure VLANs on both switch 1 and 2, some VLAN tagging, etc, and that way you gain some flexibility.

What's not clear to me is whether you actually save a switch or not. I think that depends in part on where your equipment is. If you have one subnet in the basement and one subnet on the second floor, and one cable coming from each switch, then sure, you can take something like the Qotom box, plug each cable into a different interface, and boom, you might avoid the need for a managed switch next to the Qotom.

The other point I would raise is this - how many of your subnets need >1 GbE speeds? Something like the dream machine with its 1 gig ports might be just fine for lower-traffic subnets, then you just have one subnet with a 10G switch that has all the bandwidth hungry devices. 10G networking is expensive, it's not something you really need on more than a couple machines that can actually take advantage of it.
 

synthrax

New Member
Aug 23, 2024
1
0
1
Could anyone describe how bottom fan works/behaves in Q20331G9 1U system?

When machine is powered on does fan start immediately and runs with constant speed that doesn't depend on system temperature? Or maybe it behaves differently?
When the machine is powered on, the bottom fan starts immediately and runs consistently at a noticeable speed, regardless of the system temperature. In my experience, the fan noise is quite noticeable. My unit initially came with a defective fan. However, after reporting the issue to Qotom, they promptly sent me two replacement fans, both of which are functioning correctly.
 

sadness

New Member
Jul 9, 2024
15
5
3
I bought one of these m.2 sata to sff 8087 adapters from ali https://www.aliexpress.com/item/1005006026982407.html and I used a drill with a bit of manual filing to widen the hole for the wifi antenna on the DC power side and now I can have 9 sata disks plugged into this box.

Theoretically, you could get two of these more expensive m.2 sata to dual sff 8087 https://www.aliexpress.com/item/1005006164051397.html and populate the two m.2 slots with them and have 20 sata disks, 22 if you count the onboard sata connectors, kind of ridiculous but still possible.

If you wonder how I power them, https://www.aliexpress.com/item/1005007124951752.html? with a 12V adapter and a dc jack splitter, since these are ssds, all 8 barely use 10w at idle.
 

Attachments

Last edited:
  • Like
Reactions: hotgeek and Antioch

fefifochizzle

New Member
Aug 30, 2024
1
0
1
I have one of these with an Atom C3758R, and put 2 SFP+ 10G adapters in there and everything slows to a crawl. The 2.5 GB ports seems to work fine without any issues. I've tried with OPNsense, VyOS and OpenWRT, and facing the same issues. Any thoughts?
How did you get OpenWRT to work? I tried using an M.2 SSD and just dd-ing the image. It boots fine but for some reason the VGA output suddenly drops and won't come back on. I went to configure the network interfaces for WAN and LAN and once I exited vim it lost VGA output. I reimaged and it worked fine, but I can't get past the initial configuration issue. I've never done an x86 OWRT install, does it auto-detect WAN and LAN on first boot?
 

blunden

Well-Known Member
Nov 29, 2019
883
293
63
So the network speed comes to a crawl. I see excellent upload speeds but horrible download speeds (around 100 Mbps) This is with a 5 Gbps google fiber connection. But upload speeds seem to be good and accurate. I tried with OPNsense, VyOS and having the same issue.
Did you ever solve your issue? :)

I finally deployed mine and I can get 9.1 - 9.4 Gbps in each direction on speedtest.net on a device behind it when the Qotom acts as a VyOS router, and that's with NAT. That's with one Flexoptix 10GBASE-T 100 m transceiver on WAN and then a DAC connected to my Hasivo switch, which in turn uses fiber to connect to my testing device.

I'm still leaning towards some transceiver incompatibility. Either that or your device is somehow defective, but the former is probably more likely. Also make sure to enable Flow Control as your transceiver might require that if it's the kind that does speed limiting internally. :)
 
  • Like
Reactions: farmerj

straylit

New Member
Feb 4, 2019
3
1
1
I see the C3758R variant is now unavailable and has been seemingly superceded by a new C3808 "Q20342G9" model which is absent from the website but in-stock on AliExpress. Can anyone weigh in on the differences? It seems to be slower and smaller cache but with more cores, couldn't find much info on it.
 

blunden

Well-Known Member
Nov 29, 2019
883
293
63
I see the C3758R variant is now unavailable and has been seemingly superceded by a new C3808 "Q20342G9" model which is absent from the website but in-stock on AliExpress. Can anyone weigh in on the differences? It seems to be slower and smaller cache but with more cores, couldn't find much info on it.
The models they have seem to vary based on when they decide to make a new production run, and that's also what they told me when I asked about the C3758R variant last year that was out of stock everywhere at the time. I therefore wouldn't be surprised if it turns up again later.

Unfortunately, I don't know how well networking workloads scale across all those cores, so I can't say how well the additional cores compensate for the lower clock speed (and cache). I would also be interested to know though. :)
 

Antioch

New Member
Sep 4, 2024
10
3
3
Anyone tried updating AST2400 Advanced PCIe Graphics & Remote Management Processor(IPMI)? (stock firmware is 1.09.00 dated 29/01/19)
As far as I can tell, these computers don't contain an IPMI or Aspeed device in them?
 

Antioch

New Member
Sep 4, 2024
10
3
3
Has anyone put together a power tuning guide for this mini server? I have picked one up and it runs considerably hot to the touch sitting at idle, and it seems stuck at C2 state. I was hoping to be able to find settings that the community has used to get things tuned better (as I've seen for n100 systems, for example).

Cheers!