Qotom Denverton fanless system with 4 SFP+

[renewgeorgia]

Has anyone used the Qotom with the ATT fiber SPF+ WAS 110 bypass? I read somewhere that someone had trouble getting it to work with Qotom for some reason. In case that does not make sense, the WAS110 is an SPF+ that you plug your fiber into and then into the Qotom bypassing the ATT crappy box so you have full control of the firewall. I want to buy the Qotom if it works, anyone know if the WAS110 works? Thanks in advance

 

[blunden]

I think the lack of a GPU might make transcoding a weak spot on this unit. Can anyone else confirm my noob opinion?

Agreed. Since we're only talking 1080p it can probably use software decoding and encoding though, but it will almost certainly be less efficient.

can anyone reccomend me a heatsink?

any standard ones fit this?

whats the spacing?

A heatsink for what? Also, are you talking about the fanless model or the rackmount one?

• As of Oct 2024, what's the support like for QAT on pfSense CE? The old Atom-based system I'm replacing ran OpenVPN w/ AES-NI fast enough to me, so I assume this newer CPU will be at least that fast?
• For folks running pfSense on this system, can I expect at least 2Gbps of routing? This assumes no snort, suricata, etc, but definitely a set of firewall rules.

I'm pretty sure QAT remains a pfSense Plus feature, so the answer would be "none". I would assume this would also use AES-NI when QAT isn't available.

OPNsense includes QAT in the free version from what I understand, but OPNsense seems to have fallen behind in other areas. I don't use either of them though so feel free to look into it more deeply.

Anyone still running OpenVPN should also look into OpenVPN DCO to increase performance.

The C3758R model should be basically equivalent to the Netgate 8200 appliance so you can presumably look at those performance numbers (except for the VPN numbers unless using pfSense Plus).

pfSense Plus How to buy

Purchase pfSense Plus software on a Netgate Appliance, on AWS or Azure as a cloud instance , or as a virtual machine

www.netgate.com

The Qotom box has more network connectivity though (2 extra 10G SFP+ ports and 1 extra 2.5 Gbps RJ45).

I just got my 1U Q20331G9 from Amazon delivered, before I order a VGA to HDMI converter, anybody know if the USB-C port can be used for a monitor?

Almost definitely not. I highly doubt the old ASPEED "GPU" supports anything more modern than VGA. If it did, I'm sure they would've put an HDMI or DisplayPort port on it.

This is the adapter I used. Also available for less from AliExpress if you're not in a hurry.

https://www.amazon.se/dp/B0B15FLN9K/

Has anyone used the Qotom with the ATT fiber SPF+ WAS 110 bypass? I read somewhere that someone had trouble getting it to work with Qotom for some reason. In case that does not make sense, the WAS110 is an SPF+ that you plug your fiber into and then into the Qotom bypassing the ATT crappy box so you have full control of the firewall. I want to buy the Qotom if it works, anyone know if the WAS110 works? Thanks in advance

It probably depends on the power consumption of the WAS110. If it's above 2.5W I wouldn't be so sure. We've already seen someone having issues with the older 30 m 10GBASE-T transceivers that was solved by going for the modern more power efficient ones. I actually had an issue getting one of those more power efficient ones to work, but it ended up being the EEPROM data that was the problem in my case. I thankfully had a FlexBox to reprogram it.

You might be able to work around that by putting one of those cheap Realtek 2 * 10G + 4 * 2.5G switches (from HiSource, YuanLey, Davuaz, etc.) in front of it.

 

[adaptivesystems]

A heatsink for what? Also, are you talking about the fanless model or the rackmount one?

you can buy the motherboard without the case, you know....costs about £50 less in shipping. and i wanted a rack model, but those weren't an option, wothout paying *crazy* prices for shipping it.

 

[sko]

• As of Oct 2024, what's the support like for QAT on pfSense CE? The old Atom-based system I'm replacing ran OpenVPN w/ AES-NI fast enough to me, so I assume this newer CPU will be at least that fast?

qat

man.freebsd.org

HISTORY
     The qat driver first appeared in FreeBSD 13.0.

I wouldn't use anything that hides a basic function of the underlying base OS behind a paywall...

• For folks running pfSense on this system, can I expect at least 2Gbps of routing? This assumes no snort, suricata, etc, but definitely a set of firewall rules.

I'm routing part of my local 10G traffic* through the gateway jail and seeing 9.x Gbps numbers. CPU overhead for simple routing/forwarding is negligible, especially if no NAT is involved.

*) the major 'heavy lifting' is done at the switch; anything that doesn't easily translate into "understandable 2 years after I set this up"-PBR rulesets/ACLs is handed over to the gateway.

 

[krby]

I just got my 1U Q20331G9 from Amazon delivered, before I order a VGA to HDMI converter, anybody know if the USB-C port can be used for a monitor?

This page from the STH review says:

There is also an ASPEED AST2400 BMC chip onboard. That is an older chip (we are currently on the AST2600 version) that is used for VGA and console output. The GPU IP in this is ancient, so think of something to allow for a management display, but not much else beyond that.

That makes me think the GPU can only drive the VGA output, nothing on USB-C

 

[cyben76]

Does anyone know if the small M.2 slot for wifi/sim could be used with Coral AI m.2?

 

[bugmen0t]

Does anyone know if the small M.2 slot for wifi/sim could be used with Coral AI m.2?

No, it's not recognized at all (even if I unplug every USB device as suggested elsewhere, i.e. in case PCIe lines are shared).
I also tried m.2 B-key → miniPCIE adapter for a wifi card and it didn't recognize it either.
My Coral m.2 B+M key works fine on regular m.2 M-key port, though.

 

[cyben76]

No, it's not recognized at all (even if I unplug every USB device as suggested elsewhere, i.e. in case PCIe lines are shared).
I also tried m.2 B-key → miniPCIE adapter for a wifi card and it didn't recognize it either.
My Coral m.2 B+M key works fine on regular m.2 M-key port, though.

Thank you so much for the info.

 

[adaptivesystems]

could always buy a minipcie -> usb adapter, and a usb coral tpu if you were out of other options....

 

[cyben76]

could always buy a minipcie -> usb adapter, and a usb coral tpu if you were out of other options....

currently using USB Coral, was hoping to have everything internal to avoid accidental unplugging.

 

[sequoia10]

Has anyone been able to get 'decent' full-duplex throughput on the 10G (SFP+) NICs on these?

If running iperf3 against my test unit (Qotom Q20332G9-S10 w/Atom C3808) speeds are 'ok' when doing normal tests (or reverse with -R or --reverse option). However, when testing "full-duplex" (bi-directional), transmit speeds towards Qotom drop to less than half (compared to when doing "half duplex" testing)....

Bi-directional test (with --bidir option to iperf3):

[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec  4.45 GBytes  3.83 Gbits/sec  1235             sender
[  5][TX-C]   0.00-10.00  sec  4.45 GBytes  3.82 Gbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec  10.8 GBytes  9.27 Gbits/sec    0             sender
[  7][RX-C]   0.00-10.00  sec  10.8 GBytes  9.26 Gbits/sec                  receiver

If I run same test against ancient Lenovo server (with old Mellanox ConnectX NIC), I get expected results (both servers are on same Cisco switch connected with DAC cables):

[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec  10.5 GBytes  9.06 Gbits/sec  281             sender
[  5][TX-C]   0.00-10.00  sec  10.5 GBytes  9.06 Gbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec  10.9 GBytes  9.32 Gbits/sec    0             sender
[  7][RX-C]   0.00-10.00  sec  10.9 GBytes  9.32 Gbits/sec                  receiver

Testing against Qotom unit in 'half-duplex' mode results are ok (but throughput towards Qotom is still noticeably worse):

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  10.1 GBytes  8.72 Gbits/sec  2545             sender
[  5]   0.00-10.00  sec  10.1 GBytes  8.72 Gbits/sec                  receiver

While reverse (-R) is slightly faster:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  10.9 GBytes  9.39 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  10.9 GBytes  9.39 Gbits/sec                  receiver

Test was done on Proxmox with the currently latest kernel:

Linux pve1 6.8.12-2-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.12-2 (2024-09-05T10:03Z) x86_64 GNU/Linux

 

[Brent Geery]

Has anyone used the Qotom with the ATT fiber SPF+ WAS 110 bypass? I read somewhere that someone had trouble getting it to work with Qotom for some reason.

I've got my WAS 110 plugged into my Q20331G9S10 and it seems to boot up and work fine with OPNsense. However, I'm waiting for someone to actually write a guide to getting access to the web interface through the firewall (so I can configure it using existing guides) and then moving the WAN interface from the current port over to the WAS 110. Over my head.

I have a feeling if one had more things hooked up to the SPF+ ports, then one might run into power issues.

 

[krby]

I have a feeling if one had more things hooked up to the SPF+ ports, then one might run into power issues.

This felt like it isn't related to everything else you said, what makes you think this? I'm not challenging you, just trying to figure out what caused this instinct. I'm also concerned about the heat and power on this device before buying it.

 

[Brent Geery]

This felt like it isn't related to everything else you said, what makes you think this? I'm not challenging you, just trying to figure out what caused this instinct. I'm also concerned about the heat and power on this device before buying it.

Purely a hunch. If some are having issues and others not, it's the first thing I'd suspect, lacking further evidence. These are on the more hungry end (~2.3w IIRC).

 

[zebra]

Thanks, that helped me a lot.

What I get in VyOS:

Interface numbering (Looking at port side):

SFP+:

eth0 eth2
eth1 eth3

RJ45:

eth7 eth6 eth8
eth5 eth4

 

[adaptivesystems]

currently using USB Coral, was hoping to have everything internal to avoid accidental unplugging.

tbf that would achieve that....although it also does just have 4 internal usbs, as well....

 

[sequoia10]

Has anyone been able to get 'decent' full-duplex throughput on the 10G (SFP+) NICs on these?

If running iperf3 against my test unit (Qotom Q20332G9-S10 w/Atom C3808) speeds are 'ok' when doing normal tests (or reverse with -R or --reverse option). However, when testing "full-duplex" (bi-directional), transmit speeds towards Qotom drop to less than half (compared to when doing "half duplex" testing)....

Bi-directional test (with --bidir option to iperf3):

[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec  4.45 GBytes  3.83 Gbits/sec  1235             sender
[  5][TX-C]   0.00-10.00  sec  4.45 GBytes  3.82 Gbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec  10.8 GBytes  9.27 Gbits/sec    0             sender
[  7][RX-C]   0.00-10.00  sec  10.8 GBytes  9.26 Gbits/sec                  receiver

If I run same test against ancient Lenovo server (with old Mellanox ConnectX NIC), I get expected results (both servers are on same Cisco switch connected with DAC cables):

[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec  10.5 GBytes  9.06 Gbits/sec  281             sender
[  5][TX-C]   0.00-10.00  sec  10.5 GBytes  9.06 Gbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec  10.9 GBytes  9.32 Gbits/sec    0             sender
[  7][RX-C]   0.00-10.00  sec  10.9 GBytes  9.32 Gbits/sec                  receiver

Testing against Qotom unit in 'half-duplex' mode results are ok (but throughput towards Qotom is still noticeably worse):

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  10.1 GBytes  8.72 Gbits/sec  2545             sender
[  5]   0.00-10.00  sec  10.1 GBytes  8.72 Gbits/sec                  receiver

While reverse (-R) is slightly faster:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  10.9 GBytes  9.39 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  10.9 GBytes  9.39 Gbits/sec                  receiver

Test was done on Proxmox with the currently latest kernel:

Linux pve1 6.8.12-2-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.12-2 (2024-09-05T10:03Z) x86_64 GNU/Linux

Seems like these units can't actually handle full 10Gps full-duplex at wire-speed on the SFP+ ports? (Or is this just some issue with the Linux kernel in Proxmox 8)

Looking lspci output on the X553 NICs, they only get 2.5GT/s link:

                LnkSta: Speed 2.5GT/s, Width x1
                TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

While I checked couple other Linux servers with 10G NICs (where there is no issue with performance), those report 5GT/s:

                LnkSta: Speed 5GT/s, Width x4 (downgraded)
                TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-

Looks like not enough PCIe lanes routed to the X553 chipset on this (Q20332G9-S10) motherboard, for SFP+ NICs to be able to handle 10Gbps transmit and receive simultaneously? Since iperf3 test results are okay until one tests with --bidir option....

 

[blunden]

Looking lspci output on the X553 NICs, they only get 2.5GT/s link:

                LnkSta: Speed 2.5GT/s, Width x1
                TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

While I checked couple other Linux servers with 10G NICs (where there is no issue with performance), those report 5GT/s:

                LnkSta: Speed 5GT/s, Width x4 (downgraded)
                TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-

Looks like not enough PCIe lanes routed to the X553 chipset on this (Q20332G9-S10) motherboard, for SFP+ NICs to be able to handle 10Gbps transmit and receive simultaneously? Since iperf3 test results are okay until one tests with --bidir option....

Are you sure it's not just downclocking for power saving reasons when not heavily used? That's pretty common for PCI-E devices after all.

The X553 NICs are integrated in the C3000 SoCs so I don't think that it's up to the motherboard makers to allocate PCI-E lanes to them. See more details here:

Intel®Atom™ Processor C3000 Product Family Integrated 10 GbE LAN Controller Programmer's Reference Manual (PRM)

This document describes the external architecture for Intel® Ethernet Connection X553, a dual-port 10 GbE LAN controller integrated into the Intel® Atom™ Processor C3000 Product Family.

www.intel.com

What if you try multiple streams?

 

[cyben76]

you can buy the motherboard without the case, you know....costs about £50 less in shipping. and i wanted a rack model, but those weren't an option, wothout paying *crazy* prices for shipping it.

would the mobo alone fit a standard ITX case? Qotom didn't specify the form factor of the mobo, but their AliExpress store is under Nano ITX.

 

[adaptivesystems]

would the mobo alone fit a standard ITX case? Qotom didn't specify the form factor of the mobo, but their AliExpress store is under Nano ITX.

according to the manual:
3.5 inch Mini-ITX
and
165mm x115mm

honestly, this in a standard mini itx size, with a full bmc would be incredible...


it doesn't have standard mounts if thats what you're asking, but with a bit of hot snot, and some screws, it'll all be good....even if it looks terrible inside -