Qotom Denverton fanless system with 4 SFP+

Pheckphul · Feb 27, 2024

scampower said:
Hey all, did anyone managed to get the serial console port working with Opnsense? Is there a setting I need to enable it? I have enabled serial console in Opnsense but I ain't getting a output from the serial console port.

Not the answer you are looking for, but it is a data point.

I got console redirection to work by enabling it in the BIOS and connecting a standard DTE RJ45 to DB9 cable using SecureCRT on Windows with terminal emulation set to "OEM-US" to get the DEC Special Graphics/VT100 line drawing characters to display the BIOS menus correctly, and the serial port set to 115,200 8N1, and everything was copacetic.

zimrigeek · Mar 1, 2024

I've observed everyone discussing the Qotom Q20332G9 s10 on the forums, and I also purchased one via Amazon. After approximately 10 days, I received my shipping notification and eventually my unit.

Several main points have been raised, including temperature, auto-boot jumpers, and interfaces. After a few days of use, I have some experiences to share:

1. Temperature: If you live in an area where the average temperature exceeds 25 degrees Celsius, you might want to consider adding a fan to the unit to prevent overheating, which could result in frequency reduction and performance impact.

2. Jump-start: There has been limited sharing on this topic, and I was unable to locate the exact jumper position. However, in the BIOS North Bridge chipset settings, you can set the power status to either "Last State" or "Auto Start".

3. Interfaces: The device comes with various ports. If you have a console wire, you can even get quasi-IPMI capabilities in a pinch. Just a single RJ45 console wire is needed, no need for a monitor or keyboard for system setup and BIOS configuration.

Moving on to my main point and questions, it seems like the bus bandwidth of this device is unable to accommodate the combined speed of four 10G SFP+ interfaces running simultaneously:
I performed some simple iperf loopback performance tests (127.0.0.1) to evaluate whether the C3758R can run at the theoretical limit speed of 10G SFP+ *4 simultaneously.

Code:

W1290 iperf 127.0.0.1 
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  6]   0.00-1.00   sec  1.79 GBytes  15.4 Gbits/sec                  
[  8]   0.00-1.00   sec  1.79 GBytes  15.4 Gbits/sec                  
[ 10]   0.00-1.00   sec  1.79 GBytes  15.4 Gbits/sec                  
[ 12]   0.00-1.00   sec  1.79 GBytes  15.4 Gbits/sec                  
[SUM]   0.00-1.00   sec  7.16 GBytes  61.5 Gbits/sec  

i3-12100 iperf 127.0.0.1
[ ID] Interval           Transfer     Bitrate
[  6]   0.00-1.00   sec   3.43 GBytes  27.2 Gbits/sec                  
[  8]   0.00-1.00   sec   3.43 GBytes  27.2 Gbits/sec                  
[ 10]   0.00-1.00   sec   3.43 GBytes  27.2 Gbits/sec                  
[ 12]   0.00-1.00   sec   3.43 GBytes  27.2 Gbits/sec                  
[SUM]   0.00-1.00   sec  13.7 GBytes  109 Gbits/sec 

N5105 iperf 127.0.0.1
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  6]   0.00-1.00   sec   694 MBytes  5.81 Gbits/sec    0   1.12 MBytes       
[  8]   0.00-1.00   sec   694 MBytes  5.81 Gbits/sec    0   1.12 MBytes       
[ 10]   0.00-1.00   sec   694 MBytes  5.81 Gbits/sec    0   1.12 MBytes       
[ 12]   0.00-1.00   sec   694 MBytes  5.81 Gbits/sec    0   1.12 MBytes       
[SUM]   0.00-1.00   sec  2.71 GBytes  23.3 Gbits/sec    0 

ATOM  C3758R iperf 127.0.0.1
[ ID] Interval           Transfer     Bitrate
[  6]   0.00-1.00   sec   526 MBytes  4.41 Gbits/sec                  
[  8]   0.00-1.00   sec   526 MBytes  4.41 Gbits/sec                  
[ 10]   0.00-1.00   sec   526 MBytes  4.41 Gbits/sec                  
[ 12]   0.00-1.00   sec   525 MBytes  4.40 Gbits/sec                  
[SUM]   0.00-1.00   sec  2.05 GBytes  17.6 Gbits/sec

I know my tests were too simplistic, but the results have given me some insights. In computer networking, the overall data transfer rate is usually the total speed that all ports can reach under full load.
Take the ATOM C3758R, for example. Despite having four G SFP+ ports, if all four ports are running at full speed simultaneously, their total data transfer rate would only reach 17.6 Gbits/sec. As such, it seems that it can only operate: one 10G SFP+ port + three 2.5G RJ45 ports simultaneously.
I would like to hear from anyone who has carried out practical tests and is willing to share their findings.
Coming up, I will purchase a Mellanox MCX4121A-ACAT to add to my PC, and use a DAC copper cable to interconnect two devices. Some people have noted issues with DAC copper cables. Once it arrives, I'll ascertain for myself. I hope it supports and doesn't give me any trouble.

zimrigeek · Mar 1, 2024

scampower said:
Hey all, did anyone managed to get the serial console port working with Opnsense? Is there a setting I need to enable it? I have enabled serial console in Opnsense but I ain't getting a output from the serial console port.

You might need to explore some settings in your BIOS. When I was using Linx, it took me numerous attempts.
These attempts involved adjustments to:
Baud rate
Flow control
Resolution
Encoding type

Of course, most importantly, you should pay attention to the settings for the second serial port - some systems may be dependent on these settings.

Pheckphul · Mar 3, 2024

zimrigeek said:
<SNIP>

Coming up, I will purchase a Mellanox MCX4121A-ACAT to add to my PC, and use a DAC copper cable to interconnect two devices. Some people have noted issues with DAC copper cables. Once it arrives, I'll ascertain for myself. I hope it supports and doesn't give me any trouble.

I have had zero problems using DAC cables. I've used some no-name generic, a 10Gtek, and a Cisco without a single hiccup going to a Dell PowerConnect 8024.

As for the x553 performance, the Intel Denverton C3xxx series processors are specifically aimed at network-intensive tasks, thus the inclusion of the x553 on the SoC. I would hope that we could trust Intel not to market a device as having 10x its actual performance, and for vendors to promulgate the lie so widely. I would double-check your methodology and setup. Perhaps look for testing on the web, such as that pointed to by @blunden, to see their results.

blunden · Mar 3, 2024

Pheckphul said:
I have had zero problems using DAC cables. I've used some no-name generic, a 10Gtek, and a Cisco without a single hiccup going to a Dell PowerConnect 8024.

Same here. The only time I've had issues it has seemingly depended entirely on the device at the other end, which also had issues with DACs connected to my main computer's unlocked Intel X710-DA2 NIC.

zimrigeek · Mar 5, 2024

I appreciate the response from the previous contributor regarding DAC copper cables. My equipment and accessories have finally arrived, and I have been eager to run some tests.

Unfortunately, the results were a bit disappointing. I conducted limited tests using the W1290 platform with Mellanox MCX4121A-ACAT + 10G DAC copper cable, mainly using iperf3 for throughput tests.
The main test directions are as following:

Loopback test of the W1290 platform when Mellanox MCX4121A-ACAT is connected.
Direct test of the W1290 platform when two Mellanox MCX4121A-ACAT are connected.
Testing with ATOM C3758R as the client connecting to the W1290 platform via Mellanox MCX4121A-ACAT.
Testing with W1290 platform as a client connecting to ATOM C3758R via Mellanox MCX4121A-ACAT.

Test conclusion:
It seems that even one 10G SFP+ interface on the ATOM C3758R can't be fully loaded, with a top speed of only 8.51 Gbits/sec, which is quite disappointing.

The following are some of my test records:

Code:

Mellanox MCX4121A-ACAT  W1290 CPU Loopback

root@iperf3:~# iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[  5] local 127.0.0.1 port 32860 connected to 127.0.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  7.48 GBytes  64.3 Gbits/sec    0   2.19 MBytes       
[  5]   1.00-2.00   sec  7.39 GBytes  63.5 Gbits/sec    0   2.19 MBytes       
[  5]   2.00-3.00   sec  7.50 GBytes  64.4 Gbits/sec    0   2.19 MBytes       
[  5]   3.00-4.00   sec  7.67 GBytes  65.9 Gbits/sec    0   2.19 MBytes       
[  5]   4.00-5.00   sec  7.76 GBytes  66.6 Gbits/sec    0   2.19 MBytes       
[  5]   5.00-6.00   sec  7.75 GBytes  66.6 Gbits/sec    0   2.19 MBytes       
[  5]   6.00-7.00   sec  7.76 GBytes  66.7 Gbits/sec    0   2.19 MBytes       
[  5]   7.00-8.00   sec  7.72 GBytes  66.4 Gbits/sec    0   2.19 MBytes       
[  5]   8.00-9.00   sec  7.75 GBytes  66.6 Gbits/sec    0   2.19 MBytes       
[  5]   9.00-10.00  sec  7.75 GBytes  66.6 Gbits/sec    0   2.19 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  76.5 GBytes  65.7 Gbits/sec    0             sender
[  5]   0.00-10.04  sec  76.5 GBytes  65.5 Gbits/sec                  receiver

iperf Done.



Mellanox MCX4121A-ACAT Port1(Ubuntu22.04)192.168.20.125 ----> Mellanox MCX4121A-ACAT Port2(Ubuntu22.04) 192.168.20.126
Connecting to host 192.168.20.125, port 5201
[  5] local 192.168.20.126 port 37976 connected to 192.168.20.125 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  1.25 GBytes  10.0 Gbits/sec    0   1.11 MBytes       
[  5]   1.00-2.00   sec  1.26 GBytes  10.1 Gbits/sec    0   1.12 MBytes       
[  5]   2.00-3.00   sec  1.25 GBytes  10.0 Gbits/sec    0   1.11 MBytes     
[  5]   3.00-4.00   sec  1.25 GBytes  10.0 Gbits/sec    0   1.11 MBytes       
[  5]   4.00-5.00   sec  1.25 GBytes  10.0 Gbits/sec    0   1.11 MBytes       
[  5]   5.00-6.00   sec  1.25 GBytes  10.0 Gbits/sec    0   1.11 MBytes       
[  5]   6.00-7.00   sec  1.25 GBytes  10.0 Gbits/sec    0   1.11 MBytes       
[  5]   7.00-8.00   sec  1.26 GBytes  10.1 Gbits/sec    0   1.12 MBytes       
[  5]   8.00-9.00   sec  1.26 GBytes  10.1 Gbits/sec    0   1.12 MBytes       
[  5]   9.00-10.00  sec  1.25 GBytes  10.0 Gbits/sec    0   1.11 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  10.15 GBytes  10.0 Gbits/sec    0             sender
[  5]   0.00-10.04  sec  10.15 GBytes  10.0 Gbits/sec                  receiver

iperf Done.





############ ATOM  C3758R as Client ##############
ATOM  C3758R ----> Mellanox MCX4121A-ACAT(Ubuntu22.04)

Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.20.1, port 37282
[  5] local 192.168.20.125 port 5201 connected to 192.168.20.1 port 37294
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   971 MBytes  8.14 Gbits/sec  2910    711 KBytes       
[  5]   1.00-2.00   sec  1022 MBytes  8.58 Gbits/sec  1174   1.05 MBytes       
[  5]   2.00-3.00   sec  1010 MBytes  8.47 Gbits/sec  1357    908 KBytes       
[  5]   3.00-4.00   sec  1.01 GBytes  8.67 Gbits/sec  174   1005 KBytes       
[  5]   4.00-5.00   sec   996 MBytes  8.36 Gbits/sec  904    928 KBytes       
[  5]   5.00-6.00   sec   952 MBytes  7.99 Gbits/sec  610    802 KBytes       
[  5]   6.00-7.00   sec   914 MBytes  7.67 Gbits/sec  717    874 KBytes       
[  5]   7.00-8.00   sec   991 MBytes  8.32 Gbits/sec  683    918 KBytes       
[  5]   8.00-9.00   sec   988 MBytes  8.28 Gbits/sec  362    957 KBytes       
[  5]   9.00-10.00  sec   994 MBytes  8.34 Gbits/sec  507   1.04 MBytes       
[  5]  10.00-10.04  sec  43.8 MBytes  8.58 Gbits/sec    0   1.06 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.04  sec  9.68 GBytes  8.28 Gbits/sec  9398             sender
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------


############ ATOM  C3758R  as Server ############
Mellanox MCX4121A-ACAT(Ubuntu22.04) ----> ATOM  C3758R

root@iperf3:~# iperf3 -c 192.168.20.1
Connecting to host 192.168.20.1, port 5201
[  5] local 192.168.20.125 port 58598 connected to 192.168.20.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   976 MBytes  8.18 Gbits/sec  3825    816 KBytes       
[  5]   1.00-2.00   sec   914 MBytes  7.66 Gbits/sec  644    830 KBytes       
[  5]   2.00-3.00   sec   948 MBytes  7.95 Gbits/sec  1462    912 KBytes       
[  5]   3.00-4.00   sec   992 MBytes  8.33 Gbits/sec  857    913 KBytes       
[  5]   4.00-5.00   sec  1010 MBytes  8.47 Gbits/sec  308   1.15 MBytes       
[  5]   5.00-6.00   sec  1002 MBytes  8.41 Gbits/sec  227   1.31 MBytes       
[  5]   6.00-7.00   sec  1015 MBytes  8.51 Gbits/sec  232   1.09 MBytes       
[  5]   7.00-8.00   sec   954 MBytes  8.00 Gbits/sec   58    387 KBytes       
[  5]   8.00-9.00   sec   945 MBytes  7.93 Gbits/sec    1    489 KBytes       
[  5]   9.00-10.00  sec   931 MBytes  7.81 Gbits/sec  601    895 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  9.46 GBytes  8.13 Gbits/sec  8215             sender
[  5]   0.00-10.04  sec  9.46 GBytes  8.09 Gbits/sec                  receiver

iperf Done.

blunden · Mar 5, 2024

zimrigeek said:
I appreciate the response from the previous contributor regarding DAC copper cables. My equipment and accessories have finally arrived, and I have been eager to run some tests.

Unfortunately, the results were a bit disappointing. I conducted limited tests using the W1290 platform with Mellanox MCX4121A-ACAT + 10G DAC copper cable, mainly using iperf3 for throughput tests.
The main test directions are as following:

Loopback test of the W1290 platform when Mellanox MCX4121A-ACAT is connected.

Direct test of the W1290 platform when two Mellanox MCX4121A-ACAT are connected.

Testing with ATOM C3758R as the client connecting to the W1290 platform via Mellanox MCX4121A-ACAT.

Testing with W1290 platform as a client connecting to ATOM C3758R via Mellanox MCX4121A-ACAT.

Test conclusion:
It seems that even one 10G SFP+ interface on the ATOM C3758R can't be fully loaded, with a top speed of only 8.51 Gbits/sec, which is quite disappointing.

First of all, if you have a Q20332G9 you don't have a C3758R, you have a C3758. The Q20331G9 is the C3758R. It shouldn't make more than a few percent difference, but it's always better to be correct.

Second of all iperf3 (unlike iperf2) is single threaded so you should try with 2 or more threads (-P2, etc.).

Third, your results don't match what I've seen elsewhere so I'd say you might have to take a another look at your testing methodology or configuration.

Some official benchmarks from Netgate (Netgate 8200):

pfSense Plus How to buy

Purchase pfSense Plus software on a Netgate Appliance, on AWS or Azure as a cloud instance , or as a virtual machine

www.netgate.com

zimrigeek · Mar 5, 2024

blunden said:
First of all, if you have a Q20332G9 you don't have a C3758R, you have a C3758. The Q20331G9 is the C3758R. It shouldn't make more than a few percent difference, but it's always better to be correct.

Second of all iperf3 (unlike iperf2) is single threaded so you should try with 2 or more threads (-P2, etc.).

Third, your results don't match what I've seen elsewhere so I'd say you might have to take a another look at your testing methodology or configuration.

Some official benchmarks from Netgate (Netgate 8200):

pfSense Plus How to buy

Purchase pfSense Plus software on a Netgate Appliance, on AWS or Azure as a cloud instance , or as a virtual machine

www.netgate.com

1. I've taken a close look at my specific device model: Q20331G9 S10 C3758R

2. Although I'm using iperf3 as a non-authoritative test, regarding threads, I've actually done many tests and found that the results are simply the sum of a single thread. If you have better suggestions, I'm very willing to test according to your suggestions.

3. If you have time, I would like to ask you to provide results suggested elsewhere for my reference.
Thank you very much!

Code:

root@Ubuntu22:~# iperf3 -c 192.168.20.1 -P 4
Connecting to host 192.168.20.1, port 5201
[  5] local 192.168.20.125 port 36686 connected to 192.168.20.1 port 5201
[  7] local 192.168.20.125 port 36700 connected to 192.168.20.1 port 5201
[  9] local 192.168.20.125 port 36714 connected to 192.168.20.1 port 5201
[ 11] local 192.168.20.125 port 36716 connected to 192.168.20.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   223 MBytes  1.87 Gbits/sec   54    430 KBytes       
[  7]   0.00-1.00   sec   223 MBytes  1.87 Gbits/sec  192    368 KBytes       
[  9]   0.00-1.00   sec   223 MBytes  1.87 Gbits/sec   45    580 KBytes       
[ 11]   0.00-1.00   sec   225 MBytes  1.88 Gbits/sec   14    489 KBytes       
[SUM]   0.00-1.00   sec   893 MBytes  7.49 Gbits/sec  305             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.00-2.00   sec   212 MBytes  1.78 Gbits/sec    0    430 KBytes       
[  7]   1.00-2.00   sec   212 MBytes  1.78 Gbits/sec    0    379 KBytes       
[  9]   1.00-2.00   sec   212 MBytes  1.78 Gbits/sec    0    580 KBytes       
[ 11]   1.00-2.00   sec   212 MBytes  1.78 Gbits/sec    0    489 KBytes       
[SUM]   1.00-2.00   sec   848 MBytes  7.11 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   2.00-3.00   sec   218 MBytes  1.83 Gbits/sec    0    430 KBytes       
[  7]   2.00-3.00   sec   217 MBytes  1.82 Gbits/sec    0    379 KBytes       
[  9]   2.00-3.00   sec   217 MBytes  1.82 Gbits/sec    0    580 KBytes       
[ 11]   2.00-3.00   sec   218 MBytes  1.83 Gbits/sec    0    489 KBytes       
[SUM]   2.00-3.00   sec   870 MBytes  7.30 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   3.00-4.00   sec   210 MBytes  1.77 Gbits/sec    0    430 KBytes       
[  7]   3.00-4.00   sec   212 MBytes  1.77 Gbits/sec    0    379 KBytes       
[  9]   3.00-4.00   sec   211 MBytes  1.77 Gbits/sec    0    581 KBytes       
[ 11]   3.00-4.00   sec   211 MBytes  1.77 Gbits/sec    0    492 KBytes       
[SUM]   3.00-4.00   sec   844 MBytes  7.08 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   4.00-5.00   sec   212 MBytes  1.78 Gbits/sec    0    430 KBytes       
[  7]   4.00-5.00   sec   211 MBytes  1.77 Gbits/sec    0    385 KBytes       
[  9]   4.00-5.00   sec   212 MBytes  1.78 Gbits/sec    0    581 KBytes       
[ 11]   4.00-5.00   sec   211 MBytes  1.77 Gbits/sec    0    492 KBytes       
[SUM]   4.00-5.00   sec   845 MBytes  7.09 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   5.00-6.00   sec   211 MBytes  1.77 Gbits/sec    0    430 KBytes       
[  7]   5.00-6.00   sec   212 MBytes  1.78 Gbits/sec    0    385 KBytes       
[  9]   5.00-6.00   sec   211 MBytes  1.77 Gbits/sec    0    585 KBytes       
[ 11]   5.00-6.00   sec   211 MBytes  1.77 Gbits/sec    0    495 KBytes       
[SUM]   5.00-6.00   sec   845 MBytes  7.09 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   6.00-7.00   sec   211 MBytes  1.77 Gbits/sec    0    430 KBytes       
[  7]   6.00-7.00   sec   211 MBytes  1.77 Gbits/sec    0    396 KBytes       
[  9]   6.00-7.00   sec   211 MBytes  1.77 Gbits/sec    0    585 KBytes       
[ 11]   6.00-7.00   sec   211 MBytes  1.77 Gbits/sec    0    499 KBytes       
[SUM]   6.00-7.00   sec   844 MBytes  7.08 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   7.00-8.00   sec   211 MBytes  1.77 Gbits/sec    0    430 KBytes       
[  7]   7.00-8.00   sec   210 MBytes  1.77 Gbits/sec    0    396 KBytes       
[  9]   7.00-8.00   sec   210 MBytes  1.76 Gbits/sec    0    585 KBytes       
[ 11]   7.00-8.00   sec   211 MBytes  1.77 Gbits/sec    0    499 KBytes       
[SUM]   7.00-8.00   sec   841 MBytes  7.06 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   8.00-9.00   sec   214 MBytes  1.79 Gbits/sec    0    430 KBytes       
[  7]   8.00-9.00   sec   214 MBytes  1.79 Gbits/sec    0    396 KBytes       
[  9]   8.00-9.00   sec   214 MBytes  1.80 Gbits/sec    0    587 KBytes       
[ 11]   8.00-9.00   sec   213 MBytes  1.79 Gbits/sec    0    499 KBytes       
[SUM]   8.00-9.00   sec   855 MBytes  7.17 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   9.00-10.00  sec   211 MBytes  1.77 Gbits/sec    0    430 KBytes       
[  7]   9.00-10.00  sec   210 MBytes  1.76 Gbits/sec    0    399 KBytes       
[  9]   9.00-10.00  sec   210 MBytes  1.76 Gbits/sec    0    587 KBytes       
[ 11]   9.00-10.00  sec   211 MBytes  1.77 Gbits/sec    0    502 KBytes       
[SUM]   9.00-10.00  sec   842 MBytes  7.07 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.08 GBytes  1.79 Gbits/sec   54             sender
[  5]   0.00-10.04  sec  2.08 GBytes  1.78 Gbits/sec                  receiver
[  7]   0.00-10.00  sec  2.08 GBytes  1.79 Gbits/sec  192             sender
[  7]   0.00-10.04  sec  2.08 GBytes  1.78 Gbits/sec                  receiver
[  9]   0.00-10.00  sec  2.08 GBytes  1.79 Gbits/sec   45             sender
[  9]   0.00-10.04  sec  2.08 GBytes  1.78 Gbits/sec                  receiver
[ 11]   0.00-10.00  sec  2.08 GBytes  1.79 Gbits/sec   14             sender
[ 11]   0.00-10.04  sec  2.08 GBytes  1.78 Gbits/sec                  receiver
[SUM]   0.00-10.00  sec  8.33 GBytes  7.15 Gbits/sec  305             sender
[SUM]   0.00-10.04  sec  8.31 GBytes  7.11 Gbits/sec                  receiver

iperf Done.

zimrigeek · Mar 5, 2024

blunden said:
First of all, if you have a Q20332G9 you don't have a C3758R, you have a C3758. The Q20331G9 is the C3758R. It shouldn't make more than a few percent difference, but it's always better to be correct.

Second of all iperf3 (unlike iperf2) is single threaded so you should try with 2 or more threads (-P2, etc.).

Third, your results don't match what I've seen elsewhere so I'd say you might have to take a another look at your testing methodology or configuration.

Some official benchmarks from Netgate (Netgate 8200):

pfSense Plus How to buy

Purchase pfSense Plus software on a Netgate Appliance, on AWS or Azure as a cloud instance , or as a virtual machine

www.netgate.com

That's a great approach! Opting for real-world scenario testing such as enabling SMB/CIFS or HTTP aligns more accurately with day-to-day use and often yields more applicable results. Your observation that these results are highly consistent with the iperf3 score of 7.49 Gbits/sec - 8 Gbits/sec makes complete sense.

To illustrate, a 7.49 Gbits/sec speed would approximately correlate to a real-world transfer speed of around 0.93625 GB/s (936 MB/s).

Below, the average speed achieved via wget is approximately (912 MB/s). This indeed appears to match the iperf3 outputs closely. It's a good practice to run such real-world scenario tests alongside benchmarking tools for a more comprehensive performance assessment.

Code:

root@Ubuntu22:~# wget http://192.168.20.1:5050/openSUSE-Leap-15.5-DVD-x86_64-Build491.1-Media.iso
--2024-03-05 23:03:44--  http://192.168.20.1:5050/openSUSE-Leap-15.5-DVD-x86_64-Build491.1-Media.iso
Connecting to 192.168.20.1:5050... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4413456384 (4.1G) [application/octet-stream]
Saving to: ‘openSUSE-Leap-15.5-DVD-x86_64-Build491.1-Media.iso.1’

openSUSE-Leap-15.5- 100%[===================>]   4.11G   948MB/s    in 4.6s    

2024-03-05 23:03:49 (912 MB/s) - ‘openSUSE-Leap-15.5-DVD-x86_64-Build491.1-Media.iso.1’ saved [4413456384/4413456384]

blunden · Mar 5, 2024

zimrigeek said:
1. I've taken a close look at my specific device model: Q20331G9 S10 C3758R

2. Although I'm using iperf3 as a non-authoritative test, regarding threads, I've actually done many tests and found that the results are simply the sum of a single thread. If you have better suggestions, I'm very willing to test according to your suggestions.

3. If you have time, I would like to ask you to provide results suggested elsewhere for my reference.
Thank you very much!

1. Ok, good. Then it was just a model name typo, not the other way around.

2. I see. That wasn't obvious from what you posted so I figured I should clarify that.

3. STH did iperf3 tests between two of these recently for instance.

https://www.servethehome.com/intel-x553-networking-and-proxmox-ve-8-1-3/

The Netgate benchmark I linked previously also suggest your results should be higher. I think I've seen other results as well, but I don't remember if it was in this thread or elsewhere.

There are some offloading features you could enable too. Pfsense has good documentation about them if I recall correctly. I think those are disabled by default when running those OS:es in a VM though so the STH test should be without them.

Btw. when doing HTTP testing you should really download to /dev/null to avoid the risk of being affected by storage speed.

TheGeekn°72 · Mar 5, 2024

Gibson05 said:
Got my 1U version in today. Order from the Qotom store on Aliexpress and it took 11 days to get here in the states from the day I ordered. It actually came decently packed. It has a heatsink on the bottom and exhaust out the side. Also added a pic of the power supply. It does come with rack ears and a power cord.

I asked Qotom for a board-less rack case kit and I just received it ! I probably should have checked those pictures closely, I'd have noticed that the PSU's lead were soldered onto the board, I'd have gotten a proper soldering iron... But lucky me ! The 12v barrel plug is there ! And there's a hole for it in the front panel ! Hurray ! FYI : price of kit VAT incl. + shipping to France, 90€ so y'all muricans will probably get a cheaper bill out of it if you guys want to switch cases for rack operations

Pictures to come soon !

farmerj · Mar 5, 2024

Looking at the 1U version of these which comes with a fan... Wondering if the fan noise is low enough for a bedroom when lightly loaded. Appears to be a laptop style fan.
Thanks!

TheGeekn°72 · Mar 5, 2024

farmerj said:
Looking at the 1U version of these which comes with a fan... Wondering if the fan noise is low enough for a bedroom when lightly loaded. Appears to be a laptop style fan.
Thanks!

It's... bearable ? It really depends I think
Fan ramps up and down every 5mn, as far as I'm aware, idk if there's a fan curve panel in the BIOS, I'll have to look it up but besides that, when it's spinning up, it's more like low white noise than anything else particularly disturbing though you do hear it distinctly if you've got sensitive ears

kr33 · Mar 6, 2024

Hi All,

At the risk of this question being repeated, apologies in advance

. I also don't mean to take things of track either.

I just received the Q20331G9-S10 (Atom C3758R version), I am wonder if there are any recommended SFP to RJ45 modules that perhaps work best as I am not full sure if the SFP port are locked to a specific "brand".

I've tried Ipolex, Wiitek & Qsfptek, although taking a few moments to power up they seemed to work in OPNSense even though during the opnsense setup the sfp light were flashing, the installer could detect and link so I have to manual pick a specific port. So when I started up Ubuntu initially just to explore abit, the ports are recognized but it shows as "Not Connected" and not light activity.

Perhaps is SFP Module or just a driver issue, I'm just not sure to be honest.

P.S. Reason for using SFP Module instead of DAC is my ONT has a 10G RJ45 port for my fiber connection. I plan on using a passive DAC to my switch from the mini pc.

Much appreciated for any guidance/recommendations.

blunden · Mar 6, 2024

kr33 said:
Hi All,

At the risk of this question being repeated, apologies in advance . I also don't mean to take things of track either.

I just received the Q20331G9-S10 (Atom C3758R version), I am wonder if there are any recommended SFP to RJ45 modules that perhaps work best as I am not full sure if the SFP port are locked to a specific "brand".

I've tried Ipolex, Wiitek & Qsfptek, although taking a few moments to power up they seemed to work in OPNSense even though during the opnsense setup the sfp light were flashing, the installer could detect and link so I have to manual pick a specific port. So when I started up Ubuntu initially just to explore abit, the ports are recognized but it shows as "Not Connected" and not light activity.

Perhaps is SFP Module or just a driver issue, I'm just not sure to be honest.

P.S. Reason for using SFP Module instead of DAC is my ONT has a 10G RJ45 port for my fiber connection. I plan on using a passive DAC to my switch from the mini pc.

Much appreciated for any guidance/recommendations.

Based on other people's testing in here and by STH, the SFP+ slots are not vendor locked.

The in-kernel driver for that NIC in Linux 6.x is broken, which probably what you are running into. Look a few pages back.

Intel's out-of-tree driver works fine, which VyOS switched back to for that very reason.

OPNsense isn't affected because the BSD kernel driver doesn't have an equivalent broken change.

farmerj · Mar 7, 2024

TheGeekn°72 said:
It's... bearable ? It really depends I think
Fan ramps up and down every 5mn, as far as I'm aware, idk if there's a fan curve panel in the BIOS, I'll have to look it up but besides that, when it's spinning up, it's more like low white noise than anything else particularly disturbing though you do hear it distinctly if you've got sensitive ears

I don't hear well... and it's a guest bedroom so we're all good. Thanks!

Khadgar · Mar 9, 2024

zimrigeek said:
2. Jump-start: There has been limited sharing on this topic, and I was unable to locate the exact jumper position. However, in the BIOS North Bridge chipset settings, you can set the power status to either "Last State" or "Auto Start".

I also bought one of these almost exactly the same time you did. I found the thread where images were shared of where the jumper should be put to enable jump-start (https://forums.servethehome.com/ind...n-fanless-system-with-4-sfp.41331/post-393864), but it doesn't work. When using a screwdriver on mine I can get it to boot by bridging not the first two pins as that image suggests but the next two. Boots fine with a screwdriver, but if I use a jumper it attempts to boot and then turns off.

I don't see anything in my BIOS for power status. Not sure how to get jump-start to work.

Brent Geery · Mar 9, 2024

Take another look at the BIOS settings. It's there. North Bridge chipset S3 power state or something like that. The BIOS option labels are all very technical and unhelpful.

Khadgar · Mar 9, 2024

Brent Geery said:
Take another look at the BIOS settings. It's there. North Bridge chipset S3 power state or something like that. The BIOS option labels are all very technical and unhelpful.

Thanks for your help. I went digging for "S3" and found it in the South Bridge config. Here's a screenshot for others so they'll know where it is:

Brent Geery · Mar 9, 2024

Ah, OK. Yeah, that's the one. Glad you found it! Why Power On or Last State isn't the default on a server device I'll never understand.

Qotom Denverton fanless system with 4 SFP+

Better than being feckless.

New Member

New Member

Better than being feckless.

Active Member

New Member

Active Member

New Member

New Member

Active Member

Member

New Member

Member

New Member

Active Member

New Member

New Member

New Member

New Member

New Member