Fujitsu Futro S920 Thin Client as opnsense firewall

[Pietro395]

@Pietro395
I think you ‘ll be disappointed because 2 port card has a lower bandwidth and ids/ips processes are cpu hungry

Yes I know IPS/IDS are cpu hungry and I will probably not keep them on, it is my first OPNsense build and I would like to start with something cheap.

What do you mean by the bandwidth of the card? I should still reach in each port 2.5Gbps right?

 

[gianry]

That card has a pci gen 3 x4 connector with a 4 gbit max bandwidth

 

[Pietro395]

That card has a pci gen 3 x4 connector with a 4 gbit max bandwidth

So you say I won't be able to achieve 2.5Gbps simultaneously in the two ports?

One port is for LAN and the other for WAN, I think I've seen people reaching +3Gbps per port with 10Gbps cards

 

[boerni666]

you can archive 2.5gbps simultaneously on 2 ports.

 

[gianry]

So you say I won't be able to achieve 2.5Gbps simultaneously in the two ports?

in my experience it is not possible with that card.

 

[boerni666]

oh yeah, saw that card, which uses a singel PCIex1 3.0 interface. Ive got a similiar card for the minipcie slot. 2x Realtek 8125 behind a ASM PCIe Switch. i saw 2.1gbit/s NAT performance on that one. But still tight.

Check out the 4 Port 2.5G Realtek 8125 Cards on Aliexpress, they got a PCIex4 interface and work flawlessly in that machine.

 

[boerni666]

Couldnt sleep so i got bored and started up one of my Futros for some 10G Testing.


Slid in a Mellanox Connect-X3, connected it via SFP+ to my cheap chinese 10G switch and booted up some live-USB linux (grml).

Spoiler: 10G Results

root@grml ~ # lspci -s 01:00.0 -vvv
01:00.0 Ethernet controller: Mellanox Technologies MT26448 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0)
Subsystem: Mellanox Technologies MT26448 [ConnectX EN 10GigE, PCIe 2.0 5GT/s]
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 33
Region 0: Memory at fe900000 (64-bit, non-prefetchable) [size=1M]
Region 2: Memory at fc800000 (64-bit, prefetchable) [size=8M]
Expansion ROM at fe800000 [disabled] [size=1M]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [48] Vital Product Data
Product Name: HP ConnectX-2 SFP+
Read-only fields:
[PN] Part number: 666172-001
[EC] Engineering changes: A1
[SN] Serial number: IL214802FU
[V0] Vendor specific: PCIe Gen2 x8
[RV] Reserved: checksum good, 0 byte(s) reserved
Read/write fields:
[V1] Vendor specific: N/A
[YA] Asset tag: N/A
[RW] Read-write area: 102 byte(s) free
End
Capabilities: [9c] MSI-X: Enable+ Count=128 Masked-
Vector table: BAR=0 offset=0007c000
PBA: BAR=0 offset=0007d000
Capabilities: [60] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 unlimited
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- FLReset-
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #8, Speed 5GT/s, Width x8, ASPM L0s, Exit Latency L0s unlimited
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x4 (downgraded)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [100 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 0
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [148 v1] Device Serial Number 00-02-c9-03-00-52-9f-e2
Kernel driver in use: mlx4_core
Kernel modules: mlx4_core

root@grml ~ # iperf3 -c 192.168.0.2
Connecting to host 192.168.0.2, port 5201
[ 5] local 192.168.0.202 port 39032 connected to 192.168.0.2 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 690 MBytes 5.78 Gbits/sec 0 687 KBytes
[ 5] 1.00-2.00 sec 694 MBytes 5.82 Gbits/sec 0 687 KBytes
[ 5] 2.00-3.00 sec 675 MBytes 5.66 Gbits/sec 0 687 KBytes
[ 5] 3.00-4.00 sec 700 MBytes 5.87 Gbits/sec 0 687 KBytes
[ 5] 4.00-5.00 sec 689 MBytes 5.78 Gbits/sec 0 687 KBytes
[ 5] 5.00-6.00 sec 696 MBytes 5.84 Gbits/sec 0 687 KBytes
[ 5] 6.00-7.00 sec 689 MBytes 5.78 Gbits/sec 0 687 KBytes
[ 5] 7.00-8.00 sec 699 MBytes 5.86 Gbits/sec 0 687 KBytes
[ 5] 8.00-9.00 sec 697 MBytes 5.85 Gbits/sec 0 687 KBytes
[ 5] 9.00-10.00 sec 706 MBytes 5.92 Gbits/sec 0 915 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 6.80 GBytes 5.84 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 6.80 GBytes 5.84 Gbits/sec receiver

iperf Done.
iperf3 -c 192.168.0.2 0.06s user 9.94s system 99% cpu 10.016 total
root@grml ~ # iperf3 -c 192.168.0.2 -P 4
Connecting to host 192.168.0.2, port 5201
[ 5] local 192.168.0.202 port 51314 connected to 192.168.0.2 port 5201
[ 7] local 192.168.0.202 port 51326 connected to 192.168.0.2 port 5201
[ 9] local 192.168.0.202 port 51332 connected to 192.168.0.2 port 5201
[ 11] local 192.168.0.202 port 51348 connected to 192.168.0.2 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 286 MBytes 2.40 Gbits/sec 0 757 KBytes
[ 7] 0.00-1.00 sec 285 MBytes 2.38 Gbits/sec 0 781 KBytes
[ 9] 0.00-1.00 sec 285 MBytes 2.39 Gbits/sec 0 817 KBytes
[ 11] 0.00-1.00 sec 287 MBytes 2.40 Gbits/sec 0 754 KBytes
[SUM] 0.00-1.00 sec 1.12 GBytes 9.57 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 1.00-2.00 sec 282 MBytes 2.36 Gbits/sec 0 796 KBytes
[ 7] 1.00-2.00 sec 283 MBytes 2.38 Gbits/sec 0 781 KBytes
[ 9] 1.00-2.00 sec 282 MBytes 2.37 Gbits/sec 0 817 KBytes
[ 11] 1.00-2.00 sec 282 MBytes 2.37 Gbits/sec 0 754 KBytes
[SUM] 1.00-2.00 sec 1.10 GBytes 9.47 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 2.00-3.00 sec 283 MBytes 2.37 Gbits/sec 0 796 KBytes
[ 7] 2.00-3.00 sec 282 MBytes 2.36 Gbits/sec 0 781 KBytes
[ 9] 2.00-3.00 sec 283 MBytes 2.37 Gbits/sec 0 817 KBytes
[ 11] 2.00-3.01 sec 286 MBytes 2.37 Gbits/sec 0 754 KBytes
[SUM] 2.00-3.00 sec 1.11 GBytes 9.51 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 3.00-4.00 sec 282 MBytes 2.36 Gbits/sec 0 796 KBytes
[ 7] 3.00-4.00 sec 282 MBytes 2.37 Gbits/sec 0 781 KBytes
[ 9] 3.00-4.00 sec 278 MBytes 2.33 Gbits/sec 0 817 KBytes
[ 11] 3.01-4.00 sec 278 MBytes 2.36 Gbits/sec 0 793 KBytes
[SUM] 3.00-4.00 sec 1.09 GBytes 9.40 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 4.00-5.00 sec 282 MBytes 2.36 Gbits/sec 0 796 KBytes
[ 7] 4.00-5.00 sec 282 MBytes 2.37 Gbits/sec 0 781 KBytes
[ 9] 4.00-5.00 sec 283 MBytes 2.38 Gbits/sec 0 817 KBytes
[ 11] 4.00-5.00 sec 283 MBytes 2.37 Gbits/sec 0 793 KBytes
[SUM] 4.00-5.00 sec 1.10 GBytes 9.48 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 5.00-6.00 sec 193 MBytes 1.62 Gbits/sec 0 834 KBytes
[ 7] 5.00-6.00 sec 372 MBytes 3.12 Gbits/sec 0 891 KBytes
[ 9] 5.00-6.00 sec 372 MBytes 3.12 Gbits/sec 0 961 KBytes
[ 11] 5.00-6.00 sec 192 MBytes 1.61 Gbits/sec 0 793 KBytes
[SUM] 5.00-6.00 sec 1.10 GBytes 9.47 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 6.00-7.00 sec 188 MBytes 1.58 Gbits/sec 0 834 KBytes
[ 7] 6.00-7.00 sec 376 MBytes 3.15 Gbits/sec 0 980 KBytes
[ 9] 6.00-7.00 sec 376 MBytes 3.16 Gbits/sec 0 961 KBytes
[ 11] 6.00-7.00 sec 188 MBytes 1.58 Gbits/sec 0 793 KBytes
[SUM] 6.00-7.00 sec 1.10 GBytes 9.46 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 7.00-8.00 sec 189 MBytes 1.59 Gbits/sec 0 834 KBytes
[ 7] 7.00-8.00 sec 377 MBytes 3.16 Gbits/sec 0 980 KBytes
[ 9] 7.00-8.00 sec 377 MBytes 3.16 Gbits/sec 0 961 KBytes
[ 11] 7.00-8.00 sec 189 MBytes 1.58 Gbits/sec 0 793 KBytes
[SUM] 7.00-8.00 sec 1.11 GBytes 9.50 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 8.00-9.01 sec 189 MBytes 1.58 Gbits/sec 0 834 KBytes
[ 7] 8.00-9.01 sec 378 MBytes 3.16 Gbits/sec 0 1.00 MBytes
[ 9] 8.00-9.01 sec 378 MBytes 3.16 Gbits/sec 0 961 KBytes
[ 11] 8.00-9.01 sec 190 MBytes 1.58 Gbits/sec 0 793 KBytes
[SUM] 8.00-9.01 sec 1.11 GBytes 9.47 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 9.01-10.00 sec 187 MBytes 1.57 Gbits/sec 0 834 KBytes
[ 7] 9.01-10.00 sec 374 MBytes 3.15 Gbits/sec 0 1.00 MBytes
[ 9] 9.01-10.00 sec 373 MBytes 3.14 Gbits/sec 0 1008 KBytes
[ 11] 9.01-10.00 sec 187 MBytes 1.58 Gbits/sec 0 793 KBytes
[SUM] 9.01-10.00 sec 1.10 GBytes 9.44 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.31 GBytes 1.98 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 2.30 GBytes 1.98 Gbits/sec receiver
[ 7] 0.00-10.00 sec 3.22 GBytes 2.76 Gbits/sec 0 sender
[ 7] 0.00-10.00 sec 3.21 GBytes 2.76 Gbits/sec receiver
[ 9] 0.00-10.00 sec 3.22 GBytes 2.76 Gbits/sec 0 sender
[ 9] 0.00-10.00 sec 3.21 GBytes 2.76 Gbits/sec receiver
[ 11] 0.00-10.00 sec 2.31 GBytes 1.98 Gbits/sec 0 sender
[ 11] 0.00-10.00 sec 2.30 GBytes 1.98 Gbits/sec receiver
[SUM] 0.00-10.00 sec 11.0 GBytes 9.49 Gbits/sec 0 sender
[SUM] 0.00-10.00 sec 11.0 GBytes 9.48 Gbits/sec receiver

iperf Done.
iperf3 -c 192.168.0.2 -P 4 0.15s user 25.80s system 258% cpu 10.026 total

PCIe Speed was set to PCIe2.0 over the editCmos hack.

As you can see, single iperf3 is heavily bottlenecked by single-thread performance on the Futro.

With four parallel streams over the -P4 switch and cpufreq governor set to performance, i've archived full 10G, utilizing 258% CPU, so each core had like 65% Load. I still need to find a dual 10G card and do a loopback or nat test to see if it could route (or at least bridge) 10G.

Power consumption was ~18W around testing with the card. When idle (still 10G network connected) around 12W. Used a cheap USB-C Powermeter behind the PSU, used a 20V Trigger cable for Power.

 

[gianry]

I don’t see any Speedtest in your post . You have to consider that some routeros doesn’t allow multicpu in network traffic

 

[boerni666]

iperf3 is a speedtest. also, i don't have a 10G internet uplink. If your routeros doesn't allow multithreading, i would stick to OpenWRT.

 

[Chrysen]

Is it possible to save more energy with a different GPU driver?

I use the futro s920 with the strongest CPU GX 424-CC with an Intel i350T4 and pfsense as system.

In idle I get just under 10W. Under Windows in idle it is only about 7.5W. I read in another forum that this is due to the GPU driver.

I use the cmos hack for pcie 2
The BIOS is up to date and the firmware of the network card is also up to date, this brought a saving of about 0.3W.

 

[tbfutro22]

I also use Windows Server 2022 on a Futro s720 or s920, I get down to 3W with the s720. Linux is more like 5-6W. I did not have much success to use "powertop" or something to reduce power consumption on Linux, I was shaving half a Watt maybe, but nothing substantial to get anywhere near the efficient Windows idle power usage.

I guess for both operating systems, it is important to NOT connect a display or make sure, the GPU switches off the output completely when not used (not just screen blank). With output enabled (so you see something), power consumption is around 2W more, each USB device for mouse/keyboard or thumb drive, will consume 1W+ as well.

I also assume, that the very bright white power LED consumes quite a bit of power as well. I have not tested yet, how the power consumption goes down if the LED was desoldered, but worth a try I guess. o)

Sorry, can't say something about specific Linux / GPU driver, if this really is the cause for the higher power consumption when using Linux, it's worth investigating. Also Kernel 6.13 came out recently with 30% power saving potential for specific network load scenarios, not sure it also helps on an idle system, but worth a try as well I assume.

 

[Chrysen]

I didn't get it any lower, neither powerd undervolting helps, and if you set the power settings in pfsense to minimum I get 400-500mbit and 13-14w with wireguard and iperf.
If you set it to maximum, I get about 950mbit and about 20-25w

Of course, only power and network were connected during all tests.

Maybe there is a way to disable the igpu completely.

Possibly with a custom bios.
You can reset the BIOS even without output if something doesn't work.

 

[supermegazord]

Hello guys i read this theader but not found any infos about RAM problem with S920 therefore i decide to write this post
I bought a Futro S920 with an AMD GX-424CC and 2GB RAM in one module. According to the documentation, this model has a D3313-E motherboard, which supports up to 16GB of RAM. However, I've tested two different RAM package , and neither works. When I install the 2GB module, it works correctly, but when I install any 16GB package (which are two 8GB DDR3L modules with 1.35V), I can't load any live Linux distro. It always resets when switching to graphical mode. Has anyone experienced a similar issue?

 

[boerni666]

it completely resets? like no kernelpanic, or just X crashes, where you at least investigate the error.
Its not a PSU problem?

 

[supermegazord]

Yes exactly, completely restart, no kernel panic or any others errors,

1. Booting from USB Lubuntu or some other live distro.

2. From the Grub menu, I choose Run Lubuntu (first option).

3. On screen, I see "_" underscore.

4. Machine restarting



When I use the 2GB module Lubuntu load correctly, I don't think it is PSU problem because with the 2GB module and the same PSU load Lubuntu correctly.

Exist any info about what modules works correctly, or maybe if you use an 8GB module, what is a module (name, parameters) maybe I use modules that is not compatible with S920 from some reason

BTW, Memtest not showing any error with the memory

 

[Samir]

Sounds like an issue with the linux distribution. Try using a 'failsafe mode' if there's an option for one.

I run 8GB x2 in one of my S920s and 16GB x2 in another with no issues. Both of mine have the graphics card.

 

[tbfutro22]

I did not have any issues so far with 8GB modules in S720 and S920 (dual cores), can't say anything about the quad core versions.

If the system works fine with Windows or an installed variant of Linux-XY, I guess you can blame the live / USB boot system. Maybe it helps to toggle some options in the BIOS, like UEFI vs. Legacy Mode, VRAM size, USB legacy modes etc..

 

[supermegazord]

I try to run.
- Lubuntu
- EndevourOS
- Manjaro
- SystemRescueCD
and finally Win10 PE
On Linux, always restart when the image is loaded to RAM; on Win PE, I have a BSOD with info like "Memory Manage" and classily restart.
Which RAM are you using? (model)

I try to change the memory size for the GPU (if this is the option you have in mind) and load the default settings of BIOS, but it does not affect

Edit:
In Failsafe Mode, still have a restart.

 

[tbfutro22]

My little database tells me that I have these 8GB modules around, I used them here and there. The Elixir is mounted in my S930 / GX-424CC currently (it seems, this is what the database tells me at least). I assume it works in there as expected. Did not mention the S930 regarding memory modules yet, because I have not used it very much. I did not notice any difference with memory compatibility yet. I would need to power up the S930 and make specific tests with 2x8GB configuration to tell more.



RAM_DDR3-SO_8GB_PC3-10600_1333Mhz_Corsair_CMSA16GX3M2A1333C9_MacMemory_1of2
RAM_DDR3-SO_8GB_PC3-10600_1333Mhz_Corsair_CMSA16GX3M2A1333C9_MacMemory_2of2
RAM_DDR3-SO_8GB_PC3-12800_1600Mhz_Elixir_Nanya_M2S8G64CC8HD5N-DI
RAM_DDR3-SO_8GB_PC3-12800_1600MHz_Kingston_KN2M64-ETBS
RAM_DDR3-SO_8GB_PC3-12800_1600MHz_Samsung_M471B1G73QH0-YK0

 

[supermegazord]

@tbfutro22
Thanks for the list.
I found a cheap Samsung_M471B1G73QH0-YK0 module, I will buy it and test it.
At the moment I have to wait for a package with a RAM.
When I get this module, write how it works with my S920.
Thanks for your help.