Futro s920 pci-e limited to x1

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

hb76

New Member
Feb 13, 2023
4
0
1
Hallo,
I set up a Fujitsu futro S920 gx-222gc as a Linux router. I added a dual port 1Gb Ethernet card in the pci-e extension slot. What I noticed, is that the pci-e card uses only one lane.

During boot I get the following kernel message:
Code:
# dmesg | grep limit
[0.786196] pci 0000:01: 00.0: 2.000 Gb/s available PCIE bandwidth, limited by 2.5 GT/s PCIE link at 0000: 00:02.1 capable of 8.000 Gb/s with 5.0 GT/S PCIe X2 link)
01:00.0 being my Ethernet card and 00:02.1 seems to be the pci-e bridge of the soc. Checking with lspci shows indeed the host bridge runs only at x1 (LnkSta) but could run at x4 (according to LnkCap)

Has anyone seen a similar behaviour. Is this a reason to get a gx-415 instead?

Any ideas how I could change that ?
 

hb76

New Member
Feb 13, 2023
4
0
1
I just tested under load. I get aprox. 760 MBit/s transfer speed (in and out through the PCIe dual port NIC).

However, the limit of the NIC, using only one PCIe lane still persists. If I interpret the dmesg messages, the host bridge is limited to x1.
As also shown by
Code:
# lspci -s 00:02.1 -vv
00:02.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 16h Processor Functions 5:1 (prog-if 00 [Normal decode])
        Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1234
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 25
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: [disabled] [32-bit]
        Memory behind bridge: fea00000-feafffff [size=1M] [32-bit]
        Prefetchable memory behind bridge: fc900000-fc9fffff [size=1M] [32-bit]
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
        BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16+ MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [50] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [58] Express (v2) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0
                        ExtTag+ RBE+
                DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
                        RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 256 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us
                        ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1
                        TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
                        Slot #0, PowerLimit 0W; Interlock- NoCompl+
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                        Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
                        Changed: MRL- PresDet- LinkState-
                RootCap: CRSVisible+
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible+
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
                         10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
                         EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
                         FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd-
                         AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS-
                DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled, ARIFwd-
                         AtomicOpsCtl: ReqEn- EgressBlck-
                LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS-
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis+
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
                         EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
                         Retimer- 2Retimers- CrosslinkRes: unsupported
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
                Address: 00000000fee01004  Data: 0021
        Capabilities: [b0] Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1234
        Capabilities: [b8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Kernel driver in use: pcieport
LnkCap has width x4, but LnkSta is at width x1 - even during the speed test.
 

hb76

New Member
Feb 13, 2023
4
0
1
Thank you - that was the information I needed.

It was the PSPP (PCIE Speed Power Policy) set to "balanced-low". By setting it to "balanced-high", speed increased to 5 GT/s and 4 Gb/s.

BTW: I used the "SetBios GUI" Windows 10 tool from the Fujitsu homepage (search for DeskView).
 

fanoush

New Member
Mar 11, 2023
8
8
3
@hb76 Do you see some real speed increase in speed test with 5GT/s?

As for 1x vs 4x the slot definitely can do 4x, I tried with nvme in pcie 4x adapter. If you do
lspci -vvv then in "LnkSta:" I see (downgraded) on the bridge and (ok) on the card if the card can only do 1x (wifi, builtin ethernet). If the card could do more than the bridge allows the (downgraded) is on the card output. I see it on my ssd, speed is downgraded to 5GT/s, SSD can do 16GT/s.

Good tip about SetBios GUI. I actually had only linux and freedos available, don't have windows on my S920. Did you also find linux version on Fujitsu site?
 
Last edited:

hb76

New Member
Feb 13, 2023
4
0
1
@fanoush I think there is no linux version of Biosset. In the Fujitsu DeskView documentation only DOS and Windows clients are mentioned.

So I resolved the 2.5 GT/s vs. 5GT/s issue, but my Broadcom card still uses only one lane:
Code:
#lspci -s 01:00.0 -vvv #( Broadcom NIC)
[...]
               LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <1us, L1 <2us
                        ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp-
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1 (downgraded)
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
[...]
The bridge does not say "(downgraded)":
Code:
#lspci -s 00:02.1 -vvv #(AMD PCIe Bridge)
[...]
                LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <512ns, L1 <64us
                        ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1
                        TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
[...]
Well, maybe the card is PCI-E 2.1 x1 anyway - thats what some webites claim, but then the lspci output is wrong (capability x2 - downgraded to x1).
 
Last edited:

fanoush

New Member
Mar 11, 2023
8
8
3
Well if (downgraded) is on card the card wanted more. If it is on the bridge the card wants less than the bridge allows, that's how I understand it. So here really it looks like the bridge decides it is OK to give only one lane. Wonder if the second pcie lane on the card or slot is physically disconnected (or even traces are missing on the card) despite the the chip on the card believing it can do 2x.

Could be interesting to test another pci-e card that is 2x or 4x