EU ASRock Rack B450D4U-V1LQ5 mATX AM4

[gdv-it]

Hey,

i have not read the whole thread but i hope this informations are helpful.

Just tested my bunch of boards. Due to missing Overclock/Underclock bios settings, you can underclock/overclock (undervolt/overvolt) the CPU and set the missing Memory Settings by RyzenMaster (Windows) or ryzen_smu (Linux).

( ryzen_smu/docs/rsmu_commands.md at master · leogx9r/ryzen_smu )

for example (on my ryzen 3600), set the freq to 4000mhz @ 1,2V

# enable OC
printf '%0*x' 48 0 | fold -w 2 | tac | tr -d '\n' | xxd -r -p | sudo tee /sys/kernel/ryzen_smu_drv/smu_args && printf '\x5A' | sudo tee /sys/kernel/ryzen_smu_drv/rsmu_cmd

# vcore 1,2v (VID 56)
printf '%0*x' 48 56 | fold -w 2 | tac | tr -d '\n' | xxd -r -p | sudo tee /sys/kernel/ryzen_smu_drv/smu_args && printf '\x61' | sudo tee /sys/kernel/ryzen_smu_drv/rsmu_cmd

#overclock freq 4000
printf '%0*x' 48 4000 | fold -w 2 | tac | tr -d '\n' | xxd -r -p | sudo tee /sys/kernel/ryzen_smu_drv/smu_args && printf '\x5C' | sudo tee /sys/kernel/ryzen_smu_drv/rsmu_cmd

i try to find a way to setup the memory timings without windows because sometimes the settings get lost by bios.

 

[luckylinux]

Hey,

i have not read the whole thread but i hope this informations are helpful.

Just tested my bunch of boards. Due to missing Overclock/Underclock bios settings, you can underclock/overclock (undervolt/overvolt) the CPU and set the missing Memory Settings by RyzenMaster (Windows) or ryzen_smu (Linux).

( ryzen_smu/docs/rsmu_commands.md at master · leogx9r/ryzen_smu )

Interesting, I am pretty sure I tried (and failed) with another Tool (cannot remember which one exactly) on Linux.

Thanks for the Tip .

Did you also try that with a 5000 Series CPUs such as e.g. AMD Ryzen 7 5700X ?

 

[gdv-it]

Yes, i will try it with a 5600G and a 5950X. With this tool, you access the in-Processor Power Management Unit directly, so it should be work with any Ryzen that boots on this Platform.

Any idea to set the Memory Timings? I have set it with RyzenMaster on windows and it seems to store it in the UEFI to survive a reboot. After Reboot the AMI Bios shows 3600Mhz Memory Clock. But i want to avoid to start Windows on each Machine every time the BIOS drops this setting.

 

[gdv-it]

Accidentialy checked out, that BIOS 2.02 has the AMD Overclocking menu in bios. !?!

EDIT: Last bios with AMD OC is L2.08
in any later bios, the OC Menu and any undervolt/overvolt settings were removed from the bios. Verified with UEFITOOL. WTF? WHY?

 

[RolloZ170]

WTF? WHY?

you should not overclock a server motherboard, to much RMAs if available.

 

[luckylinux]

you should not overclock a server motherboard, to much RMAs if available.

I agree on the Overclock Part, but to be able to UNDERvolt it would be good to have that Option.

Probably too many Calls from Customers that ran into Stability Issues maybe.

 

[luckylinux]

@Cruzader: for the 1U Case with Riser mount that you showed in this Post

https://forums.servethehome.com/index.php?threads/asrock-rack-b450d4u-v1lq5-matx-am4.47305/page-28#post-467337

Is there space for a 10mm Heatsink (or possibly 3mm Heatsink) plus thermal Pad for the NVMe Device, especially the one facing bottom ?

I tried running some Samsung PM983 960gb and they get quite Hot even at Idle (45-50°C according to one Sensor, 82-88°C according to another Sensor [probably the Controller]).

Thanks in Advance

 

[luckylinux]

I'm curious if anybody managed to get ASPM working on this ***** Board.

I did a quick Test with a 360W Seasonic 80+ Gold PSU.

I run some ASPM Enabler Script then powertop --auto-tune but there are apparently still 3 Devices which do NOT want to play along:

03:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset SATA Controller (rev 01) (prog-if 01 [AHCI 1.0])
    Subsystem: ASMedia Technology Inc. 400 Series Chipset SATA Controller
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin B routed to IRQ 81
    IOMMU group: 15
    Region 5: Memory at fc380000 (32-bit, non-prefetchable) [size=128K]
    Expansion ROM at fc300000 [disabled] [size=512K]
    Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Address: 00000000fee00000  Data: 0000
    Capabilities: [78] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=55mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
        Status: D3 NoSoftRst+ PME-Enable+ DSel=0 DScale=0 PME-
    Capabilities: [80] Express (v2) Legacy Endpoint, MSI 00
        DevCap:    MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 <64us
            ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
        DevCtl:    CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
            RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
            MaxPayload 256 bytes, MaxReadReq 512 bytes
        DevSta:    CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr+ TransPend-
        LnkCap:    Port #0, Speed 8GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <2us, L1 <32us
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl:    ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta:    Speed 8GT/s, Width x4
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Not Supported, TimeoutDis- NROPrPrP- LTR-
             10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS-
             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled,
             AtomicOpsCtl: ReqEn-
        LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
             EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
             Retimer- 2Retimers- CrosslinkRes: unsupported
    Capabilities: [100 v1] Advanced Error Reporting
        UESta:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt:    DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
        CEMsk:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
        AERCap:    First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap- ECRCChkEn-
            MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
        HeaderLog: 00000000 00000000 00000000 00000000
    Kernel driver in use: ahci
    Kernel modules: ahci


28:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 (prog-if 02 [NVM Express])
    Subsystem: Samsung Electronics Co Ltd SSD 970 EVO
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin A routed to IRQ 44
    IOMMU group: 18
    Region 0: Memory at fcb10000 (64-bit, non-prefetchable) [size=16K]
    Expansion ROM at fcb00000 [disabled] [size=64K]
    Capabilities: [40] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [50] MSI: Enable- Count=1/32 Maskable- 64bit+
        Address: 0000000000000000  Data: 0000
    Capabilities: [70] Express (v2) Endpoint, MSI 00
        DevCap:    MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
            ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 75W
        DevCtl:    CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
            RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
            MaxPayload 256 bytes, MaxReadReq 512 bytes
        DevSta:    CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
        LnkCap:    Port #0, Speed 8GT/s, Width x4, ASPM not supported
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl:    ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta:    Speed 8GT/s, Width x4
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
             10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS- TPHComp- ExtTPHComp-
             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ 10BitTagReq- OBFF Disabled,
             AtomicOpsCtl: ReqEn-
        LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
        LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
        LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
             EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
             Retimer- 2Retimers- CrosslinkRes: unsupported
    Capabilities: [b0] MSI-X: Enable+ Count=33 Masked-
        Vector table: BAR=0 offset=00003000
        PBA: BAR=0 offset=00002000
    Capabilities: [100 v2] Advanced Error Reporting
        UESta:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt:    DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
        CEMsk:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
        AERCap:    First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
            MultHdrRecCap+ MultHdrRecEn- TLPPfxPres- HdrLogCap-
        HeaderLog: 00000000 00000000 00000000 00000000
    Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00
    Capabilities: [158 v1] Power Budgeting <?>
    Capabilities: [168 v1] Secondary PCI Express
        LnkCtl3: LnkEquIntrruptEn- PerformEqu-
        LaneErrStat: 0
    Capabilities: [188 v1] Latency Tolerance Reporting
        Max snoop latency: 1048576ns
        Max no snoop latency: 1048576ns
    Capabilities: [190 v1] L1 PM Substates
        L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates-
        L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
        L1SubCtl2:
    Kernel driver in use: nvme
    Kernel modules: nvme

29:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 (prog-if 02 [NVM Express])
    Subsystem: Samsung Electronics Co Ltd SSD 970 EVO
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin A routed to IRQ 45
    IOMMU group: 19
    Region 0: Memory at fca10000 (64-bit, non-prefetchable) [size=16K]
    Expansion ROM at fca00000 [disabled] [size=64K]
    Capabilities: [40] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [50] MSI: Enable- Count=1/32 Maskable- 64bit+
        Address: 0000000000000000  Data: 0000
    Capabilities: [70] Express (v2) Endpoint, MSI 00
        DevCap:    MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
            ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 75W
        DevCtl:    CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
            RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
            MaxPayload 256 bytes, MaxReadReq 512 bytes
        DevSta:    CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
        LnkCap:    Port #0, Speed 8GT/s, Width x4, ASPM not supported
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl:    ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta:    Speed 8GT/s, Width x4
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
             10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS- TPHComp- ExtTPHComp-
             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ 10BitTagReq- OBFF Disabled,
             AtomicOpsCtl: ReqEn-
        LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
        LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
        LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
             EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
             Retimer- 2Retimers- CrosslinkRes: unsupported
    Capabilities: [b0] MSI-X: Enable+ Count=33 Masked-
        Vector table: BAR=0 offset=00003000
        PBA: BAR=0 offset=00002000
    Capabilities: [100 v2] Advanced Error Reporting
        UESta:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt:    DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
        CEMsk:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
        AERCap:    First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
            MultHdrRecCap+ MultHdrRecEn- TLPPfxPres- HdrLogCap-
        HeaderLog: 00000000 00000000 00000000 00000000
    Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00
    Capabilities: [158 v1] Power Budgeting <?>
    Capabilities: [168 v1] Secondary PCI Express
        LnkCtl3: LnkEquIntrruptEn- PerformEqu-
        LaneErrStat: 0
    Capabilities: [188 v1] Latency Tolerance Reporting
        Max snoop latency: 1048576ns
        Max no snoop latency: 1048576ns
    Capabilities: [190 v1] L1 PM Substates
        L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates-
        L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-

Without NVMe / Without Mellanox ConnectX-4 LX NIC: approx 25W Idle

With NVMe / Without Mellanox ConnectX-4 LX NIC: approx 33-35W Idle

With NVMe / With Mellanox ConnectX-4 LX NIC: approx 40W Idle

I find this VERY high considering that a similar Intel Xeon System idles around 20W (granted without NVMe but with SATA SSDs, and Intel X710 instead of the Mellanox ConnectX-4 LX), partly due to this Ryzen Chiplet Design.

Still, the fact that we are unable to turn on ASPM is a Bummer. I might need to give setup_var.efi (and/or BIOS Patch) another try and turn on Aggressive Link Power Management for SATA as well as manually set ASPM to L0s+L1.

Weird is that the Samsung SSD claims to NOT support ASPM (Latency is unlimited), yet it claims it within its Capabilities

 L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates-

        L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-

I even managed to update Firmware to latest EDA7902Q Firmware, but ASPM stayed disabled .

 

[b3rrytech]

Weird is that the Samsung SSD claims to NOT support ASPM (Latency is unlimited), yet it claims it within its Capabilities

Does that Samsung 970 EVO NVMe SSD support ASPM in any of your other systems? I’ve had problems with Samsung NVMes in the past but that was a long time ago.

 

[luckylinux]

Does that Samsung 970 EVO NVMe SSD support ASPM in any of your other systems? I’ve had problems with Samsung NVMes in the past but that was a long time ago.

I don't know, that is a different Model.

I tried the PM983 in a Supermicro X11SCZ-F and after the usual lovely ACPI Patching, ASPM auto-tuning Script and even FORCING ASPM to be enabled on the PM983 (where the Device clearly doesn't state it as being supported), the CPU is stuck at Package C3 (PC3) anyways according to turbostat / cpupower.

So, BIOS or not BIOS on this ASROCK Rack B450D4U-V1LQ5 Motherboard, it still would NOT work I believe .

The only Thing that might help (on OTHER Systems, NOT an Option on this specific Ryzen Motherboard) is to try a PCH connected PCIe Slot instead of a PCIe Slot directly tied to the CPU.

ASPM seems to be easier/more forgiving via the PCH Slots, for instance with the Mellanox ConnectX-4 LX.

I did NOT test without NVMe and WITH the Mellanox ConnectX-4 LX though, but anyways the SATA Aggressive Link Power Management (hidden in BIOS) might need to be turned on anyways first, because that is preventing ASPM as well.

 

[gdv-it]

Did you also try that with a 5000 Series CPUs such as e.g. AMD Ryzen 7 5700X ?

test results with a 5600G: no GPU (ASPEED), no OC Settings on bios > 2.0, no rear USB.
It doesnt boot with Bios below 2.05 and below.
but CPU Works, onboard USB-3 Header works, pcie VGA works, ryzen_smu works for OC/UC. Memory Timing is configurable in BIOS (AMD CBS/PBS)

test result with a 2200G / 3200G: no GPU (ASPEED) on BIOS > 2.0 but with 1.14, same with rear USB

 

[dbram]

A little update on the CSE512 1u case mount I'm trying for these boards :

• Ryzen 3700x
• 2x16 = 32GB UDIMM
• Gelid 1U cooler + stock 1u supermicro case fan (Delta blower style)
• Supermicro 1u 200w psu (old non-gold certified)
• On demand scaling governor
• nvme ssd 1TB Samung 980
• dual 10gb x710

When idle, cpu temps are around 39° C


Under stress-ng -c 16 load:


So the Gelid cooler manages fine on its own when not under full load. When putting some workload on it I suggest to keep the rather noisy blower style delta fan of the case. they can be mounted together.

I tried without the Delta fan too. under full load the cpu temps go to 95-97° C, with the Delta fan, it remains just below 90°
FAN Noise included
actually the most noisy thing in this 1u case, when idle is the PSU fan. under full load the delta or gelid can be heared quite clearly.


Mounting wise, I just used the rack ears, no sliders, simply fixed. the bottom one is on top of a 1U shelf plate (that is mounted up side down).

 

[luckylinux]

@dbram: I think we need to Undervolt and possible Underclock a bit with that Cooler

Did the Fans ramp up to 100% ? To be honest I'm NOT 100% sure the "Smart Fan" Points in the BIOS Table are set correctly / intuitively.

Did you try using e.g. a Molex 4-pin to Fan Adapter to make sure they run at full speed ?

Did you try removing the existing Thermal Paste and installing your own across ALL the CPU Heatspreader / Cooler Base ?

 

[dbram]

something i'd have to look into. for now the workloads don't seem to require 100% cpu over longer periods.
i didn't look at any fan speeds while doing the stress-ng, will have to look later on that. they did start to make a lot more noise when running for a minute or so
the cpu in this system had some thermal paste from being mounted in a midi desktop case before, i just put the gelid cooler with the pre-applied thermal paste on top

I should use the generated heat and transfer it into a hot water tank ...

 

[luckylinux]

something i'd have to look into. for now the workloads don't seem to require 100% cpu over longer periods.
i didn't look at any fan speeds while doing the stress-ng, will have to look later on that. they did start to make a lot more noise when running for a minute or so
the cpu in this system had some thermal paste from being mounted in a midi desktop case before, i just put the gelid cooler with the pre-applied thermal paste on top

I should use the generated heat and transfer it into a hot water tank ...

I think Linus (LTT) did that quite a few Years back.

Not sure I would reccomend it due to all the extra Complexity, Parts (Pumps, Heat Exchangers, Pipes/Tubes, ...), Risk of Leakage (and Corrosion) and lack of flexibility if you ever have to pull out a Server to Fix something / Troubleshoot. I guess there are Water Cooling Quick-Connect Fittings and whatnot, but I think the extra work is not really worth the Trouble. Only Exception could be the Fan Noise ...

 

[dbram]

maybe we should go into the direction that Hetzner did for this boards. they have an open 2U kind of custom frame just one fan combined with a shroud voor memory, cpu and vrm doing the cooling. hot-isle, cold-isle however is less trivial for a homelab

 

[luckylinux]

maybe we should go into the direction that Hetzner did for this boards. they have an open 2U kind of custom frame just one fan combined with a shroud voor memory, cpu and vrm doing the cooling. hot-isle, cold-isle however is less trivial for a homelab

Not now that I bought so many SC512 Cases

 

[dbram]

Some efforts in Kicad for a pcb version of the prototype. Still very early stage and my pcb design skills are beginner level, through-hole components only for the moment.
production of pcbs at a Chinese pcb factory is done and shipped. waiting on arrival (estimated a week or two).

I probably should re-orient the esp32's usb connection and ethernet jack. this prototype should allow for mounting it on a modified io-shield and have the usb connection connected to the one of internal usb headers of the mb (that means that 5V standby power connection might conflict with usb power).



I probably should avoid restricting too much cooling air flow

 

[luckylinux]

Some efforts in Kicad for a pcb version of the prototype. Still very early stage and my pcb design skills are beginner level, through-hole components only for the moment.
production of pcbs at a Chinese pcb factory is done and shipped. waiting on arrival (estimated a week or two).

I probably should re-orient the esp32's usb connection and ethernet jack. this prototype should allow for mounting it on a modified io-shield and have the usb connection connected to the one of internal usb headers of the mb (that means that 5V standby power connection might conflict with usb power).

View attachment 44055

I probably should avoid restricting too much cooling air flow

You say PCB Design Skills at zero Level but I only played a bit around with Kicad and my PCB Skills are archived after 15 Years or so of University .

To be honest I made ZERO Progress here.

My Smartphone broke down and besides the Inconvenience I also need to repair that to recover some Data .

And on the Topic of this Board I also didn't decide what to do yet . Without knowing if you will Opensource it or not, I will probably have to go through all of the work and reinvent the Wheel and do pretty much what you did.

Probably with some Variations e.g. add a Fan Header for having Fan control via the OS, some 16 Pin IDC Connector to directly plug the Supermicro Front Panel (plus either 4 x 2 Pin for the Power/Reset/HDD LED/Power LED Connectors, or some IDC Connector to go to the ASRock Board, etc).

Plus of course all the Software Development. I stopped a couple Weeks ago testing the MS2130 HDMI -> USB Capture Device on a RockPro64 aarch64 Device and it works OK with ustreamer. I still would need to do the entire things after that (kvmd + the USB-HID Adapter).

Although for Security Reasons, even if you plan to go with MQTT, I'd set that up as Authenticated + Encrypted on say 20 different Containers each with their own MQTT Server and Network, so they are (as much as possible) decoupled.

EDIT 1: I was thinking of actually making 2 x IDC Connectors, one for the Main ASRock Front Panel Header, one for the AUX ASRock Header (where we could get the 5Vsb from, in particular).

 

[dbram]

Some updates:

• I'll put everything on a github page and opensource it soon, after some licensing is sorted out (I'll never take any responsibility whatsoever for any damages, liabilities)
  • some cleaning up to, esphome etc.
• the pcb factory deliverd, got the soldering station out
  • 
  • 
• tested on an old ASrock motherboard H110M-HDV for now
  • managed to power on and off using ATX PWR_SW
  • connected to the usb2 port below the onboard network jack, that is providing 5V standby

some lessons learned:

• use a dual row of pins e.g. 2x6, as the ones on motherboards for the connectors,
  • the 2x2 arrangement is more hassle during soldering, one of the individual 2pin headers fell out during soldering creating a solder-filled through-hole.
  • maybe even mimic the same layout as used on motherboards or similar to CSE512 connector. attention point is to make sure the signal is not handled differently on those custom connectors.
  • use 90° angle connectors, uses less space
• double, triple check the design AND check again when the boards come in, I somehow managed create an isolated 3.3V island
  • that resulted in the ethernet not receiving power.
  • by mistake i probably didn't upload the very latest pcb/drill exports when ordering the pcb.
  • for now, I left out the display and using that 3.3V line to connect the island on a header of the esp32-s3
  • it's not a real v0.1 if there are no jumper wires soldered on

edit: typo and test on older motherboard