BKHD-NVR-N5105 Motherboard - C States and Crashes

[xpc]

Hi all, wondering if anyone can help with some c state questions, and some crashes that has resulted from my tinkering

I recently bought the BKHD N5105 motherboard from the Hi ElectronicC store on Ali Express - this seems to be identical to the popular Topton N5105 board, that one seems to have the model number CW-N5105-NAS (does the CW stand for CWWK?) but it seems identical to my BKHD one - e.g. has a JMB585 SATA adaptor and 4 x intel I226-V LAN ports. Out of interest, what's the relationship between Topton, CWWK, BKHD etc - seems a lot of people make these boards.

Anyway - intention is to build a low power server with TrueNAS or similar. The motherboard is actually running fine day to day but I notice from PowerTop that the CPU package is only ever running in C2, being as this will be on 24/7 and with the cost of electricity these days I'm trying to reduce the consumption as much as possible, so I used powertop to apply some tunables - that's the first time I noticed weird crashes where the HDMI display shows a scrambled multicolour image for a while and sometimes reboots. It seems it does this under a couple of circumstances at least:

• If I enable sata link power management for host0 (which is the onboard SATA, not the JMB585) it seems to always die
• If I disable the onboard sata controller in the BIOS and launch powertop it seems to die

Also another one - if I enable runtime PM for the Intel I226-V I get basically no network and messages regarding tasks being blocked with call traces output to the console.

So yeh - basically the questions are:

• What's with the random multicoloured crashes?
• Has anyone got this motherboard to run at a lower package C state?

I do have a feeling this is related to both the onboard SATA and the JMB585 - if I disable both and boot to a USB stick arch linux install I may get a crash when I load powertop, but if I don't then I will see the CPU has gone to C8. I did once have the JMB585 on and the onbaord SATA off and I think I got to C3.

 

[xpc]

Just ran 3 passes on MemTest86 and all is fine, the onboard SATA and JMB585 were disabled at the time as well.

I did a bit more testing today, very raw notes below!

Attempt 1 - onboard SATA enabled but port SATA port 2 disabled, that achieves c3 - 13.9w. Seems to be unstable on certain bootups though:

• boot 1 was ok
• boot 2 garbled screen straight after login
• boot 3 garbled after powertop
• boot 4 was ok even with repeated powertop, oh no it blew up midway through powertop, then it rebooted

Attempt 2 - Onboard SATA enabled, ports 1 and 2 also enabled, stuck on c2 - 14.8w, appears to be stable

Attempt 3 - Onboard SATA and JMB585 disabled, can reach c8 - 12.8w until it died on powertop (about 10 seconds)

So it seems to me that if I disable any of the ports (well I only tried port 2) or the SATA controller itself I get instability. Might try with only the JMB585 disabled actually, see what happens there.

 

[vvkvvk]

You can disable the JMB585 by disabling the PCIe root port, see my post here https://forums.servethehome.com/index.php?threads/topton-nas-motherboard.37979/post-399748 and this post why I re-enabled it and only run default settings and C-states: https://forums.servethehome.com/index.php?threads/topton-nas-motherboard.37979/post-400347

 

[xpc]

You can disable the JMB585 by disabling the PCIe root port, see my post here https://forums.servethehome.com/index.php?threads/topton-nas-motherboard.37979/post-399748 and this post why I re-enabled it and only run default settings and C-states: https://forums.servethehome.com/index.php?threads/topton-nas-motherboard.37979/post-400347

Aha awesome thank you! You have very much helped me! Scrolled down from your second post and found this post which describes the video corruption and a fix - I've enabled that and can now run with the onboard SATA disabled and only the JMB585 enabled. Unfortunately it seems the JMB585 does not allow the system to go any deeper than C3 (as opposed to the ASM1166 which seems to allow this according to this article) however at least it's a small saving and it's interesting to have fixed it regardless!

My current BIOS settings I think are now defaults with the exceptions of:

• C states enabled
• SAGV disabled
• Serial port disabled
• Onboard SATA disabled
• PCI E root ports 5, 6 and 8 disabled (NICs and second M.2)
• Wake on LAN disabled

Not sure that the serial port, PCI e root port and WOL disable save any power but I did them anyway.

Disabling SAGV also allows me to run powertop with the auto tune flag, basically it seems SAGV was responsible for all the issues where my video output became corrupted.

Overall the system now seems to run between 13.4 and 14 watts idle-ish, this is with all the powertop tunes applied, seems the devices stopping lower C states are the I226-V ethernet, the Phison E16 controller in my M.2 drive, the JMB585 and the JasperLake UHD card - not sure what to expect really tbh, seems reasonable that at least some of those would prevent lower c states anyway, not sure about all of them.

 

[xpc]

Interestingly whilst poking around in lspci I notice that the JMB585 is only connected via what seems to be 8GT/s x2:

LnkCap:    Port #0, Speed 8GT/s, Width x2, ASPM not supported

And I also notice the NVMe is connected by 16GT/s x4:

LnkCap:    Port #1, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 unlimited

Though the link status seems to show downgraded:

LnkSta:    Speed 8GT/s (downgraded), Width x1 (downgraded)

I haven't checked the second M.2 slot yet, but I wonder if this is also connected in the same manner? If it is then you could get an ASM1166 6 port SATA controller and benefit from deeper c states and an increased link speed, and still have an M.2 slot for an NVMe for your OS.

I didn't think the M.2 slots were this well connected - the listing page on AliExpress states they are PCI3.0x1.

Am I wrong somehow? Is the LnkSta "Downgraded" figures the max? it seems the max read I can get with hdparm on my rocket 4x4 is about 600MB/s so I assume we are indeed working only at 3x1, presume that somehow the LnkSta figure is the accurate maximum then in this case?

I do notice that the JMB585 though is also downgraded:

LnkSta:    Speed 8GT/s, Width x1 (downgraded)

So then actually if I wanted to use an ASM1166 on the M.2 interface I'm no worse off - it seems it is 3x1 whatever you use? Seems a bit naff being as 5 ports of 6Gb/s SATA 3 on the JBM585 tops out at 3.75GB/s and 3x1 is just under 1GB/s - am I missing something there?

 

[vvkvvk]

Please update the thread if your setup manages more than 2 days of uptime, I never got past that.

 

[xpc]

Please update the thread if your setup manages more than 2 days of uptime, I never got past that.

So it's been up now for 2 days and 8 hours during which time it has been doing quite a bit, so it seems stable.

 

[vvkvvk]

So it's been up now for 2 days and 8 hours during which time it has been doing quite a bit, so it seems stable.

With deeper C-states than 3 and JMB585 disabled?

 

[xpc]

With deeper C-states than 3 and JMB585 disabled?

No I only have the onboard SATA disabled, the JMB585 is enabled as I need it for a SATA drive I have and will need it for future additional drives. I was tempted to disable it and purchase an ASM1166 M.2 to 5 x SATA purely to allow deeper C states because I'm finding it fun to see how low power the board can go but I doubt I will do it because it seems a bit ridiculous!