Hang on Supermicro X11SCH-F when running Xeon E-2186G with all cores

[ph0t0nix]

I've got a home server based on a Supermicro X11SCH-F motherboard with 2× 32GB DDR4 2666MHz RAM. I bought it new several years ago, with a Core i3-9300 F CPU and it worked fine. About 2 years ago, I needed more power and bought a Xeon E-2236. Again, it worked fine.

A few weeks ago I decided I wanted to try hardware transcoding in Jellyfin, for which I needed a (i)GPU. So I bought a second hand E-2186G which I found for a reasonable price. And this time it's a headache .

TLDR: Linux won't boot properly when I enable all 6 cores in the BIOS settings. With only one core and hyperthreading enabled, the CPU works fine.

The longer version:

The BIOS recognizes the CPU (correct frequency and nr. of cores). The boot procedure works fine, including loading Grub up to the point where the Linux kernel takes over. Whenever I have all cores enabled in the BIOS, the kernel hangs. Sometimes just a black screen, sometimes with a kernel message about loading smpboot:
smpboot: x86: Booting SMP configuration:

Here's a screenshot:



If I have only one core (with or without hyperthreading) enabled in the BIOS the machine boots into the OS without problem.

Here's what I have done so far, all to no avail:

• Updated the BIOS to the latest version (v2.5 dated 07/05/2024), as well as the latest IPMI firmware (v1.74.15)
• Reset the BIOS to Optimized Defaults (at least) two times.
• Removed the E-2186G, replaced it with the E-2236, which booted fine. Put the 2186G in again.
• Tried to boot via both UEFI and BIOS;
• I've tried to boot my regular OS (Ubuntu 24.04 server), but also various other versions of Ubuntu (including the latest 24.10) and other Linuxes (all using a USB stick with Ventoy). All fail to load unless I restrict the CPU to one core.
• I even tried the Windows 11 installer, but that failed to load as well when all cores were enabled.
• I removed the E-2186G again, cleaned its contacts with alcohol and put it back: still no luck
• Although I don't suspect the RAM to be a problem I started a memtest run, but that seemed all fine (although I didn't let it finish). Interestingly, memtest did seem to use all 6 cores, even if I restrict the nr. of cores in the BIOS .

When running on 1 core, the iGPU works fine and I get hardware transcoding in both Jellyfin and Nextcloud Memories. So the iGPU settings seem to be fine.

Unfortunately, I don't have another board in which I could try the 2186G. but I'm starting to think something may be wrong with the CPU. Any suggestions or help is much appreciated.

 

[ph0t0nix]

Well, it turns out I have to adjust the title: I just tried to boot with 5 out of 6 cores and that works... .

It's bed time here now, I'll try to find time for more tests later this week.

 

[RolloZ170]

I just tried to boot with 5 out of 6 cores and that works... .

have had a 8176 which boots only with 10 cores, 11(and more) are defective,
can't figure out because there was no bitmask feature in the bIOS.

 

[RolloZ170]

Might need a (manual) microcode update

Xeon E-2236 was working, so BIOS should be up to date
2186G is not mobile/embedded processor , just iGPU

 

[Stephan]

Try to disable one core at a time using what I wrote in https://forums.servethehome.com/index.php?resources/machine-check-exception-mce-workaround.55/ i.e. use isolcpus for both HT thread siblings per core in boot loader, and then set core to off using echo 0 to /sys/devices/system/cpu/cpu${c}/online once Linux is booted.

 

[ph0t0nix]

Thanks for the input! I'll give it another go later next week as this weekend's taken up by the Ubuntu Summit .

Or I might just contact the vendor and return the CPU...

 

[ph0t0nix]

Unfortunately, I couldn't get @Stephan's suggestion with the isolcpus kernel parameter to work, so I decided to return the CPU.

Thanks for the suggestion, though. I learned something new

 

[Stephan]

Sellers usually know when the things they sell are defective. CPU will go on sale again, until a greater fool buys it, leaves it in a drawer for 19 months and when it is time for the project, seller disclaims any warranty.