Xeon Platinum P-8124 & ASROCK EPC621D8A

[iamnypz]

Hi,

I've read in some previous thread that the aforementioned motherboard is per default compatible with the 8124m but did anyone succeed in getting the P-8124 work with this board?

I did reach out to another forum and asked for a modded BIOS file with the 50653 microcode added (as mentioned in another thread on this forum) but no luck in getting the CPU work. All i got to was this during POST:

intel reference code execution b1

Any help on the subject would be extremely appreciated

 

[RolloZ170]

have heared it worked now
, can you please provide a Screenshot(HWinfo) of the P-8124 working ?

 

[iamnypz]

Big thanks RolloZ170 for helping me out on this one!

If some one do have a working BIOS (se issues below) please hit me up!

I got it partially working with a modded BIOS V1.60. I can't get it working with a GPU (GTX 1050 Ti LP) - which is sad, but the mobo got integrated VGA at least (I was planning to use mobo/cpu combo on unRAID with GPU passthrough to a gaming VM)

These are the specs for this current testing setup:

MOBO: ASROCK EPC621D8A with Modded BIOS ver 1.6 (tested a modded version of 2.18 but could not even POST with the P-8124 on that version. It worked with a Bronze 3106 though)
CPU: Intel Pentium P-8124 3GHz/3.4GHz
RAM: 1x Samsung 64GB RDIMM (M386A8K40BM1-CRC5Q)
CPU Cooler: Noctua NH-D9 DX-3647 4U with two 90mm fans @ 100%
PSU: Corsair CV650

As seen on screenshots below, the (at least MY) P-8124 runs HOT. For the record, I tested it in an open frame with the aforementioned cooler which is specced to 205W vs the 240W P-8124 beast, but I think its notable still.


Cinebench R23.200 Single Core, 756p


Cinebench R23.200 Multi Core, 19290p


Passmark (partial since missing GPU), 18416p


HW Info 64, note the max CPU temp of 106C (actually got 107C first time)


CPU-Z Proof

 

[Arphy]

Have you got geekbench score?
My P-8124 on Asrock MB-after modded, only gets a measly 680 single core score on geekbench5.

Generic - Geekbench

Benchmark results for a Generic with an Intel Xeon Platinum P-8124 processor.

browser.geekbench.com

Other 8124M usually produces around a 1000.

A slight base clock OC to 102MHz dramatically raise the score to 800. I do not undrstans why such a small increment give such an improvement.

Still it is miles slower than 8124m.
What should I look to improve the performance?

 

[iamnypz]

Have you got geekbench score?
My P-8124 on Asrock MB-after modded, only gets a measly 680 single core score on geekbench5.

Generic - Geekbench

Benchmark results for a Generic with an Intel Xeon Platinum P-8124 processor.

browser.geekbench.com

Other 8124M usually produces around a 1000.

A slight base clock OC to 102MHz dramatically raise the score to 800. I do not undrstans why such a small increment give such an improvement.

Still it is miles slower than 8124m.
What should I look to improve the performance?

No, unfortunately I couldn't get it stable at all. Would only run for like sub 30 mins then all the suddenly the machine would freeze so i decided to retire it. I'm going for a 8124m later on i think.

 

[RolloZ170]

No, unfortunately I couldn't get it stable at all.

skylake-ep is not soldered. need delid and new TIM.

 

[iamnypz]

skylake-ep is not soldered. need delid and new TIM.

Agree. But that does not explain why I can't use any PCIe-devices with that CPU. It simply won't POST with any PCIe device.

 

[RolloZ170]

that does not explain why I can't use any PCIe-devices with that CPU. It simply won't POST with any PCIe device.

another broken arrow.

 

[iamnypz]

Have you got geekbench score?
My P-8124 on Asrock MB-after modded, only gets a measly 680 single core score on geekbench5.

Generic - Geekbench

Benchmark results for a Generic with an Intel Xeon Platinum P-8124 processor.

browser.geekbench.com

Other 8124M usually produces around a 1000.

A slight base clock OC to 102MHz dramatically raise the score to 800. I do not undrstans why such a small increment give such an improvement.

Still it is miles slower than 8124m.
What should I look to improve the performance?

Got another one to work now. Not much better scores than you though...

Generic - Geekbench Browser

 

[iamnypz]

UPDATE 21 AUG 2022:

Sorry for wall of text, felt I needed to decompress.

So, since i apparently hate money, I ordered another Platinum P-8124 from a German seller this time (Damned Yankees can't be trusted ) for about 150€ shipped. I contacted the uttermost helpful member RolloZ170 and told him that I got another P-8124 which didn't cook itself due to faulty IHS or whatever it was.

We tried some BIOSes, the best out of the box was a modded 2.18 with added B1. It did boot into windows and the GPU (or any PCIe card for that matter) showed up! Nice! Well, not so nice actually because the CPU didn't turbo boost at all. When under load, it went to 1,99GHz straight, even if it actually boosted to almost 3GHz when just starting windows. There were no thermal issues or so. The CPU just bent over when stressed. It could go from 1,4GHz to about 3GHz when doing background tasks according to task manager but no dice when there was some load.Back to the drawing board.

I then tried a 1.63 BIOS that RolloZ170 had modded and added microcode for B1 stepping. This was really promising! I couldn't downgrade from 2.18 to 1.60 with the P-8124 so I had to switch to my vanilla stock Bronze 3106, downgrade then back with the P-8124. It didn't boot straight out of the box with a GPU though, so I had to get my hands dirty in BIOS.

Before that i wanted to check if the CPU did boost under load, so I ran a passmark benchmark and I got about 18 000 cpumark and it did turbo boost! (Previous, without turbo boost enabled I got about 13 000 cpumark @ 2GHz) Here's a link to my two current benchmarks. The previous - faulty CPU - actually did better, 18400 cpumark.

After ALOT of tweaking with the BIOS settings i now have it all under control. A little bit of testing is left, maybe some settings could be reverted back to default, but as it is working now i'm not too keen to revert certain settings back to default.

Settings in BIOS that have been altered from default:

Chipset-> Every PCIe-link is changed from auto to Gen3 (image is from the manual, not current settings in my bios) Could possibly be reverted to auto but i will leave it there for now since it's working.


Everything is UEFI booted. The board can't initialize anything if it's in legacy mode with the P-8124. CSM should be disabled and change from AUTO to UEFI on the rest. Remember to have a UEFI install of your OS, I missed that


I did notice when I installed Windows that Win10 has a hard time finding correct drivers so the NIC was down under the install. Make sure you have NIC / Realtek Audio on a USB when first booting your clean install.

After doing some benching i was digging around in the device manager and there's ALOT missing / in error state which Win10 can't possible find out what it is. PCIe devices, Base devices etcetera. Here's how to fix that:

Double click in the device->Details, choose "Hardware IDs". The numbers after "VEN_", in this case 8086 is the vendor number (actually it's Intel - 8086 - your get it? ) and the numbers after "DEV_" is the device number.


Remember those and head to PCILookup.com and enter them respectively. You'll get to THIS page and it says it's "Sky Lake-E CBDMA Registers"

Google that and you probably end up on THIS SITE. What are you waiting for, download the latest "Intel Chipset Device Software" (in this case 07 MAY 2020). Voila! Now you have your freaking P-8124 up n running as it should!

What about the benchmarks then? Well here they are:

CPU-Z I actually got 101% of Threadripper 1950X the first time, but it seems to vary a bit.

Cinebench R23:


Geekbench 5:


I really hope this sheds some light on this very particular piece of hardware. I do not encourage anyone to actually buy this CPU if you have the same mobo - EPC621D8A - Go for the Xeon Platinum 8124m instead which seems to 1) be totally compatible with the board and stock bios and 2) better performance.

EDIT: I almost forgot: a BIG, BIG thanks to RolloZ170 for being there and answering all the stupid questions I had and taking his time to actually help an anonymous brother in need!

Have a good one,

 

[iamnypz]

UPDATE 22 AUG 2022

Since I've only used spare parts for the build, I wanted to know how the system behaved with 4x 2666 DDR4 REG DIMMS. Well it seems like the P-8124 liked it

27 000+ CPUMARK! :O

Edit:


Geekbench 5 Score

Single-Core Score:
926

Multi-Core Score:
13835

Geekbench 5.4.5 Tryout for Windows x86 (64-bit)

 

[jasonsansone]

I gotta try changing all bios settings for the asrock epc621d8a because windows runs fine, but linux craps out (especially proxmox) so maybe it's my bad luck. If anyone can get proxmox working let me know

How does Proxmox crap out? I have ordered 3x P-8136 and X11SPL-F’s for a proxmox cluster. Hoping I don’t regret the P-8136 decision vs a retail chip.

 

[jasonsansone]

Ive been using the asrock EPC621D8A board. When I start it up with the custom bios using the additional microcode it will just randomly crash for no reason. It sometimes runs for 3 seconds to 15 and rarely past a minute. It reports cpu configuration errors in the sel logs and hard cycles and goes back to the post screen

Which Proxmox version and which kernel?
Which microcode release for 50653?

 

[jasonsansone]

I did a little searching on Geekbench for P-8124 and P-8136. There are results submitted using various Linux flavors including Debian, Ubuntu, and Fedora. Obviously someone out there has figured out a way to boot Linux and have it stable enough to run a geekbench test. I’ll test and report once my chips arrive. This gives me some hope there is a way to make it work.

 

[jasonsansone]

So of course the 8275cl works like a beauty... now to find a way to get 8124's to run without crashing running proxmox.. I just want it to run smoothly for me

Just a few random ideas, of which you may have tried them all. On your Proxmox install, try these things while it is stable with the 8275cl's before testing the 8124's:

1. Update the Proxmox install to all newest packages.
2. Install the new 5.19 kernel.
3. Install the Intel microcode package for Debian.

If it keeps crashing, post the kernel dump log with backtraces for review.

 

[jasonsansone]

After updating bios with custom microcode and then putting the p-8124 back in it seems to still be failing. The issues I get are the same as before. SEL logs on IPMI report Critical Interrupts: "Bus Currectable Error - Asserted" and Processor - Configuration Error - Asserted which results in a hard crash. I can try to upload the bios file i'm using.

That isn’t the crash information from Debian but what is reported by the BMC. You should be able to view the dump with the remote console, but here is an alternate method.

 

[jasonsansone]

The last error I get and it somehow is showing up when no pci slots are even populated and all I have is the cpu, ram, motherboard, psu and that's it. ethernet is plugged in to a single port and the ipmi dedicated port...

http://imgur.com/qyKEArg

the device ID are the PCIe root controller.

what is storage? NVMe?

what are the BIOS settings related to PCIe?

 

[Stephan]

Just a cautionary note about all those cheap 81xx 82xx Xeons:

There is literally a ton of them out there on ebay for cheap, but I think 50% or so are defective in one way or another. The failure modes vary. I have a hunch they were ejected from the datacenter for a reason. Sometimes the board they sat on might have been defective and not the CPU, but the datacenter didn't bother to check so entire system gets dumped.

Chips from the west coast of the USA all seem to come from a recycler and are badly dinged on corners due to their weight when thrown into a bucket. The 3647 socket itself is delicate. Frankly if you get PCIe errors like that, something is very very wrong. If it doesn't show up with your Bronze CPU, it hints towards the CPU. ProxMox should not require anything special like installing Debian raw and then ProxMox on top, instead of just using their ISO. Also there shouldn't be any bugs resolved in microcode since maybe 2 years. If anything, the new workarounds only introduce MORE bugs.

If the CPU isn't the reason, I had to RMA an Asrock Rack 3647 board because of a DIMM socket signal integrity problem. So board can be an issue too, prepare to swap around alot of gear to figure it out.

What to do... When I test new hardware, first thing I do is run Memtest Pro by Passmark for a few hours. Very good to detect RAM and board errors because the commercial version appears to know about the CPU's IMC and can display fishy errors. Then I boot from USB stick that also starts rasdaemon and then I run "nice -n19 stress-ng --vm $(nproc) --vm-bytes 86% --vm-keep --vm-populate --vm-madvise willneed --verify -v -t 4h --tz --perf" while simultaneously tailing the system log to watch for rasdaemon errors. This will show errors that Memtest might not have been able to trigger in the CPU/RAM/board complex. To test PCIe, I recommend to install a ConnectX3 with two ports into a slot and configure them for loopback in Linux using namespaces.

Server:

#!/bin/sh

echo Setup
ip netns add ns_server
ip netns add ns_client

ip link set enp1s0 netns ns_server
ip netns exec ns_server ip addr add dev enp1s0 192.168.1.1/24
ip netns exec ns_server ip link set dev enp1s0 up
ip netns exec ns_server ethtool -s enp1s0 speed 56000 autoneg off

ip link set enp1s0d1 netns ns_client
ip netns exec ns_client ip addr add dev enp1s0d1 192.168.1.2/24
ip netns exec ns_client ip link set dev enp1s0d1 up
ip netns exec ns_client ethtool -s enp1s0d1 speed 56000 autoneg off

ip netns exec ns_server iperf -s -B 192.168.1.1 -w 16M

echo ""
echo Teardown

killall iperf
killall bwm-ng
sleep 0.5

ip netns del ns_server
ip netns del ns_client

echo Done

exit 0

Client:

#!/bin/sh

while :; do
    ip netns exec ns_client iperf -c 192.168.1.1 -B 192.168.1.2 -P 2 -w 16M -t 300
    sleep 0.1
done

Here I am using a 50cm 56 Gbps-capable "FDR" Mellanox cable with suitable card, an Oracle 7046442 rev A3 or A4 or A5, flashed to generic MCX354A-FCBT with custom Lenovo 2.42.5032 firmware mashup. I.e. to really push the PCIe slot the card sits on by creating traffic between the physical interfaces. Card has to be cooled with some airflow. The mlx4 driver reports "63.008 Gb/s available PCIe bandwidth (8.0 GT/s PCIe x8 link)" so if you max out the 56 Gbps link speed, PCIe 3.0 x8 will be close to the limit. In theory 7880 MByte/s of slot per direction vs. 7000 MByte/s of a single 56 Gbps link per direct. Traffic has to pass PCIe in+out so both in+out of each x8 should be quite busy.

See if one of the slots craps out...

 

[RolloZ170]

Frankly if you get PCIe errors like that, something is very very wrong. If it doesn't show up with your Bronze CPU, it hints towards the CPU

this is specialy a issue with OEM stepping B1 not regular SKUs.
B1 is not B0 anymore and still far away from H0 stepping, no support from Intel because OEM which run special code as usual.
so unfotunaly we should handle P-8124/P-8136 as ES in the future.
my chinese supplier told me that this B1 SKUs have always PCIe errors and many more issues if the board/OS don't have special support for this stepping.

 

[jasonsansone]

UPS and USPS claim my motherboards and processors should deliver today. Hopefully I’ll soon be able to join in the testing and report.