ES Xeon Discussion

[alex.forencich]

Thanks, good to know. It's a big thread, I did see that supermicro is no good for ES parts, but I guess I missed the comments on Tyan.

What about the ASRock Rack SPC741D8-2L2T or GIGABYTE MS03-CE0? I'm trying to find something relatively economical that has lots of usable PCIe x16 slots, and I don't really need more than one CPU. MS33-AR0 is not very interesting because half the slots are blocked by the RAM. MS73-HB1 I guess might be a decent option, but I don't really need two CPUs and with only one CPU installed presumably some of the PCIe slots will not be usable. I suppose ASRock Rack SPC741D16X-2T might also be an option worth considering, similar setup as the MS73-HB1.

 

[RolloZ170]

What about the ASRock Rack SPC741D8-2L2T or GIGABYTE MS03-CE0?

ASRock C741 series supports ONLY E-Stepping ES/QS/PRD processors, same than supermicro/Tyan.
(the BIOS of ASRockRack/Gigabyte/Tyan can be modded,)
Currently ALL Gigabyte C741 boards support D0 stepping SPR-SP ES. if you update BIOS, don't update Intel SPS/ME.
edit: Intel SPS/ME allows SPR D0 up to latest BIOS

 

[alex.forencich]

Got it. Are there any options other than gigabyte that are worth considering? Or is that the only option at the moment that doesn't involve modding the BIOS?

Also, is there a summary/compilation of this sort of information somewhere?

 

[RolloZ170]

Got it. Are there any options other than gigabyte that are worth considering? Or is that the only option at the moment that doesn't involve modding the BIOS?

8470 ES Q0KL stepping E0 2.1Ghz 52cores.
runs on gigabyte,supermicro and asus. asrock rack not tested.

 

[mumim0]

Hello everyone, I just bought an ASRock EPC621D8A from ebay. I also bought a 8251 QS (QPLC). I put it in my motherboard and plug in my psu and the leds turn on but the computer will not start when I press the power button. Did I mess up and pick the wrong CPU? I also bought two more: QPKD and QMQ9. If it is my CPU that is faulty or incompatible, do you think those two QS CPUs would be fine? Thanks in advance

 

[RolloZ170]

I just bought an ASRock EPC621D8A from ebay. I also bought a 8251 QS (QPLC).

If it is my CPU that is faulty or incompatible, do you think those two QS CPUs would be fine?

all of them should work. processor not properly installed or other things wrong (bad mobo/cpu/psu)

 

[mumim0]

all of them should work. processor not properly installed or other things wrong (bad mobo/cpu/psu)

Hello, thank you for your response. I just tried a different cpu that came in today (the QS QPKD) and my computer is not booting when I short the power switch pins. Do you think it's possible that it will not start due to an outdated BIOS? Here is a pic of my setup. When the PSU is turned on, two green LEDS are on.

 

[Kizune]

Hello, thank you for your response. I just tried a different cpu that came in today (the QS QPKD) and my computer is not booting when I short the power switch pins. Do you think it's possible that it will not start due to an outdated BIOS? Here is a pic of my setup. When the PSU is turned on, two green LEDS are on.

Where is the cpu cooler? I assume you are not trying to turn on your system with CPU just laying in the socket, right? Right?…

 

[mumim0]

Where is the cpu cooler? I assume you are not trying to turn on your system with CPU just laying in the socket, right? Right?…

Right sorry, I took that off. The cooler is an AlphaCool Eisbaer Aurora Pro 240mm, but even without it installed I won't get anywhere... even the PSU fan will not start when trying to power on

 

[RolloZ170]

Hello, thank you for your response. I just tried a different cpu that came in today (the QS QPKD) and my computer is not booting when I short the power switch pins. Do you think it's possible that it will not start due to an outdated BIOS? Here is a pic of my setup. When the PSU is turned on, two green LEDS are on.

the LEDs are power led and BMC ASPEED heartbeat led. should start blinking after some seconds.
If the FAN is not turn on the CPU gets no power and no x86 code is executed, so can not BIOS.
BIOS ver. is printed on the sticker of the socketed BIOS chip.
i see you use not the CPU carrier/frame, maybe bent some socket pins now ?
installation of CPU needs the screws 3&4 turned to the end to lift up the leaf-springs topmost.

 

[mumim0]

the LEDs are power led and BMC ASPEED heartbeat led. should start blinking after some seconds.
If the FAN is not turn on the CPU gets no power and no x86 code is executed, so can not BIOS.
BIOS ver. is printed on the sticker of the socketed BIOS chip.
i see you use not the CPU carrier/frame, maybe bent some socket pins now ?
installation of CPU needs the screws 3&4 turned to the end to lift up the leaf-springs topmost.

BIOS ver. seems valid and cpu socket pins seem okay, pictures:

http://imgur.com/a/7aooQbo

I will buy the cpu frame since the mobo did not come with it when i bought it
EDIT: By the way do you have any recommendations for where to buy the cpu carrier?
EDIT2: I ended up buying this one: SuperMicro BKT-0126L CPU Processer Package Carrier 10122321 | eBay

 

[sam55todd]

The question RolloZ170 is asking - does it have a life (right led signal) in BMC if you power it without CPU?
Then next debugging step - can you connect to BMC? (still without CPU installed).
Only after all above are fine - we can start CPU install/debug stage
(including checking boot codes even if there are CPU, RAM or peripheral related issues)

 

[Syr]

So I also got one of kizune's E3 8461Vs off ebay, but try as I might, it doesnt seem to have AMX enabled.
HWINFO64 doesn't show AMX (See screenshot), /proc/cpuinfo doesnt show it under flags either:

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm pni pclmulqdq dtes64 est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave osxsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl avx512vbmi umip avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid ibrs ibpb stibp ssbd

Trying to compile Intel's AMX demo (GitHub - intel/AMX-TMUL-Code-Samples: Code samples related to Intel(R) AMX) with their default -march=native on gcc-12 doesnt work, claiming that the platform doesnt support it. Overriding it with -march=sapphirerapids does get it to compile, but it crashes upon trying to run the build program.

Any tips for getting AMX enabled? I have it in an X13SEM-F since its supposed to be for a portable system. I tried updating the bios from 1.1 (what it shipped with) to 1.4, which did fix a number of significant bugs with the platform including an inability to turn off, but did not fix the missing AMX instructions. I see that sam55todd previously got AMX working though, also on a supermicro board and using one of the same E3 8461Vs from Kizune, a few pages back.

Also here is the version of microcode running in bios 1.4 for that board:

Date: 09.05.2023
CPU signature: 000806F8h
Revision: 2B0004B1h
CPU flags: 87h
Checksum: 515E81B0h, valid
Extended header entries: 5
Extended header checksum: 68FF5210h, valid
CPU signature #1: 000806F8h
CPU flags #1: 87h
Checksum #1: 515E81B0h, valid
CPU signature #2: 000806F7h
CPU flags #2: 87h
Checksum #2: 515E81B1h, valid
CPU signature #3: 000806F6h
CPU flags #3: 87h
Checksum #3: 515E81B2h, valid
CPU signature #4: 000806F5h
CPU flags #4: 87h
Checksum #4: 515E81B3h, valid
CPU signature #5: 000806F4h
CPU flags #5: 87h
Checksum #5: 515E81B4h, valid

 

[Kizune]

So I also got one of kizune's E3 8461Vs off ebay, but try as I might, it doesnt seem to have AMX enabled.
HWINFO64 doesn't show AMX (See screenshot), /proc/cpuinfo doesnt show it under flags either:

Hmm. Just got the fresh CPU out of the box, the motherboard is Supermicro X13SEI-F, BIOS 1.4 (latest) factory default settings.
Windows 11 Pro, again - no special settings or drivers installed, fresh copy with the standard chipset drivers downloaded from Supermicro.
That all is very concerning. I would try to reset BIOS settings to the factory default and make sure you installed the chipset driver from Supermicro support page for the motherboard. If you think that the problem is with the CPU - ping me, i will replace it with the one that i just tested.

 

[Kizune]

Also on your screenshot it shows a number of other technologies as unsupported. Not even disabled - unsupported. That is strange. Like VMX - it can be disabled in BIOS but then HWInfo would show it as a red (supported but disabled) rather than gray (unsupported). Also SMX, SGX and TME shown as unsupported. All this is very strange.

 

[RolloZ170]

@Kizune @Syr can anyone provide this area ? TMUL is required for AMX-TMUL-Code-Samples, this is accel.

 

[Kizune]

@Kizune @Syr can anyone provide this area?

Sure! Here it is:

 

[Syr]

Hmm. Just got the fresh CPU out of the box, the motherboard is Supermicro X13SEI-F, BIOS 1.4 (latest) factory default settings.
Windows 11 Pro, again - no special settings or drivers installed, fresh copy with the standard chipset drivers downloaded from Supermicro.
That all is very concerning. I would try to reset BIOS settings to the factory default and make sure you installed the chipset driver from Supermicro support page for the motherboard. If you think that the problem is with the CPU - ping me, i will replace it with the one that i just tested.

Ok, I've tried the factory reset of the bios options and installing the chipset driver (at least on win 10 22H2, since that is the drive I have in it right now. I also have a ubuntu 22.04 drive but its not currently installed. Also just to double check, I tried the AMX-TMUL samples in ubuntu 22.04 in WSL2 - same behavior to bare metal ubuntu 22.04). Neither of those have fixed it.

@Kizune @Syr can anyone provide this area ? TMUL is required for AMX-TMUL-Code-Samples, this is accel.

Here are my chip's turbo ratios, incl TMUL which appear to be identical to Kizune's:

 

[Syr]

Right sorry, I took that off. The cooler is an AlphaCool Eisbaer Aurora Pro 240mm, but even without it installed I won't get anywhere... even the PSU fan will not start when trying to power on

Just another thing to check, but you might over over/under torquing or unequally torquing the mounting screws. The sockets since lga3647 can sometimes be varying degrees of finicky - depending on manufacturing tolerance - about having just the right amount of mounting pressure evenly distributed over the whole socket, otherwise you run into issues such as the CPU not being detected, missing memory channels, or missing pcie lanes. Same goes for AMD's SP3/SP3r2 sockets, hence the included screwdriver with the retail threadripper chips. Its something I've run into before, and I believe Patrick or at least someone on the STH crew have mentioned it being a problem before as well in an article, so it could be worth checking.

Also for that reason, running an lga3647/4189/4677 chip without a cooler wont work because the cooler is designed to provide all the mounting pressure, unlike in most sockets with a retention bracket (except SP5 - the retention bracket there is just to stop the chip from falling out of the socket/coming out with the cooler, AMD is using the cooler to provide adequate mounting pressure there as well).

 

[Kizune]

Ok, I've tried the factory reset of the bios options and installing the chipset driver (at least on win 10 22H2, since that is the drive I have in it right now. I also have a ubuntu 22.04 drive but its not currently installed. Also just to double check, I tried the AMX-TMUL samples in ubuntu 22.04 in WSL2 - same behavior to bare metal ubuntu 22.04). Neither of those have fixed it.




Here are my chip's turbo ratios, incl TMUL which appear to be identical to Kizune's:View attachment 32011

All I can offer at this point is that you DM me your address and I send you a replacement chip hoping that the problem is not in the motherboard. Seems like our setups are as similar as it can be and at that I can’t see any setting or upgrade that might affect the missing instructions set.