ES Xeon Discussion

[Shiitake]

tried the USB in other PC ? i used sandisk 128GB stick.
maybe you have 4GB faked to 16GB USB stick ??

Yes, that's right. It was confirmed that it was installed normally on another PC.

I own one Samsung 64GB product and one Transcend product, and since i were purchased from a Korean distributor, I don't think they are counterfeit products.

If I had purchased it from eBay or China, I think I would have been suspicious of the USB first, as you mentioned.

I'm worried that I'm messing up the thread by dragging in a case that's too hard to find. haha too sad..

 

[RolloZ170]

This is what happens when I try to install W11 on my MS73.

After completing the BIOS post -> Press the F10 key to select the OS USB device -> The loading part spins around and then a BSOD appears immediately.

i noticed something important:
only if i try to boot the USB via F10 i have no display.
if i set the USB as first device first in BIOS, i have onboard VGA display.
please try that too, NOT F10, 1st Boot device = UEFI USB part1 (or similar)

you could also downgrade to earlier BIOS(BMCWebGUI,not Afu) to check if not a bug issues your problem.

 

[Shiitake]

i noticed something important:
only if i try to boot the USB via F10 i have no display.
if i set the USB as first device first in BIOS, i have onboard VGA display.
please try that too, NOT F10, 1st Boot device = UEFI USB part1 (or similar)

you could also downgrade to earlier BIOS(BMCWebGUI,not Afu) to check if not a bug issues your problem.

Mr. RolloZ170, thank you for your sympathy and advice regarding my problem.

I tried many things based on my experience and the advice you gave, but unfortunately, it didn't work out as we expected. As with previous ES CPUs, I changed the C State and it was the same.

After worrying about this problem for a few days, I thought about it a little before going to bed yesterday. Is there something wrong with the virtualization-related options or command set?

The first thing that came to mind was NUMA and vNUMA. I turned off vNUMA first.
Then, I was able to run the Windows 11 Pro installation media as if nothing had happened. This is something to be happy about, right?

I remember that this part was disabled by default, so most people may not have seen an error like mine. Of course, if I turn off this option, the L3 Cache will be cut in half, but since the reason for moving to W11 is to escape from W10's CPU grouping, I decided to take it.

Once again, I would like to thank you for your interest in my problem. I also leave a screen image of that option.

 

[RolloZ170]

I remember that this part was disabled by default, so most people may not have seen an error like mine. Of course, if I turn off this option, the L3 Cache will be cut in half, but since the reason for moving to W11 is to escape from W10's CPU grouping, I decided to take it.

Once again, I would like to thank you for your interest in my problem. I also leave a screen image of that option.

nice find. try to set this option after w11 installation, maybe only a issue with installation media(WinPE) ???

 

[Shiitake]

nice find. try to set this option after w11 installation, maybe only a issue with installation media(WinPE) ???

That's right, sir. I thought that might be the case too. I had that hope haha.

So, after installing Windows, I reinstalled all drivers, updated the latest updates, and changed vNUMA to enable again, but unfortunately, I was greeted with a kernel panic.

Anyway, I achieved what I wanted. In Windows 10, more than 64 multiprocessors are grouped, so except for programs that can distribute the load by group, only one group is loaded.

In Windows 11, this setting has been changed again so that regular programs are automatically load separated.

I am attaching two photos. In Windows 10, which I was using until yesterday, when benchmarking 7zip with QYFQ 2S 96C HT ON, the load is concentrated on only one node (group) and the load is not distributed properly. However, Windows 11 even utilizes spare resources such as photos. If the 7zip thread was capable of up to 192 threads, I think the score would have been higher. Anyway, there was the same problem not only with 7zip but also with cinebench2024.

This is where my problem ends. The process of reading here and configuring the system for several months was fun thanks to you.

 

[RolloZ170]

So, after installing Windows, I reinstalled all drivers, updated the latest updates, and changed vNUMA to enable again, but unfortunately, I was greeted with a kernel panic.

works fine here with MS33-AR0 and QYFQ. must be an issue with dual socket CPU ? maybe you go better with NUMA, not vNUMA ?

 

[Shiitake]

works fine here with MS33-AR0 and QYFQ. must be an issue with dual socket CPU ? maybe you go better with NUMA, not vNUMA ?

I learned about the concept of NUMA while using this system, so I don't have anything to comment on. As far as I know, NUMA came out after the concept of UMA, and Virtual NUMA came out after that. Additionally, machines for which V NUMA is recommended must have high-density cores and memory per socket.

Since you mentioned it, I tried booting with NUMA disabled and V NUMA enabled. Then I entered Windows without a kernel panic. I didn't expect this. haha

In Windows 10, I enabled both and used them well. I don't know what part caused it.
I don't think it's a multi-socket problem, as technologies like NUMA should be used well in multi-sockets.

It may not be of much help, but I am attaching a benchmark image showing all NUMA disabled, only NUMA enabled, and only V NUMA enabled. I'm currently using a lower core profile, so it's 32 cores, but I think you'll get different results if you compare it with 48 cores with HT turned on.
I think people who work in virtual computing will know the most about this.

 

[RolloZ170]

In Windows 10, I enabled both and used them well. I don't know what part caused it.
I don't think it's a multi-socket problem, as technologies like NUMA should be used well in multi-sockets.

possibly Windows 11 is buggy ? can that be LOL ?
time for a specialised thread for this issue thought.
this is helpful, applications need to manage the processor groups now.

Processor Groups - Win32 apps

The 64-bit versions of Windows 7 and Windows Server 2008 R2 and later versions of Windows support more than 64 logical processors on a single computer. This functionality is not available on 32-bit versions of Windows.

learn.microsoft.com

 

[Shiitake]

possibly Windows 11 is buggy ? can that be LOL ?
time for a specialised thread for this issue thought.

Looking at the progress of Windows 11 so far, it smells plausible enough, right? That's my opinion too. But I also feel like it was divided on purpose. With Server 2022. hahaha

 

[Kizune]

But I also feel like it was divided on purpose. With Server 2022.

If you mean divided into separate logical NUMA groups - yes, that was done on purpose. When Windows threading API was created processors with more than 16 threads were unthinkable so when core affinity variable was introduced it was decided that a 64-bit wide bitmask is more than enough. Unfortunately that meant that if processor would have more than 64 logical cores this API does not have any way to schedule thread execution onto all of them, only first 64 cores will be addressable. When such processors (like Intel Xeon Phi) happened it was decided that instead of breaking API in an incompatible way it is easier to introduce virtual NUMA nodes and that will allow NUMA-aware programs to do the thread scheduling using these groups. For the NUMA-unaware processes they were assigned a random NUMA node and all of its threads would be scheduled on that node.

 

[RolloZ170]

"Starting with Windows 11 and Windows Server 2022, it is no longer the case that applications are constrained by default to a single processor group. Instead, processes and their threads have processor affinities that by default span all processors in the system, across multiple groups on machines with more than 64 processors."

Processor Groups - Win32 apps

The 64-bit versions of Windows 7 and Windows Server 2008 R2 and later versions of Windows support more than 64 logical processors on a single computer. This functionality is not available on 32-bit versions of Windows.

learn.microsoft.com

 

[Kizune]

"Starting with Windows 11 and Windows Server 2022, it is no longer the case that applications are constrained by default to a single processor group. Instead, processes and their threads have processor affinities that by default span all processors in the system, across multiple groups on machines with more than 64 processors."

Processor Groups - Win32 apps

The 64-bit versions of Windows 7 and Windows Server 2008 R2 and later versions of Windows support more than 64 logical processors on a single computer. This functionality is not available on 32-bit versions of Windows.

learn.microsoft.com

Yep, the scheduler in Win11 (and Server 2022) had to be rewritten to be less dependent on the input from programs because of the new Intel architecture with performance and efficiency cores. You can not just expose them to the user and let them schedule their threads on any core they want. Well, they’ve done it with Win10 and that’s the reason Lakefield notebooks had such a terrible reputation regarding performance and power efficiency. Microsoft had to fully rewrite scheduler introducing new API that allowed to get rid of the old limitations and to drop the old default behavior.

 

[nkw]

I have two X11SPH-NCTPF based systems currently populated with Xeon Silver 4110s. I would like to spend some eBay money and upgrade these (moar CORES! moar POWER!). These are in my personal lab so I don't mind playing around with ES cpus. I understand some of the OEM 'special' cpus may require doing the VRM mod on this board.

What is everyone's opinion on the sweet spot right now for a 2nd gen Xeon scalable for this board? Both if I do the VRM mod and if I don't. I've been looking at maybe the QQ89 or QQ8Q?

 

[Shiitake]

"Starting with Windows 11 and Windows Server 2022, it is no longer the case that applications are constrained by default to a single processor group. Instead, processes and their threads have processor affinities that by default span all processors in the system, across multiple groups on machines with more than 64 processors."

Processor Groups - Win32 apps

The 64-bit versions of Windows 7 and Windows Server 2008 R2 and later versions of Windows support more than 64 logical processors on a single computer. This functionality is not available on 32-bit versions of Windows.

learn.microsoft.com

It was that link that made me decide. After seeing that post, I decided to upgrade from Windows 10 to Windows 11.

In W10, if the number of logical processors exceeds 64, they are grouped, but even though ordinary programs are designed to support multiprocessors, they only operate within the group in which they are running, and the resources of other groups are idle.

For example, 48C/2S/HTON=192 logical processors, but W10 divides this into four groups of 48 processors each. If you run a normal program under these conditions, only one group of logical processors (48T) will be used, and the remaining three groups (144T) will be idle.

Anyway, all issues have been resolved now that I upgraded to W11. If anyone operates the W10 under the same conditions as the few photos posted before, they will be able to see that the benchmark scores differ by up to half. The performance improvement is enormous. It's magic magic magic.

 

[RolloZ170]

Ice Lake QWAT 8368 ES2 step 6, runs fine on X12SPM even with latest BIOS 1.6...

 

[zorg33]

Hi!
I've been here in read-only mode for a few years now. I've done a lot of bios mods and hw mods in desktops and also servers.
But i'm lack of knowledge about how Intel ME restricts CPU support.
My desire lately is building a 4th gen dual Xeon rig with some cheap D0 stepping ES CPUs.
But I come with 2nd gen Xeon experiences and it surprises me that Supermicro and Asrock boards no longer support early ES's.
WHAT IF I simply desolder the 256MB flash chip from the X13DEI/DAI board and write in an unsigned modded BIOS that contains ES microcodes with a programmer?
We should really find a solution, guys
Update: hmm I guess the first problem is that we cannot decrypt a signed BIOS to see its contents?

 

[RolloZ170]

WHAT IF I simply desolder the 256MB flash chip from the X13DEI/DAI board and write in an unsigned modded BIOS that contains ES microcodes with a programmer?

the BIOS is checked at every power up by RoT mechanism. if validation fails the BIOS is restored from Backup(flashdisk)
this prevents the whole system for modifying of FW even with physical access. if there is a way it will be closed soon.
there are some unofficial early BIOS for some X13 boards with D0 ES support. but you have to stay on old buggy BIOS forever.

Update: hmm I guess the first problem is that we cannot decrypt a signed BIOS to see its contents?

it is not encrypted, it is signed with a Private Key (Cert,supermicro)
health checked with the Public Key.
there is a way for OEMs. you can sign the BIOS with your own cert key and provide the key to the SUM.efi to accept your OEM BIOS.
Afaik only supermicro can re-sign the BIOS.

 

[RolloZ170]

My desire lately is building a 4th gen dual Xeon rig with some cheap D0 stepping ES CPUs.

you go with Gigabyte MS73-HBx, latest BIOS still support D0 ES.
i run QYFP,QYFQ and more on MS33-AR0 BIOS F19 i.e.

 

[zorg33]

you go with Gigabyte MS73-HBx, latest BIOS still support D0 ES.
i run QYFP,QYFQ and more on MS33-AR0 BIOS F19 i.e.

Yes I already learned that but I had bad experience with buggy GB boards from the LGA3647 era. Might give it a chance again.
BTW are there any blade form LGA4677 boards available? I converted some blades from previous generations to standalone operation.

 

[RolloZ170]

My desire lately is building a 4th gen dual Xeon rig with some cheap D0 stepping ES CPUs.

check the step 4, E0 steppings, costs a little more but much better compatibility.
Q0KL i.e.