I think most of us are using them with Linux (+ Nvidia GPUs) for ML stuff. Why else pay the huge tax to get 112 lanes. You see similar issues with Epyc chips too. Of course, there are exceptions but then the argument is, how many people want Windows, SPR, and care enough about power they want to use efficient settings.
ES Xeon Discussion
- Thread starter britinpdx
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
My issue mostly pulling 700 watts at plug on CPUs alone for dual board, but pairing with 4 4090s with no rules for LLM is 2.5kw draw. Even if offshoot to V100s which are same price with worse token/s
But 3 systems drawing 7.0kw average (different workloads) for 150+ kw/h in 24 period... even on Linux performance pref from PS to balance drops generation significantly due to scheduler getting wakelocked for backend/frontend tasks that are not LLM or at forefront, even with loaded affinity.
Container/VM performance is abyssmal as well as scheduler on main OS does not handle this well at all either on SPR. This am not able to replicate on ROME/GENOAs as bad,even remotely.
SPR is not good right now unless you CAN afford energy to workload.
Yeah, but if you can afford a dual 4677 motherboard AND 4x 4090s (and the ram for that!), do you really care about saving power? At that point you are starting to creep close to the scale where I'd say just use runpod/another cloud! That's what, a 10K-20K system? So at 2$ per hour for an H100, that's ~1 year of 24/7 use. Then you also don't have to cool 7KW in your house. Cooling 1.5KW in the summer means spending 1.8KW of AC... Maybe 50% less if you have a really efficient heat pump?
These extreme HEDT really are not tuned for home use. Dual socket+ never worked great on windows, ever? Surprisingly when used in the home, they behave worse than consumer systems designed for home use.
And for Epic/Milan, the issues have been worked out on Linux/Windows. Try using FreeBSD with a Milan chip...weird issues at every turn. Basically they sit in C1 and never down clock. ~20C temp difference using Linux vs. FreeBSD. Perhaps I misconfigured something, but in general, the less eyes on something, the less easy your path will be...
For professional systems servers that are on 24/7 The price off the power usage for about 4 years of system use can be as high as the total value of the system. Lets take this case example 2.5kWh in germany price depending on location can be 40 cents per KWh. so 1 Euro per hour. times 24 times 365 =8760 Euro per year. Say we have a load factor of 0.7 we still get about 6100 Electricity cost per year. (Without considering Airconditioning power). My personal load factor is closer to 0.3-0.4 and wall powerddraw about 1.2kW, but still Electricity cost are really significant and generally also a reason to upgrade to more modern hardware, and to use ES cpu's. But honestly a dual cpu set up with these SPR (ES) xeons are really power hungry, if they are only used to feed gpu's. For many load cases single cpu with multi gpu is probably better. Unfortiunatly i also need the cpu power, and yes for me dual socket performance is also a bit disappointing.
Last edited:
Thanks for the Link of the MS33-CPA i got my hands on a QYFP and the price will fit in my budget
@RolloZ170
Unable to find in MS73-HB0 to disable cores, other than performance level 1-4 with level 4 going from 54 cores to 48 cores to get higher all core clock.
Looking to get larger boost lock out of disabling cores, but am not finding option in Gigabyte manual or just in BIOS. Not supported?
Unable to find in MS73-HB0 to disable cores, other than performance level 1-4 with level 4 going from 54 cores to 48 cores to get higher all core clock.
Looking to get larger boost lock out of disabling cores, but am not finding option in Gigabyte manual or just in BIOS. Not supported?
Pre-Socket Configuration Press [Enter] to configure advanced items. CPU Socket 0 Configuration – Core Disable Bitmap(Hex) • Number of Cores to enable. 0 means all cores. FFFFFFF means to disable all cores. The maximum value depends on the number of CPUs available. Press the numeric keys to adjust desired values.
CPU Socket 0 Configuration – Core Disable Bitmap(Hex) • Number of Cores to enable. 0 means all cores. FFFFFFF means to disable all cores. The maximum value depends on the number of CPUs available. Press the numeric keys to adjust desired values.
Per-Socket Configuration (Press [Enter] to configure advanced items)
CPU Socket 0 Configuration – Core Disable Bitmap(Hex) • Number of Cores to enable. 0 means all cores. FFFFFFF means to disable all cores. The maximum value depends on the number of CPUs available. Press the numeric keys to adjust desired values.
Suppose I thought I was in the right area, and Per-Socket Configuration offers no adjustments Watch 20240902_112421 | Streamable , on bios R07 from July, should back and attempt different revisions?
I noticed this, they are both QYFRs, and did not look much into it since they have identical processor ID... Could be the issue? I can try disabling socket 1 and see if anything adjusts with per-socket config.
Okay figured out it was Intel SSP and Dynamic Intel SSP.
Reset defaults, changed bitmap. BIOS reports "16" Fs, adjusted bitmap from 16 to 11, but does not register disabled cores until 11, and registers 71 cores...
However weirdest thing, socket 1 registers 16 detected cores but socket 0 displaying 55 with same applied bitmap????
Base clock and boost remain
played around more, does not seem disabling cores in socket 0 bitmap works
discovered adding 16 '0's in place of F's on socket 0 for 16 0's makes base clock 1.7
Reset defaults, changed bitmap. BIOS reports "16" Fs, adjusted bitmap from 16 to 11, but does not register disabled cores until 11, and registers 71 cores...
However weirdest thing, socket 1 registers 16 detected cores but socket 0 displaying 55 with same applied bitmap????
Base clock and boost remain
played around more, does not seem disabling cores in socket 0 bitmap works
discovered adding 16 '0's in place of F's on socket 0 for 16 0's makes base clock 1.7
Last edited:
if you use the bitmap shown, all cores are disabled.
ther must be 60bit = 4x 15 bit representing the XCC cores.
its a good idea to disable same count of cores in each tile.
Bitmap not applying for socket 0, but can assign OS to detected socket 1, just.. base clock, all core, and boost clock remain the same as if all 56 were enabled...
don't understand. you need to edit both socket bitmaps.
HWinfo will show you less cores per socket. if not it don't work or you done it wrong.
