ES Xeon Discussion
- Thread starter britinpdx
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
There were some concerns about power draw by accelerators, but seems like it's not an issue (although I'm not sure about if invocation logic cuts it automatically).
There's another potential flaw - these programs might not be adopted for SR and new PCH/drivers, therefore reporting could be wrong.
in my experience existence of the accelerators do NOT increase IDLE package power.
you used latest HWinfo verision, the reporting is correct within the tolerances.
you can check/compare with wattage at the wall.
Is there a motherboard which supports dual 84xx QS samples and also S3 sleep?
I'd like to build a new workstation at some point, and need S3 support, lots of high clocked cores + RAM, and lots of pci-e (multiple x16 GPUs, x16 100GbE, multiple x8 and x4 NVme)
I've had great experience with QS Xeons of old - I still have 3 dual QGN7 workstation systems (EP2C612 WS, X10DAi) with S3 support and plenty of pci-e lanes (80)/slots(7,6), which have served me really well since ~2016, but are now getting very long in the tooth in terms of performance.
On the other hand, my H11DSi-NT + ZS1406E2VJUG5 system performs well for e.g. building code, but has been disappointing both for lacking S3 and lack of exposed pci-e (58/66 lanes: 5 slots, only 2 x16, and an x2 m.2, plus two disabled x4 oculink; despite the platform having ample lanes available!), limiting its usefulness for lots of primary daily driver workstation use-cases that the dual-2011 Xeon systems excel at.
It may be that Genoa-X is the thing to wait for, but it seems like QS Xeons could probably compete on performance with my Rome system, if there is a good workstation combination (but I also noticed that some QS are missing UPI links needed for dual socket)?
I'd like to build a new workstation at some point, and need S3 support, lots of high clocked cores + RAM, and lots of pci-e (multiple x16 GPUs, x16 100GbE, multiple x8 and x4 NVme)
I've had great experience with QS Xeons of old - I still have 3 dual QGN7 workstation systems (EP2C612 WS, X10DAi) with S3 support and plenty of pci-e lanes (80)/slots(7,6), which have served me really well since ~2016, but are now getting very long in the tooth in terms of performance.
On the other hand, my H11DSi-NT + ZS1406E2VJUG5 system performs well for e.g. building code, but has been disappointing both for lacking S3 and lack of exposed pci-e (58/66 lanes: 5 slots, only 2 x16, and an x2 m.2, plus two disabled x4 oculink; despite the platform having ample lanes available!), limiting its usefulness for lots of primary daily driver workstation use-cases that the dual-2011 Xeon systems excel at.
It may be that Genoa-X is the thing to wait for, but it seems like QS Xeons could probably compete on performance with my Rome system, if there is a good workstation combination (but I also noticed that some QS are missing UPI links needed for dual socket)?
Last edited:
Probably there are some options to push it even lower (BIOS or Operating System power configuration).
@RolloZ170 as per your request - turbo states for Sapphire Rapids / LGA4677 Xeon 8461V QS Q16Z sold by @Kizune on ebay
As you've predicted some time ago - there are already off-roadmap Prod units starting appearing on a market, for instance this affordable 8469 52-core from China for only $1.8K (I've kept an eye on this supplier for a while now since they do offer quite good options sometimes). The reason I didn't go with their listing this time - 8469 doesn't have any index after model name suggesting there aren't all accelerators included (most likely only one DSA is present), hence testing horizon is reduced.
Not sure if it's question of configuration (I'm not an expert on Power Sleep States) , default state on X13DEI looks like this (without any configuration of BIOS or in Windows, only fresh install with drivers):
At the same time some devices (don't have anything plugged into PCIe unfortunately) do have some S3 capabilities and mappings:
They do have Workstation version too - X13DAI , that might have a bit more broad/flexible choices.
For comparison my i7-11700 Dell Workstation/Precision 3450 system shows this (S3 Power State is supported):
As for Dual-Socket EATX/EATX+ motherboards for LGA4677 platform - haven't seen any MB with all PCIe slots (or lanes) implemented from any known brand yet (for instance previous CPU generation SuperMicro had X12DPG extended board, but for X13 they haven't even bothered to channel excess/remaining lanes via NVMe connectors which would fit on MB just nicely, not like PCIe requiring lots of space).
Theoretically Chipset/PCH has some PCIe lanes for slower-speed devices too, but those are hardly implemented/wired these days either.
Last edited:
no effect.
have disabled all cores except 8 (of 48 cores) and just 10watts lower.
lowered DDR5 from 4800 to 3600Mt = 5 watts lower.
don't realy know, must have something todo with emib/mesh.
intel's next gen Emerald Rapids do not use 4 tiles anymore, only 2 identical DIEs like the Sapphire Rapids MCC SKUs.
DDR5 RDIMM is powered with +12V. your PSU has to stay ON. the hardware EFI driver must be up to date because the EFI BIOS does not init.from scratch up if wakeup,
i.e. my SM X11SPM sleeped/waked up well until i use PCIe GPU
How about Icelake-SP ES (Stepping 5，QVxx), especially when working with supermicro single slot motherboards with 1.0bios, will there be malfunctions (such as memory channel drop, BSOD) in specific situations? Or are they completely unable to be used as production tools at all?
afaik only ES1 are affected. note the mem.channels vanish and never come back.
some say it is a silicon issue, some say due the lack of chemicals, intel doesn't use die/substrat sealing for ES1(corrosion)
there are ES1 and ES2 stepping C0 (step ID 5)
QVM7,QVM8 are ES2.
QV2E,QV2E are ES1.
you want to live with a buggy BIOS ? on any problems you can't ask supermicro because they force you to upgrade to latest BIOS first.
if you want to run non PRD stepping ES stay away from supermicro.
i have sold my X13SEI and moved to ASUS because i can mod the BIOS there, gigabyte is fine too.
tyan and asrock-rack removed non PRD stepping ES support.
But I have already buy a x12spl with 1.0bios and have a Qvm7. You mentioned that there are defects in the 1.0 version of the bios, are they serious? This will make me decide whether to return this supermicro motherboard.Alternatively buy QWAx (step 6) and upgrade bios
Last edited:
there are bugs in any older BIOS, for that reason they provide updates.
Code:
BIOS X12SPL release Notes summary
Release Version 1.5 (4/21/2023)
1. Updated 5.22_WhitleyCrb_0ACMS_ICX_075 Intel 2023.2 IPUPV, please check header for firmware revisions.
2. No boot option for single HDD under RAID mode.
3. Added ECM RNDIS support.
4. Improved VROC key detection
1. Fixed the SUM TC: 2020 test fail problem.
1.4b (1/9/2023)
1. Updated 5.22_WhitleyCrb_0ACMS_ICX_74 Intel BKCWW46 IPU2023.1.
2. Added enhancements to resolve the issue where the SUM TC: 220/271/356/457/2071/3056/4057
with Windows OS caused the boot order compare fail after flashing the BIOS.
3. Updated the SA50157 Boot_Variables_Access.
4. Updated the SEL for processor error.
5. Updated Intel Server Platform Services for Whitley Server Platforms IPU2023.1 4.4.4.301.
1.4 (7/14/2022)
1. Updated BIOS version to 1.4.
2. Updated 5.22_WhitleyCrb_0ACMS_ICX_72 Intel BKCWW23 PLR3, please check header for
firmware revisions. For INTEL-SA-00657 Security Advisory to address CVE-2022-21233(6.0,
Medium) security issue. For INTEL-SA-00686 Security Advisory to address CVE-2021-33060(7.8,
High) security issue.
3. Updated VROC SATA/sSATA EFI driver to VROC PreOS v7.7.6.1004 to address INTEL-TA-00692.
4. Added "CSM Support" setup item into SMCISBForm page.
5. Updated ucode M87606A6_0D000375 to address below issue.
6. Added Link Retrain per port in BIOS.
7. Fixed SATA device, functioning abnormally.
8. If chassis type of FRU0 is not 1(other) or 2(unknown), sync it to SMBIOS type 3.
1.2 (5/3/2022)
1. Update BIOS version to 1.2.
2. Update AMI 5.22_WhitleyCrb_0ACMS_ICX_070_BETA RC27P52 for BKC 2021_WW52 (PLR1), and
RC27P56 for PLR1 HF.
3. Changed string "VMX" to "Intel Virtualization Technology".
4. Removed 1G option from MMCFG base to avoid system hang.
5. Fixed the SMBIOS event log ERROR CODE not display correctly under BIOS menu issue (EFI error
type).
6. Rolled back VROC SATA/sSATA EFI driver to VROC PreOS v7.6.0.1012 to fix system hang when
VMD is enabled.
7. Fixed issue with password that can't be preserved after loading default via Supermicro Update
Manager (SUM).
1. Updated 5.22_WhitleyCrb_0ACMS_ICX_069 Beta Intel BKCWW39
2021 PV MR7.
2. Updated Dx/Mx microcode from Intel-Generic-Microcode20210927_NDA.
3. Extended NVMe OPROM control options to 24 slots.
4. Turned on Shutdown Suppression and Log MCA IERR to fix crash
dump error.
5. Removed support for the default CA-Cert, Client-Cert, and PrivateKey.
6. Added multiple KMS Server Support for SmcKMS.
7. Fixed the wording of the TCG NVMe KMS Policy.
8. Changed some Setup Item types from String type to Password type.
9. Added SmcKMS Probe function.
10. Updated Asp2600 module.
11. Updated the strings on the Setup Menu related to SmcKMS.
12. Added "DelCaCert", "DelClientCert", and "DelClientPriKey" into
SmcOOB Interface for SUM to remove CERT keys.
13. Set Enhanced PPR from Enabled to Disabled.
14. Displayed item "Data Link Feature Exchange" on PCIe SLOT pages.
1. Fixed inability to change COM port resource and failure of item's
behavior.
2. Updated boot status to BMC at first power-on for test scenario
"flashing other ROM immediately".
3. Removed 1G option from MMCFG base to prevent system hang.
4. Fixed problem of only one CERT key updating successfully when
updating all CERT keys (CA, Client Cert, and Private Key) by BIOSCfg at
the same time.
5. Implemented restoration of ROM-Hole and NVRAM Variables that
should be kept after checking if GUID-HOB
(NvramLoadDefaultModeGuid) exists.
6. Set bits of IPMI CMD 30_A0_15 to be cleared according to usage of
location instead of all bytes.
7. Updated SmcOutBand in SmcPKG/Module/SmcOOB to fix the
failure to store DMI data when executing "SUM -c
LoadDefaultBiosCfg".
1.1a (07/30/2021)
1. Updated 5.22_WhitleyCrb_0ACMS_ICX_066_BETA Intel BKCWW24 2021 PV MR4.
2. Updated Dx/Mx microcode from Intel-Generic-Microcode-20210701_NDA.
3. Updated SATA/sSATA EFI driver to VROC PreOS v7.6.0.1012.
4. Updated SPS 4.4.4.56 PV MR2.
5. Fixed the issue with TPM 1.2/2.0 disappearing when enabling Intel "TXT Support" without
provisioning Intel TXT requiring NV indices to TPM.
6. Changed "Hard Drive Security Frozen" default setting to disabled.
7. Added support to SUM upload/delete HTTPS TLS certificate. (Default disabled by TOKEN
"Sum_UploadTlsKey_SUPPORT")
8. Disabled EFI iSCSI support.
9. Path BMC Redfish Host Interface was renamed as ethX for the case where CDN was disabled under
Linux OS.
10. Fixed the issue that WHLK TPM 2.0 Supplemental test failed.
11. Fixed the issue with SGX settings not getting preserved after updating BIOS. The function cannot
support IPMI web updating BIOS.
12. Fixed the issue due to which wrong FW version and vendor were shown on Trusted computing
page.
1.1 (4/9/2021)
1. Updated RC 20.P95 for PV RC update.
2. Updated BIOS ACM 1.0.9 and SINIT ACM 1.0.9.
3. Updated SPS 4.4.4.53.
4. Automatically disabled and hid ADDDC with x8 width DIMM.
5. Extended memory DIMM serial number information (Samsung, Micron, Hynix).
6. Enhanced SMC DCPMM feature.
7. Automatically disabled and grayed out ADDDC, UMA-Base Clustering, and mirror mode and enabled
NUMA when SGX is enabled.
8. Set relation setup to restore setting after "Factory Mode" is disabled.
9. Removed 4G limit of Intel LAN memory if boot mode is not legacy.
10. Set all OPROM control items to Legacy when boot mode is set to Dual.
11. Updated SATA/sSATA EFI driver to VROC PreOS v7.5.0.1152.
12. Updated BPS firmware to 2.2.0.1553.
13. Set BootGuard to enabled by default.
14. Removed iSCSI option from LAN OPROM item.
15. Removed 4G limit of Intel LAN memory if boot mode is EFI.
16. Set PCH PCIe ASPM to disabled if CPU PCIe global ASPM is disabled.
17. Stopped sending major code 0x2A (no memory ) SEL.
18. Added support for VROC OOB.
19. Changed "SMCI PMem Formset" to "SMCI PMem Configuration".
20. Fixed problem of BIOS initialization showing IPV6 address when IPV6 is disabled in the IPMI GUI.
21. Filtered Dynamic HDD Security pages to patch failure of SUM ChangeBiosCfg.
22. Set AFU to stop if there are no parameters.
23. Fixed mismatch of memory device in IPMI and in BIOS setup when some memory DIMMs are
mapped out.
24. Corrected display of IPv6 when IPv6 status is not active.
25. Fixed inability to upload all OOB files on the first BMC boot.
26. Fixed failure of Secure Boot Append/Update Keys.on LGA4189 platform recently ASRock Rack and Gigabyte removed non PRD stepping ES support.
but there are BIOS you can go back to or someone can mod. BIOS.
note: supermicro's BIOS is signed and protected by RootOfTrust. no way for modding.
I tested x12spl+qvm7 and they didn't work well. When opening hwinfo and cpu-z sometimes blue screen (memory management). And c-state must be turned off otherwise the system will be very slow. This should be a problem caused by the early BIOS and CPU, because the c-state bug disappeared after replacing qwas (D0) but hwinfo blue screen still exists, I think icx step5 is unusable.
HWinfo: settings, safety, safe scan of potentially problematic PCI devices.
same on SPR-SP
it is not ES issue.
Last edited:
Did you also encounter hwinfo blue screen on the SPR-SP platform?The c-state bug is acceptable, but hwinfo frustrates me so much that even though the cpu has passed all the performance tests and stress tests, I still returned the x12spl, maybe I should have kept them, they are probably the best production tool you can buy at this price.
Performance tests are still normal, but all cores display at 3600mhz(Scheduling bugs?), normal at full cpu load, and significantly slow response at empty load (requires 3s to open task manager)
yes with some HWinfo versions and with unbalanced DIMM population(i.e. only one DIMM)
it is more a HWinfo issue.
1..18 cores can do 3600 on QVM7, displaying ths core clocks is still to slow to get that.
windows likes it hopping the threads from core to core to prevent wear out of single cores.
LOL. same on SPR. maybe latest Win drivers solve that issue.
x12 and ES step 4 or 5 is bad solution. you stuck on buggy BIOS forever.
at the risk of repeating myself:
avoid supermicro if you want to run non PRD stepping ES