Finally: Overclocking EPYC Rome ES

Zhang

Member
Sep 18, 2018
32
22
8
Furthermore there’s the ‘lock frequency’ checkbox. This can help lift multi-core performance when not using the EDC “bug” on all the CPUs.
@ExecutableFix Thank you for the great overclocking tool!

I am running Linux and I am using the Linux tool by @I.nfraR.ed. Unfortunately there is currently no such option to lock frequency on Linux. @ExecutableFix could you please let me know how this "lock frequency" feature is implemented? Which SMU register it is changing to enable locked frequency? I can add it to the Linux tool.

Thanks!:)
 
  • Like
Reactions: efschu3

Layla

Game Engine Developer
Jun 21, 2016
63
25
18
36
I've finally got my hands on an H11DSi-NT (rev 1) board (after mistakenly receiving a non-NT board first). I have two 2S1404E2VJUG5 (64-core) CPUs.

After updating the BIOS using Rome_H11DSI_Rev1 via IPMI web interface, with a single CPU installed on the left (CPU1) socket (plus RAM), the CPU fan starts to spin, but the board does not POST.

Are there any known issues with trying to use one of these CPUs at a time? Has anyone run into an issue like this? The Supermicro IPMI website doesn't update any of the sensor readings or other CPU info (it reports still having Dual 7551 installed with BIOS 1.3c from the previous owner).
 
  • Like
Reactions: Spartus

Layla

Game Engine Developer
Jun 21, 2016
63
25
18
36
I've finally got my hands on an H11DSi-NT (rev 1) board (after mistakenly receiving a non-NT board first). I have two 2S1404E2VJUG5 (64-core) CPUs.

After updating the BIOS using Rome_H11DSI_Rev1 via IPMI web interface, with a single CPU installed on the left (CPU1) socket (plus RAM), the CPU fan starts to spin, but the board does not POST.

Are there any known issues with trying to use one of these CPUs at a time? Has anyone run into an issue like this? The Supermicro IPMI website doesn't update any of the sensor readings or other CPU info (it reports still having Dual 7551 installed with BIOS 1.3c from the previous owner).
Interesting findings:
1. Can't set the date and time (I set it via IPMI Web UI, but it doesn't "stick"). I assume this needs a POST to stick.
2. Sensors all report "Not Present!"
3. IKVM HTML5 just has a grey box.
4. Post Snooping doesn't update - stuck at d1, but this seems to be from last owner.
 

Layla

Game Engine Developer
Jun 21, 2016
63
25
18
36
Interesting findings:
1. Can't set the date and time (I set it via IPMI Web UI, but it doesn't "stick"). I assume this needs a POST to stick.
2. Sensors all report "Not Present!"
3. IKVM HTML5 just has a grey box.
4. Post Snooping doesn't update - stuck at d1, but this seems to be from last owner.
Tried resetting CMOS. This reset post code to 0xFF (not updating, though), but nothing else appears to have changed.
The speaker was broken off the board during shipping (yay), so I can't hear audible POST beeps for errors, either.
The CPU does get warm (it has a heatsink+fan on it, and still gets warm).
 

mirrormax

Member
Apr 10, 2020
48
25
18
I've finally got my hands on an H11DSi-NT (rev 1) board (after mistakenly receiving a non-NT board first). I have two 2S1404E2VJUG5 (64-core) CPUs.

After updating the BIOS using Rome_H11DSI_Rev1 via IPMI web interface, with a single CPU installed on the left (CPU1) socket (plus RAM), the CPU fan starts to spin, but the board does not POST.

Are there any known issues with trying to use one of these CPUs at a time? Has anyone run into an issue like this? The Supermicro IPMI website doesn't update any of the sensor readings or other CPU info (it reports still having Dual 7551 installed with BIOS 1.3c from the previous owner).
Yea it needs both cpus to boot
 
  • Like
Reactions: Spartus and Layla

Layla

Game Engine Developer
Jun 21, 2016
63
25
18
36
P1_SOCRUNUpper Critical1.174 Volts

Is this anything to worry about?
Added 40mm Noctua fans on the VRMs and seeing this for now at POST.

VRMCpu1 TempNormal34 degrees C
VRMCpu2 TempNormal34 degrees C
VRMSoc1 TempNormal62 degrees C
VRMSoc2 TempNormal60 degrees C
VRMP1ABCD TempNormal36 degrees C
VRMP1EFGH TempNormal35 degrees C
VRMP2ABCD TempNormal34 degrees C
VRMP2EFGH TempNormal33 degrees C
P1-DIMMA1 TempNormal31 degrees C


 

Layla

Game Engine Developer
Jun 21, 2016
63
25
18
36

Attachments

Last edited:
  • Like
Reactions: eatbuckshot

Layla

Game Engine Developer
Jun 21, 2016
63
25
18
36
28k Cinebench R20 using High Multi-Core Preset with 128T:

Settings:
* SMT disabled (2xZS1406E2VJUG5 --> 128 cores/128 threads)
* Otherwise default BIOS settings (IOMMU Disabled, Above 4G Enabled)
* 16x sticks of DDR4 2133 memory (but one memory channel [P1-DIMMG1] not working)

30965 (~31k) Cinebench R20 using High Multi-Core Preset with 256T:

Settings:
* SMT Enabled (128 cores/256 threads)
* IOMMU Enabled, Above 4G Enabled
* 16x sticks of DDR4 2133 memory (but one memory channel [P1-DIMMG1] not working)
 

Attachments

Last edited:

Layla

Game Engine Developer
Jun 21, 2016
63
25
18
36
28k Cinebench R20 using High Multi-Core Preset.

Settings:
* SMT disabled (2xZS1406E2VJUG5 --> 128 cores/128 threads)
* Otherwise default BIOS settings
* 16x sticks of DDR4 2133 memory (but one memory channel [P1-DIMMG1] not working)
Wanted to to the test again with SMT enabled, but the system hangs during POST with code 0x55 (Memory not installed) after changing back to SMT Enabled. Anybody encountered this?

Note that I've already updated Windows Server 2019 and applied the SP3 driver package from SuperMicro.
 

Layla

Game Engine Developer
Jun 21, 2016
63
25
18
36
Wanted to to the test again with SMT enabled, but the system hangs during POST with code 0x55 (Memory not installed) after changing back to SMT Enabled. Anybody encountered this?

Note that I've already updated Windows Server 2019 and applied the SP3 driver package from SuperMicro.

[URL='https://forums.servethehome.com/index.php?goto/post&id=259693']ExecutableFix[/URL] said:
Oh yes that's right. I forgot to mention that there are a couple of bios settings that need to be set on the DSI for the ZS cpu's, it's finicky. So turn on IOMMU and disable Above 4G decoding and it should work
So, I just turned on IOMMU (but did not turn off Above 4G Decoding), and now my system boots again. In-case anyone else runs into this.
 
Last edited:

fobus

New Member
May 17, 2020
14
0
1
Layla,

Can You please run Corona benchmark test with SMT turned on?
And it will be great if You can tell us frequency on all cores during the test. With multicore preset and with standart settings.
 

Zhang

Member
Sep 18, 2018
32
22
8
Based on @ExecutableFix's open source Windows tool, I modified @I.nfraR.ed's ZenStates for Rome ES on Linux and added a few missing features. My version comes with these improvements:

1. Read/Write PCI config space directly in Python instead of using setpci. I found that setpci sometimes does not work reliably (probably due to race conditions), and in one occasion it wrote a wrong VID (0x0, =1.55V) that almost killed my CPUs.
2. Dual socket support.
3. Implement some missing features, including frequency lock/unlock and reverting to default voltage/frequency.
4. Add a few preset scripts (e.g., see preset_multicore.sh) that work well on my H11DSi with dual ZS1406E2VJUG5 (untested on other systems, so you may need to adjust them for your use case).

Here is my modified ZenStates on Linux. Note that I only used/tested the CLI on a headless server and I did not use the GUI.
 

Attachments

  • Like
Reactions: efschu3

vangoose

Active Member
May 21, 2019
268
70
28
Canada
So, I just turned on IOMMU (but did not turn off Above 4G Decoding), and now my system boots again. In-case anyone else runs into this.
Isn't that is mentioned in manual? Turn it off during installation and turn it back on after for win 10.