Finally: Overclocking EPYC Rome ES

ExecutableFix

Active Member
Nov 25, 2019
114
45
28
In November of 2019 I first got my hands on an AMD EPYC Naples engineering sample. I thought it was really cool you could overclock 32-cores for a lot cheaper than a 2990WX and achieve mostly the same performance. Once I saw that there were next-generation EPYC Rome engineering samples floating around I was immediately interested. It was disappointing to find out it for one didn’t boot and couldn’t be overclocked. It has been 6 months and since then I’ve been involved in a lot of awesome stuff and I’ve definitely learned a lot. Today is the day to finally unveil what 4 months of research and bricking expensive CPU’s has yielded.

Overclocking your AMD EPYC Rome engineering sample. (And a bonus on how to run them on any motherboard)

Enough chitchat, how do I overclock this thing?
So first off, please note that this only works on CPU’s which have the “Engineering Sample” tag in their name, E.g. ZS1406E2VJUG5 (64-core), ZS1711E3VIVG5 (32-core), 2S1404E2VJUG5 (64-core), 2S1705E3VIVG5 (32-core) but also the OEM EPYCs (the ones with a normal production number with -04 at the end of the number). It will not work on normal retail EPYCs.

Another thing to note is that the 32-core variants will be better overclockers in terms of frequency and relative performance. For absolute performance the 64-core models will obviously be the best.

I’ve created a simple program to make the process a lot easier. Once you start the software it’ll look like this:



There are a couple things you can change here like frequency, voltage, limits etc. I have also included some handy presets to get you started without having to tinker too much. Please keep in mind that the presets are not guaranteed to work for everyone as it might depend on the platform or bios settings. If you encounter any problems using these presets please report it in this thread.



So now I can just get started, right?
No, not really. There are a couple different things that you should be aware of if you’ve never overclocked a server CPU before. Even if you have, overclocking on Rome is a whole different thing compared to Naples.

First off: No, setting the frequency does not guarantee it’ll run at that frequency. This depends on the core-count and the CPU. For example: I’m able to set the frequency to 3600MHz but the performance is more like 2000MHz. How is this possible you might ask? Well, I’m not totally sure myself, but here’s a pretty good explanation: The CPU/board still has its limiters active which control the power and current draw. The platform essentially blocks it from going any higher and therefor the performance will be down. I would recommend using either task manager or HwInfo64 to check the actual frequency in this case.

Setting the voltage also affects the speed of the CPU, that is as long as you’re not using another trick that I’ll get into later. The voltage is also not reported correctly by most tools, e.g. CPU-Z. I recommend checking the IPMI for the most accurate reading or HwInfo64’s CPU enhanced section.

Changing the limits is completely safe as long as you’re setting a number > 100. There is also an awesome “bug” (PBO Turbo Boost) in the AGESA these ES CPUs are running on. It has originally been discovered on another forum, I’ll let you search for that yourself if you want to know more about it. It basically involves setting the EDC to some really low value like 30 and the PPT/EDC to 0. It causes the CPU to go into some sort of weird mode where the frequency you set in the software is actually 800MHz higher than the actual speed. But the main difference here is that the actual speed is honored by the CPU. BUT there is a catch: setting the voltage over 1.05v or setting the frequency > 3800MHz in the software could actually KILL the CPU instantly (this only applies to models > 32-cores, it’s different on lower core models). I’ve done it twice :(. You’ll know when it goes bad when you get a black screen and can’t power off the system with the On/Off button. So please be careful when using this “bug”, but it should be fine as long as you don’t go over the limits mentioned above. This “bug” enables the best multi-core performance but hurts the single-core performance. I’ve also found that this only actually improves performance on the 64-core model and not the 32-core model as those chips already have great performance with ‘normal’ settings and aren’t throttled down so easily by the board.

Furthermore there’s the ‘lock frequency’ checkbox. This can help lift multi-core performance when not using the EDC “bug” on all the CPUs. It’s recommended in most scenarios and can’t hurt performance. Keep in mind that in some cases when the frequency is really high it can up performance too much and actually crash the system.

Then there are the built-in presets. Each CPU has three presets: “High multi-core”, “Best of both” and “High single-core”. If you want to apply a preset to a dual-socket setup you'll have to check the "dual socket" checkbox as well. Generally if you just want to get good performance and not worry about things, the “Best of both” preset will give you exactly what you’re after. It won’t tinker with the EDC ”bug” and will generally be safe to use. The “High single-core” preset is mostly the “Best of both” with a higher frequency. This also sets the voltage to a higher value so this is recommended in cases where no heavy multi-core applications are used. The “High multi-core” preset uses the EDC “bug” to get the most multi-core performance as possible. The 64-core ‘ZS14’ CPU actually performs like a stock 3990x and beats an $8000 7H12 in this configuration.

The presets might not be suited for every task out there so feel free to tinker around and share your configuration in this thread.

But the CPU doesn't post on my -insert brand- motherboard!!!
That's what these things do. Engineering sample CPUs just aren't supported by every board out of the box. There is a simple trick to make it work on any board, even on Rev. 1 boards. The only thing you need to do is copy the second half of the bios (the Rome part) and duplicate it on Rev. 2 boards or just upload that on Rev. 1 boards. Make sure the bios has an Agesa < 1.0.0.3 because otherwise it will not work. The engineering samples do work out of the box on the Asus KRPA-U16. I’ve included some pre-‘modded’ bioses below.

Some disclaimers
Please keep in mind that I haven’t tested anything for 24/7 use, although I don’t think there should be a problem as long as the voltage isn’t too high. Please check the voltage via the IPMI to confirm it’s not too high and if it is, dial back the settings. Please be careful when overclocking your CPU, the last thing you want is to destroy it like I did. I am not responsible for any damage done to the CPU so use this software at your own risk.

Screenshots
Here are some screenshots of the true power of these engineering samples :cool:




If you have any questions, suggestions or want to share anything, feel free to post it in this thread or DM me. For now, enjoy the massive performance of these things and keep an eye on your VRM temps :D

tldr; Read the whole post you lazy because your CPU might DIE!

Download link for the software

- ExecutableFix

Massive thanks to @nero243 and @I.nfraR.ed for their contributions to this project as well!
 
Last edited:

hrvoje.sedlic

New Member
Dec 23, 2019
21
3
3
Wow, now this is big! So you have managed to get 100% more power on these 1.4 ghz es? Do you have hwinfo of these oc-ed?

We have a few -4 2.6 ghz samples and will try it tomorrow.

Thanks!
 
  • Like
Reactions: ExecutableFix

Nabladel

Member
Jan 27, 2017
34
5
8
33
Very nice work to everyone involved! I'm surprised you were able to get that much more perf from those early 64c ES. Is this currently supported on windows only or can existing tools on linux be modified to make OC work?

Thanks!
 

ExecutableFix

Active Member
Nov 25, 2019
114
45
28
Very nice work to everyone involved! I'm surprised you were able to get that much more perf from those early 64c ES. Is this currently supported on windows only or can existing tools on linux be modified to make OC work?

Thanks!
It's definitely possible to port it to Linux as well. I think @I.nfraR.ed will be the one who'll get an app out for that. He's definitely more of a Linux guy than I am
 
  • Like
Reactions: Nabladel

I.nfraR.ed

New Member
Aug 20, 2019
19
23
3
Bulgaria
Linux tools can be modified, however I'm working on a better detection, since Epyc and Desktop Threadripper 3000 series (Castle Peak) share the same basic cpuid, so another more complicated detection mechanism should be implemented - at least socket type detection is needed.

If you want, you can try the modified code, however for now it works on Epyc or on TR - not on both at the same time, since command IDs are different.
Attached is the modified version of my Zenstates-Linux fork: irusanov/ZenStates-Linux

It had been tested on a ES Epyc and it worked, but this version is unofficial until I sort out the cpu detection. Ofcourse, as always, everyone is free to contribute.

You can first try to set a different frequency and if it works, proceed following the guide and mess with rest of the things. For the test, please use the CLI, then you can use the GUI as well.

Code:
sudo ./zenstates.py --no-gui --oc-frequency=2000
Frequency works in 25MHz steps.
Be extremely careful with voltage. Make sure you have something to monitor it with - preferably a multimeter to check the actual voltage. Zenmonitor seems to be the best currently available on Linux, but I'm not sure if it displays correct info on Epyc: ocerman/zenmonitor


MD5: 0c2fe08b1dd06bfb29f834206d95ba18
 

Attachments

Last edited:

Nabladel

Member
Jan 27, 2017
34
5
8
33
Awesome, thanks! @I.nfraR.ed just did a quick test, looks like its working.
Tested on 64c ES (QS?) 100-000000053-04_32/20_N and was able to get 3.4GHz, previously these were capped at 3.2.
 
Last edited:
  • Like
Reactions: I.nfraR.ed

ari2asem

Active Member
Dec 26, 2018
565
92
28
The Netherlands, Groningen
@ExecutableFix question about your modded bios's for H11DSI...i have rev.1.01 of h11dsi...do i understand correctly that i can put epyc rome es cpu's on my rev1.01 board with your modded bios??

because rev.1.01 boards support only epyc naples (7001 serie), no epyc rome support (7002 serie).

btw...great job you have done !!!
 

Nabladel

Member
Jan 27, 2017
34
5
8
33
catch: setting the voltage over 1.05v or setting the frequency > 3800MHz in the software could actually KILL the CPU instantly
This only applies if the PPT/EDC values are set as well right?
 

ExecutableFix

Active Member
Nov 25, 2019
114
45
28
@ExecutableFix question about your modded bios's for H11DSI...i have rev.1.01 of h11dsi...do i understand correctly that i can put epyc rome es cpu's on my rev1.01 board with your modded bios??

because rev.1.01 boards support only epyc naples (7001 serie), no epyc rome support (7002 serie).

btw...great job you have done !!!
Thank you! Yes, this is exactly what you need to do. The supermicro revision 2 boards are nothing more than a bios chip change to support both Naples and Rome. If you only need support for Rome this is an option without having to get a rev. 2 board
 
  • Like
Reactions: ari2asem

hrvoje.sedlic

New Member
Dec 23, 2019
21
3
3
Hi, I have tried to test it on our samples, but get actually lower Corona bench 1.3 score (which is the software we use so is referrent for us). We tried both presets and setting it manualy. CB scores are higher though. Any insights on this?
 

ExecutableFix

Active Member
Nov 25, 2019
114
45
28
Hi, I have tried to test it on our samples, but get actually lower Corona bench 1.3 score (which is the software we use so is referrent for us). We tried both presets and setting it manualy. CB scores are higher though. Any insights on this?
Some behaviour is still weird, you sometimes might have better luck setting the clocks lower. If it's a multi-threaded and single-threaded application I would recommend the best of both preset, but you'll have to play around with it a bit.
 

ExecutableFix

Active Member
Nov 25, 2019
114
45
28
H11DSI 2.0 BIOS,I can light up ZS1406E2VJUG5,cannot start the system, win10
H11DSI 2.1 BIOS,000000053-04 7742 QS,I can overclock to full core 3.0g
Oh yes that's right. I forgot to mention that there are a couple of bios settings that need to be set on the DSI for the ZS cpu's, it's finicky. So turn on IOMMU and disable Above 4G decoding and it should work