Finally: Overclocking EPYC Rome ES

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

bayleyw

Active Member
Jan 8, 2014
291
95
28
@Nabladel Thanks! What CPU and board were you using, and were you able to hit 3800MHz under load or did the board throttle the CPU?
 

Nabladel

Member
Jan 27, 2017
38
5
8
37
It was high 60s if I remember correctly. But I also had a fan blowing directly at the vrm heatsinks.
 

efschu3

Active Member
Mar 11, 2019
160
61
28
So am I rigth that there does not exist any <32C rome EPYCs which are overclockable?

I have found this:

https://www.google.com/url?q=https:...FjAIegQIBhAB&usg=AOvVaw2oKsSLohNHcrLE4-qlo068

which seems to be a list of all AMD production codes. If so, then not all epycs have a -04 OEM counterpart but there seems to be -00 -01. Maybe this helps someone with better knowledge to identify overclockable sub 32core rome epycs.
 
Last edited:

efschu3

Active Member
Mar 11, 2019
160
61
28
Never mind - I think I will look for a ZS1711E3VIVG5 32C EPYC and a H11SSL-i REV2.0 and for testing a single 8GB RDIMM (UDIMM does not work with epycs, correct?), does this combo work together?
 

ExecutableFix

Active Member
Nov 25, 2019
123
64
28
Never mind - I think I will look for a ZS1711E3VIVG5 32C EPYC and a H11SSL-i REV2.0 and for testing a single 8GB RDIMM (UDIMM does not work with epycs, correct?), does this combo work together?
Yes, that will work just fine with the custom bios
 

Jon

Member
Feb 28, 2016
77
18
8
42
Was looking around the forum to see if anyone had started playing with the new epyc rome ES chip and found this gem in the forum. Thanks for all the effort.

Anyone else getting virus detected when downloading from chrome?
 

efschu3

Active Member
Mar 11, 2019
160
61
28
Yes, that will work just fine with the custom bios
Do you mean the custom bios mention in the start post here:
https://forums.servethehome.com/index.php?threads/overclock-your-amd-epyc.23422/

V2.0 Rome tested:
H11SSLv2.0SSPMod

?

Then the EDC-Bug is confusing me. What is the benefit of overclock it by edc bug? Can I use the EDC-Bug OC on linux at all? The overclock via EDC-Bug higher then 3,8GHz might (or probably) will kill the 64C, but not the 32C EPYC?

A while ago I have read, that there exist some Ryzen per CCX overclock for 3900X/3950X - does there exist something similar for EPYCs - would be aweseome to clock one CCX to maybe 4 GHz and let the rest on some "low clock".
 
Last edited:

Layla

Game Engine Developer
Jun 21, 2016
215
177
43
40
In November of 2019 I first got my hands on an AMD EPYC Naples engineering sample. I thought it was really cool you could overclock 32-cores for a lot cheaper than a 2990WX and achieve mostly the same performance. Once I saw that there were next-generation EPYC Rome engineering samples floating around I was immediately interested. It was disappointing to find out it for one didn’t boot and couldn’t be overclocked. It has been 6 months and since then I’ve been involved in a lot of awesome stuff and I’ve definitely learned a lot. Today is the day to finally unveil what 4 months of research and bricking expensive CPU’s has yielded.

Overclocking your AMD EPYC Rome engineering sample. (And a bonus on how to run them on any motherboard)

Enough chitchat, how do I overclock this thing?
So first off, please note that this only works on CPU’s which have the “Engineering Sample” tag in their name, E.g. ZS1406E2VJUG5 (64-core), ZS1711E3VIVG5 (32-core), 2S1404E2VJUG5 (64-core), 2S1705E3VIVG5 (32-core) but also the OEM EPYCs (the ones with a normal production number with -04 at the end of the number). It will not work on normal retail EPYCs.

Another thing to note is that the 32-core variants will be better overclockers in terms of frequency and relative performance. For absolute performance the 64-core models will obviously be the best.

I’ve created a simple program to make the process a lot easier. Once you start the software it’ll look like this:



There are a couple things you can change here like frequency, voltage, limits etc. I have also included some handy presets to get you started without having to tinker too much. Please keep in mind that the presets are not guaranteed to work for everyone as it might depend on the platform or bios settings. If you encounter any problems using these presets please report it in this thread.



So now I can just get started, right?
No, not really. There are a couple different things that you should be aware of if you’ve never overclocked a server CPU before. Even if you have, overclocking on Rome is a whole different thing compared to Naples.

First off: No, setting the frequency does not guarantee it’ll run at that frequency. This depends on the core-count and the CPU. For example: I’m able to set the frequency to 3600MHz but the performance is more like 2000MHz. How is this possible you might ask? Well, I’m not totally sure myself, but here’s a pretty good explanation: The CPU/board still has its limiters active which control the power and current draw. The platform essentially blocks it from going any higher and therefor the performance will be down. I would recommend using either task manager or HwInfo64 to check the actual frequency in this case.

Setting the voltage also affects the speed of the CPU, that is as long as you’re not using another trick that I’ll get into later. The voltage is also not reported correctly by most tools, e.g. CPU-Z. I recommend checking the IPMI for the most accurate reading or HwInfo64’s CPU enhanced section.

Changing the limits is completely safe as long as you’re setting a number > 100. There is also an awesome “bug” (PBO Turbo Boost) in the AGESA these ES CPUs are running on. It has originally been discovered on another forum, I’ll let you search for that yourself if you want to know more about it. It basically involves setting the EDC to some really low value like 30 and the PPT/EDC to 0. It causes the CPU to go into some sort of weird mode where the frequency you set in the software is actually 800MHz higher than the actual speed. But the main difference here is that the actual speed is honored by the CPU. BUT there is a catch: setting the voltage over 1.05v or setting the frequency > 3800MHz in the software could actually KILL the CPU instantly (this only applies to models > 32-cores, it’s different on lower core models). I’ve done it twice :(. You’ll know when it goes bad when you get a black screen and can’t power off the system with the On/Off button. So please be careful when using this “bug”, but it should be fine as long as you don’t go over the limits mentioned above. This “bug” enables the best multi-core performance but hurts the single-core performance. I’ve also found that this only actually improves performance on the 64-core model and not the 32-core model as those chips already have great performance with ‘normal’ settings and aren’t throttled down so easily by the board.

Furthermore there’s the ‘lock frequency’ checkbox. This can help lift multi-core performance when not using the EDC “bug” on all the CPUs. It’s recommended in most scenarios and can’t hurt performance. Keep in mind that in some cases when the frequency is really high it can up performance too much and actually crash the system.

Then there are the built-in presets. Each CPU has three presets: “High multi-core”, “Best of both” and “High single-core”. If you want to apply a preset to a dual-socket setup you'll have to check the "dual socket" checkbox as well. Generally if you just want to get good performance and not worry about things, the “Best of both” preset will give you exactly what you’re after. It won’t tinker with the EDC ”bug” and will generally be safe to use. The “High single-core” preset is mostly the “Best of both” with a higher frequency. This also sets the voltage to a higher value so this is recommended in cases where no heavy multi-core applications are used. The “High multi-core” preset uses the EDC “bug” to get the most multi-core performance as possible. The 64-core ‘ZS14’ CPU actually performs like a stock 3990x and beats an $8000 7H12 in this configuration.

The presets might not be suited for every task out there so feel free to tinker around and share your configuration in this thread.

But the CPU doesn't post on my -insert brand- motherboard!!!
That's what these things do. Engineering sample CPUs just aren't supported by every board out of the box. There is a simple trick to make it work on any board, even on Rev. 1 boards. The only thing you need to do is copy the second half of the bios (the Rome part) and duplicate it on Rev. 2 boards or just upload that on Rev. 1 boards. Make sure the bios has an Agesa < 1.0.0.3 because otherwise it will not work. The engineering samples do work out of the box on the Asus KRPA-U16. I’ve included some pre-‘modded’ bioses below.

Some disclaimers
Please keep in mind that I haven’t tested anything for 24/7 use, although I don’t think there should be a problem as long as the voltage isn’t too high. Please check the voltage via the IPMI to confirm it’s not too high and if it is, dial back the settings. Please be careful when overclocking your CPU, the last thing you want is to destroy it like I did. I am not responsible for any damage done to the CPU so use this software at your own risk.

Screenshots
Here are some screenshots of the true power of these engineering samples :cool:




If you have any questions, suggestions or want to share anything, feel free to post it in this thread or DM me. For now, enjoy the massive performance of these things and keep an eye on your VRM temps :D

tldr; Read the whole post you lazy because your CPU might DIE!

Download link for the software

- ExecutableFix

Massive thanks to @nero243 and @I.nfraR.ed for their contributions to this project as well!
Is there a GitHub repository or anything where we could experiment with and/or contribute to the development of the software? This seems really interesting, but I'm not a fan of losing power-management and running at maximum frequency all the time. I would be interested in writing some software to detect the OS load and change the p-states accordingly (along with giving myself an API which my own software could use to lock clocks, set configurations (e.g. single-thread-performance vs. multi-core-performance vs. balanced), etc.)
 

Layla

Game Engine Developer
Jun 21, 2016
215
177
43
40
Is there a GitHub repository or anything where we could experiment with and/or contribute to the development of the software? This seems really interesting, but I'm not a fan of losing power-management and running at maximum frequency all the time. I would be interested in writing some software to detect the OS load and change the p-states accordingly (along with giving myself an API which my own software could use to lock clocks, set configurations (e.g. single-thread-performance vs. multi-core-performance vs. balanced), etc.)
Oh, to second-guess my own post: it looks like the VRMs on the Epyc boards might not be up to the task of quickly changing frequency from C-States to P-States at those high frequencies and requisite core voltages... Might be the case that locking clocks to high or low for extended periods of time are the only option after-all. :/
 

ExecutableFix

Active Member
Nov 25, 2019
123
64
28
Is there a GitHub repository or anything where we could experiment with and/or contribute to the development of the software? This seems really interesting, but I'm not a fan of losing power-management and running at maximum frequency all the time. I would be interested in writing some software to detect the OS load and change the p-states accordingly (along with giving myself an API which my own software could use to lock clocks, set configurations (e.g. single-thread-performance vs. multi-core-performance vs. balanced), etc.)
It's not open-souce yet as I wanted to await some feedback on the forum first. It's not really possible to implement what you're suggesting because P-States do not work on Rome. The software is entirely based on the SMU. The commands I'm using are most likely debug commands built into engineering samples for testing purposes. This also means that it doesn't like frequencies being changed often as it just works differently than setting p-states.

Oh, to second-guess my own post: it looks like the VRMs on the Epyc boards might not be up to the task of quickly changing frequency from C-States to P-States at those high frequencies and requisite core voltages... Might be the case that locking clocks to high or low for extended periods of time are the only option after-all. :/
I don't think they're not up for the task as long long as you're not going crazy with high frequencies. This because the frequency set by the smu can still be throttled by the motherboard without reflecting it in the shown frequency. Eg. you've set the frequency at 3.3GHz but it performs worse than setting it to 2.6GHz in certain workloads. So yes, you'll have to settle for a single clockspeed. If you're looking for the best single-threaded and multi-threaded EPYC, the 32-core would be your best bet. The 64-core will be the better choice for low single-core but high-multi core performance.
 

ExecutableFix

Active Member
Nov 25, 2019
123
64
28
Do you mean the custom bios mention in the start post here:
https://forums.servethehome.com/index.php?threads/overclock-your-amd-epyc.23422/

V2.0 Rome tested:
H11SSLv2.0SSPMod

?

Then the EDC-Bug is confusing me. What is the benefit of overclock it by edc bug? Can I use the EDC-Bug OC on linux at all? The overclock via EDC-Bug higher then 3,8GHz might (or probably) will kill the 64C, but not the 32C EPYC?

A while ago I have read, that there exist some Ryzen per CCX overclock for 3900X/3950X - does there exist something similar for EPYCs - would be aweseome to clock one CCX to maybe 4 GHz and let the rest on some "low clock".
No, the bioses I attached to the post contains a "custom" bios for the H11SSL-i. You will need that one to make it work on that board. The EDC bug is usable in every OS because it's a firmware level bug (AGESA), but you will only see a performance difference when using it on the 64-core or the 32-core "2S1705" CPU. There's no per-CCX overclock possible on Rome as of now.
 

efschu3

Active Member
Mar 11, 2019
160
61
28
No, the bioses I attached to the post contains a "custom" bios for the H11SSL-i. You will need that one to make it work on that board. The EDC bug is usable in every OS because it's a firmware level bug (AGESA), but you will only see a performance difference when using it on the 64-core or the 32-core "2S1705" CPU. There's no per-CCX overclock possible on Rome as of now.
Thanks for clarification. Does your modified UEFIs have CBS menu unlocked? Can we overclock RAM (i.e. running 2133 specified RAM on higher clocks) and can we set infinity fabric speed? Btw. what is the stock infinity fabric speed on EPYC Rome?
 

bayleyw

Active Member
Jan 8, 2014
291
95
28
This because the frequency set by the smu can still be throttled by the motherboard without reflecting it in the shown frequency. Eg. you've set the frequency at 3.3GHz but it performs worse than setting it to 2.6GHz in certain workloads. So yes, you'll have to settle for a single clockspeed. If you're looking for the best single-threaded and multi-threaded EPYC, the 32-core would be your best bet. The 64-core will be the better choice for low single-core but high-multi core performance.
Do we know what causes the board to throttle the frequencies yet? The obvious first guess would be the VRM's see that the load is too high and refuses to output the requested voltage, but I'm quite unfamiliar with how the latest AMD CPU's work in this regard.

@efschu3 Fabric is capped at 1467MHz (DDR4-2933 speeds) - if the fabric clock doesn't match the RAM clock you incur an extra cycle penalty for memory transfers.
 
Last edited:

efschu3

Active Member
Mar 11, 2019
160
61
28
Yeah I know about IF and RAM clock. This is why I asked if we can clock RAM manualy to what we want it to be, or is it just set by reading the rams specs?

Edit:
Btw, does the ASUS KNPA-U16 work too? Atm kinda cheaper then the supermicros.
 
Last edited:

bayleyw

Active Member
Jan 8, 2014
291
95
28
Doing a little more digging, it looks like Matisse (and presumably Castle Peak and Rome as well) clock stretch when they detect downward voltage transients, so if the VRM's can't quite keep up with the load the CPU will slow down to avoid crashing.
Has anyone instrumented their board yet to keep track of temperatures and actual Vcore under load? My board and 64c ES show up soon but it will be some time before I can take any serious measurements - all of my interesting test equipment is stuck on the wrong coast right now. Plus knowing these 'esoteric' builds it will probably take a couple tries to get things running.
 

kaixin

New Member
Apr 24, 2020
4
0
1
hello @ExecutableFix
I have a very important question:
Is there any way to avoid the overclocking behavior of the tool? Such as by setting the BIOS to reject the function, or Ask the motherboard manufacturer to update the BIOS to solve the problem?
Because I have a lot servers running of ROME 32-core ES(100-000000054-04_32/24_N) CPUs with supermicro h11dsi rev2.0, I need to avoid someone damaging my cpus.