Finally: Overclocking EPYC Rome ES

dragonian · Feb 17, 2021

I have 2 H11SSL-i v2 boards running with ZS1711E3VIVG5 (Rome ES 32c) cpus

I was able to update the IPMI to the latest with the older Rome BIOS 32MB (on the 1st page of this thread) I unchecked all of the "preserve" boxes -- I believe this is important.

Firmware Revision: 01.52.00
Firmware Build Time: 11/18/2019
BIOS Version: 2.0a
BIOS Build Time: 08/08/2019
Redfish Version: 1.0.1
CPLD Version: 02.b4.02

I do get sensor readings:

Status	Reading
CPU Temp	Normal	45 degrees C
System Temp	Normal	43 degrees C
Peripheral Temp	Normal	41 degrees C
M2NVMeSSD Temp1	N/A	Not Present!
VRMCpu Temp	Normal	41 degrees C
VRMSoc Temp	Normal	51 degrees C
VRMABCD Temp	Normal	40 degrees C
VRMEFGH Temp	Normal	42 degrees C
DIMMA1 Temp	N/A	Not Present!
DIMMB1 Temp	N/A	Not Present!
DIMMC1 Temp	Normal	38 degrees C
DIMMD1 Temp	Normal	39 degrees C
DIMME1 Temp	N/A	Not Present!
DIMMF1 Temp	N/A	Not Present!
DIMMG1 Temp	Normal	39 degrees C
DIMMH1 Temp	Normal	37 degrees C
FAN1	N/A	Not Present!
FAN2	Normal	600 R.P.M
FAN3	Normal	1600 R.P.M
FAN4	Normal	700 R.P.M
FAN5	Normal	1100 R.P.M
FANA	Normal	1000 R.P.M
FANB	Normal	1000 R.P.M
12V	Normal	11.984 Volts
5VCC	Normal	4.96 Volts
3.3VCC	Normal	3.259 Volts
VBAT		Battery presence detected.
VDDCR	Normal	1.165 Volts
VMEMABCD	Normal	1.225 Volts
VMEMEFGH	Normal	1.214 Volts
VDD_5_DUAL	Normal	5.099 Volts
VDD_33_DUAL	Normal	3.242 Volts
SOCRUN	Normal	1.095 Volts
SOCDUAL	Normal	0.886 Volts

RolloZ170 · Feb 17, 2021

dragonian said:
I was able to update the IPMI to the latest with the older Rome BIOS 32MB (on the 1st page of this thread) I unchecked all of the "preserve" boxes -- I believe this is important.

and are you able to see sensors(i.e. CPU voltages) in the IPMI frontend ?

dragonian · Feb 17, 2021

RolloZ170 said:
and are you able to see sensors(i.e. CPU voltages) in the IPMI frontend ?

Yes. I ended up editing my previous post, I ended up hitting post prematurely. You can see them if you hit the spoiler button.

alex_stief · Mar 6, 2021

Sooo... Epyc Milan aka Zen3 Epyc should be right around the corner for retail availability. At least I hope so.
What might be the chances of getting those CPUs to run on a rev.1 Supermicro H11DSi. It's still the same socket after all, and some of you managed to get Rome CPUs running on rev. 1 boards.

RolloZ170 · Mar 6, 2021

alex_stief said:
What might be the chances of getting those CPUs to run on a rev.1 Supermicro H11DSi.

the upper 16MB of a BIOS H11DSi rev.2(naples/rome)) is flashed on the rev. 1 board.
if there will be a BIOS rome/milan for the H11DSi then there is IMHO a chance.

Brainbug · Mar 11, 2021

@I.nfraR.ed Thanks for making the Linux version.

From the k10temp sources, it looks like tctl has a 27C offset. Does this mean that the actual CPU temperatures are 27C lower than reported? Curious, as the idle temp seems high, but matches what is shown in IPMI. (Load gets up to mid 90s, which seems accurate based on the 3990x reviews, but ** really ** high for a server CPU. If it was actually 90-27C = 63C, that would make me feel much better...)

For others using Linux with ZS processors, newer kernels have better sensor support for Zen2 systems - However, they aren't correctly detected by the kernel as the Processor version for Rome ZS is 0x30 vice 0x31.

For improved temp support, open "drivers/hwmon/k10temp.c" as sudo/root and search for "0x31". Add "case 0x30:" between "break;" and "case 0x31"

Code:

        } else if (boot_cpu_data.x86 == 0x17 || boot_cpu_data.x86 == 0x18) {
                data->temp_adjust_mask = ZEN_CUR_TEMP_RANGE_SEL_MASK;
                data->read_tempreg = read_tempreg_nb_zen;
                data->show_temp |= BIT(TDIE_BIT);       /* show Tdie */
                data->is_zen = true;

                switch (boot_cpu_data.x86_model) {
                case 0x1:       /* Zen */
                case 0x8:       /* Zen+ */
                case 0x11:      /* Zen APU */
                case 0x18:      /* Zen+ APU */
                        k10temp_get_ccd_support(pdev, data, 4);
                        break;
                case 0x30:
                case 0x31:      /* Zen2 Threadripper */
                case 0x71:      /* Zen2 */
                        k10temp_get_ccd_support(pdev, data, 8);
                        break;
                }
        }

To see the power usage by core, open "drivers/hwmon/amd_energy.c" and search for "0x31". In the "x86_cpu_id_cpu_ids[]" struct, change the 0x31 to 0x30. (Note this will stop this driver from working for retail Zen2 CPUs, and only work for ES)

Code:

static const struct x86_cpu_id cpu_ids[] __initconst = {
        X86_MATCH_VENDOR_FAM_MODEL(AMD, 0x17, 0x30, NULL),
        {}
};

Remove the compiled drivers, then rebuild, and reload

Code:

 $>rm drivers/hwmon/amd_energy.o drivers/hwmon/amd_energy.mod.o drivers/hwmon/amd_energy.ko drivers/hwmon/k10temp.o drivers/hwmon/k10temp.ko drivers/hwmon/k10temp.mod.o
$> make
$> make modules_install
$> modprobe k10temp amd_energy

When you run sensors, you should now get output that resembles:

Code:

k10temp-pci-00c3
Adapter: PCI adapter
Tctl: +44.8 C
Tdie: +44.8 C
Tccd1: +43.2 C
Tccd2: +42.8 C
Tccd3: +42.8 C
Tccd4: +42.8 C
Tccd5: +41.5 C
Tccd6: +45.0 C
Tccd7: +41.0 C
Tccd8: +43.5 C

amd_energy-isa-0000
Adapter: ISA adapter
Ecore000: 2.76 kJ
Ecore001: 2.57 kJ
Ecore002: 2.63 kJ
Ecore003: 2.57 kJ
Ecore004: 2.63 kJ
Ecore005: 2.54 kJ
Ecore006: 2.62 kJ
...
Ecore062: 3.54 kJ
Ecore063: 3.47 kJ
Esocket0: 1.13 MJ

Wasmachineman_NL · Mar 11, 2021

Milan ES overclocking when :O

bayleyw · Mar 12, 2021

Wasmachineman_NL said:
Milan ES overclocking when :O

Probably after we get Milan ESes. Should be good though, with a mature 7nm process even the early ES revisions of Milan will clock quite well.

irgen · Mar 13, 2021

Guys still trying to make it work with zs1711e3vivg5 , now i replaced h11ssl v1.0 with rev. 2.0 board, it boots into windows fine but when i try to give it more-less significant load like CB it goes black screen and reboots after 1 sec. What could it be? Currently im using just one 8gb 2400 stick and for windows 10 i just borrowed ssd of old lenovo pc - so it is not fresh install and thap pc was intel.. so i have some devices with yellow mark it does not found drivers for, could that be the issue?
forgot to mention i did not overclock it yet

RolloZ170 · Mar 13, 2021

irgen said:
zs1711e3vivg5 , now i replaced h11ssl v1.0 with rev. 2.0 board

which BIOS version ?

irgen said:
so i have some devices with yellow mark it does not found drivers for, could that be the issue?

if you install the Rome drivers there will be some yellow marks left, that are dummy devices. you can manualy install them or leave it as it is.

irgen · Mar 13, 2021

RolloZ170 said:
which BIOS version ?

ver. 2.0a from page 1 of this thread. I didnt installed any Rome drivers - AMD platform is something new to me - do i need some drivers for the CPU?

RolloZ170 · Mar 13, 2021

irgen said:
AMD platform is something new to me - do i need some drivers for the CPU?

any CPU needs drivers. they are normaly included in the chipset drivers. EPYC is system on chip.
download the SP3 IO Driver from the supermicro page of your motherboard.

irgen · Mar 13, 2021

RolloZ170 said:
any CPU needs drivers. they are normaly included in the chipset drivers. EPYC is system on chip.
download the SP3 IO Driver from the supermicro page of your motherboard.

thank you i will try to install this driver and see if anything changes!

RolloZ170 · Mar 13, 2021

irgen said:
AMD platform is something new to me

which Power supply ? 8 Pin (EPS) connected ?

irgen · Mar 14, 2021

RolloZ170 said:
which Power supply ? 8 Pin (EPS) connected ?

Yes 8 pin is connected, but 4 pin is not. I tried to run Vraybench software on just one core , using Set Affinity, and going one-by-one cores found out that it immediately crashes if running on core 6-7 (which is core 3 in hwinfo)
Does that means i have a broken cpu? Also RomeOverclock.exe does not run on my system its just says -program stopped working - right after i launch
edit RomeOverclock 1.0 works but RomeOverclock 1.0.1 does not. So i was able to finish the benchmark with 3300 al cores but 3 disabled with SetAffinity. Is there a way i can disable that in bios?

RolloZ170 · Mar 14, 2021

irgen said:
Yes 8 pin is connected, but 4 pin is not.

~~Attention ! the 4 Pin is NOT a power input.~~
the 4 Pinmust connected to.

RolloZ170 · Mar 14, 2021

irgen said:
I tried to run Vraybench software on just one core , using Set Affinity, and going one-by-one cores found out that it immediately crashes if running on core 6-7 (which is core 3 in hwinfo)
Does that means i have a broken cpu?

that would be my next suspection. a broken core. yes, highly possible unfortunaly

irgen · Mar 14, 2021

RolloZ170 said:
that would be my next suspection. a broken core. yes, highly possible unfortunaly

I believe i can disable that core in bios with core bitmap?

RolloZ170 · Mar 14, 2021

irgen said:
I believe i can disable that core in bios with core bitmap?

needs a little luck, never done that on EPYCs.

irgen · Mar 14, 2021

RolloZ170 said:
needs a little luck, never done that on EPYCs.

Yes i found manual for Skylake how Downcore Bitmap shoud work, but so far cant get the logic how it works for AMD (if at all)
Did someone tried to use that?

Finally: Overclocking EPYC Rome ES

Member

Well-Known Member

Member

Well-Known Member

Well-Known Member

New Member

Wittgenstein the Supercomputer FTW!

Active Member

Member

Well-Known Member

Member

Well-Known Member

Member

Well-Known Member

Member

Well-Known Member

Well-Known Member

Member

Well-Known Member

Member