Hey guys, I have my first real server hardware here, also 2 epyc 32core ES.
Been a whole road to get it all working but as a experienced tuner I got a lot of thoughts and experiences with the setup I wanted to share.
First of got the pair of 2s1705e3vivg5 BB ES 32 core samples second hand.
Bought a H11DSI-NT V2 supermicro board because, its the only dual socket sold separately and the v2 version for the 10gb lan.
I got half of the dimms populated with 2933mhz rdimms, just the money was not sufficient to get everything filled for now.
Running windows server 2019.
The samples run by default run at 1,7ghz, kinda slow but with the help of the program I could get it up to 3ghz/1v program settings.
In reality this meant that it was running at 2,4ghz on 1.34V, and that's fast enough for me, also seem to be a bit at the max here of what can be done with ease.
I know from TRX40, TR4 and ryzen how important memory and IF speed is for the end result.
So I was very surprised to read that SP3 IF is up to maximum 1467Mhz because TRX40 for example goes to 1800+
This would mean that up to 2933mhz the fabric is clocked 1:1 and at 3200mhz there is a divider active.
After testing it out with AIDA cache and memory tester you can see that at 2933 there is the least latency, at 3200 latency increases and bandwith increases.
So that would be a clear sign of the clock divider becoming active.
Its a shame we cant 'modify' the IF max clock for 1:1 ratio, this is something I have done on ryzen mobile where it also should not be possible.
Maybe there's some hope. If we could get the IF up to 1600 with a 1:1 locked ratio would be a huge improvement in performance and latency.
I can advice everyone to buy or get ur ram to 2933, not so much for the bandwidth but for the decrease in latency and overall improvement because of the faster running Infinity fabric. To be clear, 2933 is the best speed vs latency option.
In the bios you can also tweak the speed between the two sockets, by default its running at 10,66gbps.
I got it up to 16gbps, at 18gbps its very unstable. But a decent improvement in CB score.
Also forced in bios that the infinity fabric should run max speed all the time, not going in sleep mode to prevent jitter from cycles needed to reup the clock.
Damn, what are these server boards crappy the settings. After reading the complete amd guideline document about rome and all the options and functionality's I should be able to adjust, I can honestly say supermicro has not implemented even half of the functions.
Compared to a trx40 system this feels like a 10 dollar OEM mobo bios.
At the Moment I am close to the 19K points in CB, but with only half the dimms per socket filled. So I think there is still a lot of headroom for improvement when more dimms arrive.
Things that I would really love:
1 Memory timing adjustment, one of the big gains on desktop is to do every dramm timing custom, primary, secondary and tertiary timings. I know it can be done because when placing the same DDR4 ecc Rdimm in a trx system all the memory timings are tunable.
2 More IF control and options. I will see if the hack that works on ryzen mobile could maybe also work on EPYC, if the IF could go to 1600 1;1 running 3200mhz would actually give a big performance increase instead of the mixed bag it is now.
3 Like with all zen, latency is the biggest enemy, if you decrease it performance goes up drastic
4 Noctua coolers named TR4-SP3 are not really sp3 proof............ something with flow direction.
Anyway, always open to feedback or suggestions about all.
Greetings.
