And so it begins... First AMD Ryzen AM4 server motherboard.
- Thread starter McKajVah
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
This is, again, one of the things where a similarly geared EPYC Embedded would be good at.
A Motherboard close to the X470D4U but based on that instead of standard AM4 could be a prosumer monster.
BIOS v3.04 has just been posted (along with v3.02 for the X470D4U2-2T) listing 7nm Ryzen support, so that pretty much confirms current BIOS won't work with Ryzen2 off the bat*... so, assuming benches and power levels are all I'm hoping for, likely a couple of months before I get myself one of these... so there's still time for Supermicro to get my money by announcing a mATX Epyc 3000 
Still though... for less than £500 I'm looking at getting an 8C/16T IPMI-enabled home server platform... even with the high cost of ECC UDIMMs it's still a bargain (and I doubt an Epyc 3000 would be that affordable either).
* Many X470 desktop boards are flashable without a CPU installed but I've no idea if that's possible on this board, the manual certainly didn't mention it and I don't know if it's possible through IPMI-sans-CPU either.
Still though... for less than £500 I'm looking at getting an 8C/16T IPMI-enabled home server platform... even with the high cost of ECC UDIMMs it's still a bargain (and I doubt an Epyc 3000 would be that affordable either).
* Many X470 desktop boards are flashable without a CPU installed but I've no idea if that's possible on this board, the manual certainly didn't mention it and I don't know if it's possible through IPMI-sans-CPU either.
I had put together a parts list this weekend, but then realized that the 32GB DIMMs were RDIMM not UDIMM
I haven't seen Micron announce any 32GB ECC UDIMMs yet. Samsung announced two, with the M391A4G43MB1-CTD 2666 MHz variant being the only one listed in the US, however ofc it's OOS.
I'd love to use this as a compute server, or even as a FreeNAS build seeing as all the Intel vulnerabilities being quite a turn off. The Threadripper mobo seems nice too, however I like to separate my compute and storage servers.
One very minor annoyance for Ryzen is the non-APU CPUs don't have an iGPU for transcodes. A Ryzen APU for a server is a no-go for me since it only exposes a x8 link rather than a x16 link from the CPU itself.
I haven't seen Micron announce any 32GB ECC UDIMMs yet. Samsung announced two, with the M391A4G43MB1-CTD 2666 MHz variant being the only one listed in the US, however ofc it's OOS.
I'd love to use this as a compute server, or even as a FreeNAS build seeing as all the Intel vulnerabilities being quite a turn off. The Threadripper mobo seems nice too, however I like to separate my compute and storage servers.
One very minor annoyance for Ryzen is the non-APU CPUs don't have an iGPU for transcodes. A Ryzen APU for a server is a no-go for me since it only exposes a x8 link rather than a x16 link from the CPU itself.
I hadn't yet installed the CPU into the motherboard, so I gave this a try. The IPMI does have an option to flash the BIOS, which I was able to use to flash v3.04 without the CPU being installed. Interestingly, I did discover that the system can't actually power on without a CPU installed. With power connected, the motherboard's LEDs light up and I am able to access IPMI, but the fans don't spin and the Dr. Debug module doesn't show anything. I have no idea whether this is the case with all motherboards, as I've never not installed the CPU.
So it looks like you'll be able to buy the board with a Ryzen 3000 CPU without any worries.
Superb info, thank you! I assume you're going for a Ryzen 2000 build as opposed to waiting for the 3000 series?
It's also confirmed that the ASRock motherboards also don't need abash one-liner £20 license to allow the BIOS flashing via IPMI
It's also confirmed that the ASRock motherboards also don't need a
I've not confirmed this with a rep from ASRockRack who tested it themselves and confirms the BIOS is flashable via BMC without a CPU installed.
They also pointed me at their own stab at a centralised system management tool here, available as a linux/windows electron package; apparently BIOS flashes will work through this too.
I had a conversation with ASRock the other day and they confirmed they will not be making a mITX X470 board with IPMI.
However, they confirmed that a mITX X570 board with onboard dual 10GBASE-T using the Intel X550-AT2 will be available sometime 2019Q4 or later. This is very exciting!
However, they confirmed that a mITX X570 board with onboard dual 10GBASE-T using the Intel X550-AT2 will be available sometime 2019Q4 or later. This is very exciting!
So just to confirm the prior posts, ASRock support and my own optimism... my motherboard arrived today, as expected with the 1.50 BIOS. Plugged in the IPMI port, scoured the DHCP logs to find the IP address, logged in, upgraded BIOS to v3.04 (no license hack needed, yay!).
Installed the CPU and memory and we're able to go straight in.
Initial thoughts:
As ever, more work needed. As per other messages in this thread, I'll try the same fio SSD test when I get a SATA drive plugged in but don't expect me to be moving too quickly
Installed the CPU and memory and we're able to go straight in.
Initial thoughts:
- Lots of swishy animations in the IPMI GUI that I'm really not a fan of, and the GUI seems to be trying to cram everything into a phone-style display that makes no use of horizontal space, but functionality-wise it's an improvement on their previous offerings. I think I might even prefer their HTML5 GUI to Supermicro's.
- Many of the onboard sensors don't appear to work in the IPMI interface yet (most importantly CPU and memory temperatures)
Code:root@frogstar:~# ipmitool sensor 3VSB | 3.380 | Volts | ok | 2.800 | 2.980 | na | na | 3.620 | 3.780 5VSB | 5.010 | Volts | ok | 4.260 | 4.500 | na | na | 5.490 | 5.730 VCPU | 0.190 | Volts | ok | na | na | na | na | 1.650 | 1.730 VSOC | 1.030 | Volts | ok | 0.340 | 0.360 | na | na | 1.540 | 1.610 VCCM | 1.190 | Volts | ok | 1.020 | 1.080 | na | na | 1.320 | 1.380 APU_VDDP | 0.950 | Volts | ok | 0.770 | 0.810 | na | na | 1.160 | 1.210 3V | 3.360 | Volts | ok | 2.800 | 2.980 | na | na | 3.620 | 3.780 5V | 5.130 | Volts | ok | 4.260 | 4.500 | na | na | 5.490 | 5.730 12V | 12.100 | Volts | ok | 10.200 | 10.800 | na | na | 13.200 | 13.800 MB Temp | na | degrees C | na | na | na | na | 55.000 | na | na Card side Temp | na | degrees C | na | na | na | na | 68.000 | na | na CPU Temp | na | degrees C | na | na | na | na | 91.000 | na | na DDR4_A2_Temp | na | degrees C | na | na | na | na | 84.000 | 85.000 | na DDR4_A1_Temp | na | degrees C | na | na | na | na | 84.000 | 85.000 | na DDR4_B2_Temp | na | degrees C | na | na | na | na | 84.000 | 85.000 | na DDR4_B1_Temp | na | degrees C | na | na | na | na | 84.000 | 85.000 | na FAN1 | 1000.000 | RPM | ok | na | na | 100.000 | na | na | na FAN2 | na | RPM | na | na | na | 100.000 | na | na | na FAN3 | na | RPM | na | na | na | 100.000 | na | na | na FAN4 | na | RPM | na | na | na | 100.000 | na | na | na FAN5 | na | RPM | na | na | na | 100.000 | na | na | na FAN6 | na | RPM | na | na | na | 100.000 | na | na | na ChassisIntr | 0x0 | discrete | 0x0080| na | na | na | na | na | na CPU_PROCHOT | 0x0 | discrete | 0x0080| na | na | na | na | na | na CPU_THERMTRIP | 0x0 | discrete | 0x0080| na | na | na | na | na | na PSU1 Status | 0x0 | discrete | 0x0080| na | na | na | na | na | na PSU1 AC lost | na | discrete | na | na | na | na | na | na | na PSU2 Status | 0x0 | discrete | 0x0080| na | na | na | na | na | na PSU2 AC lost | na | discrete | na | na | na | na | na | na | na 1.05V_PROM_S5 | 1.060 | Volts | ok | 0.890 | 0.950 | na | na | 1.160 | 1.210 2.5V_PROM | 2.560 | Volts | ok | 2.120 | 2.260 | na | na | 2.740 | 2.880 1.05V_PROM_RUN | 1.010 | Volts | ok | 0.890 | 0.950 | na | na | 1.160 | 1.210 BAT | 3.000 | Volts | ok | 2.000 | 2.700 | na | na | 3.400 | 3.560 PSU2 PIN | na | Watts | na | na | na | na | na | na | na PSU2 POUT | na | Watts | na | na | na | na | na | na | na PSU1 PIN | na | Watts | na | na | na | na | na | na | na PSU1 VIN | na | Volts | na | na | na | na | na | na | na PSU2 VIN | na | Volts | na | na | na | na | na | na | na PSU2 IOUT | na | Amps | na | na | na | na | na | na | na PSU1 IOUT | na | Amps | na | na | na | na | na | na | na PSU1 POUT | na | Watts | na | na | na | na | na | na | na - I'm using the Noctua NH-L9x65 SE-AM4 HSF and the space between it and the first DIMM slot is really tight, to the extent the heatpipe touches the DIMM to the extent you need to bend it slightly
- A test debian stretch netinstall on to an M2 went with zero problems (even using what some would consider the odd config of "legacy" MBR boot from an NVME drive, which works without issue)
- Ryzen 3000 is fast. Like, really nicely fast. I already disabled PBO in the BIOS as I don't plan to use (power efficiency ) it but here's some quick'n'dirty openssl scores compared with my current Haswell E3 (same version of openssl also under debian stretch):
Code:type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes md5 131680.77k 312282.73k 552217.26k 685280.60k 731400.87k 739469.99k sha1 141524.19k 335635.35k 694369.54k 925579.61k 1019554.47k 1038150.31k sha256 84687.84k 186115.69k 344207.19k 437095.42k 472416.26k 471684.44k sha512 54593.59k 218337.73k 384408.40k 571728.55k 680763.39k 684119.38kCode:type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes md5 165881.11k 376279.10k 654155.95k 805793.11k 863455.91k 868013.40k sha1 289492.20k 744160.96k 1513553.49k 2047567.87k 2281201.66k 2300051.46k sha256 263302.86k 666966.55k 1388905.47k 1905337.69k 2137448.45k 2156341.93k sha512 80800.28k 323225.73k 543556.52k 796167.51k 918145.71k 928879.96k - Idle power usage of the out-of-the-box config is pretty good at ~31W (that's with a single ECC DIMM). I dare say this'll be reduced considerably with some tweaking - especially given that the idle power usage of some of the X570 systems was pushing north of 60W in many of the benches I read (granted, almost all with a GPU as well)
- ECC is seemingly working, or at least reported as available; lshw and dmidecode report multi-bit ECC although I'm not sure that's correct given this is unbuffered memory (I suspect it's reading what the memory controller/motherboard is reporting rather than what's actually "negotiated"). edac-util doesn't recognise the Ryzen 2 driver when used with kernel 4.9 or 4.19 from backports. I was somewhat surprised to see dmidecode reporting a data width of 128bits rather than 72 so I'm not entirely sure what's going on here:
Code:Handle 0x0019, DMI type 17, 40 bytes Memory Device Array Handle: 0x000F Error Information Handle: 0x0018 Total Width: 128 bits Data Width: 64 bits Size: 16384 MB Form Factor: DIMM Set: None Locator: DIMM 1 Bank Locator: P0 CHANNEL A Type: DDR4 Type Detail: Synchronous Unbuffered (Unregistered) Speed: 2666 MHz Manufacturer: Micron Technology Serial Number: xxxxxxxxxxxxx Asset Tag: Not Specified Part Number: 18ASF2G72AZ-2G6D1 Rank: 2 Configured Clock Speed: 2666 MHz Minimum Voltage: 1.2 V Maximum Voltage: 1.2 V Configured Voltage: 1.2 V
As ever, more work needed. As per other messages in this thread, I'll try the same fio SSD test when I get a SATA drive plugged in but don't expect me to be moving too quickly
Yup, just as long as you update the BIOS beforehand - this was the first time I've updated a BIOS without even installing the CPU in the socket, doing so through IPMI was rather cool. As per my comment further up the thread there's also ASRock's management utility that can do the same thing indirectly.
It's going to be a while before it achieves replacement status for me but I'm optimistic as of the last hour or two; suspect a couple of BIOS updates are all that's needed to fix the IPMI sensors and a newer kernel to support ECC detection for the memory controller.
P.S. One thing I noticed in the dmidecode output is that the CPU reported supporting a maximum of 128GB so I'm cautiously optimistic that support for 32GB UDIMMs is already baked in to the controller and will arrive at some point in the future.
It's going to be a while before it achieves replacement status for me but I'm optimistic as of the last hour or two; suspect a couple of BIOS updates are all that's needed to fix the IPMI sensors and a newer kernel to support ECC detection for the memory controller.
P.S. One thing I noticed in the dmidecode output is that the CPU reported supporting a maximum of 128GB so I'm cautiously optimistic that support for 32GB UDIMMs is already baked in to the controller and will arrive at some point in the future.
Code:
root@frogstar:~# edac-util
edac-util: Error: No memory controller data found.
root@frogstar:~# dmidecode --type memory
# dmidecode 3.0
Getting SMBIOS data from sysfs.
SMBIOS 3.2.0 present.
# SMBIOS implementations newer than version 3.0 are not
# fully supported by this version of dmidecode.
Handle 0x000F, DMI type 16, 23 bytes
Physical Memory Array
Location: System Board Or Motherboard
Use: System Memory
Error Correction Type: Multi-bit ECC
Maximum Capacity: 128 GB
Error Information Handle: 0x000E
Number Of Devices: 4I suppose you could use some other monitoring software to check temps unless those in the IPMI are that important to you. I personally won't need more than even 16GB with my server, but it is nice to know it may support more for people in this thread that want to run a lot of VMs. Can you select any faster speeds for memory in BIOS or is it stuck at 2666MHz?
Definitely nice that you can flash through IPMI in case you don't have an older CPU. I know some AM4 boards have a BIOS flashback feature so you can do this with no CPU/memory and some have dual BIOS like GPUs sometimes have.
Definitely nice that you can flash through IPMI in case you don't have an older CPU. I know some AM4 boards have a BIOS flashback feature so you can do this with no CPU/memory and some have dual BIOS like GPUs sometimes have.
2666MHz is the speed of the memory, it's the fastest ECC UDIMM I could find and the fastest supported by ASRock. There's a trillion options in the BIOS that I suspect I'll need days to read up on to understand, but ultimately memory performance is much less of a big deal on Ryzen 2 than it was on previous generations so I'm not overly concerned in that regard.
I certainly didn't intend to run it any faster than 2666, and it's possible it might drop down to 2400 when I add the second DIMM anyway, but even so the new architecture seems to mitigate the need for high-bandwidth, low-latency RAM in order for it to perform well. All the benches I've seen so far (granted, all windows, none for linux) have shown the performance difference between 2400 and 3200 to be under 5% (at least for the kind of workloads I care about which are all compute-heavy and not memory-heavy). Indeed, it was the reviews that showed how much Ryzen was no longer dependent on memory that convinced me to pull the trigger so soon after launch (although I've been planning to upgrade my haswell for over a year now). Great to see AMD back in the game with such aplomb.
I'm not usually the sort of person who worries about temps (quite the opposite really) but given that it's a brand new proc that I bought precisely to chuck a fairly CPU-heavy workload at (mostly ffmpeg) it'd be good to be reasonably sure I'm not cooking things.
The BIOS flash routines via USB were only present on a few of the X470 boards but yeah it's great to see the ASRock kit taking advantage of it. Since I got my Supermicro IPMI BIOS flashing working I've been hooked on it, so it's great that ASRock provide it for free and without making you jump through any hoops.
I certainly didn't intend to run it any faster than 2666, and it's possible it might drop down to 2400 when I add the second DIMM anyway, but even so the new architecture seems to mitigate the need for high-bandwidth, low-latency RAM in order for it to perform well. All the benches I've seen so far (granted, all windows, none for linux) have shown the performance difference between 2400 and 3200 to be under 5% (at least for the kind of workloads I care about which are all compute-heavy and not memory-heavy). Indeed, it was the reviews that showed how much Ryzen was no longer dependent on memory that convinced me to pull the trigger so soon after launch (although I've been planning to upgrade my haswell for over a year now). Great to see AMD back in the game with such aplomb.
I'm not usually the sort of person who worries about temps (quite the opposite really) but given that it's a brand new proc that I bought precisely to chuck a fairly CPU-heavy workload at (mostly ffmpeg) it'd be good to be reasonably sure I'm not cooking things.
The BIOS flash routines via USB were only present on a few of the X470 boards but yeah it's great to see the ASRock kit taking advantage of it. Since I got my Supermicro IPMI BIOS flashing working I've been hooked on it, so it's great that ASRock provide it for free and without making you jump through any hoops.
I wonder what the memory speed will be with 4 x DR ECC UDIMM and Ryzen 3000’s improved IMC. Since 32GB ECC UDIMMs are still magical unicorns, acceptable speed with 16GB modules might be the way to go since the maximum memory is 64GB anyway.
I’ve been trying to source this board for a while here in the US, and not hot on paying Newegg’s $400+ current list price. If it takes too long for the price to go back to $250-270 the X570 based ASRock Rack boards might be out by then (Q4).
I’ve been trying to source this board for a while here in the US, and not hot on paying Newegg’s $400+ current list price. If it takes too long for the price to go back to $250-270 the X570 based ASRock Rack boards might be out by then (Q4).
It is on Amazon for $270. Not much benefit in getting X570 though except for PCIE 4.0. It looks like the all have active cooling on the chipset and use a lot of power too.
https://www.amazon.com/ASRock-Rack-X470D4U-Asrock-Motherboard/dp/B07PNFTPGB
Last edited: