And so it begins... First AMD Ryzen AM4 server motherboard.

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

TomUK

Member
Aug 30, 2017
67
12
8
38
Apparently this will get a follow-up release with 10gb ethernet according a French hardware news site (can't find the bloody link now though!)
 
  • Like
Reactions: damienr

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
Ryzen itself is limited on PCIe lanes; the chips have 24 lanes themselves, of which 16x usually go to the graphics card, 4x for NVME and 4x to the chipset. Other PCIe lanes vary depending on the chipset (IIRC X470 provides an additional 8x PCIe 2.0) but ASRock helpfully provide a block diagram on page 13 of the manual (PDF warning or you can see an image of it here); you can see they've split the 16x into two 8x and the 4x goes into a slot rather than into NVME.

The remaining 8 lanes from the chipset all look to be used which makes me somewhat sceptical of a 10GbE version being possible - TTBOMK there's no 10Gb MAC on the Ryzen die and you need PCIe 2.0 2x to max out a single 10GbE connection, so one of the other peripherals would need to be chopped... and at that point why not just tell people to use a card in one of the PCIe slots?

Certainly going to be interesting to see how this behaves with Ryzen 3000 chips and potentially PCIe 4.0 - especially if 32GB ECC UDIMMs work with it.
 

ReturnedSword

Active Member
Jun 15, 2018
526
235
43
Santa Monica, CA
The lack of PCIe lanes is a constraint of a consumer platform vs an enterprise one. Intel's enthusiast platform (Z390) only appears to have "more" lanes, but if we take a closer look they're not real PCIe lanes.

AMD X470 with Zen/Zen+

  • CPU: PCIe 3.0 x16 or PCIe 3.0 x8/x8 or PCIe 3.0 x8/x4/x4 or PCIe 3.0 x4/x4/x4/x4
  • CPU: PCIe 3.0 x4
  • CPU -> X470 chipset: PCIe 3.0 x4 interconnect @ 3.94 GB/s
    • X470: Configurable 8x PCIe 2.0 lanes
Intel Z390 with CFL/CFL-R
  • CPU: PCIe 3.0 x16 or PCIe 3.0 x8/x8
  • CPU -> Z390 chipset: 4 lane DMI 3.0 @ 3.93 GB/s
    • Z390: Configurable 24x PCIe 3.0 lanes
As you can see here, both chipsets are connected to the CPU by basically the same bandwith. While it's nice to have the flexibility of more fake PCIe lanes on the Intel platform, once you push over the equivalent of PCIe 3.0 x4, you're bottle-necked. Coincidentally, note that the 8x PCIe 2.0 lanes on the AMD X470 platform adds up to 4.0 GB/s, which is roughly the bandwidth provided by the CPU-Chipset link.
 

zir_blazer

Active Member
Dec 5, 2016
355
128
43
Some notes:

The CPU Zen dies has 32 PCIe Lanes, not 24. AM4 doesn't expose 8 lanes from the SERDES Controller (For PCIe, SATA or 10G NIC), and always wastes 4 for the Chipset, which is the main reason why I consider Embedded Ryzen/EPYC to be superior (Note that Embedded Ryzen is a Raven Ridge APU and it is missing 8 PCIe Lanes from the pure PCIe Controller, but those are missing in AM4 too. Is 32 or 24 vs 24 or 16 on CPU and APU, respectively). Also, the embedded Zens DOES expose the 10G NICs, which takes two lanes each.
Zen also can bifurcate each 16 lanes controller to up to 16 1x, but I'm not sure if AM4 Ryzens can do that. You could theorically have 8x/4x/1x/1x/1x/1x if you want to.

Intel consumer CPUs can do 8x/4x/4x. Note that for PCI Passthrough scenarios they require hacks since the Processor PCIe Controller does NOT support PCIe ACS, whereas AMD supports it.


While Intel has much better Chipset connectivity, a single high end NVMe drive can saturate DMI. Ryzen can have it hooked directly to it, albeit chances are that you are using the Chipset in some way (Moving NVMe SSD data to the Chipset NIC?). Actually, a full 32 lanes Embedded EPYC should be able to pull 16x or 8x/8x, 2 10G NICs, 2 NVMe and 4 SATAs with no multiplexing. A serious I/O beast.
 
  • Like
Reactions: Nayef and TomUK

Mam89

Member
Jan 14, 2016
58
11
8
34
SoCal
I think you're all arguing apples and oranges.

This is targeted at replacing aging entry level compute/storage servers. Read SMB (nowhere near enterprise).

EPYC/Ryzen embedded is embedded, which is a whole other ballgame for use cases and taget market. The price tags alone will tell that story.

The Intel E-2100 series is actually a closer compare, but, likewise, still far off. E-2100 simply doesn't offer the raw compute power (read core count) and flexibility of these devices.

Hint:
There is a reason that savy buyers arent running out for those E-2100s: the value vs used last gen equipment just isn't there.

The Ryzen (socketed) CPUs on the other hand offer so much more value per $ NEW it's tantalizing! Heck, I can swap this board into my 3U Supermicro 836, swap over my HBA and 10Gb NIC, and not sweat pennies or ever second guess performance. Getting even better compute for less than half the power usage as a bonus.
 

e97

Active Member
Jun 3, 2015
323
193
43
You guys nailed it. These NEW components are nicer in terms of price / performance / efficiency / cost vs last gen used stuff of similar tier.

FYI:

DDR4 ECC Unbuffered UDIMM / EUDIMM Crucial - 16 GB (1 x 16 GB) DDR4-2666 Memory (CT16G4WFD8266) - PCPartPicker now at $117. The price has been dropping nicely, I'm waiting until they're sub $99 myself.

Re: AMD Embedded :

Sapphire Unveils Motherboard With AMD Ryzen Embedded V1000 APU

UDOO BOLT - UDOO

Around $350 - $450 for a a powerful small form factor (5" x 5")
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
No word on whether this board and a current Ryzen CPU will work with 32GB ECC UDIMMs though, unofficially. Officially, it's limited to 4x16GB EUDIMMs.
 

TomUK

Member
Aug 30, 2017
67
12
8
38
Even with the 64GB limit - it's still very competitive against the E3 Xeons.

If 128GB is needed - first gen threadripper chips are cheap now, and the new Asrock phantom board is as cheap as chips
 

ReturnedSword

Active Member
Jun 15, 2018
526
235
43
Santa Monica, CA
The spec sheet says it supports up to 64GB EUDIMM. Not sure if that's going to the limited by the Ryzen/Ryzen+ IMC or not. Officially Ryzen/Ryzen+ supports up to 64GB. I haven't been able to find any information on configuration. It would be nice if it supports 2 x 32GB DR.
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
A minor point but, as I've been eyeing this board as a potential replacement for my current hardware when the Zen2 chips come out; no new BIOS releases or updates on the memory compatibility yet, but they are at least confirming it'll work with "7mm Ryzen" chips; when last I looked they were only showing compatibility with the Pinnacle Ridge (Zen+) chips.

Then:
AMD Ryzen™ (Pinnacle Ridge) 2nd Generation Series Processors

Now:
AMD AM4 Socket Ryzen Series CPUs (Raven Ridge and Pinnacle Ridge) and Ryzen 7nm CPUs

They're available in the UK now but I'm likely going to be waiting until boards with a compatible BIOS are in the channel, and it'd be nice to see an update on memory compatibility as well. Sadly the Zen2 IMC still only seems to be rated for 16GB DIMMs so still limited to 64GB max memory officially. Eagerly awaiting benches and especially TDP numbers of the new chips as well, the 3700X is looking like an ideal sweet-spot for me.
 
  • Like
Reactions: gigatexal

ReturnedSword

Active Member
Jun 15, 2018
526
235
43
Santa Monica, CA
Same here on Zen2 @EffrafaxOfWug.

I’ve switched all my workstations over to Zen/Zen+ and it’s been great so far. My only Intel PC left is my Dell XPS 9350, and with all the previous vulnerabilities + the new vulnerabilities out this month recommending disabling HT, it’s meh for the blue team.

My main gripe about Zen/Zen+ is lack of downstream PCIe connectivity. Though as I highlighted before that in terms of raw CPU-Chipset interconnect bandwidth it’s basically the same for Intel vs AMD consumer platforms, it’s somewhat more convenient for the chipset to act as a PCIe switch for “more” lanes. Looks like the X570 chipset basically fixes this issue.

I can’t be sure, but based on observation it looks like for the Ryzen 3000 AM4 is taking much more seriously by the motherboard manufacturers. I wonder what sort of effort it would take to update this motherboard to the X570, beyond the chipset being in-house AMD, and needing redrivers for the PCIe_2 and onward slots?
 

saivert

Member
Nov 2, 2015
138
18
18
40
Norway
The problem with the mainstream platforms has been the low pin count on the socket both for AMD and Intel. They need to up their game in that department but I'm sure it costs too much. People criticize the HEDT platform as a cash grab but it was necessary to provide more lanes and memory channels. If it was just consumer variant of the server platform so be it.

Seems like we are stuck with this setup of some lanes coming directly from the CPU and some via the chipset for the forseeable future. There has been no mention of any other layout for mainstream like just having a beefy link between CPU and chipset and having good PCIe switching capability in the chipset where everything connects including that GTX 2080 Ti.
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
As it is, I can get away with the connectivity restraints of the Ryzen desktop platform - I only need PCIe slots for an HBA and maybe a 10Gb card. IO is expensive both in terms of power and additional complexity so I don't begrudge the additional cost of the proper workstation/server platforms. The vast majority of Ryzen purchasers will at most use one PCIe card, an NVME drive and maybe a SATA drive or two.

My only disappoint is has the IMC, given that the Epyc can comfortably handle 128GB DIMMs but I guess that's more a function of registered ECC than anything else.

If there'd been something like an Epyc 3000 in a mATX format - lots of PCIe capability and support for much more memory being the huge advantages there - I'd have leapt on that in a shot. But as it is, no-one seems to have anything like that in their roadmaps and I'm in the market for some more high-performance cores for not a huge amount of money; this board in conjunction with an 8+ core Ryzen2/Ryzen+ fits the bill perfectly even with the constraints. HEDT boards just don't have the IPMI or anything like it.
 

zir_blazer

Active Member
Dec 5, 2016
355
128
43
If there'd been something like an Epyc 3000 in a mATX format - lots of PCIe capability and support for much more memory being the huge advantages there - I'd have leapt on that in a shot.
You want exactly what I want, I have ranted out loud to see if I can generate interest in such a product.

EPYC Embedded does a far better use of the Zen SoCs that what AM4 + Chipset does. Supermicro has Motherboards that use it almost the way that I want, but they're all mITX, so at most one slot, and they don't have consumer/workstation features that we're used to like Azalia audio. There is nothing stopping a Motherboard manufacturer to try to make a Ryzen Embedded or EPYC Embedded prosumer facing product.
Zen SoC has a single weakness: It has only 4 USB Ports. Even Supermicro EPYC Embedded Motherboards have some USB Hubs integrated to fanout a single USB 3.0 to two USB 2.0, it is really that starved. I hope that now that they have a dedicated I/O die, AMD can put at least eight, as that would make it far more viable.
 

TomUK

Member
Aug 30, 2017
67
12
8
38
@zir_blazer might be worth looking at Asrock X399m? Has 3 full NVME M2 slots that support NVME RAID and 3 full x16 slots in M-ATX. Reasonably cheap, supports ECC and HCC Threadripper - only downside is it's limited to 64 GB of RAM total (though this might go up to 128 with a new BIOS when 32gb Udimms become more common)
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
The 10GbE version of the X470D4U2-2T has been kicking around for a while, it was shown in their 2019 catalogue which was published a few weeks back.

That threadripper board... wasn't. That's one helluva board, sadly doesn't scratch my itch for a home server board though.
 

zir_blazer

Active Member
Dec 5, 2016
355
128
43
@zir_blazer might be worth looking at Asrock X399m? Has 3 full NVME M2 slots that support NVME RAID and 3 full x16 slots in M-ATX. Reasonably cheap, supports ECC and HCC Threadripper - only downside is it's limited to 64 GB of RAM total (though this might go up to 128 with a new BIOS when 32gb Udimms become more common)
ThreadRipper based platforms should balloon too much the price, whereas AMD price for EPYC Embedded series seems to be comparable the consumer AM4 parts (The 8C/16T 3251 was like, 330 U$D, same than R7 1700 on release) but with more SoC features, that is why I find them more attractive. Besides, TR on mATX has pretty much the same issue than single die EPYC Embedded on mITX: Too big for that size to use all its I/O. The X99D8A-2T posted before suits better my tastes.

Sadly, first world tastes, third world wallet...