And so it begins... First AMD Ryzen AM4 server motherboard.

Discussion in 'Processors and Motherboards' started by McKajVah, Feb 26, 2019.

  1. TomUK

    TomUK Member

    Joined:
    Aug 30, 2017
    Messages:
    59
    Likes Received:
    10
    Apparently this will get a follow-up release with 10gb ethernet according a French hardware news site (can't find the bloody link now though!)
     
    #41
    damienr likes this.
  2. EffrafaxOfWug

    EffrafaxOfWug Radioactive Member

    Joined:
    Feb 12, 2015
    Messages:
    1,048
    Likes Received:
    351
    Ryzen itself is limited on PCIe lanes; the chips have 24 lanes themselves, of which 16x usually go to the graphics card, 4x for NVME and 4x to the chipset. Other PCIe lanes vary depending on the chipset (IIRC X470 provides an additional 8x PCIe 2.0) but ASRock helpfully provide a block diagram on page 13 of the manual (PDF warning or you can see an image of it here); you can see they've split the 16x into two 8x and the 4x goes into a slot rather than into NVME.

    The remaining 8 lanes from the chipset all look to be used which makes me somewhat sceptical of a 10GbE version being possible - TTBOMK there's no 10Gb MAC on the Ryzen die and you need PCIe 2.0 2x to max out a single 10GbE connection, so one of the other peripherals would need to be chopped... and at that point why not just tell people to use a card in one of the PCIe slots?

    Certainly going to be interesting to see how this behaves with Ryzen 3000 chips and potentially PCIe 4.0 - especially if 32GB ECC UDIMMs work with it.
     
    #42
  3. ReturnedSword

    ReturnedSword Member

    Joined:
    Jun 15, 2018
    Messages:
    106
    Likes Received:
    20
    The lack of PCIe lanes is a constraint of a consumer platform vs an enterprise one. Intel's enthusiast platform (Z390) only appears to have "more" lanes, but if we take a closer look they're not real PCIe lanes.

    AMD X470 with Zen/Zen+

    • CPU: PCIe 3.0 x16 or PCIe 3.0 x8/x8 or PCIe 3.0 x8/x4/x4 or PCIe 3.0 x4/x4/x4/x4
    • CPU: PCIe 3.0 x4
    • CPU -> X470 chipset: PCIe 3.0 x4 interconnect @ 3.94 GB/s
      • X470: Configurable 8x PCIe 2.0 lanes
    Intel Z390 with CFL/CFL-R
    • CPU: PCIe 3.0 x16 or PCIe 3.0 x8/x8
    • CPU -> Z390 chipset: 4 lane DMI 3.0 @ 3.93 GB/s
      • Z390: Configurable 24x PCIe 3.0 lanes
    As you can see here, both chipsets are connected to the CPU by basically the same bandwith. While it's nice to have the flexibility of more fake PCIe lanes on the Intel platform, once you push over the equivalent of PCIe 3.0 x4, you're bottle-necked. Coincidentally, note that the 8x PCIe 2.0 lanes on the AMD X470 platform adds up to 4.0 GB/s, which is roughly the bandwidth provided by the CPU-Chipset link.
     
    #43
  4. zir_blazer

    zir_blazer Active Member

    Joined:
    Dec 5, 2016
    Messages:
    212
    Likes Received:
    63
    Some notes:

    The CPU Zen dies has 32 PCIe Lanes, not 24. AM4 doesn't expose 8 lanes from the SERDES Controller (For PCIe, SATA or 10G NIC), and always wastes 4 for the Chipset, which is the main reason why I consider Embedded Ryzen/EPYC to be superior (Note that Embedded Ryzen is a Raven Ridge APU and it is missing 8 PCIe Lanes from the pure PCIe Controller, but those are missing in AM4 too. Is 32 or 24 vs 24 or 16 on CPU and APU, respectively). Also, the embedded Zens DOES expose the 10G NICs, which takes two lanes each.
    Zen also can bifurcate each 16 lanes controller to up to 16 1x, but I'm not sure if AM4 Ryzens can do that. You could theorically have 8x/4x/1x/1x/1x/1x if you want to.

    Intel consumer CPUs can do 8x/4x/4x. Note that for PCI Passthrough scenarios they require hacks since the Processor PCIe Controller does NOT support PCIe ACS, whereas AMD supports it.


    While Intel has much better Chipset connectivity, a single high end NVMe drive can saturate DMI. Ryzen can have it hooked directly to it, albeit chances are that you are using the Chipset in some way (Moving NVMe SSD data to the Chipset NIC?). Actually, a full 32 lanes Embedded EPYC should be able to pull 16x or 8x/8x, 2 10G NICs, 2 NVMe and 4 SATAs with no multiplexing. A serious I/O beast.
     
    #44
    Nayef and TomUK like this.
  5. Mam89

    Mam89 Member

    Joined:
    Jan 14, 2016
    Messages:
    52
    Likes Received:
    7
    I think you're all arguing apples and oranges.

    This is targeted at replacing aging entry level compute/storage servers. Read SMB (nowhere near enterprise).

    EPYC/Ryzen embedded is embedded, which is a whole other ballgame for use cases and taget market. The price tags alone will tell that story.

    The Intel E-2100 series is actually a closer compare, but, likewise, still far off. E-2100 simply doesn't offer the raw compute power (read core count) and flexibility of these devices.

    Hint:
    There is a reason that savy buyers arent running out for those E-2100s: the value vs used last gen equipment just isn't there.

    The Ryzen (socketed) CPUs on the other hand offer so much more value per $ NEW it's tantalizing! Heck, I can swap this board into my 3U Supermicro 836, swap over my HBA and 10Gb NIC, and not sweat pennies or ever second guess performance. Getting even better compute for less than half the power usage as a bonus.
     
    #45
  6. ReturnedSword

    ReturnedSword Member

    Joined:
    Jun 15, 2018
    Messages:
    106
    Likes Received:
    20
    I think you're right there @Mam89. This platform would compete with the Xeon E-21xx. Quite interesting for homelabbing or SMB.

    Here's the French hardware site link referring to a variant by this summer that would have 10 GbE:
    Google Translate

    They have a review up as well, but it's paywalled:
    Google Translate
     
    #46
  7. e97

    e97 Active Member

    Joined:
    Jun 3, 2015
    Messages:
    191
    Likes Received:
    61
    You guys nailed it. These NEW components are nicer in terms of price / performance / efficiency / cost vs last gen used stuff of similar tier.

    FYI:

    DDR4 ECC Unbuffered UDIMM / EUDIMM Crucial - 16 GB (1 x 16 GB) DDR4-2666 Memory (CT16G4WFD8266) - PCPartPicker now at $117. The price has been dropping nicely, I'm waiting until they're sub $99 myself.

    Re: AMD Embedded :

    Sapphire Unveils Motherboard With AMD Ryzen Embedded V1000 APU

    UDOO BOLT - UDOO

    Around $350 - $450 for a a powerful small form factor (5" x 5")
     
    #47
  8. ReturnedSword

    ReturnedSword Member

    Joined:
    Jun 15, 2018
    Messages:
    106
    Likes Received:
    20
    Shame there are no 32GB EUDIMM available yet. If 4 x DR EUDIMMs are used the memory speed drops to 1866 MHz. The motherboard can do 2400 MHz with 2 x DR EUDIMM.
     
    #48
  9. EffrafaxOfWug

    EffrafaxOfWug Radioactive Member

    Joined:
    Feb 12, 2015
    Messages:
    1,048
    Likes Received:
    351
    No word on whether this board and a current Ryzen CPU will work with 32GB ECC UDIMMs though, unofficially. Officially, it's limited to 4x16GB EUDIMMs.
     
    #49
  10. TomUK

    TomUK Member

    Joined:
    Aug 30, 2017
    Messages:
    59
    Likes Received:
    10
    Even with the 64GB limit - it's still very competitive against the E3 Xeons.

    If 128GB is needed - first gen threadripper chips are cheap now, and the new Asrock phantom board is as cheap as chips
     
    #50
  11. ReturnedSword

    ReturnedSword Member

    Joined:
    Jun 15, 2018
    Messages:
    106
    Likes Received:
    20
    The spec sheet says it supports up to 64GB EUDIMM. Not sure if that's going to the limited by the Ryzen/Ryzen+ IMC or not. Officially Ryzen/Ryzen+ supports up to 64GB. I haven't been able to find any information on configuration. It would be nice if it supports 2 x 32GB DR.
     
    #51
  12. EffrafaxOfWug

    EffrafaxOfWug Radioactive Member

    Joined:
    Feb 12, 2015
    Messages:
    1,048
    Likes Received:
    351
    A minor point but, as I've been eyeing this board as a potential replacement for my current hardware when the Zen2 chips come out; no new BIOS releases or updates on the memory compatibility yet, but they are at least confirming it'll work with "7mm Ryzen" chips; when last I looked they were only showing compatibility with the Pinnacle Ridge (Zen+) chips.

    Then:
    AMD Ryzen™ (Pinnacle Ridge) 2nd Generation Series Processors

    Now:
    AMD AM4 Socket Ryzen Series CPUs (Raven Ridge and Pinnacle Ridge) and Ryzen 7nm CPUs

    They're available in the UK now but I'm likely going to be waiting until boards with a compatible BIOS are in the channel, and it'd be nice to see an update on memory compatibility as well. Sadly the Zen2 IMC still only seems to be rated for 16GB DIMMs so still limited to 64GB max memory officially. Eagerly awaiting benches and especially TDP numbers of the new chips as well, the 3700X is looking like an ideal sweet-spot for me.
     
    #52
    gigatexal likes this.
  13. ReturnedSword

    ReturnedSword Member

    Joined:
    Jun 15, 2018
    Messages:
    106
    Likes Received:
    20
    Same here on Zen2 @EffrafaxOfWug.

    I’ve switched all my workstations over to Zen/Zen+ and it’s been great so far. My only Intel PC left is my Dell XPS 9350, and with all the previous vulnerabilities + the new vulnerabilities out this month recommending disabling HT, it’s meh for the blue team.

    My main gripe about Zen/Zen+ is lack of downstream PCIe connectivity. Though as I highlighted before that in terms of raw CPU-Chipset interconnect bandwidth it’s basically the same for Intel vs AMD consumer platforms, it’s somewhat more convenient for the chipset to act as a PCIe switch for “more” lanes. Looks like the X570 chipset basically fixes this issue.

    I can’t be sure, but based on observation it looks like for the Ryzen 3000 AM4 is taking much more seriously by the motherboard manufacturers. I wonder what sort of effort it would take to update this motherboard to the X570, beyond the chipset being in-house AMD, and needing redrivers for the PCIe_2 and onward slots?
     
    #53
  14. saivert

    saivert Member

    Joined:
    Nov 2, 2015
    Messages:
    118
    Likes Received:
    12
    The problem with the mainstream platforms has been the low pin count on the socket both for AMD and Intel. They need to up their game in that department but I'm sure it costs too much. People criticize the HEDT platform as a cash grab but it was necessary to provide more lanes and memory channels. If it was just consumer variant of the server platform so be it.

    Seems like we are stuck with this setup of some lanes coming directly from the CPU and some via the chipset for the forseeable future. There has been no mention of any other layout for mainstream like just having a beefy link between CPU and chipset and having good PCIe switching capability in the chipset where everything connects including that GTX 2080 Ti.
     
    #54
  15. EffrafaxOfWug

    EffrafaxOfWug Radioactive Member

    Joined:
    Feb 12, 2015
    Messages:
    1,048
    Likes Received:
    351
    As it is, I can get away with the connectivity restraints of the Ryzen desktop platform - I only need PCIe slots for an HBA and maybe a 10Gb card. IO is expensive both in terms of power and additional complexity so I don't begrudge the additional cost of the proper workstation/server platforms. The vast majority of Ryzen purchasers will at most use one PCIe card, an NVME drive and maybe a SATA drive or two.

    My only disappoint is has the IMC, given that the Epyc can comfortably handle 128GB DIMMs but I guess that's more a function of registered ECC than anything else.

    If there'd been something like an Epyc 3000 in a mATX format - lots of PCIe capability and support for much more memory being the huge advantages there - I'd have leapt on that in a shot. But as it is, no-one seems to have anything like that in their roadmaps and I'm in the market for some more high-performance cores for not a huge amount of money; this board in conjunction with an 8+ core Ryzen2/Ryzen+ fits the bill perfectly even with the constraints. HEDT boards just don't have the IPMI or anything like it.
     
    #55
  16. zir_blazer

    zir_blazer Active Member

    Joined:
    Dec 5, 2016
    Messages:
    212
    Likes Received:
    63
    You want exactly what I want, I have ranted out loud to see if I can generate interest in such a product.

    EPYC Embedded does a far better use of the Zen SoCs that what AM4 + Chipset does. Supermicro has Motherboards that use it almost the way that I want, but they're all mITX, so at most one slot, and they don't have consumer/workstation features that we're used to like Azalia audio. There is nothing stopping a Motherboard manufacturer to try to make a Ryzen Embedded or EPYC Embedded prosumer facing product.
    Zen SoC has a single weakness: It has only 4 USB Ports. Even Supermicro EPYC Embedded Motherboards have some USB Hubs integrated to fanout a single USB 3.0 to two USB 2.0, it is really that starved. I hope that now that they have a dedicated I/O die, AMD can put at least eight, as that would make it far more viable.
     
    #56
  17. TomUK

    TomUK Member

    Joined:
    Aug 30, 2017
    Messages:
    59
    Likes Received:
    10
    @zir_blazer might be worth looking at Asrock X399m? Has 3 full NVME M2 slots that support NVME RAID and 3 full x16 slots in M-ATX. Reasonably cheap, supports ECC and HCC Threadripper - only downside is it's limited to 64 GB of RAM total (though this might go up to 128 with a new BIOS when 32gb Udimms become more common)
     
    #57
  18. McKajVah

    McKajVah Member

    Joined:
    Nov 14, 2011
    Messages:
    46
    Likes Received:
    8
    #58
    ejecthawk, TomUK and rekd0514 like this.
  19. EffrafaxOfWug

    EffrafaxOfWug Radioactive Member

    Joined:
    Feb 12, 2015
    Messages:
    1,048
    Likes Received:
    351
    The 10GbE version of the X470D4U2-2T has been kicking around for a while, it was shown in their 2019 catalogue which was published a few weeks back.

    That threadripper board... wasn't. That's one helluva board, sadly doesn't scratch my itch for a home server board though.
     
    #59
  20. zir_blazer

    zir_blazer Active Member

    Joined:
    Dec 5, 2016
    Messages:
    212
    Likes Received:
    63
    ThreadRipper based platforms should balloon too much the price, whereas AMD price for EPYC Embedded series seems to be comparable the consumer AM4 parts (The 8C/16T 3251 was like, 330 U$D, same than R7 1700 on release) but with more SoC features, that is why I find them more attractive. Besides, TR on mATX has pretty much the same issue than single die EPYC Embedded on mITX: Too big for that size to use all its I/O. The X99D8A-2T posted before suits better my tastes.

    Sadly, first world tastes, third world wallet...
     
    #60
Similar Threads: begins Ryzen
Forum Title Date
Processors and Motherboards Budget Motherboard for Ryzen 5 3600 Aug 9, 2019
Processors and Motherboards Ryzen 3000 and PCH Slot Video Card. No Video Jul 11, 2019
Processors and Motherboards Suggestions for reviews with Ryzen 2700 / 3700 (future) AM4 + ECC Jan 3, 2019
Processors and Motherboards Ryzen crypto co-processor Sep 10, 2018
Processors and Motherboards Unexpected benefit with Ryzen - reducing power for home server Sep 3, 2018

Share This Page