2024 AM5 Server builds (limited motherboard overview)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

olliver

New Member
Jan 6, 2024
8
2
3
Europe
Hey all,

With great interest I read all 7 pages (though probably only 3 useful) about the AM5 offerings.

To keep the spirit of the thread (less so on the original post) going, a new thread to have some discussions about server idea's offerings for the AM5 platform. Which should be interesting with the upcomming CES announcements (hopefully :p)

I'm also in the market for a new server build, trying to replace my ancient Phenom II x6 server. My idea is to get something pretty cool now, so it'll last me another 10 - 12 years. Hence PCI5 isn't too big an overkill, as hopefully in 8 years, there'll be some cool upgrades in that space. _personally_ I'd need space for some old HBA's and expanders for SATA storage, SFP NIC (2x 10G cages is super afordable now), and a DVB tuner (pcie 1.0 1x) which is an exotic requirement on my end :p

Epyc's, while amazing, are too expensive generally. In power consumption etc. So for now, I'd like to keep this thread focused on AM5.

Obviously there's different usecases for everybody. Some want a render farm with lots of pci slots, and some want compile/storage servers. Sadly, in the AM5 (pro/con)sumer world, there aren't that many options it seems, though they are better then what I've found so far for TRX50.

Further more, lets focus on ATX only boards, as getting some speciallized super-micro form factor, isn't realistic for most of us. E.g. H13SRD-F | Motherboards | Products | Supermicro isn't even cool with its single PCIe slot, but not ATX.

Finally, one more point to make, is chipsets. I think, especially for server builds, the x670 chipset is mostly useless. It basically just offers extra sata ports and even more USB ports, at the cost of PCIe bandwidth. Granted, some users may be perfectly happy with 8 sata connectors, but for most, they'd end up with an HBA anyway. I think the B650 is nice, the A620 is cool because it the most power efficient, overclocking isn't needed anyway, but there's practical no PCIe5 based motherboards there.

Since we know we only have 24 PCIe 5x links, that limits what we can expect, but more importantly, on the chipset side, we have an 4 lane PCIe5 link. I think the A620 downgrades this from the get-go to PCIe 4 speeds. Contraray to what some posted elsewhere, the X670 doesn't give you any more PCIe 4x lanes then what the B650 offers. Actually, the available bandwith is less, as we have USB and Sata controllers to deal with, and I think internally, the two chips use 4 PCIe 4 lanes as well. Nasty really. That yields us with at best, 7 PCIe 4lanes; as 1x lane would be needed by the chipset itself to offer sata, usb etc, probably by downgrading it into 4x PCIe 2 or even PCIe 1, for those onboard NIC's, soundcards and WiFi too!

The onboard BMC is usually nice, but with PiKVM getting more fancy, I think for most that could be a good alternative.

Looking at the most obvious two contenders (they are not really imo)

SuperMicro H13SAE-MF pricerange $700+?
uATX
BMC: ASpeed 2600
ECC: Yes
M.2 (pcie5): 2 (no built in heatsink)
PCIe 5.0 x16 slots: 2 (16/NA or 8/8)
PCIe 4.0 x4 slots: 1
LAN: 2x 1Gbit (+dedicate BMC port)

ASRock Rack B650D4U $500+?
uATX
BMC: ASpeed 2600
ECC: Yes
M.2 (pcie5): 1 (no built in heatsink)
M.2 (pcie4): 1 (no built in heatsink)
PCIe 5.0 x16: 1
PCIe5.0 x4: 1
PCIe4.0 x1: 1
LAN: 2x 1Gbit (+dedicated BMC port; +2 10G optional)

I think that's pretty much the prosumer boards? Happy to learn of more :)

Then looking at the consumer boards, we suffer from as mentioned in the other thread of issues. RGB, tons of fan connectors (though that doesnt' have to be bad for monitoring), other useless gaming features, way too many VRM's etc. All these things heavily drive up prices of boards, which imo are already heavily overpriced.

To kick off the consumer boards, I looked at 4x M.2 slots; ideally PCIe 5. In the next 10 years, this may be crucial for expantion, allows us to do RAID10 on NVMe (the HBA would do raid on sata already anyway), and we gotta keep that 10 - 100 GBit links saturated, even if that raid10 NVMe is partially only used for cache of the big raid ;)
Secondly, atleast PCIe 5.0 x8 for a storage controller in the future, and PCIe 5.0 for a second port for an HBA; but this is already a problem, as we're short on PCIe 5.0 links from the CPU :(

Gigabyte Aorus master B650 $375?
ATX
BMC: No
ECC: Unkown (needs validation, should work in theory)
M.2 (pcie5): 4 (with built in heatsink) Note, that 2 M.2 slots are shared with the main PCIe slot
PCIe 5.0 x16: 1 (16 and 2 M.2 slots or 8 and 4 M.2 slots)
PCIe 4.0 x4: 1 (from southbridge)
PCIe4 .0 x2: 1 (from southbridge)
LAN: 1x 2.5Gbit

Asus ROG strix B650 gaming wifi $325?
ATX
BMC: No
ECC: Yes
M.2 (pcie5): 2 (with built in heatsink)
M.2 (pcie4): 2 (with built in heatsink)
PCIe 5.0 x16: 2 (16 or 8/4)
PCIe 4.0 x4: 1 (from southbridge)
LAN: 1x 2.5Gbit
(having some doubts about some combinations, as it doesn't quite add up to what's possible, so some PCIe 5x links probably get downgraded. I also doubt Asus put a PCIe 5 2x -> PCIe 4 x4 or similar chip on there.

So what now? What other options are afordable and worth exploring? This post was just to start/continue the earlier discussions on AM5 server builds and I'm very interested in opinions!
 
  • Like
Reactions: Lix

olliver

New Member
Jan 6, 2024
8
2
3
Europe
Maybe I should have put some kind of wishlist in there too :p

But what i'd be looking for, is an uATX - EATX (which afaik still fits in my case, it's a 4U one), with ECC support, 4 M.2 PCIe 5.0, 4 or 5 PCIe slots, 8, 4, 4, 2, 1 in terms of lanes would be great; PCIe 5 not needed, but nice to have, heck PCIe 2 and 3 would be even fine for some of the ports. gbit lan is plenty, as I'd use a PCIe card anyway. SATA ports 1 - 4 is fine. USB, nothing fancy, a couple of 1.1 for keyboard/mouse; PS/2 is even fine for that. Some USB ports to run some USB2 stuff, maybe 1 or 2 USB3 ports, but doesn't need to be fancy. Ideally though; 'old-skool' pricing though, for $125 you'd be able to get a pretty nice board in the past ...

Technically possible; but I doubt any manfacturer is gonna build something this niche. Though if I look at the Gigabyte offering, if it would just offer a couple more PCIe slots, 3x being fine; it would be not that bad; 'cept for price ...
 

stanos4

New Member
Sep 2, 2020
17
2
3
well, there are, although apparently hard to get and expensive (400+$), 2 chipsetless X300 boards with pcie5 from asrock. Recently, one of them was reviewed here as well.
AM5D4ID-2L+/BCM
AM5D4ID-2T/BCM
 

stanos4

New Member
Sep 2, 2020
17
2
3
it would be nice if vcc3 could retest or somebody reproduce the results with a more suitable ssd (apparently the used kingston drive does not support low power modes consuming itself almost 2W in idle according the specs and thus also likely preventing deeper power saving states/more interrupts of cpu), but there the lowest value was 33W. Assuming you can get down to ~25W yes, that is nice/equivalent in comparison to non-apu ryzen systems. Anything else there was pretty much >2x more.

But I don't know what prices you see, but opening ebay, the combo you mentioned is 600+$. I wouldn't call that cheap. I guess you can say it's relatively cheap (given what it offers).
 

zer0sum

Well-Known Member
Mar 8, 2013
862
483
63
it would be nice if vcc3 could retest or somebody reproduce the results with a more suitable ssd (apparently the used kingston drive does not support low power modes consuming itself almost 2W in idle according the specs and thus also likely preventing deeper power saving states/more interrupts of cpu), but there the lowest value was 33W. Assuming you can get down to ~25W yes, that is nice/equivalent in comparison to non-apu ryzen systems. Anything else there was pretty much >2x more.

But I don't know what prices you see, but opening ebay, the combo you mentioned is 600+$. I wouldn't call that cheap. I guess you can say it's relatively cheap (given what it offers).
Check the prices of the B650D4U and the H13SAE :p
 

stanos4

New Member
Sep 2, 2020
17
2
3
ok, I must admit, I don't really get what the OP is searching for/wants to collect in this thread (as besides "pro-/con-sumer" terms he didn't give any "guidelines"), so my comment on price/power cons of epyc vs ryzen was meant in a broader aspect.
 

olliver

New Member
Jan 6, 2024
8
2
3
Europe
well, there are, although apparently hard to get and expensive (400+$), 2 chipsetless X300 boards with pcie5 from asrock. Recently, one of them was reviewed here as well.
AM5D4ID-2L+/BCM
AM5D4ID-2T/BCM
I saw that one as well, and figured 'its the other asrocks little brother'. very similar spec, but deep-mini ITX, so left it out.

I really don't think it's worth it when a H12SSL and Epyc 7302 is so cheap, and Ryzen AM4/5 doesn't really blow it away in terms of power consumption. Just go Epyc and love the expandability for a decade to come :cool:

This is a great thread - https://forums.servethehome.com/index.php?threads/amd-epyc-milan-idle-power-consumption.34118/
That motherboard starts at 522 from amazon (and amazon probably incorrectly quotes me 165 shipping :p)

And that CPU is 800 as well.

That said, that supermicro board IS sexy :p

But I don't know what prices you see, but opening ebay, the combo you mentioned is 600+$. I wouldn't call that cheap. I guess you can say it's relatively cheap (given what it offers).
I admit my google foo was lazy, and I took the first amazon hit; but $600 is not unreasonable, if obtainium. Would have been nice if @zer0sum would have mentioned some price indications or some links with his claims :)

ok, I must admit, I don't really get what the OP is searching for/wants to collect in this thread (as besides "pro-/con-sumer" terms he didn't give any "guidelines"), so my comment on price/power cons of epyc vs ryzen was meant in a broader aspect.
It's not so much as what I specifically need, though that's my 'guideline' to see what's out there. But as I wrote, the other thread was interesting, and actually distracted from the OP's ask, to have a more interesting discussion on what's out there.

So I suppose, while I'm in the market for a new server; "what is out there" for us DYI server builders that want a somewhat cheap, power efficient (due to cost) server that'll last a decade or so, based on zen something.
 

NPS

Active Member
Jan 14, 2021
147
45
28
I saw that one as well, and figured 'its the other asrocks little brother'. very similar spec, but deep-mini ITX, so left it out.
I think the question of having a B650 chipset or no extra chip at all is a huge difference (power, connections). I feel ignoring them is a big oversight.
 

zer0sum

Well-Known Member
Mar 8, 2013
862
483
63
...

That motherboard starts at 522 from amazon (and amazon probably incorrectly quotes me 165 shipping :p)

And that CPU is 800 as well.

That said, that supermicro board IS sexy :p

...
Try searching on Ebay - $722 for the H12SSL and an Epyc 7302 as a combo :cool:

 
  • Like
Reactions: Auggie

olliver

New Member
Jan 6, 2024
8
2
3
Europe
I think the question of having a B650 chipset or no extra chip at all is a huge difference (power, connections). I feel ignoring them is a big oversight.
Oh absolutely! If this would lead to a cost reduction, power reduction that's of course amazing.

Very sadly (for my specific use case, which is why I initially excluded it), is that it only has a single PCIe slot, single M.2. 4 slots would be great (4x4, 2x8 and 16x1 configuration options exist). Which is weird, as (not sure if it is possible at all), those PCIe 5.0 x4 lanes used 'exclusively for the chipset' could have been used for 2 * 1x, 1 * 2x (or whatever configuration) (or use it as a x4 M.2 bus, and use other lanes for said configuration). So imo that's a big miss.

The oculink helps of course, but having it 'switch' between a real port and the link would have been better.

In conclusion,t his board felt a bit like a HTPC board, or pure compute board to me.

But yes, chipsetless!
 

olliver

New Member
Jan 6, 2024
8
2
3
Europe
So CES came and went, but nothing exciting there. Actually a worry if anything. "BTF" with connectors now on the backside, making them incompatible with what we currently do/use :(
 

KingKaido

New Member
Nov 24, 2023
20
16
3
So CES came and went, but nothing exciting there. Actually a worry if anything. "BTF" with connectors now on the backside, making them incompatible with what we currently do/use :(
I think the new motherboard 'cycle' for Ryzen 7000 is over, everyone is waiting for Zen 5 which is due at Q2/3 this year, but yeah i agree a server/workstation motherboard with 3 x X8 slots and 2 M.2s would be amazing for me, or having x8 x8 x4 and 2 M.2 slots then with an x8 chipset having a shared X4 X4 slot and M.2s + usbs, and support for 64GB ECC udimms for 256GB Max, it would be a homelabbers dream

There was a leak on next gen intel 'arrow lake' and it looks like the pcie lanes has gone up from 28 to 32 (24 + 8 for the chipset), so hopefully Zen5 follows suit or even goes above that with 36 (wishful thinking) but that would allow alot of X8 or X4(pcie 4.0 x4 has alot of bandwidth for most pcie cards) slots in a motherboard , without the need for expensive threadripper and older EPYC
 

olliver

New Member
Jan 6, 2024
8
2
3
Europe
I don't think they can up the southbridge connection. A) it's been a x4 link since forever, and B) aren't those pins more or less 'set in stone' for AM5? The only way they'll get more PCIe slots, is to change the socket.

So AM6 is probably more PCIe slots and DDR6 :p
 
  • Like
Reactions: KingKaido

KingKaido

New Member
Nov 24, 2023
20
16
3
I don't think they can up the southbridge connection. A) it's been a x4 link since forever, and B) aren't those pins more or less 'set in stone' for AM5? The only way they'll get more PCIe slots, is to change the socket.

So AM6 is probably more PCIe slots and DDR6 :p
That's a very good point it would suck to have to wait until Zen 6 to get more PCIe lanes, check this article that just got released
Hopefully X690e supports 128GB ECC R-DIMMS for 512GB total (hehehe)
Looks like a competitor to W680
 
Last edited:

olliver

New Member
Jan 6, 2024
8
2
3
Europe
I have doubts on the usefulness of the X690e. It's not the 90's anymore, where chipsets where powerful and responsible for features :p It's the CPU that really had to support that RAM, as the CPU has the memory controller.

If we compare B650E to X670E (the E imo is bs, it's basically a 'license bit' to unlock features. e.g. the PCIe links are v4 or v5. The pins are the same, the chip is the same, just the fact that the mobo was design in a way to handle the speeds, and the chipset says it's okay to use these speeds), anyway, we see that it's just 1 Promontory vs 2 Promontory cores. What do these Promontory cores do? Just offer more USB and SATA links!! So it's imo a huge waste of electricity :p

Sadly, there was never a X590E, so we can't compare; but looking at the evolution of A300 - X570, better PCI breakout support, e.g. those x4 PCIe4.0 links somehow turned into a PCIe4.0 x16 link on the X570. How that is possible, beats me :p But the B550 took the x4 PCIe4.0 link and turned it into 10x PCIe3.0 links, vs PCIe2.0 links before. Other then that, just more sata and USB ports (and CPU support obviously).

So for the chipset, I'm not holding much hope up :)
 

KingKaido

New Member
Nov 24, 2023
20
16
3
I feel you, it does seem like AMD is constantly just playing catch-up and is just raw performance/efficiency king (due to TSMC) but they don't really innovate or validate configurations for niche usecases like intel

AMD does have bifurcation support intel could level the playing field if they really wanted to,

Intel needs to bring back cheaper HEDT for AMD to take it seriously, the threadripper cpu and motherboard combo is just too expensive, the 7950X is enough for alot usecases, especially for single thread, all it needs is just better ECC RAM support or support for higher capacities and a few more pcie lanes
 

xyvyx

New Member
Apr 22, 2020
4
1
3
question re: these B650 boards -
(although not really exclusive to these boards... maybe more of a general CPU/AM5 question)

If I used that Supermicro board with a pcie Gen4 device plugged into the 1st x16 slot and a pcie Gen3 device plugged into the 2nd x16 slot, I understand that they'd only get 8 pcie lanes/each. But do the lanes/slots negotiate their speed / generation independently?

In my case, it'd be an nvidia 4090 in one slot and a 25gb NIC in the other. I have an existing mellanox ConnectX-4 card, but it's a PCI-e Gen3 device and I wouldn't want it to slow the graphics card down. (beyond already cutting the lanes in half). There are newer ConnectX-6 cards that are PCI-E Gen4 x8, but if I can mix Gen3/4 cards with them both using 8 lanes at their own native speeds, I can use the hardware I already have.

I kinda hope that each pcie lane or group negotiates their device speed independently... That way, I could have a Gen5 m.2 nvme drive, the 4090 running at x8 Gen4 speed + the NIC operating at x8 Gen3. But I don't know if having that MUX between the two slots complicates things.
 
  • Like
Reactions: RDshift