Ryzen motherboard or start over?

[86turbodsl]

I'm reconfiguring my whole server stack at home, been using a Ryzen 3600 on an Asrock motherboard to run a bunch of VM containers on proxmox, virtualized pfsense, email, a bunch of other things, with a supermicro backplane and SAS drives attached. After replacing the ASrock MB *TWICE* i am sick of the problems and am moving back to a discrete supermicro X10 board for pfsense firewall duty in a 1U case, and an X11 Supermicro board with a couple of SAS HBA for NAS duty, which leaves me with a nice hole for the VM stuff i want to do in a big 4U case with a Ryzen 3600 and bunch of DDR4 ram sitting around with no use.

So does it make sense to try to find a Ryzen motherboard OTHER than ASrock to run my VMs on with what i have, or just start over with Intel? I have been digging around on Newegg for a couple of weeks looking for AM4 motherboards that are more server-like, but practically everything is gaming focused.

I'd like something power efficient if possible, my power bill is already over 500 a month with all these computers running 24/7. The high availability stuff is the NAS and the pfsense box. I don't need the server uptime to be bulletproof, but it would be nice to leave it on 24/7 without a crash every 2 days like the Asrock has gotten to be. I just don't have any confidence in them anymore. I'd just jump straight to a Supermicro board but they have very little in AMD offerings. Maybe it does make sense to abandon the AMD chip.

Looking for advice. Thank you!

 

[louie1961]

I don't know the answer and do have a question. Why are you avoiding the non-server boards? Some of the consumer boards support some server like features (ECC memory, PCI bifurcation). Is there a general assumption these boards won't handle running 24x7?

 

[Markess]

I was just pondering something similar myself. In my case, the system is actually quite stable, but its a consumer board (Asus Prime B350 Plus & Ryzen 5 1400) so has no IPMI, which is inconvenient. Especially so, since "non-G" Ryzens have no iGPU. So I was pondering if its best to eventually get an AM4 board with IPMI, or switch platforms. In my case, the CPU is two generations back from yours, so I'm leaning toward "starting over". Sharing what I've found here, and interested to see what others think!

been using a Ryzen 3600 on an Asrock motherboard to run a bunch of VM containers on proxmox, virtualized pfsense, email, a bunch of other things, with a supermicro backplane and SAS drives attached. After replacing the ASrock MB *TWICE*

I don't know the answer and do have a question. Why are you avoiding the non-server boards? Some of the consumer boards support some server like features (ECC memory, PCI bifurcation). Is there a general assumption these boards won't handle running 24x7?

Yes, it would help if you shared which ASRock board(s) those were? There's a range of "quality" in their product line. Their ASRock Rack line, for example, is oriented more toward business/enterprise and may be more suited to your needs.

So does it make sense to try to find a Ryzen motherboard OTHER than ASrock to run my VMs on with what i have, or just start over with Intel? I have been digging around on Newegg for a couple of weeks looking for AM4 motherboards that are more server-like, but practically everything is gaming focused.

Yeah, that is a quandary, isn't it? You want to be economical, keep as much of what you already have as you can, and just drop in a new motherboard to update/repair the system. But sometimes it actually costs more to work around what you have. What memory do you have in your current system? Is it ECC? Also, AMD systems of that generation could take 32GB UDIMMs, while Intel couldn't till much later. So, if you have 32GB UDIMMs, for example, your options are even more limited.

I can't say with absolute certainty, but generally, the only AM4 boards with a server/workstation focus that you can actually find to buy right now are ASRock Rack X470 & X570 series. They have IPMI for example. Although I know from reading reports that the X470 at least had some teething pains of its own upon release. Since you mention Newegg, the lowest priced version of these (X470D4U) there is going for ~$275 shipped.

Assuming you can re-use your RAM, that $275 can probably get you close to a new Intel platform Workstation/Server Motherboard and (probably used) CPU of the same general generation as your current Ryzen. Newegg's Ebay store for example has an ASRock Rack C242 board (E3C242D4U) for $167.10 currently.

Plus, if you're willing to buy used enterprise gear, then you've probably seen that there's a whole range of Intel stuff out there that can take much cheaper RDIMMs. With that option, you'd be switching everything, but depending on your needs it could be the most economical way to go!

 

[86turbodsl]

I don't know the answer and do have a question. Why are you avoiding the non-server boards? Some of the consumer boards support some server like features (ECC memory, PCI bifurcation). Is there a general assumption these boards won't handle running 24x7?

I've been running an ASrock board for a little over 2 yrs. In that time i've blown 2 of the same board. The current one is failing ram slots slowly and having random reboots. It's also losing bios settings randomly. And it's not a power sag thing. I've got a nice high quality power supply, a BIG eaton UPS and high quality brand name ram. I'm trying to get something new up and running before it completely quits. Its been my experience that consumer level gaming type boards just don't like being powered on and under load for 24/7/365 duty for years. The best luck i've had to date was a Supermicro board w/Q6600 that ran like that for around 10 years as my main server. The move to the Asrock was supposed to be the modern upgrade to that system that has been nothing but headaches and downtime. I wanted to like Asrock, but right now i have lost confidence in them for this type of duty.

 

[86turbodsl]

Yes, it would help if you shared which ASRock board(s) those were? There's a range of "quality" in their product line. Their ASRock Rack line, for example, is oriented more toward business/enterprise and may be more suited to your needs.



Yeah, that is a quandary, isn't it? You want to be economical, keep as much of what you already have as you can, and just drop in a new motherboard to update/repair the system. But sometimes it actually costs more to work around what you have. What memory do you have in your current system? Is it ECC? Also, AMD systems of that generation could take 32GB UDIMMs, while Intel couldn't till much later. So, if you have 32GB UDIMMs, for example, your options are even more limited.

I can't say with absolute certainty, but generally, the only AM4 boards with a server/workstation focus that you can actually find to buy right now are ASRock Rack X470 & X570 series. They have IPMI for example. Although I know from reading reports that the X470 at least had some teething pains of its own upon release. Since you mention Newegg, the lowest priced version of these (X470D4U) there is going for ~$275 shipped.

Assuming you can re-use your RAM, that $275 can probably get you close to a new Intel platform Workstation/Server Motherboard and (probably used) CPU of the same general generation as your current Ryzen. Newegg's Ebay store for example has an ASRock Rack C242 board (E3C242D4U) for $167.10 currently.

Plus, if you're willing to buy used enterprise gear, then you've probably seen that there's a whole range of Intel stuff out there that can take much cheaper RDIMMs. With that option, you'd be switching everything, but depending on your needs it could be the most economical way to go!

I can't remember which one it was, and it's not here at work. It's one of the X570 boards, as far away as i could get from the "gamer" type boards with the RGB, overclocking, pciX16 focus. Around $200+ish. After first one suddenly died one day, completely dead i thought it might be a fluke, and to save time i ordered a 2nd just to reduce downtime and not have to reinstall all the software. After #2 started SMOKING i thought maybe Asrock was not the path i should be on.

Memory is bog standard DDR4 3200 8GB. Nothing crazy. Quality, nonECC. I am willing to get into enterprise gear. 1-2 generations back no problem, just trying to stay somewhat efficient since i'm footing the AC and power bills. I've got 2 supermicro boards on the way to handle NAS and firewall duty. Just need to try and figure out what the VM machine should look like. Whatever it is, it'll be a complete rebuild with the focus on doing all the heavy lifting while the other 2 boxes hopefully sit quietly doing their thing in the background. Not that i can hear them over the JET ENGINE that the switch is.

I feel like the Asrock RACK boards are the only real AM4 boards out there with a focus on 24/7 operation. From my few days of searching anyway. I'm hoping i'm wrong and can find something, but starting to think jumping back to intel is my only real option for some choice.
Ebay searches of Supermicro boards are looking like something in the X11 or X12 range are what i should be looking for. LOTS of choices there. I am really not up to date on Intel offerings as last time i did this research ended up in a CoreQuad heater and i really need to focus on getting bang for buck and power as my power bill can attest to.

As far as IPMI, it's not critical. I can always do the Rpi IPMI thing if i had to. The server is for internal use at home, it's not necessary to have remote access although it's nice.

For context, it runs mythtv, asterisk, email, zoneminder, mysql, and a few other odds and ends, all on Debian stable with proxmox over top.
I'll move the drive pool over to the nas i'm building. Pfsense will stop being virtual. I was really tired of losing everything in the house when the asrock board burped.

 

[alaricljs]

If it's of any help... I work for a business that purchases specialty overclocked servers from a company that uses ASRock/rack motherboards. We run them hard 24x7 until the CPUs die of heatstroke 2-4yrs later. We have had an occasional dimm slot failure.

 

[rtech]

If it's of any help... I work for a business that purchases specialty overclocked servers from a company that uses ASRock/rack motherboards. We run them hard 24x7 until the CPUs die of heatstroke 2-4yrs later. We have had an occasional dimm slot failure.

If the CPUs die what about motherboards? Apart from DIMM slot failures.
How high are the overclocks?
What temps you achieve during workloads?

 

[alaricljs]

We only strip ram/disk from our systems when they die everything else gets e-wasted.
OC's up to 5Ghz, they are fine tuned specific to the entire package (CPU, RAM, ...), some only hit 4.6 but they typically have more cores.
90-95C when new, when things start breaking down that ramps up to 105C. After 105C for a while it's just spirals out of control and dies because you're still hitting 100C+ without an OC or a load.

 

[alaricljs]

Another interpretation for your 'what about motherboards' question... DIMM slot failure is pretty much the only mobo failure mode we experience w/ rock/rack. On other brands we have IPMI/BMC die for some reason but those are quite old (8yr+). RAID controller failures, but we have standardized on mdraid 1 now so they are disappearing.

 

[86turbodsl]

If it's of any help... I work for a business that purchases specialty overclocked servers from a company that uses ASRock/rack motherboards. We run them hard 24x7 until the CPUs die of heatstroke 2-4yrs later. We have had an occasional dimm slot failure.

When the dimm slots start to die, does the machine boot cycle when there's a dimm in that slot? I'm down at least 2 slots on my current asrock board. Even bought new ram to check if that was it. But our biggest issue is when the power cycles on the board, the bios just decides to disable SR-IOV so my passthru doesn't work anymore and the VMs start failing. If i go back into bios and re-enable, then i'm back up, but i can't power cycle it unless i'm there. I can't deal with that.

And note: I do not overclock.

 

[alaricljs]

Nope, OS hang or full on crash and reboot but we service the box inside of 8hrs so I haven't seen more than 1 reboot.

 

[86turbodsl]

ok. Mine doesn't make it past the bios with ram in a bad slot. Just into bios test, then shut down and restart.

 

[matt_garman]

Random thought: in what kind of case is the failing motherboard? Many many years ago, I had an ASRock Rack mini-itx motherboard that mysteriously died. I thought it was just a dud board, until I realized the case had some poorly-placed metal bits protruding from the area under the motherboard (i.e. the area that should be perfectly flat except for the actual standoff screws). Now I'm quite certain that those metal bits shorted something on the board. I'm actually still using that case, but I've since secured some plastic over the protruding bits (i.e. so any contact with the motherboard will be non-conductive).

Anyway, based on my experience, two motherboard failures as described is suspicious, unless they just happened to be manufactured at the exact same time there was a production flaw.

I would look to rule out some potential environmental trigger: e.g. stray wires or other conductive material floating about the chassis (similar to what I described above). Also, are temperature and humidity reasonably well-controlled? If there's any humidity in the air, combined with any temperature deltas (e.g. cool surface in a warm environment), and/or rapid temperature shifts, condensation will form and lead to all kinds of grief. Anything loose combined with moving air (i.e. fans) can potentially lead to intermittent shorts (again, causing all kinds of grief).

When the systems were assembled, did you wear a static ground strap? I'm being flagrantly hypocritical here, as I've assembled countless systems and never worn a strap. But I've read about people who won't even touch a computer without their ground strap after having learned the hard way about what static electricity can do.

 

[linuxmint826]

Considering your needs for power efficiency and stability, it might be worth exploring options beyond ASRock for your VM setup. Check for AM4 motherboards with server-like features. If you can't find a suitable AMD board, Intel could be a viable alternative.

 

[rtech]

When the systems were assembled, did you wear a static ground strap? I'm being flagrantly hypocritical here, as I've assembled countless systems and never worn a strap. But I've read about people who won't even touch a computer without their ground strap after having learned the hard way about what static electricity can do.

You don really need to ground yourself. All you need to do is to touch metal object before touching anything ESD sensitive.
During pulling out of multilayer ESD bags open the bag grab it firmly and the pull out with second hand (you can touch ground before sticking the hand in the multilayer ESD bag).
The reasoning for all this is multilayer ESD bag outer shell is electrically resistive thus you can charge yourself on it.

 

[alaricljs]

The chip tech that's highly sensitive to static is CMOS... Everything else is pretty robust. I can't think of any computer stuff that's still CMOS, last thing I recall is Sun's Sparc and I don't even remember which models.

 

[86turbodsl]

Random thought: in what kind of case is the failing motherboard? Many many years ago, I had an ASRock Rack mini-itx motherboard that mysteriously died. I thought it was just a dud board, until I realized the case had some poorly-placed metal bits protruding from the area under the motherboard (i.e. the area that should be perfectly flat except for the actual standoff screws). Now I'm quite certain that those metal bits shorted something on the board. I'm actually still using that case, but I've since secured some plastic over the protruding bits (i.e. so any contact with the motherboard will be non-conductive).

Anyway, based on my experience, two motherboard failures as described is suspicious, unless they just happened to be manufactured at the exact same time there was a production flaw.

I would look to rule out some potential environmental trigger: e.g. stray wires or other conductive material floating about the chassis (similar to what I described above). Also, are temperature and humidity reasonably well-controlled? If there's any humidity in the air, combined with any temperature deltas (e.g. cool surface in a warm environment), and/or rapid temperature shifts, condensation will form and lead to all kinds of grief. Anything loose combined with moving air (i.e. fans) can potentially lead to intermittent shorts (again, causing all kinds of grief).

When the systems were assembled, did you wear a static ground strap? I'm being flagrantly hypocritical here, as I've assembled countless systems and never worn a strap. But I've read about people who won't even touch a computer without their ground strap after having learned the hard way about what static electricity can do.

It's a custom case i assembled using an old case removable motherboard tray. The supermicro SAS backplane is mounted vertical and drives drop in ala-backblaze.

I'm pretty careful with ESD, my first job out of college was in a circuit board fab. We had to use wrist straps for everything.

I can't imagine there'd be any stray wires floating around in there. There's tons of empty space and not much in there but the sas board and the motherboard with all the sata cables. It's about a 6U size.

The environment is my basement in my climate controlled house. it's so dry in there that the sump pump has never run once in 13 years.

Manufacturing was years apart. the first one ran for about a year before just shut off and refused to ever turn on again. Current one has the bios weirdness and dead ram slots.

It's an Asrock X570 Pro4 since i'm wfh today and can check the box.

 

[86turbodsl]

Considering your needs for power efficiency and stability, it might be worth exploring options beyond ASRock for your VM setup. Check for AM4 motherboards with server-like features. If you can't find a suitable AMD board, Intel could be a viable alternative.

Yes, i certainly do not want another asrock. trying to find a modernish supermicro board that doesn't break the bank and still can support cpu that has decent performance with at least 4 and preferably 6 or more cores.

As far as AM4 motherboards that are server like that is the real question isn't it? Asrock seems to have the market cornered on this.

 

[name stolen]

Every feature most people want except IPMI, Asus ROG Crosshair VIII Hero. X570, PCIe 4.0 everywhere. Consumer but insanely solidly built.

Don't like Asus? Like mATX? GIGABYTE B550M AORUS ELITE AM4 AMD B550 Micro-ATX Motherboard

Just to give you a couple of ideas that aren't ASRock that I personally use. Seems a lot easier to just replace the motherboard, plus AM4 has cheap future upgrade options in terms of CPU, and using the PCIe wisely. I would mention server-y stuff with IPMI, but it's ASRock Rack.

 

[86turbodsl]

That asus is an expensive mb. $300+ I have an Aorus elite for a desktop. Not a bad board. Not perfect, but not bad. Thank you for the suggestions.