MicroATX All in one zfs upgrade

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Nemesis_001

Member
Dec 24, 2022
36
3
8
Hi,

I currently run a home server since 2014, based on a SuperMicro X10SLL+-F and a Xeon E3-1230v3.
Running ESXI 7, hosting an OmniOS guest for ZFS and a Windows OS for applications. Everything has been running silky smooth with close to 0 maintenance or problems during software upgrades.

As the hardware is getting a little long in the tooth, I've been looking for an upgrade.
I have been unable to find a reasonably recent upgrade that doesn't cost too much.

I was paid around $700 at the time for the current hardware, so I was looking to pay up to 800-850$ for a new current gen system that will last me 8-9 years, or around 500-550$ for a coffee lake gen system or better that will last me 6-7 years. Without DRAM, for a 6 core part at least.

Everything seems old and expensive. Focusing on server market hardware as everything just works rather smoothly, and there are usually no compatibility issues.

Limited to Micro-ATX or Mini-ITX board sizes due to the compact chassis.

Anyone has any ideas?

Thanks
 
  • Like
Reactions: abq

juma

Member
Apr 14, 2021
64
34
18
How about a DIY AMD system?
  • CPU: AMD Pro 4750GE - ~170 used on ebay
  • Motherboard: AsRockRack X470D4U - $260 new
  • RAM: 64GB ECC UDIMM - $240 new
This should be much more performant and energy efficient, and leave you enough for a chassis and storage.
 

Nemesis_001

Member
Dec 24, 2022
36
3
8
How about a DIY AMD system?
  • CPU: AMD Pro 4750GE - ~170 used on ebay
  • Motherboard: AsRockRack X470D4U - $260 new
  • RAM: 64GB ECC UDIMM - $240 new
This should be much more performant and energy efficient, and leave you enough for a chassis and storage.
Thanks. My brother tried this board at the time with a consumer CPU, had some issues with stuttering when using ESXI.
How is ESXI support? I'm looking for something that is pretty hassle free to install and maintain.

Supermicro X11-SCL-F with Xeon E2246G
Running ESX 7 with 4-5 VMs.
That's the main issue, it's a 3 year old platform that will cost $700-800, if I can even find that cpu, right now 2236 is easier to find, and brings the bundle price to around $700.

I paid about $450 for CPU and Mobo combo in 2014, it's unfathomable I need to pay almost double for a 3 year old platform.

1671991334262.png
 
  • Like
Reactions: abq

juma

Member
Apr 14, 2021
64
34
18
Thanks. My brother tried this board at the time with a consumer CPU, had some issues with stuttering when using ESXI.
How is ESXI support? I'm looking for something that is pretty hassle free to install and maintain.
Technically ESXI isn't supported on consumer/Pro Ryzen since there aren't any validated systems for it, but there's plenty of people out there that have got it working. Not sure about the stuttering though. Since the Pro GE line has only 35W TDP, there should be much less stress on the motherboard, PSU, cooler, etc. Don't see any significant points of failure that would differ from the newer Ryzen lineups. While the X470 chipset is a bit dated, it's the cheapest board out there that has IPMI, which should save you a lot of time if there's any troubleshooting.
 

gea

Well-Known Member
Dec 31, 2010
3,141
1,182
113
DE
In general I would prefer Supermicro mainboards with a server class chipset (and a suited CPU) to support ECC RAM, a model number that ends with "F" to include ipmi. To support next gen ESXi, use a mainboard from the generation 11 or newer, Server Boards | Supermicro ex X11 S..-F up to the quite expensive X12 SPM-TF with 10G or a similar board from Asrock. For an AiO you should either have a mainboard with many Sata ports on different pci lanes to allow a passthtough of some like on some Epyc boards. Best is when you use an LSI HBA like sas 9300 onboard or per pci card. 10G onboard is also a nice to have. Sadly only older cheaper Supermicro boards like the older X11SSH-CTF have 10G and SAS 9300 onboard. As you should have a disc controller option for ESXi and for pass-through the modern and propably best alternative is to use Sata + NVMe. (one for ESXi boot and local datastore, the other for high performance storage passthrough).

To select options I use geizhals.de or skinflint.co.uk (englisch) due the superiour filter options, ex Motherboards Price Comparison Skinflint UK
English pound is at around 1.15 usd or if you use geizhals.de usd and euro are around the same.

ESXi works well with current Intel or AMD server boards, only care about nic support in ESXi. HBA support is also critical ex older 6G LSI HBA no longer supported under ESXi. If you passthrough the HBA this does not matter.
 
Last edited:
  • Like
Reactions: Nemesis_001

Nemesis_001

Member
Dec 24, 2022
36
3
8
Hi gea,
Thanks.
I currently have a LSI sas9220-8i that I am passing through to OmniOS and it serves me well.
X12SPM-TF or either modern Epyc board + CPU are extremely expensive for my modest needs.
X12STH-F with an E-2336 would probably make more sense, but even that is a bit pricy for a modest platform.
I'm just running it as an AIO system, one Windows guest OS and that's it.
I'd probably run another Linux OS as guest, but I'm limited by the 32GB of RAM on my current board.

Honestly, a solution like TrueNAS would probably suit my needs and work on much cheaper hardware like the ASROCK Rack X470 with a Ryzen PRO and ECC RAM. I do like the extreme reliability of my current AIO and the simplicity of moving hardware, but maybe TrueNAS is not too bad nowadays and makes more sense for my needs.

If I could find 2-3 year old hardware for under $500 moving along with inertia would be easy, but that is hard to find.
 

gea

Well-Known Member
Dec 31, 2010
3,141
1,182
113
DE
Newer 12/13gen SuperMicro serverboards are ultrasolid but quite expensive and currently lack sub 500 $ options that were available in the X9-X10 gen boards. Without 10G and SAS and maybe IPMI the boards are cheaper. Asrock is a good alternative, https://www.asrockrack.com//general/products.asp#Server. With AMD boards ECC is also not as expensive as with Intel boards but the new Ryzen 7000 and 11/12gen Intel ones are very new and hard to find, maybe you want to wait for availability as X10 gen boards are still ok even for current ESXi8 and a board switch is quite painless with ESXi. 32GB max RAM is propable the main limit even when ESXi (2GB) and OmniOS (say 6-8GB) are more resource efficient and often faster than other solutions. With AiO check nic compatibility with ESXi.

Desktop AMD boards with desktop CPUs may be an option with ECC for a homeserver and ESXi will work but without the ipmi option. Not so sure about problems on some more highend features like pass-through or virtualisation. Intel is often better when it comes to "it just works in the server segment" you need a more expensive server chipset (C..) when you want ECC.

Just to add
You can use a Debian for ZFS storage, virtualisation and to provide services. This is a matter of philosophy. But any problem on either or after a simple security or bugfix update means the whole systerm is offline or unstable or under maintenance. The idea of AiO is to separate the three completely. ESXi is quite the most minimalistic and stable VM platform as OmniOS is for ZFS storage. All services beside VM server and base storage are located in VMs so the two most critical parts of a server are usually always up and running. A failure of the ESXi boot disk means to restore ESXi and import VMs. This is done within minutes. OmniOS can also be restored within minutes. Both without any special settings and dependencies. Minimalistic and separated storage and ESXi server with other VMs for services is the reason why AiO is so stable.
 
Last edited:

Nemesis_001

Member
Dec 24, 2022
36
3
8
Thanks gea.
Even with AIO, if the underlying OmniOS is down due to an update, everything is still down, although applications are separate.
My brother swears by TrueNAS, but he hasn't suffered a catastrophe to see how easy it is to recover from.

Guess I'll stick to my tried and true AIO which gave me close to no issues over close to 9 years.
My preferred option would be a used E-2200 or E-2100 series Xeon with a matching SuperMicro X11STL, but those are nowhere to be found.

I guess my only choice is now between:
X12STH-F + E-2336 which would run me about $750 before RAM or
Asrock Rack X470D4U + Used Ryzen Pro 4750GE which will run me about $430 before RAM.

I found a bunch of happy campers with the Ryzen system on some German forum. ECC supposedly works.
Might even be able to use my 3800X and use 5600X on my desktop instead.

Planning to move from ESXI 6.7u2 to ESXI 8.0

Any idea? What would you suggest?
 

gea

Well-Known Member
Dec 31, 2010
3,141
1,182
113
DE
The question is not if something fails but how long do you need to regain functionality and how complicate it is. It is a difference if you need 20 exact steps or simply a minimal vanilla setup.

Desktop vs server boards.
In first case you alone are responsible for fixes. It can help if others say it works for them in a similar setup. At least avoid parts that are known to have problems ex cheap Realtec nics and very new or too old parts. Prefer mainstream (server) parts that are supported in a default ESXi setup.
On common server boards VMware cares about as they are widely used in enterprises. An update to ESXi 8 should work with your current board either when booting the iso via USB stick (Rufus) or after an online update via Putty, How to update any VMware ESXi Hypervisor to the latest using ESXCLI for easy download and install
 

Octopuss

Active Member
Jun 30, 2019
411
62
28
Czech republic
Just to offtopic here a little, why do people bring Ryzen PRO CPUs up over and over again? Those extra features are actually good for anything in home environment?
 

gea

Well-Known Member
Dec 31, 2010
3,141
1,182
113
DE
Just to offtopic here a little, why do people bring Ryzen PRO CPUs up over and over again? Those extra features are actually good for anything in home environment?
Not from functionality.

What is the difference between Ryzen and Ryzen Pro?

A superset to the standard Ryzen chips,
the PRO chips have the same feature set as other Ryzen devices, but also offer enhanced security (memory encryption), 24 months availability, a longer warranty and promise to feature better chip quality
 

juma

Member
Apr 14, 2021
64
34
18
Not from functionality.

What is the difference between Ryzen and Ryzen Pro?

A superset to the standard Ryzen chips,
the PRO chips have the same feature set as other Ryzen devices, but also offer enhanced security (memory encryption), 24 months availability, a longer warranty and promise to feature better chip quality
I'd argue that being able to use ECC with an integrated GPU is a functional improvement over the non-Pro APUs.