Proxmox VE raid 1/ZFS?

ScarredPinguin

New Member
Mar 19, 2021
11
0
1
Hello everyone!
I'd like to kindly ask you for a recommendation on installing Proxmox to my server.
As written bellow, I have Ryzen 5 3600 server with 32GB 3200Mhz Cl22 ECC RAM, Two SSDs (Mx500 250GB, Samsung 850 PRO 256GB) and one new Intel Optane 16GB.
The idea n.1 is to install proxmox in Raid 1 on both SSDs but since I don't have a data raid controller, I was wondering if I can use ZFS Raid (which is very new to me) and use my 16GB intel Optane as a cache.
Idea n.2 is to install Proxmox on 16GB intel Optane (if it's possible) and create software raid 1 in Proxmox with my two SSDs and have VMs on it.

Since I want to put the server in datacentre where I won't have much physical access, Proxmox VE looks like a good OS as far as I'm concerned.

Thank you in advance!:)
Any recommendations are welcome!

Build’s Name: Willy
Operating System: Proxmox VE
CPU: Ryzen 5 3600
Motherboard: Asus prime X570-P
Chassis: 2U Supermicro SC825
Drives: 1x Crucial MX500 250GB, 1x Samsung 850 PRO 256 GB, 1x Intel Optane 16 GB
RAM: ECC Kingston 2x 16gbb DDR4 3200mhz cl22 1.2V KSM32ED8/16ME
Add-in Cards: SAS controller planed for 8 HDDs
Power Supply: Supermicro PWS-563-1H
Other Bits: SAS backplane with 8 hot swap caddies
Usage: Proxmox VE with VMs for Minecraft server, Kodi/Plex server and Nextcloud/Owncloud server
 

ttabbal

Active Member
Mar 10, 2016
767
209
43
44
Sure you could. Either would likely work fine. I'm not sure how much you would gain with the Optane drive in either case though. A ZFS mirror is already quite fast on SSDs. To really benefit from a cache, you need that cache to be faster than the drives. It is, but how much? And you would possibly be IOPS limited on it vs the pair of SSDs.

Since you are planning spinners with the SAS card, I'd use the Optane there as a cache if at all. Considering that all access will end up being over the internet, it might not matter.

Overall, it looks like the setup will do what you want it to do though. I use Proxmox for my main server and it's been working very well.
 
  • Like
Reactions: ScarredPinguin

pinkanese

New Member
Jun 19, 2014
25
8
3
30
Definitely option 1. The Proxmox installer has the option to setup ZFS, it is very easy.

I would honestly just leave out the 16GB Optane drive. It only works at PCI-e 3.0 x2 and the write speed is a pretty poor 145 MB/s. Though one interesting use for it could be as storage for the Minecraft world files, not sure it would be worth the hassle setting it up.

You might also want to consider using ZFS for your hard drive storage as well. You can set it up through the Proxmox web interface. It would tie up some of your RAM, but there are a number of benefits to ZFS over a tradition raid controller/array.
 
  • Like
Reactions: ScarredPinguin

ScarredPinguin

New Member
Mar 19, 2021
11
0
1
Thank you guys for help!:)
I was thinking if Proxmox on Optane would be a good idea since it has a very good TBW, and VMs on SSDs with ZFS Raid 1.
I know that it has 145 MB/S Write speed but what would it be writing/reading if VMs were on SSDs?
I'm not using server SSDs yet and I've been told that ZFS raid will wear my consumer SSDs quickly.

Is it wise to have Proxmox on Optane, VMs on zfs raid 1 SSDs and when they wear out, I'll just replace them with better and higher capacity ones? Or should I just stick with Proxmox with VMs on zfs raid 1 SSDs and use Optane just for Minecraft server files?
 

ttabbal

Active Member
Mar 10, 2016
767
209
43
44
If there's no I/O, why have the Optane at all? :)

IMO, the issue with consumer SSDs and ZFS is overblown. It does run them a bit harder, but it's not like they are going to die in a week. Even with the Optane as a cache, you still have to push data in/out of the SSDs.

I wouldn't use the Optane as a boot drive solo. It will work, but you now have a single point of failure for the whole server.

I think it makes more sense to drop it entirely, it's just not doing much for you in this case.

Use the SSDs as a mirror, with timed backup to the HDD array. When they fall over, which will likely take longer than you expect, swap it and rebuild the mirror (zfs replace). If both go, you still have the slower backup you can restore from.
 
  • Like
Reactions: ScarredPinguin

ScarredPinguin

New Member
Mar 19, 2021
11
0
1
If there's no I/O, why have the Optane at all? :)

IMO, the issue with consumer SSDs and ZFS is overblown. It does run them a bit harder, but it's not like they are going to die in a week. Even with the Optane as a cache, you still have to push data in/out of the SSDs.

I wouldn't use the Optane as a boot drive solo. It will work, but you now have a single point of failure for the whole server.

I think it makes more sense to drop it entirely, it's just not doing much for you in this case.

Use the SSDs as a mirror, with timed backup to the HDD array. When they fall over, which will likely take longer than you expect, swap it and rebuild the mirror (zfs replace). If both go, you still have the slower backup you can restore from.
Thank you! That makes sense