Proxmox configuration suggestions

[Scionero]

Hardware I currently have:

Dual xeon E5-2650 V2 and Asrock board
64GB ECC ram
Two 10TB NAS drives
Three 120GB S3500 Intel SSDs
All the appropriate hardware to go along with it, including a UPS.

The servers purpose will be to host maybe a dozen VMs and containers, all serving different purposes. Plex, AD, nextcloud, linux distros for learning various things, torrent box, etc.

I am thinking about installing proxmox to the mirrored 10TB drives, with 4k sectors and sync=always. After that I'll add a mirrored slog of maybe 6GB with two of the SSDs, overprovisioned. The last SSD will be a 64GB L2arc. I'll start with these values and evaluate performance, changing later as needed. I will be increasing the ram to 256GB in increments as money allows, as well as adding hard drives when possible.

My question is simply, will this give me decent performance? Would you change anything?

 

[Patrick]

I did my first RAID 10 HDD data store on Proxmox ZFS in over a year recently. It was noticable how slow it was.

Personally I would install Proxmox on one or mirrored 120GB drives. I would then use the leftover drives as read cache and think about a ZIL / SLOG device if possible.

 

[Scionero]

Thanks for your response Patrick. You think it would still be slow with the slog mirror? I'm trying to combine size and speed without having to do two separate pools.

 

[vl1969]

Hi, I am also rebuilding my Proxmox setup and not sure what the best way to do it.
I have recently did the full overhaul of the system after my PSU failed. I thought my disks were dying since they were dropping out of Zpool randomly. turned out it was the PSU. but by the time I figured it out I had bough a whole set of new components so figured I build a new server.
so as of now I have up and running

the MB have a sas controller on-board but it does not support large disks so I got the HBA and a 24 port expender

I run Proxmox on ZFS mirror using 2 120GB ssd.
no slogs or zil. would like to have it but simply have no space to put more SSDs. the case is 24 3.5 disks old SuperMicro. I have a couple of DELL 3.5 to 2.5 caddie adaptors that I want to try mount intro the SM caddies but haven't try that yet. if it works I have 2 more SSDs I can use for slog or zil, but I am not sure how to configure it.

I have 4 zfs pools setup.

2x1TB disks Mirror pool for use as Proxmox storage. I want to move the default zfs-local stor onto this disks as the boot drives too small.

the rest are grouped into mirrored pools by size.
as in tank2 useing 2x4TB
tank1 using 4x3TB
tank0 using 6 2TB.
I used to have 8 2Tb but one had failed before I add it to the tank0 so now I have a spare 2TB.

all disks Are user storage.
in my old setup I bind-mount the folders on the pools into LXC container .
I want to run
1 LXC as my Emby server
1LXC as my File Server

I want to run several LXC with Calibre server, Lazy Librarian, CoachPotato, Sick Rage, SubNZB, Deluge or Transmission and if I can figure out how to run headless jDownloader with web UI
maybe MithTV or something like it if I ever get a network tuner.

I do not download much but I want to setup the above bunch so it can help managing my media.
that is if I do download something I want to just drop it into download folder and let the apps move/rename automatically.
it worked before on small scale but I lost the setup.
speed was never an issue before, and with the new hardware it should be OK now but if some one can point out a better solution please do.
also a safer setup.
I almost lost all my files when proxmox crushed on me. not even sure what happened.
one day the reboot start take too long and at some point stop booting at all. stopped at black screen with blinking cursor.
my back up drive got screwed up too but that was my mistake. I tried to send zfs snapshot to external drive I had with EXT4.
and it wiped all my partition table. I only found out when the server went down and I checked the backup.
I am smarter now. have all my data back, but it was a nerve wrecking couple of days.

SuperMicro 846A case
SuperMicro X8DT3-LN4F Server Motherboard
Pair of Intel Xeon X5670 SLBV7 2.93GHz 12MB LGA1366 6-Core
4xHP 500207-071 - 16GB DDR3 SDRAM
4x Samsung 16gb pc3-8500r HP 500207-071
HP H220 6Gbps SAS PCI-E 3.0 HBA LSI 9207-8i P20 IT Mode for ZFS FreeNAS unRAID
HP 487738-001 468405-001 24-Bay 3Gb SAS Expander Card ProLiant ML370 & ML350 G6
2x 120GB SanDisk SDSSDA120G  SSD
6x2GB Seagate HDD
4x3TB Toshiba HDD
2x4TB Seagate
2x1TB toshiba