Hello all,
Thanks in advanced for reading and your feedback. I have a Cisco UCS server with 8 SAS drives, dual 2660 v4, 384GB RAM. I’m trying to map out the best way to utilize it for what I need. I’m coming from an UnRaid gaming machine that runs some gaming VM and all my media dockers (plex, Radarr, Sonarr, etc). I’ve done a lot with ESXi, but not really for home use.
Objectives:
ESXi with OMV/Napp-it VM for storage. RancherOS VM for Dockers. Not FreeNAS. I tried this first. FreeNAS crashed and burned with my RAID controller, even in JBOD. Have not tried OMV/Napp-it yet.
Pros:
Option 2:
Proxmox with Docker/portainer running in parallel. Check_MK container for ZFS monitoring.
Pros:
Same as option 1, but use Proxmox instead of ESXi. This would allow Proxmox to natively manage the ZFS pool, while not needing to share the pool back to the hypervisor via NFS. Basically, avoid using the SLOG. Otherwise, same pros/cons.
Questions:
Thanks in advanced for reading and your feedback. I have a Cisco UCS server with 8 SAS drives, dual 2660 v4, 384GB RAM. I’m trying to map out the best way to utilize it for what I need. I’m coming from an UnRaid gaming machine that runs some gaming VM and all my media dockers (plex, Radarr, Sonarr, etc). I’ve done a lot with ESXi, but not really for home use.
Objectives:
- Deploy a media stack of Dockers. I want Radarr to be able to hard-link files from the download client. To elaborate: Radarr sends the request of what’s to be downloaded to deluge. Deluge downloads it to a directory (NFS share?) named Downloads. Deluge extracts it in-place. Radarr sees the the downloaded file and renames/hard-links it to a plex watched media directory. I could not get hard-linking in unRaid to work to save my life. It would always copy the file creating duplicates.
- I want to use ZFS. I want alerts/monitoring of drive status. Would prefer to manage with a GUI.
- Avoid using a SLOG (already bought a 900p), but I will use it if 100% necessary.
- Run some needed VM such as GNS3, Cisco ISE, Firepower, PiHole(maybe Docker or container?).
ESXi with OMV/Napp-it VM for storage. RancherOS VM for Dockers. Not FreeNAS. I tried this first. FreeNAS crashed and burned with my RAID controller, even in JBOD. Have not tried OMV/Napp-it yet.
Pros:
- I know it (VMWare)
- More compatibility with VMs I use.
- GUI for ZFS management/alerts/status.
- Would need SLOG for NFS performance. Datastore is mounted via NFS.
- Dockers would have to run in a VM. Possible performance issue?
- OMV ZFS support would be a plug-in, not “native” support. Not sure about Napp-it.
Option 2:
Proxmox with Docker/portainer running in parallel. Check_MK container for ZFS monitoring.
Pros:
- Can manage ZFS locally and natively.
- VM storage is “local” to ZFS. Meaning datastore is not NFS.
- Very easy to deploy new Dockers with local storage. Again, no NFS share.
- Can get away with not having a SLOG from performance perspective, but not data safety. Don’t really care about VM data safety, I can just rollback a snapshot.
- Bare-metal Docker performance.
- Unsafe for production?
- Docker is out-of-band. Meaning have to run two separate GUI to manage/monitor.
- No GUI for ZFS management. Must use Check_MK for monitoring and alerts.
Same as option 1, but use Proxmox instead of ESXi. This would allow Proxmox to natively manage the ZFS pool, while not needing to share the pool back to the hypervisor via NFS. Basically, avoid using the SLOG. Otherwise, same pros/cons.
Questions:
- Any further considerations?
- How the heck do I manage shares to achieve the hard-liking I want? This part has me the most confused. I’ve got 30+ tabs open and it’s only getting worse :’(
- How does Auto-ZFS snapshot work? What is snapshotted and what isn’t? Admittedly, I haven’t researched this yet.