I'm building a new Proxmox cluster to migrate from unRAID. Looking for some direction from the community.
My hardware (3 Dell Servers):
R730 / 2x2620v3 / 96GB ECC / 2xSSD, 6x10k SAS / Multiple GPUs for pass through- "Compute 1" / C1
R720XD / 2x2609v2 / 64GB ECC / 2xSSD, 12x4TB 7.2k SAS - "Storage 1" / S1
R720XD / 2x2609v2 / 64GB ECC / 2xSSD, 10xVarious SATA - "Storage 2" / S2
All have the 2x10G+1x1G daughter board NICs. R730 has additional 4x10G/4x1G available
I intend to run all of my VMs on C1 but setup HA so that they can swing to S1/S2 if necessary.
S1 is intended as my "bulk storage", S2 as my backup server.
My current direction...
Setup ZFS pools in Proxmox on C1, run all VMs and app containers here.
Install FreeNAS or OMV on S1 and S2 in a VM, nothing else, pass all drives to it, build pools to share with C1.
I would then, for example, setup a VM for Shinobi/Zoneminder on C1 and point it's "data" directory to a shared volume on S1. Setup a VM for Plex on C1 and point it's "data" directory to a shared volume on S1. Setup a client VM for photo editing, point it's data directory so S1, etc, etc. Keeping my apps running on the more capable C1. "data drives" on S1. S2 = backup.
I would use FreeNAS or OMV's replication service to backup my "S1" to "S2" then to off-site storage. S2 will be using my SATA drives from my (about to be decommissioned) unRAID build.
My thoughts on networking...
Dedicated 1GB port per server = Proxmox cluster service
(2) 10GB ports bonded/iSCSI, dedicated for shared storage
Additional (2) 10GB ports on C1 bonded, Ethernet for client access. Aside from management and during any fail over events, clients essentially won't access S1 and S2. I'm kind of viewing these as "SAN" nodes during normal operation, accessed by C1.
Questions...
Am I headed down the right path?
How best to share the storage between nodes? i.e. does iSCSI make sense? Should I be looking at Ceph for this type of setup?
FreeNAS vs. OMV (is one preferred over the other based on my requirements)?
Any thoughts/suggestions would be appreciated!
My hardware (3 Dell Servers):
R730 / 2x2620v3 / 96GB ECC / 2xSSD, 6x10k SAS / Multiple GPUs for pass through- "Compute 1" / C1
R720XD / 2x2609v2 / 64GB ECC / 2xSSD, 12x4TB 7.2k SAS - "Storage 1" / S1
R720XD / 2x2609v2 / 64GB ECC / 2xSSD, 10xVarious SATA - "Storage 2" / S2
All have the 2x10G+1x1G daughter board NICs. R730 has additional 4x10G/4x1G available
I intend to run all of my VMs on C1 but setup HA so that they can swing to S1/S2 if necessary.
S1 is intended as my "bulk storage", S2 as my backup server.
My current direction...
Setup ZFS pools in Proxmox on C1, run all VMs and app containers here.
Install FreeNAS or OMV on S1 and S2 in a VM, nothing else, pass all drives to it, build pools to share with C1.
I would then, for example, setup a VM for Shinobi/Zoneminder on C1 and point it's "data" directory to a shared volume on S1. Setup a VM for Plex on C1 and point it's "data" directory to a shared volume on S1. Setup a client VM for photo editing, point it's data directory so S1, etc, etc. Keeping my apps running on the more capable C1. "data drives" on S1. S2 = backup.
I would use FreeNAS or OMV's replication service to backup my "S1" to "S2" then to off-site storage. S2 will be using my SATA drives from my (about to be decommissioned) unRAID build.
My thoughts on networking...
Dedicated 1GB port per server = Proxmox cluster service
(2) 10GB ports bonded/iSCSI, dedicated for shared storage
Additional (2) 10GB ports on C1 bonded, Ethernet for client access. Aside from management and during any fail over events, clients essentially won't access S1 and S2. I'm kind of viewing these as "SAN" nodes during normal operation, accessed by C1.
Questions...
Am I headed down the right path?
How best to share the storage between nodes? i.e. does iSCSI make sense? Should I be looking at Ceph for this type of setup?
FreeNAS vs. OMV (is one preferred over the other based on my requirements)?
Any thoughts/suggestions would be appreciated!