I'm planning to build out my new homelab after my house reno is done. I've been using VMware for a couple years and it wasn't until recently and reading the articles on STH that I decided to give Proxmox a real try. I have a test system configured at my parents' place with a FreeNAS as storage backend. So far I'm really liking it. I have FreeNAS sharing the zvol via iSCSI to Proxmox as the main image/container storage. I also have mpio working using Intel 10G NICs. I'm pondering to put Proxmox on my 4 node Intel H2312JFJR each with dual 10G NIC when I move in, with a similar FreeNAS setup as the storage. My question is, what is the best way to do this to have a fully HA cluster with live migration capabilities? I want to keep mpio if possible but iSCSI and ext4 would not work in this situation.
FreeNAS + 4 Proxmox Nodes Question
- Thread starter eroji
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Promox with Ceph , hyper-converged HA cluster.
You already have all the hardware ( 4 nodes with 10gb nic )
Install Ceph Server on Proxmox VE
You already have all the hardware ( 4 nodes with 10gb nic )
Install Ceph Server on Proxmox VE
My understanding was that for Ceph, each node would hold part of the storage? Can I still use it if all of it is shared from FreeNAS?
Storage: Ceph - Proxmox VE
Each Proxmox node could still access FreeNAS if you choose to add FreeNAS in the mix.
Patrick's STH ( this forum ) is running on Proxmox Ceph HA cluster.
If using Proxmox Ceph cluster, then FreeNAS is not "required".
Each Proxmox node could still access FreeNAS if you choose to add FreeNAS in the mix.
Patrick's STH ( this forum ) is running on Proxmox Ceph HA cluster.
Right, that's what I understood it as, similar to VSAN. I did consider it, but I already had a built out FreeNAS infrastructure for storage, previously running VMware. I am trying to figure out how to most efficiently adapt it to Proxmox. Since setting up the iSCSI mpio is pretty straight forward, the only thing I am not clear about is what the best practice is to prepare the volumes so that they can be safely shared. VMware made that easy with VMFS.
As a thought, I suppose I could have 4 separate volumes each shared to a respective node via iSCSI from FreeNAS. Then I can create a Ceph cluster using that. Though that seems hacky.
Last edited:
Sorry, I don't use Proxmox with iSCSI.
So I am learning about this subject. I came across this post in Proxmox forum describing the procedure.
Proxmox clustering and iscsi shared storage
So I am learning about this subject. I came across this post in Proxmox forum describing the procedure.
Proxmox clustering and iscsi shared storage
A 4-nodes setup with a FreeNAS as a single storage object is a classical inverted pyramid of doom.
You need to build a hyper-converged cluster in this case and leave FreeNAS for backups for example. In order to do this, you either use CEPH which is quite tricky or some kind of VSAN that is capable of running inside a VM on top of KVM/Proxmox. AFAIK Starwind does it StarWind Virtual Storage Appliance and its free. Furthermore it's iSCSI and even iSER, MPIO is supported so looks like a perfect fit for you.
You need to build a hyper-converged cluster in this case and leave FreeNAS for backups for example. In order to do this, you either use CEPH which is quite tricky or some kind of VSAN that is capable of running inside a VM on top of KVM/Proxmox. AFAIK Starwind does it StarWind Virtual Storage Appliance and its free. Furthermore it's iSCSI and even iSER, MPIO is supported so looks like a perfect fit for you.
