Building a hyperconverged cluster with K8S support?

thetoad

Active Member
Feb 10, 2021
106
30
28
Is their any system good for a homelab where one can have 4 2u machines with gobs of storage that are clustered together into a iaas (ala proxmox), but with the ability to easily spin up k8s clusters on demand and scale them as needed up and down, but also to have easy access to the cluster's storage (say ceph) for use within the kubernetes cluster as dynamic persistent volumes (i.e. provisioned and removed on demand per the needs of the pods one is running).

proxmox does not seem to fit this criteria, nor seems to be interested in going in that direction. TrueNAS Scale is sort of moving in that direction, but it seems only single node solution and the K8S is more about simply running predefined containerized applications than being a full fledged K8S solution (at least at the moment).

i.e. the homelab version of GCP and GKE together.
 

iGene

Member
Jun 15, 2014
33
9
8
Taiwan
From your description it seems like OpenStack is the closest one that fit to your need.

If you don't need to spin up multiple Kubernetes cluster you can take a look at harverster (GitHub - rancher/harvester: Open source hyperconverged infrastructure (HCI) software)
Although it by default uses longhorn for storage instead of Ceph.

I would say Proxmox at this time isn't a IaaS cloud solution, it's more like a hypervisor manager instead of a cloud.
 

thetoad

Active Member
Feb 10, 2021
106
30
28
I should note that I hate openstack :) harvester looks reasonable. with that said, I do like being able to run mutiple k8s clusters. A reason being, I'm doing a lot of development with validating and mutating webhooks and custom resources and thos are cluster wide resources and if i'm working on multiple branches at the same time, its hard to do the work on a single k8s cluster.
 

thetoad

Active Member
Feb 10, 2021
106
30
28
I would say Proxmox at this time isn't a IaaS cloud solution, it's more like a hypervisor manager instead of a cloud.
I should note that I refer to it as an iaas because of its cluster ability and ability to provide cluster storage across all its nodes. when it comes to terminology, I've never been known to be that accurate. :)