Easy/cheap cluster questions

[LeeMan]

I'm not really sure if I'm really understanding how to scale easily with OS as this is a new venture for me.

Say I have 2-4 cheap servers, I would like to have a way of making this use all of the processing power from all of them to do a specific task say run a Plex server.

I've read some about ESXi but have no experience with it in scalability. Main question being how would you setup server small cheap servers in a cluster to run a Plex server? And would it be possible to just add more servers to the cluster to give more processing power?

 

[Peanuthead]

LMGTFY

 

[LeeMan]

LMGTFY

Thanks for that... I already know about the distributed encoding. Plex was just an example. Also that github only supports a really old version of Plex server.

 

[Peanuthead]

I misread he question thinking it was for Plex only. In anwer, ESXi is scaleable in regard to up time not giving a single VM all the power of the cluster.

 

[Diavuno]

Distributed workloads are specific to an application.

Many small servers with identical everything might be a good candidate for a load balancing server.... If the application reports properly, I know one of my techs tried a 2 server system and a basic load balancer for plex... didnt work.

Plex does not currently support it.

On the flip side, something like Exchange or supports clustering out of the box.

 

[LeeMan]

Distributed workloads are specific to an application.

Many small servers with identical everything might be a good candidate for a load balancing server.... If the application reports properly, I know one of my techs tried a 2 server system and a basic load balancer for plex... didnt work.

Plex does not currently support it.

On the flip side, something like Exchange or supports clustering out of the box.

Thanks! For some reason I thought ESXi was able to cluster a couple server together in a VM when then you could harness that power.

I might try that github with the Plex remote transcode with two servers direct connected via SFP+ 10GbE for low latency when sharing the folders on the main drive.

 

[Diavuno]

Thanks! For some reason I thought ESXi was able to cluster a couple server together in a VM when then you could harness that power.

Nope, ESXi can cluster itself for various forms of redundancy and load balancing, but the power of a VM is limited to the host it is on. (as far as I know this is true for all hypervisors)

 

[niekbergboer]

I'll suggest Proxmox VE; it does proper multi-master clustering which removes your SPOV. Also, it comes with ZFS support out of the box, even root on ZFS.

In terms of storage sharing, a nice feature is its "built-in" (requires a few very well documented shell commands) support for Ceph. After that, you just administer the whole thing through a nice web UI (on any of the nodes!). This allows you to live-migrate VMs.

High Availability; run your critical VMs in HA mode, and even if the current host's mainboard goes poof (happened to me; my pfSense VM ran on that node), the VM will automatically be started elsewhere.

Proxmox VE runs either KVM or LXC; it doesn't support Docker at present, though you can run a KVM VM that does that.

 

[vl1969]

I'll suggest Proxmox VE; it does proper multi-master clustering which removes your SPOV. Also, it comes with ZFS support out of the box, even root on ZFS.

In terms of storage sharing, a nice feature is its "built-in" (requires a few very well documented shell commands) support for Ceph. After that, you just administer the whole thing through a nice web UI (on any of the nodes!). This allows you to live-migrate VMs.

High Availability; run your critical VMs in HA mode, and even if the current host's mainboard goes poof (happened to me; my pfSense VM ran on that node), the VM will automatically be started elsewhere.

Proxmox VE runs either KVM or LXC; it doesn't support Docker at present, though you can run a KVM VM that does that.

this is all true, but if you read carefully, and this is just an FYI, Proxmox VE (you will need version 4.3 and up ) REQUIRES a MINIMUM of 3 nodes for the this type of no SAN shared storage HA Cluster setup. Unless I misunderstand something in my research.

other than that ...