Proxmox VE 5.0 and Docker with a Web GUI

[Patrick]

Heading off to sleep, but I think I have a project tomorrow that people are going to be interested in. We have had threads on Proxmox v. Docker and periodic threads on why not Proxmox and Docker.

For a home lab proof of concept I now bring you: Docker running on Proxmox bare metal with a web GUI.

Storage server: FreeNAS Corral (Xeon D-1518 based) with bhyve virtualization and Docker support
Virtualization server: Proxmox VE 5.0 Beta1 (Xeon D-1540 based) with KVM virtualization, LXC containers (Proxmox GUI), Docker CE version 17.03.0-ce, Portainer Docker Management

Here is the hello world of this setup.


I believe this is now the ultimate virtualization lab setup.

• The FreeNAS host has bhyve and Docker (docker in a VM) plus super ZFS storage
• The Proxmox host has KVM, LXC, and Docker for running applications plus ZFS, Ceph Luminous, and Gluster. Key here is that this KVM should support running Hyper-V nested.

The Proxmox team specifically states this is a bad idea since adding Docker on metal Proxmox introduces a slew of security challenges. For lab use, the Proxmox server now has just about everything you could possibly want.

I will try to get instructions up tomorrow morning if anyone is interested.

 

[gigatexal]

Easy to run nested virtualization is a huge selling point for me. I might try this

 

[MiniKnight]

This. Made. My. Day.

Get typing.

 

[T_Minus]

I am most def. going to try and play around with this!

 

[MiniKnight]

Still waiting.

 

[Patrick]

Busy day today.

 

[gigatexal]

so on the FreeNAS side you'd run a storage container and then on the Proxmox side you'd start container(s) that point to the storage container on FreeNAS? I guess I am not sure how this all shakes out, I'll wait for the article though.

 

[Eric Faden]

This looks awesome.... I just wish proxmox would build in docker... Can't wait for instructions. Totally doing this on my boxes.

 

[Eric Faden]

Did you think about installing this inside of an LXC or VM (RancherOS, Boot2Docker) instead of on the proxmox directly?

Proxmox / LXC - Running docker inside a container | The Solaris Cookbook | The Cookbook and reference for IT Admins | container, Docker, LXC, Proxmox for example.

 

[Patrick]

Did you think about installing this inside of an LXC or VM (RancherOS, Boot2Docker) instead of on the proxmox directly?

Yes. That is very easy to do. Putting it into a VM also means you add another level of abstraction which is why I wanted bare metal.

 

[Eric Faden]

Yes. That is very easy to do. Putting it into a VM also means you add another level of abstraction which is why I wanted bare metal.

Big question I have right now is how to organize my setup. I think Proxmox + LXC Docker makes the most sense to me. The big question is how/where to put my Plex setup given it's I/O needs. Right now I have a Debian VM with Docker that has a LSI card passed through for the drives. I'm considering changing it up... but I'm not sure which would be the best.... But I already made a thread for this question.

 

[Eric Faden]

Did you ever post a guide for this? I tried to install 5 on my supermicro box using ipmi... The screen didn't detect right. The ok button was off the screen to the right. Tried resetting... Still.. nothing. Going to setup something similar.

Sent from my Pixel XL using Tapatalk

 

[Patrick]

@Eric Faden not yet. Was sick this weekend. On the docket for this week though since I want it out before Ryzen 5.

 

[Eric Faden]

@Eric Faden not yet. Was sick this weekend. On the docket for this week though since I want it out before Ryzen 5.

Awesome.

Sent from my Pixel XL using Tapatalk

 

[ecosse]

As someone who is a hell of a lot clearer about a virtualisation platform around VMware, I'd be very interested if there is a comparison of how the above is so much better than a stack built on ESXi (sorry if its a dumb comment, I realise that I'd still be layering docker etc ontop of VMware in that path)

 

[Patrick]

As someone who is a hell of a lot clearer about a virtualisation platform around VMware, I'd be very interested if there is a comparison of how the above is so much better than a stack built on ESXi (sorry if its a dumb comment, I realise that I'd still be layering docker etc ontop of VMware in that path)

Starting off, ESXi (free) is fairly basic/ limited at this point right? Most features you want are an upgrade.

Proxmox has the GUI for KVM but also has features like ZFS, Gluster, Ceph, LXC containers and the HA clustering side built in. The HCL is also larger for Proxmox than VMware out of the box (e.g. Realtek NICs) It is also "free".

Big stack with a larger budget (or free licenses), VMware has a lot of awesome tools and a great ecosystem.

If you are doing more cost sensitive work, Proxmox works very well. If you just need a basic hypervisor, VMware is great.

 

[_alex]

As someone who is a hell of a lot clearer about a virtualisation platform around VMware, I'd be very interested if there is a comparison of how the above is so much better than a stack built on ESXi (sorry if its a dumb comment, I realise that I'd still be layering docker etc ontop of VMware in that path)

ESXi / VMware will
- cost you a fortune if you want advanced features above a single node
- bind you to a HCL
- make you dependent on software/drivers/kernel modules available (i.e IB / Ofed) where you are free as a bird with proxmox as it's based on Debian / GNU Linux. Just get what you need and install from binary or compile.
- not offer HC / the Storage options proxmox has built-in without VSan (=expensive, high requirements).
- Not allow you to apply changes / work on features of the Host as it's not open. On proxmox you can add Kernel modules via dkms, Build custom Storage Plugins, replace the Kernel - whatever you want/need ...

so, as long as you don't need Horizon/VDI or any other Feature not available in Proxmox (i.e. workload-based scaling of the Cluster by bringing hosts up/down) it's maybe the better, easier and also cheaper way to go. (correct me if i'm wrong)

 

[Patrick]

@_alex the obvious major pro for VMware is that there is a large company for support which is a major pro. Likewise, there is a giant community.

 

[_alex]

yes, i know ..
but question was why proxmox would be better

Don't know how good proxmox support is as i barely need support on this type of product, especially not as long it's sort of open..
but would guess proxmox paid support also is quite good, and also the community is active.

 

[ecosse]

Thanks for the thoughts. I run VMware in my lab using the evalexperience. I can see the time when VMware are potentially cut out of the market (but of course through Proton, AWS hookups etc they are trying to stay relevant). Whilst I understand the attractions of opensource, I'm not a coder at heart - I just want the thing to work and be reliable (otherwise I get it in the neck from the wife always up in my computer room!) I've a few spare servers to test drive, I look forward to the write-up.