Proxmox VE 5.0 and Docker with a Web GUI

Patrick

Administrator
Staff member
Dec 21, 2010
11,965
4,924
113
Heading off to sleep, but I think I have a project tomorrow that people are going to be interested in. We have had threads on Proxmox v. Docker and periodic threads on why not Proxmox and Docker.

For a home lab proof of concept I now bring you: Docker running on Proxmox bare metal with a web GUI.

Storage server: FreeNAS Corral (Xeon D-1518 based) with bhyve virtualization and Docker support
Virtualization server: Proxmox VE 5.0 Beta1 (Xeon D-1540 based) with KVM virtualization, LXC containers (Proxmox GUI), Docker CE version 17.03.0-ce, Portainer Docker Management

Here is the hello world of this setup.
upload_2017-3-22_22-7-12.png

I believe this is now the ultimate virtualization lab setup.

  • The FreeNAS host has bhyve and Docker (docker in a VM) plus super ZFS storage
  • The Proxmox host has KVM, LXC, and Docker for running applications plus ZFS, Ceph Luminous, and Gluster. Key here is that this KVM should support running Hyper-V nested.
The Proxmox team specifically states this is a bad idea since adding Docker on metal Proxmox introduces a slew of security challenges. For lab use, the Proxmox server now has just about everything you could possibly want.

I will try to get instructions up tomorrow morning if anyone is interested.
 

gigatexal

I'm here to learn
Nov 25, 2012
2,766
529
113
Portland, Oregon
alexandarnarayan.com
so on the FreeNAS side you'd run a storage container and then on the Proxmox side you'd start container(s) that point to the storage container on FreeNAS? I guess I am not sure how this all shakes out, I'll wait for the article though.
 

Eric Faden

Member
Dec 5, 2016
98
6
8
38
This looks awesome.... I just wish proxmox would build in docker... Can't wait for instructions. Totally doing this on my boxes.
 
Last edited:

Patrick

Administrator
Staff member
Dec 21, 2010
11,965
4,924
113
Did you think about installing this inside of an LXC or VM (RancherOS, Boot2Docker) instead of on the proxmox directly?
Yes. That is very easy to do. Putting it into a VM also means you add another level of abstraction which is why I wanted bare metal.
 

Eric Faden

Member
Dec 5, 2016
98
6
8
38
Yes. That is very easy to do. Putting it into a VM also means you add another level of abstraction which is why I wanted bare metal.
Big question I have right now is how to organize my setup. I think Proxmox + LXC Docker makes the most sense to me. The big question is how/where to put my Plex setup given it's I/O needs. Right now I have a Debian VM with Docker that has a LSI card passed through for the drives. I'm considering changing it up... but I'm not sure which would be the best.... But I already made a thread for this question.
 

Eric Faden

Member
Dec 5, 2016
98
6
8
38
Did you ever post a guide for this? I tried to install 5 on my supermicro box using ipmi... The screen didn't detect right. The ok button was off the screen to the right. Tried resetting... Still.. nothing. Going to setup something similar.

Sent from my Pixel XL using Tapatalk
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,965
4,924
113
@Eric Faden not yet. Was sick this weekend. On the docket for this week though since I want it out before Ryzen 5.
 

ecosse

Active Member
Jul 2, 2013
379
66
28
As someone who is a hell of a lot clearer about a virtualisation platform around VMware, I'd be very interested if there is a comparison of how the above is so much better than a stack built on ESXi (sorry if its a dumb comment, I realise that I'd still be layering docker etc ontop of VMware in that path)
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,965
4,924
113
As someone who is a hell of a lot clearer about a virtualisation platform around VMware, I'd be very interested if there is a comparison of how the above is so much better than a stack built on ESXi (sorry if its a dumb comment, I realise that I'd still be layering docker etc ontop of VMware in that path)
Starting off, ESXi (free) is fairly basic/ limited at this point right? Most features you want are an upgrade.

Proxmox has the GUI for KVM but also has features like ZFS, Gluster, Ceph, LXC containers and the HA clustering side built in. The HCL is also larger for Proxmox than VMware out of the box (e.g. Realtek NICs) It is also "free".

Big stack with a larger budget (or free licenses), VMware has a lot of awesome tools and a great ecosystem.

If you are doing more cost sensitive work, Proxmox works very well. If you just need a basic hypervisor, VMware is great.
 

_alex

Active Member
Jan 28, 2016
874
94
28
Bavaria / Germany
As someone who is a hell of a lot clearer about a virtualisation platform around VMware, I'd be very interested if there is a comparison of how the above is so much better than a stack built on ESXi (sorry if its a dumb comment, I realise that I'd still be layering docker etc ontop of VMware in that path)
ESXi / VMware will
- cost you a fortune if you want advanced features above a single node
- bind you to a HCL
- make you dependent on software/drivers/kernel modules available (i.e IB / Ofed) where you are free as a bird with proxmox as it's based on Debian / GNU Linux. Just get what you need and install from binary or compile.
- not offer HC / the Storage options proxmox has built-in without VSan (=expensive, high requirements).
- Not allow you to apply changes / work on features of the Host as it's not open. On proxmox you can add Kernel modules via dkms, Build custom Storage Plugins, replace the Kernel - whatever you want/need ...

so, as long as you don't need Horizon/VDI or any other Feature not available in Proxmox (i.e. workload-based scaling of the Cluster by bringing hosts up/down) it's maybe the better, easier and also cheaper way to go. (correct me if i'm wrong)
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,965
4,924
113
@_alex the obvious major pro for VMware is that there is a large company for support which is a major pro. Likewise, there is a giant community.
 

_alex

Active Member
Jan 28, 2016
874
94
28
Bavaria / Germany
yes, i know ..
but question was why proxmox would be better ;)

Don't know how good proxmox support is as i barely need support on this type of product, especially not as long it's sort of open..
but would guess proxmox paid support also is quite good, and also the community is active.
 

ecosse

Active Member
Jul 2, 2013
379
66
28
Thanks for the thoughts. I run VMware in my lab using the evalexperience. I can see the time when VMware are potentially cut out of the market (but of course through Proton, AWS hookups etc they are trying to stay relevant). Whilst I understand the attractions of opensource, I'm not a coder at heart - I just want the thing to work and be reliable (otherwise I get it in the neck from the wife always up in my computer room!) I've a few spare servers to test drive, I look forward to the write-up.