KVM vs ESXI vs Proxmox for Home Lab

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Alex Skysilk

New Member
Nov 16, 2015
4
1
3
52
There is more content written on the subject then a lifetime of concentrated reading would allow. The first and most obvious question that you'd want to address is

why?

What is ESXi not doing for you, or not well enough, or ? There is a reason that all of those products exist, and they all provide solutions that are attractive for some use cases.
 

apnar

Member
Mar 5, 2011
115
23
18
I just made the move from ESXi to KVM (on CentOS 7). Pretty happy with it so far. There were a number of things that pushed me that direction, but primarily it was VMWare not offering decent way to manage newer versions/features and my desire to get rid of a separate storage VM to run ZFS. With KVM and ZFS now being reasonably stable on Linux I thought I'd give it a shot. Few migration pains here and there but overall fairly happy with the decision.

I did consider proxmox being pretty much the same stack just in appliance form and decided to go with CentOS mainly because that's what I use daily at work.
 

acmcool

Banned
Jun 23, 2015
610
76
28
40
Woodbury,MN
My datastore for vcenter is on Omni OS all in one appliance..Its catch 22 to change anything the Omni os VM...Can not change RAM reservation using thick vmware client..
I use the VM clone/template functionality as well..
Right now I have Vcenter linux appliance and it can not apply patches either
 

Keljian

Active Member
Sep 9, 2015
428
71
28
Melbourne Australia
I use Vt-d, usb pass through quite a bit..
I am using both of those features with esxi without any drama..

As for applying patches, it can be done using ssh into the command line, and I have done this also. A little trickier than using vcenter, granted, but it's not rocket science
 

EluRex

Active Member
Apr 28, 2015
218
78
28
Los Angeles, CA
I switched from esxi 6.0 to proxmox ve4 recently @ both work and home. The following features are missing from esxi are crucial

1. ZFS on root fs just wonderful, it makes backup of the VM hosts so much easier. VMFS is troublesome.

2. Container LXC so much lighter and faster deployment.

3. Support glusterfs! With distributed file system, my iso and lxc template can be automatically synced across all my proxmox nodes.

4. Openvswitch makes vlan much easier than on VMware's vswitch.

5. May use as a nova compute node in OpenStack liberty, but requires serious tweaking. *** work in progress.

6. The cli and bash scripting much easier on pproxmox and why can't VMware use ifconfig or route those common cli instead of vmkernel etc etc commands.

7. For esxi, you can't boot up from a passthrough hd that's attached to a pci device, or usb disk. It must boot from a vmfs or a raw physical hd (iscsi, srp)
 
Last edited:

canta

Well-Known Member
Nov 26, 2014
1,012
216
63
43
just my addition on proxmox since moved to proxmox frpm esxi 5.X early this year without many bumps.

. Openvswitch makes vlan much easier than on VMware's vswitch.
many will argue back and forward especially people are using esxi soley and have no detail on OVS :p

The cli and bash scripting much easier on pproxmox and why can't VMware use ifconfig or route those common cli instead of vmkernel etc etc commands.
Well, proxmox is real linux ehem debian that all script/python, You want can install development enviroment with apt-get or aptitude :D

Container LXC so much lighter and faster deployment.
LXC is getting common that people use on linux distro. I would jump to this 99.99% very soon for my works as developer and at home :D
 

TuxDude

Well-Known Member
Sep 17, 2011
616
338
63
I never figured out a way to backup ESXI host...KVM seems easy
Both at home and at work, I simply don't backup ESXi hosts.

At work I've got a script written that takes care of everything needed to take a freshly installed ESXi and turn it into something that will drop right into one of our production clusters. If a host ever corrupts its boot drive (get one or two a year using cheap consumer-grade micro-sd cards for boot drives) I just do a fresh ESXi install, run my script, and the host is back in the cluster ready for VMs to vmotion back onto it. Besides the micro-sd for boot there is no storage in our hosts - all VMs sit out on the fiberchannel SAN.

At home I can deal with more downtime, so its mostly the same procedure, except that my ESXi has a small internal USB drive for boot (that will be formatted/re-installed, and manual-configured as I don't have a script for home), and a single internal SATA drive formatted as VMFS for the VMs. If things really went bad I can always just move that drive to a new system to bring up my VMs.

IMHO ESXi hosts are just stateless compute resources (a tiny amount of config data that is easy to recreate) and not worth the time or storage resources to take backups of.
 
  • Like
Reactions: levifig and Patrick

EluRex

Active Member
Apr 28, 2015
218
78
28
Los Angeles, CA
Its the SDN now part of the VM Host that needs back up. In addition, some of the VM like pfsense requires same MAC for virtual adapter in order for the firewall rule to be applied to certain interface.
 

cheezehead

Active Member
Sep 23, 2012
723
175
43
Midwest, US
At home backing up my esxi host config via powercli....once every quarter, the machine boots from a thumb drive so it's pretty stateless.

@work using VDS and host profiles....pretty easy to rebuild.
 

TuxDude

Well-Known Member
Sep 17, 2011
616
338
63
Its the SDN now part of the VM Host that needs back up. In addition, some of the VM like pfsense requires same MAC for virtual adapter in order for the firewall rule to be applied to certain interface.
Host networking config is taken care of either by not being done per-host (eg. VMware dvSwitch at work, config is saved in vCenter which does get backed up), or by being scripted (trivially easy to script creation of a standard vSwitch with lots of port-groups with a variety of tools).

And per-VM settings like MAC address are saved in the VM's .vmx file - VMs do get backed up, just not hosts.