Proxmox vs. ESXi...what am I losing?

[IamSpartacus]

I'm looking for those who have experience running ESXi clusters (with full either enterprise or VMUG licensing) and have switched to Proxmox to chime in and tell me what features you (or in this case I) would lose by switching.

Features I use a lot:

• HA
• Live Migration (vMotion)
• Distributed Switching (not a deal breaker but a "would like to have")
• Hardware passthrough (HBA's and USB devices available to VM's pre-OS boot).

The biggest thing I can't seem to find a good alterative to is Veeam Backup solution. I love Veeam and use it for VM Backups as well as file by file backups (Both Windows and Linux). If I can't find a good free alternative I don't think I could make the switch.

 

[K D]

Subscribing

 

[rune-san]

It's a mixed bag.

HA works, but there is no soft-fencing installed. Proxmox will in no way check to see if one host is stepping on another hosts' toes. You have to use a hardware fencing device. They have methods for APC Switching PDUs, Dell iDRACs, SNMP from a switch or IPMI. It takes testing and a bit of trial an error but must be solid.

Live Migration works, but ironically if you choose to use HA above, Live Migration does not currently work (pressing Live Migrate will simply perform a Relocate, and stop the machine, move it, and start it again).

Distributed Switching is a no go. There's no such thing in the Proxmox space. Every Change you make to one host you'll have to make to all of them.

I'd say hardware passthrough on Linux is more solid than it is on ESXi. You should not have any problems with that as long as you have proper hardware.

There are no "slick" backup / restore tools that I'm aware of that work with Proxmox. Proxmox has vzdump built in for backups, and it isn't all that difficult if you have simple schedules or processes to make automated scripts to take care of your environment. An individual makes a a small set of apps and patches that enable Differential backups, but it does add some contingencies. That's available here: Proxmox VE differential backups | ayufan's site

I'm a major fan of Proxmox, but I do think it's important to member it's price point place in the market. It's a free product with the highest tier support costing about $800/year/socket. vSphere without the vMotion goodies is going to cost you over $500, and you'll need to re-up Service and Support for around $70 a year, + $300 per Incident. As soon as you go up to vMotion, HA, and other goodies, you're at $1,300 per socket + Support + the nearly $5,000 cost for vCenter Server and it's Service and Support. It's simply a whole different class of product.

 

[PigLover]

One small correction: with recent releases there is soft-fencing for HA. Not as mature as ESXi and, frankly, i don't use it so I can't offer experience feedback. But it is there.

You also get really good support for distributed storage using Ceph. Getting better with each release and this appears to be an area of focus for the Proxmox team.

My biggest current complaint with Proxmox is the lack of any kind of networking "fabric" view for the cluster. Networking is, solid, but from a feature perspective it is, at best, rudimentary.

 

[rune-san]

Good point PigLover. I checked through and I do see that there is a now compatibility for softdog in Linux. Like you, I don't really know anything about it's use here.

Ceph is also a great way for Proxmox to get into the HyperConverged use cases. Again, is it as good as vSAN, Storage Spaces Direct, Nutanix, or other high end enterprise HyperConverged solutions? Not even close. But for the price point, it's hard to beat.

 

[Chris Web]

I'm looking for those who have experience running ESXi clusters (with full either enterprise or VMUG licensing) and have switched to Proxmox to chime in and tell me what features you (or in this case I) would lose by switching.

Features I use a lot:

• HA
• Live Migration (vMotion)
• Distributed Switching (not a deal breaker but a "would like to have")
• Hardware passthrough (HBA's and USB devices available to VM's pre-OS boot).

The biggest thing I can't seem to find a good alterative to is Veeam Backup solution. I love Veeam and use it for VM Backups as well as file by file backups (Both Windows and Linux). If I can't find a good free alternative I don't think I could make the switch.

I would try xenserver as it is great for HA and live migration.

Sent from my HTC 10 using Tapatalk

 

[PigLover]

I would try xenserver as it is great for HA and live migration.

Sent from my HTC 10 using Tapatalk

It would be interesting to take a look at Xen again. A few years ago it had enough "issues" that it really wasn't practical for much of anything, but there has been a lot of maturing (64 bit driver support, better management tools, etc.). To be fair, a few years ago Promox was barely better.

Its probably worth a new look.

 

[Chris Web]

It would be interesting to take a look at Xen again. A few years ago it had enough "issues" that it really wasn't practical for much of anything, but there has been a lot of maturing (64 bit driver support, better management tools, etc.). To be fair, a few years ago Promox was barely better.

Its probably worth a new look.

Some people aren't fans, and if you are running obscure OSes it might not be for you. But if you are using top Linux distributions and Windows, then it's fine.

If you want a great HA solution for cheap then it's really the only one. I had 20 vms across 3 severs for several years running Xen and HA worked great and we rarely had any problems. We used SAN iscsi storage for it, which was sometimes finicky but for the most part was great.

Sent from my HTC 10 using Tapatalk

 

[Patrick]

@ PigLover - I agree on the Proxmox networking. That is the hardest part if you want to use things like OVS.

One area I do like Proxmox for is hardware support. The base Debian OS is fairly easy to work with. I use "a lot" of Ubuntu so switching between Proxmox and Ubuntu admin is very easy.

 

[cheezehead]

...vSphere without the vMotion goodies is going to cost you over $500, and you'll need to re-up Service and Support for around $70 a year, + $300 per Incident. As soon as you go up to vMotion, HA, and other goodies, you're at $1,300 per socket + Support + the nearly $5,000 cost for vCenter Server and it's Service and Support. It's simply a whole different class of product.

Depends on your contract and how the original purchased was handled. VMware has kits available (packaged esxi + vcenter licensing) and annual subscriptions available.

 

[PigLover]

Depends on your contract and how the original purchased was handled. VMware has kits available (packaged esxi + vcenter licensing) and annual subscriptions available.

I think the words "contract" and "purchase" sum up most peoples biggest issue with VMware

 

[Marsh]

I don't drink ,
But after vmware neuter the C client , then used their ESXi 4.x and 5.x flash client.
I started drinking.

 

[_alex]

A big thing i miss since i switched from xenserver to proxmox is the ability to detach disks and attach them to other vm in the GUI. No idea if this is possible in vmware, but can't understand why such a simple thing has not been done yet in proxmox.

what i absolutely like is the freedom that it's based on debian, currently playing with quagga a bit, have scst running on some hosts etc.

 

[cheezehead]

A big thing i miss since i switched from xenserver to proxmox is the ability to detach disks and attach them to other vm in the GUI. No idea if this is possible in vmware, but can't understand why such a simple thing has not been done yet in proxmox.

what i absolutely like is the freedom that it's based on debian, currently playing with quagga a bit, have scst running on some hosts etc.

We do that with VMware during system migrations or sometimes for restores.

 

[_alex]

this is how i used it, too. pretty handy.
still possible with proxmox on shell, but pita compared to how it is done in a GUI.

also, attaching a volume to multiple vm (with cluster-aware filesystem on it) would be a nice to have.

 

[rune-san]

Depends on your contract and how the original purchased was handled. VMware has kits available (packaged esxi + vcenter licensing) and annual subscriptions available.

I definitely agree, but that's a whole different can of worms and there isn't much of that that will change the figures for your average home lab user. Products like these are built with a *lot* of margin in them, and we definitely don't spend anywhere near list cost, and neither do most of our clients.

For people who can stay under 6 sockets, there's still a lot of cheap VMware + Backup software out there, but Essentials Plus is still $5,000 for Production Support, and you'll pay another $1,200 a year in Support Cost renewals.

That's where Proxmox starts to fall apart, because you would be getting close to $5,000 a year in Support Costs for those same hosts with 1 Business Day response time vs. 4 hours for VMware.

If you want to split it down the middle, as others have said, Citrix Xenserver is definitely worth the look. You get a lot of the slick features of VMware, such as Live Migration with HA, DRS (called WLB in Citrix land), and EVC (called HRP in Citrix land), as well as a Distributed Switch Controller, which is their proprietary switch management on top of OVS that lets you treat your OVS switching on your hosts as one fabric, just like DVS on VMware. Under 6 sockets, it's kind of a wash with VMware, but above 6, it starts to become substantially cheaper, especially since you do not have vCenter Maintenance Costs.

For home users, nothing will beat the $200 / year VMUG Subscription. It's really too bad Proxmox / Citrix don't have a similar thing, but then again, they both let you do far more on their free tier than VMware will with it's 1 Host, no vCenter setup.

 

[cheezehead]

For home users, nothing will beat the $200 / year VMUG Subscription. It's really too bad Proxmox / Citrix don't have a similar thing, but then again, they both let you do far more on their free tier than VMware will with it's 1 Host, no vCenter setup.

VMUG Advantage also saves 20% on training costs which some employers will pay for.

 

[IamSpartacus]

I appreciate everyone chiming in on this. It's been very informative.

From what I gather, my $200 a year (or $180 with the discount I usually apply) for VMUG Advantage is well worth it. I was more curious about Proxmox and other free offers more than I was serious about wanting to switch but if anything this discussion has crystallized my viewpoint that VMware with VMUG licensing is still the best option for me on my home network.

 

[Evan]

Hyper-V free ?
Yes I know that's off the original topic but the last posts about home lab options I think it's worth a mention, of course depends how you like windows or not

 

[pricklypunter]

I don't know why M$ hasn't done spectacular things with Hyper-V before now, it's almost an afterthought compared to what else is out there being developed. Unless they really pull out the stops and do something with it, they are always going to be bottom of the options list in my opinion