I've been doing some testing, and may make a rash decision soon, so please talk me down if I'm nuts here.
I'm thinking of taking my ESXi 5.0 all-in-one host that runs Gea's napp-it on OpenIndiana (in a VM with on-board SATA passthrough to it) and making it an OpenIndiana or OmniOS machine that runs napp-it (physical-izing it if you will) with VMs running in KVM.
• Why? For one, I don't like having to manage ESXi with a Windows machine. I know most management can be done with CLI on Linux but I did some reading and it seems daunting. On the rare occasion I need to view a VM's console or desktop I have to boot up my Windows XP Virtualbox VM on my MacBook Pro, fire up vSphere, and view the console which even then doesn't always work well with text entry or mousing.
Also, the limited feature set for ESXi free version (difficult to backup VMs). And did passthrough of onboard SATA start working again yet after the 5.1 update? VMWare could kill the free edition at any time.
Also I don't feel 100% comfortable with the whole NFS datastore thing - I'd rather have the VMs right there on native ZFS so I can snapshot them, archive them, etc.
One less layer of complexity overall.
• What am I giving up? 10Gb ethernet links, which I don't need since I'm not using networked datastores anymore. 'Pretty' gui from vSphere -- I might miss that, some value in what it can show. Memory over-commitment -- at 32Gigs of RAM I'll be fine for VMs and ARC I think, no need to over-commit RAM to VMs. Probably some other features I'm not thinking of...
• What am I gaining? As mentioned, VMs stored on local ZFS pool. Super easy and great performing VNC connections to guest OSes, more work done on the host hardware as opposed to in VMs -- my household file serving needs will be handled by napp-it/OI, ability to use 'Zones' instead of KVM machines where appropriate, fewer things to update (ESXi, Virtualbox, my Windows XP VM).
------------Will it work reliably though???
I know QEMU-KVM is pretty new on OI/OmniOS but am I correct that KVM is used by many Virtual Hosting companies, so generally it's considered stable and legit? The SmartOS people said they found no bugs at all during their porting to Illumos!
I ran some Geekbench tests and guests seem to perform well on the CPU side of things. VM to VM I got almost 800 Mbps using e1000 virtual nics (iperf). I haven't done any disk speed tests yet.
I'll probably only be running a VM doing Owncloud (for professional file sharing needs), a CentOS machine doing email (atmail I think), pfsense router, Vortexbox (media streaming/transcoding for home use), and whatever else I want to play with for a while.
And to answer the question, why not SmartOS? Well, because no napp-it -- and I have been enjoying my napp-it replication system which will continue to function with an OmniOS or OI-based system.
I'm thinking of taking my ESXi 5.0 all-in-one host that runs Gea's napp-it on OpenIndiana (in a VM with on-board SATA passthrough to it) and making it an OpenIndiana or OmniOS machine that runs napp-it (physical-izing it if you will) with VMs running in KVM.
• Why? For one, I don't like having to manage ESXi with a Windows machine. I know most management can be done with CLI on Linux but I did some reading and it seems daunting. On the rare occasion I need to view a VM's console or desktop I have to boot up my Windows XP Virtualbox VM on my MacBook Pro, fire up vSphere, and view the console which even then doesn't always work well with text entry or mousing.
Also, the limited feature set for ESXi free version (difficult to backup VMs). And did passthrough of onboard SATA start working again yet after the 5.1 update? VMWare could kill the free edition at any time.
Also I don't feel 100% comfortable with the whole NFS datastore thing - I'd rather have the VMs right there on native ZFS so I can snapshot them, archive them, etc.
One less layer of complexity overall.
• What am I giving up? 10Gb ethernet links, which I don't need since I'm not using networked datastores anymore. 'Pretty' gui from vSphere -- I might miss that, some value in what it can show. Memory over-commitment -- at 32Gigs of RAM I'll be fine for VMs and ARC I think, no need to over-commit RAM to VMs. Probably some other features I'm not thinking of...
• What am I gaining? As mentioned, VMs stored on local ZFS pool. Super easy and great performing VNC connections to guest OSes, more work done on the host hardware as opposed to in VMs -- my household file serving needs will be handled by napp-it/OI, ability to use 'Zones' instead of KVM machines where appropriate, fewer things to update (ESXi, Virtualbox, my Windows XP VM).
------------Will it work reliably though???
I know QEMU-KVM is pretty new on OI/OmniOS but am I correct that KVM is used by many Virtual Hosting companies, so generally it's considered stable and legit? The SmartOS people said they found no bugs at all during their porting to Illumos!
I ran some Geekbench tests and guests seem to perform well on the CPU side of things. VM to VM I got almost 800 Mbps using e1000 virtual nics (iperf). I haven't done any disk speed tests yet.
I'll probably only be running a VM doing Owncloud (for professional file sharing needs), a CentOS machine doing email (atmail I think), pfsense router, Vortexbox (media streaming/transcoding for home use), and whatever else I want to play with for a while.
And to answer the question, why not SmartOS? Well, because no napp-it -- and I have been enjoying my napp-it replication system which will continue to function with an OmniOS or OI-based system.