ESXI Questions

[T_Minus]

Any benefit to using a SSD vs Flash drive for ESXI install? (Other than potential reliability)
- IE: Using the SSD for logs?
-- Any performance hit doing this?
-- Any other usage for SSD ESXI host drive?

You can snag Intel 320s for $15-20 each new for 40gb, and I'd feel more comfortable with that reliable controller, and NAND vs a flash drive... but am curious if any other benefits, or if it's worth looking at an Intel 520 or Samsung 830 for it, all of which are much more $ than $15-20 but if i can double-duty it seems like it may be worth it?

 

[TuxDude]

For ESXi itself there will be very little benefit in using an SSD over a small thumb-drive or SD-card. Installing ESXi updates or additional drivers/VIB files will be faster, and booting will be faster, but during normal operation ESXi itself does no disk IO, except for logging. Which brings me to the two things that real local storage gives you (SAS/SATA disk VS tiny USB/SD boot media), local logging gets enabled, and you get to use most of the capacity of the device as a local VMFS datastore. The logging is of questionable value (I direct all my ESXi and other logging to a centralized syslog server), but the local datastore can be the perfect solution for some things. For example, if building an All-in-One server and passing through lots of drives (or an entire controller) to a NAS VM, you still need to have a VMDK file for the NAS's OS drive somewhere and that local SSD datastore is the place to put it. You could even have a second VMDK on that datastore attached to the NAS VM, and use that additional space for a log/cache device.

Also - where are you seeing those small SSD's for that cheap? I would totally pick up a pair of them to run a mirrored boot drive for my home NAS box.

 

[T_Minus]

Thanks, that answers some and makes me have more

My VMs are going on RAID10 S3700 200gb drive.

So, for logging it sounds like the SSD for $20 is a deal if I go that route, and avoid Flash storage all together.

I wish the SM mini SSD chips were actually affordable, that would be ideal!

 

[PigLover]

Also - where are you seeing those small SSD's for that cheap?

eBay - where else?

Lots of small SSDs ones listed for ~$30 obo and selling on best-offer for much less than ask. They are older (intel 320s, etc). used, YMMV, but at that price you just check them with HD sentinel, return the dogs and keep the good ones. They are perfect for boot-only drives for your hypervisor host, whether it is ESXi, Hyper-V Server, KVM or even Xen.

 

[TuxDude]

Why not toss a syslog VM onto onto the S3700-array and point your ESXi logs at that (as well as any other systems that will put out logs in syslog format), optionally put Splunk or some other log viewer/analyzer on that VM. Besides saving the $ on an SSD that ends up being used for nothing but logging it also simplifies the configuration - especially if your hardware has an internal plug to hide a thumb-drive inside the chassis. Your BIOS/UEFI boot selection becomes simpler, and you free up a SATA controller that can then be used for other things. If the thumb-drive fails theres no valuable data on it anyways, just a ESXi install and a very small amount of config data - you can have a new thumb drive in and ESXi re-installed and re-configured in under an hour if you have to, but the chances of needing it are very slim since there are no writes to the thumb-drive except during install/updates.

 

[T_Minus]

Why not toss a syslog VM onto onto the S3700-array and point your ESXi logs at that (as well as any other systems that will put out logs in syslog format), optionally put Splunk or some other log viewer/analyzer on that VM. Besides saving the $ on an SSD that ends up being used for nothing but logging it also simplifies the configuration - especially if your hardware has an internal plug to hide a thumb-drive inside the chassis. Your BIOS/UEFI boot selection becomes simpler, and you free up a SATA controller that can then be used for other things. If the thumb-drive fails theres no valuable data on it anyways, just a ESXi install and a very small amount of config data - you can have a new thumb drive in and ESXi re-installed and re-configured in under an hour if you have to, but the chances of needing it are very slim since there are no writes to the thumb-drive except during install/updates.

Because I wasn't aware of Splunk, or a "syslog vm" as an option. This I will research more, thanks.

$20 for 40gb near new Intel SSD isn't bad and is ~same for 16-32gb quality flash drives but less quality, and QC than Intel SSD I'm sure.
I'm using 0 SATA ports, so no issue there.


Good to know the drive can fail and can be swapped easily.

 

[Entz]

Flashdrive is fine imo. I push all my logs to my vCenter VM which is acting as a Syslog Collector, works great.

If I have a local SSD in a ESXi node its as a flash cache device

 

[T_Minus]

Flashdrive is fine imo. I push all my logs to my vCenter VM which is acting as a Syslog Collector, works great.

If I have a local SSD in a ESXi node its as a flash cache device

Ah, that's another good idea too.

 

[TuxDude]

Because I wasn't aware of Splunk, or a "syslog vm" as an option. This I will research more, thanks.

Splunk is pretty cool, and free as long as you stay under 500GB per day of log indexing volume. Just be careful that with ESXi's install defaults (at least up to 5.5, haven't verified with 6 yet) that logs default to 'debug' level, you will be sending a LOT of syslog data. There are two options to set under Advanced Settings on every host to control the level of logging, which I change to 'warning' on all my hosts. Vpx.Vpxa.config.log.level controls the logging output from the vCenter agent, and Config.HostAgent.log.level controls the output from pretty much everything else.

 

[T_Minus]

Thanks again Tux, that is very useful information that applies to my situation!!!

 

[Entz]

That is a very good point @TuxDude, the default setting logs a huge amount of useless (to us ) data. The default VMWare syslog collector uses 8 rotated logs of 16MB each (per host) and on verbose that can be eaten in a day or two (or less depending on your VMs).

 

[Chuckleb]

Graylog2 is free and an alternative to splunk. I had started a writeup once... Easy to install inside of a docker setup.

 

[T_Minus]

Graylog2 is free and an alternative to splunk. I had started a writeup once... Easy to install inside of a docker setup.

I'll check that out too.

 

[chinesestunna]

I've been running ESXi on a small 2GB flash drive for past 4 years and no issues. My usage is pretty light so probably helps to lessen the stress as well. One thing I will say I like for the USB approach: when I need to apply ESXi update/patch I just power down host, take flash drive to my Linux machine and dd an .IMG file. This way if the patching/update goes sideways I can easily restore back you can do same with SSD but more cables to deal with and with larger drive means more time/space for dd IMG.

 

[mrkrad]

Yeah I use SLC (HP Branded) USB 2 flash drives 2gb for ESXi. Solid - great way to avoid raid controller dependency!

 

[chinesestunna]

Yeah I use SLC (HP Branded) USB 2 flash drives 2gb for ESXi. Solid - great way to avoid raid controller dependency!

Mines a cheap freebie no name drive also been good
Another benefit, you can swap ESXi installs easily by swapping drives. Want to test esxi6? Just load up another USB drive and install, when done replace old drive and back to 5.5 or whatever production version you're running

 

[NeverDie]

FreeNAS 9.3 let's me boot from a mirrored USB flash drive. Can ESXi be configured to do that? With FreeNAS I'm doing it because I don't know what reliability to expect if booting from a non-mirrored flash drive, and because 9.3 makes the mirrored setup trivially easy.

 

[chinesestunna]

FreeNAS 9.3 let's me boot from a mirrored USB flash drive. Can ESXi be configured to do that? With FreeNAS I'm doing it because I don't know what reliability to expect if booting from a non-mirrored flash drive, and because 9.3 makes the mirrored setup trivially easy.

How do you mirror a USB drive? I'm guessing its not a hardware solution?

edit; meant to say hardware solution

 

[PigLover]

How do you mirror a USB drive? I'm guessing its not a software solution?

FreeNAS is ZFS based and supports booting from a ZFS pool. It just formats the filesystem on the USB key as a 2-disk ZFS mirror file system. Same ideas as the "boot from ZFZ pool" feature highlighted in Patrick's recent front-pager on Proxmox.

 

[NeverDie]

How do you mirror a USB drive? I'm guessing its not a software solution?

With 9.3 it's as easy as this: insert two usb keys (I'm using two different brands to reduce even further the chance of a simultaneous failure) and then, during setup, just select both of them (as opposed to just one or the other if you were doing a non-mirrored setup). That is all there is to it. It couldn't be simpler.