ESXi Boot Device - Rust, SSD, SATA DOM?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

RobertFontaine

Active Member
Dec 17, 2015
663
148
43
57
Winterpeg, Canuckistan
I'm trying to decide what makes sense as a boot device for my esxi dungeon lab.

Spinny Disk, SSD, SATA DOM, boot from network.

It appears that 8GB is a good size and 16GB would be more than enough.

ESXi 6.0 has these storage requirements:
  • 1 Gigabyte+ boot device: Installing or upgrading to ESXi 6.0 requires a minimum of a 1 GB boot device.

    Note: Although a 1 GB USB or SD device suffices for a minimal installation, you should use a 4 GB or larger device. The extra space is used for an expanded coredump partition on the USB/SD device.

  • 4 GB extra for scratch partition: When booting from a local disk, a SAN or an iSCSI LUN, a 5.2 GB disk is required to allow for the creation of the VMFS volume and a 4 GB scratch partition on the boot device.

It sounds like there is very little use of the boot device once ESXi spins up other than to store log files.

Is this correct?

Once ESXi is booted is there any effective difference in what kind of storage you boot from across the range.
It seems like any relatively reliable storage will provide the same performance once ESXi is up and running.

Is this correct?

In a lab environment is there any need at all to be concerned about write cycles? (I suspect the answer is no but am not sure).

Thanks,
Robert
 

grogthegreat

New Member
Apr 21, 2016
23
7
3
36
I've always booted ESXi from USB. If you have an internal USB port I'd suggest a quality 8GB usb stick. ESXi performance isn't related to the boot drive once it is booted so anything works. spinny rust isn't worth the space or power usage. SSD or satadom is a bit of a waste of that performance you won't be benefiting from. Others may disagree but I don't like the idea of network booting a host since it is adding complexity, cost, and a new failure point for no gain.
When choosing a usb stick find one that is a decent quality. If it is internal, you don't want it to be killed by heat. If it is external, find a shorter one so that it doesn't get hit and bent. No VM data gets stored on the boot drive so if it fails, just put in a replacement, re-install ESXi, and import your VMs. The process is fast and easy enough that I don't bother trying to back up the boot drive.
 

BlueLineSwinger

Active Member
Mar 11, 2013
176
66
28
+1 on USB. HDD/SSD/DOM costs more and gain you nothing (except maybe boot speed, but if you're rebooting the host so often this is a factor then you have other problems). PXE for one or two hosts is needlessly complex.

I use one of those tiny SanDisk Fit 8 GB USB drives plugged into the rear I/O panel. Works fine and you can barely tell it's there.
 

whitey

Moderator
Jun 30, 2014
2,766
868
113
41
Others may disagree but I don't like the idea of network booting a host since it is adding complexity, cost, and a new failure point for no gain.
Yep good sir, I'm gonna have to respectfully disagree w/ this comment. Vmware vSphere AutoDeploy is a game changer, Here's a list of benefits:
  • Reduced Storage Costs
    Because Auto Deploy installs directly into the host's memory there is no need to dedicated a boot disk for each server. This not only saves money when purchasing new hardware and storage, but when booting from SAN it helps to simplify the storage architecture by eliminating the need to map dedicated LUNs to each ESXi host.

    Fast Server Provisioning
    Deploying a new ESXi host is as simple as enabling PXE boot and powering on a new server. The Auto Deploy server will identify the new server, assign an appropriate ESXi Image Profile and Host Profile, and place the server into the proper vCenter folder or cluster.

    Eliminate configuration drift
    By sharing a standard ESXi image profile across multiple hosts you can ensure that all your ESXi hosts are running the same ESXi version. In addition, each time a host is rebooted it is like performing a fresh install of ESXi.
Now that being said for a lab with 3-4 hosts probably not worth it complexity-wise (bang for buck/juice v.s. squeeze), with 30-40 hosts...PRICELESS!
 
  • Like
Reactions: leebo_28 and Boddy

fractal

Active Member
Jun 7, 2016
309
69
28
33
I use the sancruzer fit USB sticks to boot NAS4Free and pfSense. I can not recommend them more highly.

But, I boot esx from small, 60-80 GB SSD's that I buy on sale. I was thinking of using them for swap space or something but mostly they sit around empty.

I suppose I should stuff a cruzer fit in them the next time I upgrade and save the SSD / sata port but something in me suspects the SSD is a tiny bit more reliable. That and I have the SSD and don't put much storage in my esx boxes so don't need the sata port...
 

AveryFreeman

consummate homelabber
Mar 17, 2017
413
54
28
42
Near Seattle
averyfreeman.com
Sorry to revive the necro thread, but I just wanted to mention to anyone who might stumble on this like me that SanDisk Cruzer fit disks can get super, super hot.

I have a 64gb fit I pulled out one time after negligible use and it was hot enough that I literally threw the drive. I mean it seriously was too hot to hold.

It still works, but after subsequent uses I've noticed this is a regular thing. It might have a proportional relationship to capacity, as I have a few 16gbs that don't get nearly as hot (i.e. I can still touch them when I take them out). But they still get hot AF.

The heat factor I think makes the Cruzer fit, especially larger capacities, a bad option for longevity. A normal sized drive is a lot less likely to undergo frying pan temperatures.

I use the 16GB fits for my ESXi hosts but I always keep a few backups ready to go because I expect they'll die any second.

I may try the PXE method, it sounds really cool. First I've heard of it.
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,669
1,081
113
artofserver.com
@AveryFreeman if you have heat issues with USB flash drives, what I found is that if you put them in a USB 2.0 slot, they don't generate as much heat. I know it is slower, but for boot drive, I think reliability (assuming you believe heat correlates with decreasing reliability) is higher priority than losing a few seconds during boot up.
 

cactus

Moderator
Jan 25, 2011
830
75
28
CA
The USB 3.0 ones do get super hot. I have a 64GB USB 3.0 one that I use to move/backup stuff and I have let it cool on my desk before moving it. I never noticed the older, USB 2.0 getting too hot to touch.
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,669
1,081
113
artofserver.com
The USB 3.0 ones do get super hot. I have a 64GB USB 3.0 one that I use to move/backup stuff and I have let it cool on my desk before moving it. I never noticed the older, USB 2.0 getting too hot to touch.
yeah, the idea is to use USB 3.0 flash drive in a USB 2 slot, which would limit the amount of current available to the flash drive, hence inhibiting heat generation.
 
  • Like
Reactions: cactus

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
For the $ S3500 80gb :)
If $ no object or you find a deal SATADOM are hard to beat for ease of use ;)
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,669
1,081
113
artofserver.com
For the $ S3500 80gb :)
If $ no object or you find a deal SATADOM are hard to beat for ease of use ;)
i'm not a fan of SATA-DOM... when it fails, it's not easy to just go out to the store and get a replacement. USB drives are easy to find in a pinch. regular SSDs perform better than SATADOM, so I'd rather a USB drive or a decent SSD for boot drive. just my opinion though... i suppose if you stock plenty of spares, it works out okay.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
i'm not a fan of SATA-DOM... when it fails, it's not easy to just go out to the store and get a replacement. USB drives are easy to find in a pinch. regular SSDs perform better than SATADOM, so I'd rather a USB drive or a decent SSD for boot drive. just my opinion though... i suppose if you stock plenty of spares, it works out okay.
Sure it is, it just depends where you live... not easy for me, but I know many people who can drive on over and grab one.

For me if I lived by the "I can't drive and get a replacement I won't use it" I'd not own anything since I order ALL online parts.

It's also a bit funny to complain about satadom performacne then suggest a USB drive...satadom 64gb are pretty much s3500 80gb performance too iirc, def doesn't matter either way for esxi being on RAM once booted.

Sorry to counter your points, but if you're that concerned with replacement... have one on hand already. For my ESXI host I run 2x S3500, the 2nd is empty / has a backup of my storage VM and my config should the SSD die. But, you can't fit a ssd in all hosts so satadom is winner for versatility, performance and longevity.

Plan ahead so you don't have to rush to a store... a S3500 is $35 or less, and satadom aren't that much more.
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,669
1,081
113
artofserver.com
Sure it is, it just depends where you live... not easy for me, but I know many people who can drive on over and grab one.

For me if I lived by the "I can't drive and get a replacement I won't use it" I'd not own anything since I order ALL online parts.

It's also a bit funny to complain about satadom performacne then suggest a USB drive...satadom 64gb are pretty much s3500 80gb performance too iirc, def doesn't matter either way for esxi being on RAM once booted.

Sorry to counter your points, but if you're that concerned with replacement... have one on hand already. For my ESXI host I run 2x S3500, the 2nd is empty / has a backup of my storage VM and my config should the SSD die. But, you can't fit a ssd in all hosts so satadom is winner for versatility, performance and longevity.

Plan ahead so you don't have to rush to a store... a S3500 is $35 or less, and satadom aren't that much more.
no apology needed to counter my points; we've all developed different opinions based on our own individual experiences. and I've learned to live in a world where not everyone agrees with me :).

i'm not sure what brick and mortar retail outlets you know of, or know of other people who live near by one, that sells SATA DOM? I would like to know actually... even if one isn't near me, it's nice to know what kind of shops these days stock stuff like that. for me, i would normally have to order online; and in a hurry, if i'm lucky, i might find something that is same day delivery on Amazon.

I don't live by "if I can't drive and get a replacement, I won't use it"... let's not exaggerate what I said here :p. my opinion was developed because I often help out small businesses or friends with their IT stuff and when a device goes down (like internet firewall/router) because an SSD, USB flash drive, or SATA-DOM fails, for me, getting a replacement SSD or USB flash drive has been a lot easier, usually something I can find within an hour. So, in situations like that, and in the conditions I'm near (I don't know of a B&M retail outlet near me that carry SATA_DOM), I've formed the opinion I've stated. It doesn't mean I would never use one, and having spares is certainly a way to mitigate the problem, but if the choice was between USB Flash, SSD, and SATA-DOM, for *me*, USB flash is the easiest to get replaced, followed by SSD. In situations where it's some lab equipment and if something breaks and it isn't something to sweat about, I wouldn't hesitate to use SATA-DOM, at least not based on this "easy replacement" criteria as it's not relevant. I shared my opinion just in case someone else may be in a similar situation where they can't drive to a store to buy one, and plan to use it in situation where not being able to replace it quickly/easily might cause a real inconvenience. And even if one decides to get SATA-DOM anyway, consider having spares on-hand (i.e., your "plan ahead" remark). That's all it was... not a general statement that one shouldn't use something they can't replace at a local B&M store.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
SuperBiiz - Order online, can pickup at location. There are others in Sacramento and the bay area that I can't think of their names... most are more industrial feeling, like you're picking their inevntory off a shelf or a walk-in warehouse kind of thing, at-least i my limited experience visiting them myself that's what it was like :)

There's def. places in NY too like B&H and others I once again can't recall the name as it's not someplace I live or go to lol.
 

AveryFreeman

consummate homelabber
Mar 17, 2017
413
54
28
42
Near Seattle
averyfreeman.com
@AveryFreeman if you have heat issues with USB flash drives, what I found is that if you put them in a USB 2.0 slot, they don't generate as much heat. I know it is slower, but for boot drive, I think reliability (assuming you believe heat correlates with decreasing reliability) is higher priority than losing a few seconds during boot up.
Well yeah, if I'm going to do that I should just use a USB 2.0 drive to begin with - why waste the $ on a 3.0 drive I'm not going to use to full capability?

16GB 2.0 flash drives can be had for <$5.00 ea in lots on Amazon and eBay. I usually buy them ten at a time and make labels for their contents with a label maker as I go, then I don't have to use Rufus nearly as much (!) :)
 

AveryFreeman

consummate homelabber
Mar 17, 2017
413
54
28
42
Near Seattle
averyfreeman.com
i'm not a fan of SATA-DOM... when it fails, it's not easy to just go out to the store and get a replacement. USB drives are easy to find in a pinch. regular SSDs perform better than SATADOM, so I'd rather a USB drive or a decent SSD for boot drive. just my opinion though... i suppose if you stock plenty of spares, it works out okay.
Intel 24GB SSD mSATA SSDMAEXC024G3H Solid State Drive MSATA | eBay

Intel 313 24GB SLC MSATA drives. I am using two of these in ZRAID mirror for a homemade j1800 pfSense appliance and one to boot ESXi on one of my hosts. They're made to be slammed as cache drives for ReadyBoost (remember that?), not sure what the MTBF is but I can only imagine for SLC.
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,669
1,081
113
artofserver.com
Well yeah, if I'm going to do that I should just use a USB 2.0 drive to begin with - why waste the $ on a 3.0 drive I'm not going to use to full capability?

16GB 2.0 flash drives can be had for <$5.00 ea in lots on Amazon and eBay. I usually buy them ten at a time and make labels for their contents with a label maker as I go, then I don't have to use Rufus nearly as much (!) :)
Well, i use to think that too... but then, I noticed that USB 3.0 flash drives perform better than USB 2.0 drives in a USB 2.0 slot. Of course, not USB 3.0 speeds, but full USB 2.0 speeds. Where as, USB 2.0 drives often won't even max out USB 2.0. Could just be the selection of USB flash drives I had...

And on price, I often find the 16GB 3.0 SanDisk flash drives for about $4.99 or so when on sale. I just get a bunch of those. For that price, it didn't feel worth getting USB 2.0 drives anymore. What kind of labeler do you use? I find it hard to make labels as USB flash drives have gotten smaller and smaller...
 

LaMerk

Member
Jun 13, 2017
38
7
8
33
+1 for USB flash drives - very simple and cheap solution for the lab. For the enterprise solution, I would take SSDs in RAID1.
 

hhp

New Member
Aug 3, 2016
20
3
3
55
Late to the party but I also boot my esxi blades from USB, just be sure to point your hypervisor system swap etc. to a SAN or other robust storage solution.