ESXi Boot Device - Rust, SSD, SATA DOM?

Discussion in 'VMware, VirtualBox, Citrix' started by RobertFontaine, Sep 24, 2016.

  1. RobertFontaine

    RobertFontaine Active Member

    Joined:
    Dec 17, 2015
    Messages:
    666
    Likes Received:
    148
    I'm trying to decide what makes sense as a boot device for my esxi dungeon lab.

    Spinny Disk, SSD, SATA DOM, boot from network.

    It appears that 8GB is a good size and 16GB would be more than enough.

    ESXi 6.0 has these storage requirements:
    • 1 Gigabyte+ boot device: Installing or upgrading to ESXi 6.0 requires a minimum of a 1 GB boot device.

      Note: Although a 1 GB USB or SD device suffices for a minimal installation, you should use a 4 GB or larger device. The extra space is used for an expanded coredump partition on the USB/SD device.

    • 4 GB extra for scratch partition: When booting from a local disk, a SAN or an iSCSI LUN, a 5.2 GB disk is required to allow for the creation of the VMFS volume and a 4 GB scratch partition on the boot device.

    It sounds like there is very little use of the boot device once ESXi spins up other than to store log files.

    Is this correct?

    Once ESXi is booted is there any effective difference in what kind of storage you boot from across the range.
    It seems like any relatively reliable storage will provide the same performance once ESXi is up and running.

    Is this correct?

    In a lab environment is there any need at all to be concerned about write cycles? (I suspect the answer is no but am not sure).

    Thanks,
    Robert
     
    #1
  2. grogthegreat

    grogthegreat New Member

    Joined:
    Apr 21, 2016
    Messages:
    23
    Likes Received:
    7
    I've always booted ESXi from USB. If you have an internal USB port I'd suggest a quality 8GB usb stick. ESXi performance isn't related to the boot drive once it is booted so anything works. spinny rust isn't worth the space or power usage. SSD or satadom is a bit of a waste of that performance you won't be benefiting from. Others may disagree but I don't like the idea of network booting a host since it is adding complexity, cost, and a new failure point for no gain.
    When choosing a usb stick find one that is a decent quality. If it is internal, you don't want it to be killed by heat. If it is external, find a shorter one so that it doesn't get hit and bent. No VM data gets stored on the boot drive so if it fails, just put in a replacement, re-install ESXi, and import your VMs. The process is fast and easy enough that I don't bother trying to back up the boot drive.
     
    #2
  3. BlueLineSwinger

    BlueLineSwinger Active Member

    Joined:
    Mar 11, 2013
    Messages:
    117
    Likes Received:
    43
    +1 on USB. HDD/SSD/DOM costs more and gain you nothing (except maybe boot speed, but if you're rebooting the host so often this is a factor then you have other problems). PXE for one or two hosts is needlessly complex.

    I use one of those tiny SanDisk Fit 8 GB USB drives plugged into the rear I/O panel. Works fine and you can barely tell it's there.
     
    #3
  4. dwright1542

    dwright1542 Active Member

    Joined:
    Dec 26, 2015
    Messages:
    324
    Likes Received:
    67
    #4
    Karln83 likes this.
  5. whitey

    whitey Moderator

    Joined:
    Jun 30, 2014
    Messages:
    2,734
    Likes Received:
    846
    Yep good sir, I'm gonna have to respectfully disagree w/ this comment. Vmware vSphere AutoDeploy is a game changer, Here's a list of benefits:
    • Reduced Storage Costs
      Because Auto Deploy installs directly into the host's memory there is no need to dedicated a boot disk for each server. This not only saves money when purchasing new hardware and storage, but when booting from SAN it helps to simplify the storage architecture by eliminating the need to map dedicated LUNs to each ESXi host.

      Fast Server Provisioning
      Deploying a new ESXi host is as simple as enabling PXE boot and powering on a new server. The Auto Deploy server will identify the new server, assign an appropriate ESXi Image Profile and Host Profile, and place the server into the proper vCenter folder or cluster.

      Eliminate configuration drift
      By sharing a standard ESXi image profile across multiple hosts you can ensure that all your ESXi hosts are running the same ESXi version. In addition, each time a host is rebooted it is like performing a fresh install of ESXi.
    Now that being said for a lab with 3-4 hosts probably not worth it complexity-wise (bang for buck/juice v.s. squeeze), with 30-40 hosts...PRICELESS!
     
    #5
    leebo_28 and Boddy like this.
  6. fractal

    fractal Active Member

    Joined:
    Jun 7, 2016
    Messages:
    309
    Likes Received:
    67
    I use the sancruzer fit USB sticks to boot NAS4Free and pfSense. I can not recommend them more highly.

    But, I boot esx from small, 60-80 GB SSD's that I buy on sale. I was thinking of using them for swap space or something but mostly they sit around empty.

    I suppose I should stuff a cruzer fit in them the next time I upgrade and save the SSD / sata port but something in me suspects the SSD is a tiny bit more reliable. That and I have the SSD and don't put much storage in my esx boxes so don't need the sata port...
     
    #6
  7. AveryFreeman

    AveryFreeman ESXi + ( BSD / ZFS ) = HAPPY

    Joined:
    Mar 17, 2017
    Messages:
    102
    Likes Received:
    9
    Sorry to revive the necro thread, but I just wanted to mention to anyone who might stumble on this like me that SanDisk Cruzer fit disks can get super, super hot.

    I have a 64gb fit I pulled out one time after negligible use and it was hot enough that I literally threw the drive. I mean it seriously was too hot to hold.

    It still works, but after subsequent uses I've noticed this is a regular thing. It might have a proportional relationship to capacity, as I have a few 16gbs that don't get nearly as hot (i.e. I can still touch them when I take them out). But they still get hot AF.

    The heat factor I think makes the Cruzer fit, especially larger capacities, a bad option for longevity. A normal sized drive is a lot less likely to undergo frying pan temperatures.

    I use the 16GB fits for my ESXi hosts but I always keep a few backups ready to go because I expect they'll die any second.

    I may try the PXE method, it sounds really cool. First I've heard of it.
     
    #7
  8. BLinux

    BLinux Well-Known Member

    Joined:
    Jul 7, 2016
    Messages:
    1,798
    Likes Received:
    486
    @AveryFreeman if you have heat issues with USB flash drives, what I found is that if you put them in a USB 2.0 slot, they don't generate as much heat. I know it is slower, but for boot drive, I think reliability (assuming you believe heat correlates with decreasing reliability) is higher priority than losing a few seconds during boot up.
     
    #8
  9. cactus

    cactus Moderator

    Joined:
    Jan 25, 2011
    Messages:
    792
    Likes Received:
    67
    The USB 3.0 ones do get super hot. I have a 64GB USB 3.0 one that I use to move/backup stuff and I have let it cool on my desk before moving it. I never noticed the older, USB 2.0 getting too hot to touch.
     
    #9
  10. BLinux

    BLinux Well-Known Member

    Joined:
    Jul 7, 2016
    Messages:
    1,798
    Likes Received:
    486
    yeah, the idea is to use USB 3.0 flash drive in a USB 2 slot, which would limit the amount of current available to the flash drive, hence inhibiting heat generation.
     
    #10
    cactus likes this.
  11. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,392
    Likes Received:
    1,304
    For the $ S3500 80gb :)
    If $ no object or you find a deal SATADOM are hard to beat for ease of use ;)
     
    #11
  12. BLinux

    BLinux Well-Known Member

    Joined:
    Jul 7, 2016
    Messages:
    1,798
    Likes Received:
    486
    i'm not a fan of SATA-DOM... when it fails, it's not easy to just go out to the store and get a replacement. USB drives are easy to find in a pinch. regular SSDs perform better than SATADOM, so I'd rather a USB drive or a decent SSD for boot drive. just my opinion though... i suppose if you stock plenty of spares, it works out okay.
     
    #12
  13. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,392
    Likes Received:
    1,304
    Sure it is, it just depends where you live... not easy for me, but I know many people who can drive on over and grab one.

    For me if I lived by the "I can't drive and get a replacement I won't use it" I'd not own anything since I order ALL online parts.

    It's also a bit funny to complain about satadom performacne then suggest a USB drive...satadom 64gb are pretty much s3500 80gb performance too iirc, def doesn't matter either way for esxi being on RAM once booted.

    Sorry to counter your points, but if you're that concerned with replacement... have one on hand already. For my ESXI host I run 2x S3500, the 2nd is empty / has a backup of my storage VM and my config should the SSD die. But, you can't fit a ssd in all hosts so satadom is winner for versatility, performance and longevity.

    Plan ahead so you don't have to rush to a store... a S3500 is $35 or less, and satadom aren't that much more.
     
    #13
  14. BLinux

    BLinux Well-Known Member

    Joined:
    Jul 7, 2016
    Messages:
    1,798
    Likes Received:
    486
    no apology needed to counter my points; we've all developed different opinions based on our own individual experiences. and I've learned to live in a world where not everyone agrees with me :).

    i'm not sure what brick and mortar retail outlets you know of, or know of other people who live near by one, that sells SATA DOM? I would like to know actually... even if one isn't near me, it's nice to know what kind of shops these days stock stuff like that. for me, i would normally have to order online; and in a hurry, if i'm lucky, i might find something that is same day delivery on Amazon.

    I don't live by "if I can't drive and get a replacement, I won't use it"... let's not exaggerate what I said here :p. my opinion was developed because I often help out small businesses or friends with their IT stuff and when a device goes down (like internet firewall/router) because an SSD, USB flash drive, or SATA-DOM fails, for me, getting a replacement SSD or USB flash drive has been a lot easier, usually something I can find within an hour. So, in situations like that, and in the conditions I'm near (I don't know of a B&M retail outlet near me that carry SATA_DOM), I've formed the opinion I've stated. It doesn't mean I would never use one, and having spares is certainly a way to mitigate the problem, but if the choice was between USB Flash, SSD, and SATA-DOM, for *me*, USB flash is the easiest to get replaced, followed by SSD. In situations where it's some lab equipment and if something breaks and it isn't something to sweat about, I wouldn't hesitate to use SATA-DOM, at least not based on this "easy replacement" criteria as it's not relevant. I shared my opinion just in case someone else may be in a similar situation where they can't drive to a store to buy one, and plan to use it in situation where not being able to replace it quickly/easily might cause a real inconvenience. And even if one decides to get SATA-DOM anyway, consider having spares on-hand (i.e., your "plan ahead" remark). That's all it was... not a general statement that one shouldn't use something they can't replace at a local B&M store.
     
    #14
  15. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,392
    Likes Received:
    1,304
    SuperBiiz - Order online, can pickup at location. There are others in Sacramento and the bay area that I can't think of their names... most are more industrial feeling, like you're picking their inevntory off a shelf or a walk-in warehouse kind of thing, at-least i my limited experience visiting them myself that's what it was like :)

    There's def. places in NY too like B&H and others I once again can't recall the name as it's not someplace I live or go to lol.
     
    #15
  16. AveryFreeman

    AveryFreeman ESXi + ( BSD / ZFS ) = HAPPY

    Joined:
    Mar 17, 2017
    Messages:
    102
    Likes Received:
    9
    Well yeah, if I'm going to do that I should just use a USB 2.0 drive to begin with - why waste the $ on a 3.0 drive I'm not going to use to full capability?

    16GB 2.0 flash drives can be had for <$5.00 ea in lots on Amazon and eBay. I usually buy them ten at a time and make labels for their contents with a label maker as I go, then I don't have to use Rufus nearly as much (!) :)
     
    #16
  17. AveryFreeman

    AveryFreeman ESXi + ( BSD / ZFS ) = HAPPY

    Joined:
    Mar 17, 2017
    Messages:
    102
    Likes Received:
    9
    Intel 24GB SSD mSATA SSDMAEXC024G3H Solid State Drive MSATA | eBay

    Intel 313 24GB SLC MSATA drives. I am using two of these in ZRAID mirror for a homemade j1800 pfSense appliance and one to boot ESXi on one of my hosts. They're made to be slammed as cache drives for ReadyBoost (remember that?), not sure what the MTBF is but I can only imagine for SLC.
     
    #17
  18. BLinux

    BLinux Well-Known Member

    Joined:
    Jul 7, 2016
    Messages:
    1,798
    Likes Received:
    486
    Well, i use to think that too... but then, I noticed that USB 3.0 flash drives perform better than USB 2.0 drives in a USB 2.0 slot. Of course, not USB 3.0 speeds, but full USB 2.0 speeds. Where as, USB 2.0 drives often won't even max out USB 2.0. Could just be the selection of USB flash drives I had...

    And on price, I often find the 16GB 3.0 SanDisk flash drives for about $4.99 or so when on sale. I just get a bunch of those. For that price, it didn't feel worth getting USB 2.0 drives anymore. What kind of labeler do you use? I find it hard to make labels as USB flash drives have gotten smaller and smaller...
     
    #18
  19. LaMerk

    LaMerk New Member

    Joined:
    Jun 13, 2017
    Messages:
    18
    Likes Received:
    4
    +1 for USB flash drives - very simple and cheap solution for the lab. For the enterprise solution, I would take SSDs in RAID1.
     
    #19
  20. hhp

    hhp New Member

    Joined:
    Aug 3, 2016
    Messages:
    18
    Likes Received:
    3
    Late to the party but I also boot my esxi blades from USB, just be sure to point your hypervisor system swap etc. to a SAN or other robust storage solution.
     
    #20
Similar Threads: ESXi Boot
Forum Title Date
VMware, VirtualBox, Citrix ESXi 6.7 - USB Flash to Boot and Datastore? May 30, 2018
VMware, VirtualBox, Citrix ESXi 6.7 can't change settings, don't survive reboot (passthrough issues, too!) May 1, 2018
VMware, VirtualBox, Citrix boot guest from passed through AHCI on ESXi 6.0 (can that work??) Dec 29, 2016
VMware, VirtualBox, Citrix Overprovisioning USB for ESXi boot Oct 24, 2016
VMware, VirtualBox, Citrix VM with passthrough "freezes" entire ESXi box when shutdown/rebooting guest Apr 11, 2016

Share This Page