First build: ESXi 6 all-in-one (Napp-it, pfsense, plex)

[jimmytherat]

Hi,
I'm working towards my first server build (wife and budget permitting) and I'm hoping to gain some community insight into the setup that I'm proposing.

Disclaimer: I'm new to this. I'm not even sure if what I'm proposing is doable, let alone the best way to approach it. Any and all feedback is welcomed.

Build’s Name: all-in-wonder-if-I-can-pull-this-off
Operating System/ Storage Platform: ESXi 6: napp-it, pfsense, plex
CPU: Xeon E3-1231 v3 (not purchased yet)
Motherboard: Supermicro X10SL7-F
Chassis: Fractal Designs Node 804
Drives:
No boot yet (USB)
4x WD Red 4TB
4x Samsung 850 Pro 256gb
2x Intel 320 320 160gb
RAM: 32gb Crucial ECC
Add-in Cards:
Power Supply: SeaSonic G 550w (not purchased yet)
Other Bits:

Usage Profile: ESXi host, NAS (napp-it, considering FreeNAS), plex transcoding (was planning on doing an ubuntu VM for this), PFSense (Firewall/IPS/Proxy/VPN). I may add some additional VMs down the road but no plans for anything resource hungry.

I'm trying to sort out the best way to carve up the storage.

The 4x WD drives will be passed through to napp-it on the LSI 2308 (ZFS, two mirrors). leaving 4x SAS ports for expansion when $ allows

I had planned to run the Intel SSDs on two of the 3gb/s SATA ports, mirrored for napp-it datastore.

I lucked out on the 4x Samsung SSDs. I had hoped to find 2x 500gb to mirror for the main datastore but ended up with 4x 256 instead. I don't know if I'm better off using only two of them mirrored on the 6gb/s SATA or all four striped on the 3gb/s ports. Maybe even put the SSDs on the HBA? The 8TB of storage will last me a while.

Any thoughts on how I should maximize the storage hardware on hand? I'm not sure which of the VMs would benefit more from the I/O perf of the 6gb/s ports.

Also, given the CPU/Memory that I'm proposing, how would you allocate resources to the three VMs?

Thanks.

 

[gea]

Basically everything ok. Some remarks
- one Sata SSD as ESXi bootdisk and for napp-it is ok
On problems you can reinstall from scratch within minutes.

Optionally use the second SSD as a spare cloned boot disk

The idea of two pools, one fast SSD pool for VMs and one slower disk pool is good.
I would propably use one mirror of 480 GB. Connect them to the HBA as you can use
local Sata only via RDM or as ESXi virtual disks (both not as recommended as the HBA idea).

If you haven't bought the board yet, I would propably use a socket 1151 Supermicro board X11.., optionally with the same LSI HBA as well and 10G. You may use a cheaper CPU, up from a G4400 up then.

 

[jimmytherat]

Basically everything ok. Some remarks
- one Sata SSD as ESXi bootdisk and for napp-it is ok
On problems you can reinstall from scratch within minutes.

Optionally use the second SSD as a spare cloned boot disk

The idea of two pools, one fast SSD pool for VMs and one slower disk pool is good.
I would propably use one mirror of 480 GB. Connect them to the HBA as you can use
local Sata only via RDM or as ESXi virtual disks (both not as recommended as the HBA idea).

If you haven't bought the board yet, I would propably use a socket 1151 Supermicro board X11.., optionally with the same LSI HBA as well and 10G. You may use a cheaper CPU, up from a G4400 up then.

Thanks for the feedback.
I've already got the board and memory in hand, too late for 1151.

So you're thinking run the 160gb Intel SSDs mirrored for ESXi and napp-it on 3gb/s local ports. Additionally use the 4 Samsung SSDs and 4 WD HDs on the LSI. Would I present the SSDs to ESXi directly or via napp-it?

Thanks again.

 

[gea]

When you present the SSD storage via ESXi directly you
- have a fast interconnection but without readcache support like over ZFS what means mostly slower than ZFS
-have only ESXi snaps support what means one or two shorttime snaps only
- have only limited access to the VMs what means copy/move/clone/backup is slow or require special tools

Connect them to the HBA and offer as a NFS datastore to ESXi.
This give you the advantage of ZFS readcachem the security of secure sync writes, unlimited snaps
and an easy way to backup/clone/restore with the help of Snaps, clones or SMB and Windows "Previous versions"

Over ZFS you also have secure ZFS software raid without raidhole problems like traditional software or hardware raid 1/5/6

btw.
How would you raid the SSDs without ZFS?
ESXi does not support software raid over Sata, so using ZFS is not only the best but also the only solution.

 

[jimmytherat]

When you present the SSD storage via ESXi directly you
- have a fast interconnection but without readcache support like over ZFS what means mostly slower than ZFS
-have only ESXi snaps support what means one or two shorttime snaps only
- have only limited access to the VMs what means copy/move/clone/backup is slow or require special tools

Connect them to the HBA and offer as a NFS datastore to ESXi.
This give you the advantage of ZFS readcachem the security of secure sync writes, unlimited snaps
and an easy way to backup/clone/restore with the help of Snaps, clones or SMB and Windows "Previous versions"

Over ZFS you also have secure ZFS software raid without raidhole problems like traditional software or hardware raid 1/5/6

btw.
How would you raid the SSDs without ZFS?
ESXi does not support software raid over Sata, so using ZFS is not only the best but also the only solution.

I'm showing my inexperience here, thanks for the feedback. This is saving me heaps of time.

 

[xnoodle]

Did you already have the Intel 320's on hand? They have a nasty bug that can temporarily brick the drive.

Sent from my HTC One using Tapatalk

 

[jimmytherat]

Did you already have the Intel 320's on hand? They have a nasty bug that can temporarily brick the drive.

I have already purchased them, refurbs from Intel. Haven't done anything with them yet. I did read about making sure they had the latest firmware flashed, I'll do that ahead of time.

I'll see what I can find about the bug, thanks for the heads up.