Building new file/media/VM server with ZFS

[Gallager2014]

I am building a new server that is intended to be used as a media server, file server, torrent server, and as a VM server. I want the data stored on this server to have data integrity so I want to use ZFS. I am considering using ESXi as the base software and run everything else as guest VMs.
Is this a good way to go? Can ZFS be used with ESXi?

Here is what I would like to have different ZFS sets for:
- media
- Video
- Music
- Torrents
-Misc
- Media Server
- Dev. Server

Here is the hardware I currently have:
- 24 Bay X8DT3-F Supermicro server,
- 2x X5650 cpu,
- 48GB ECC RAM
- LSI MegaRAID 9260
- LSI 9207 HBA

 

[HellDiverUK]

I'd run NAS4Free/FreeNAS on a VM in ESXi and pass through the drives. Then use another drive (probably SSD) to host whatever other servers you need.

 

[Gallager2014]

Is there a performance hit using ZFS with ESXi?

 

[HellDiverUK]

No more than any other OS/file system, though you do have to take in to account ZFS' ridiculous RAM requirements.

 

[Entz]

Is there a performance hit using ZFS with ESXi?

If you are serving up your datastores over NFS there will be a fairly large one but likely not enough to ruin things for you (depending on yours disks and how much ram you can throw at it). SLOG highly recommended in that case.

 

[gea]

ESXi + a storage VM is ok, I do this for years.

What I would do:
- you can virtualize a storage OS, do not virtualize storage
- do not pass single disks, pass the whole controller with disks to give ZFS full control
- use a SSD mirror for your VMs and a Raid-Z(1-3) for media, fileservices and backup

It would be nice to use poweloss-safe SSDs for the VM pool with secure sync writes enabled - or optionally add a dedicated fast small enterprise SSD as a ZIL log device.

Regarding RAM.
ZFS does not really need more RAM than other filesystems but it use any free RAM as read-cache.
RAM above 2 GB gives you performance but it is not needed for a working ZFS server
if you build it to be as slow as a typical lowend Linux NAS with similar RAM.

I would prefer between 6 and 16 GB RAM with 16-32 GB installed RAM.

see my configuration
http://www.napp-it.org/doc/downloads/napp-in-one.pdf
with a downloadable ready to use ESXi storage VM + Web-UI based on the free Solaris fork OmniOS