All-In-One Server design

[Obscurax]

Hi there,

I'm not really new to ZFS but I am to ESXi, so here I am for your advice.
First of all this is my current configuration:

This is my OpenIndiana NAS atm.
L3426
x8sil-f
32GB ECC RAM
2x160GB boot mirror
6x2TB 512b drives RAIDZ2 conf (< 2TB free space)
Intel SASUC8I (doesn't support > 2TB)
Norco RPC-3216 (16 hot swaps)
APC UPS

I have 2 other physical servers.
Because of the electrical bill I'd like an AiO server reusing some of the above hardware.
Keep in mind this is only for home use and testing.
It's nice to have 24/24 uptime, but when a failure occurs I can live with the server being offline for a while (they are not production servers).


What I have in mind:
L3426
x8sil-f
32GB ECC RAM
USB drive for ESXi (I will clone the drive for fast recovery)
2x40GB Intel 320 SSD (mirror for OpenIndiana) => Is it wise to use the remaining space as ZIL log?
4x500GB 2,5" sata in RAID10 (ESXi VMs datastore) => Will 5400rpm drives be fast enough?
6x3TB 4k drives (RAIDZ2 conf)
6xXTB 4k drives (RAIDZ2 conf, X because this is for the future.)
2x IBM 1015
Norco RPC-3216


With this setup I have all 16 hot swaps nicely filled. Am I right to ditch the 512b drives and replace them with 4k drives for future purpose?
What do you think about the above configuration / what would you change? All advice is welcome.
WHich drives would you buy atm and why? The less the overall cost the better I'm a poor student.
Thanks for your time and sorry for my English.

 

[Mike]

I believe you can reflash the SASUC8I to support 3tb drives. It's LSI 1068e based and plenty fast for regular spinners.

 

[Obscurax]

I believe you can reflash the SASUC8I to support 3tb drives. It's LSI 1068e based and plenty fast for regular spinners.

I hope this is true, but unfortunately I can't find any info about this.

 

[Mike]

LSI Controller FW updates IR/IT modes

Check it out. It's pretty easy to do from any linux (vm).

-
Oh wait, it says 2.2tb out of 3tb. Must've been edited lately. Never had the chance to try it out myself. Not fortunate enough to have a 3tb disk array.
I guess you are out of luck with your current controller then, sorry.

 

[Biren78]

Wondering - why do you need to do ZIL? Is write performance required? Would suggest 80gb or larger Intel drives as the 40GB are much slower.

 

[Jeggs101]

Boot off of USB.
Reuse 160GB for vm datastores
Keep 2TB drives and have one ZFS set
Add 3TB or 4TB drives for new ZFS set as needed

replace 2TB drives with larger drives when needed.

Overbuying storage just wastes money

 

[TheBay]

He will lose any mirroring or redundancy for the VM data store though if done as above, though he can just take snapshots to the other 160gb or on the zfs pool.

 

[Obscurax]

Wondering - why do you need to do ZIL? Is write performance required? Would suggest 80gb or larger Intel drives as the 40GB are much slower.

Because I thought ZIL was important for synchronous IO, the ESXi VM datastore will be shared via NFS. NFS is fully synchronous writes so I thought it would be beneficiary.
I know larger SSD's are faster but would it make that much of a difference? Considering it costs a lot more and I could use the remaining space for ZIL anyway?

Boot off of USB.
Reuse 160GB for vm datastores
Keep 2TB drives and have one ZFS set
Add 3TB or 4TB drives for new ZFS set as needed

replace 2TB drives with larger drives when needed.

Overbuying storage just wastes money

-Boot off USB: check
-Reuse 160GB for vm datastores: This is no option, I will loose redundancy for the VMs.
-I don't want multiple pools that's why I want to ditch the 512b drives. I want 1 pool with multiple vdevs.
Combining 512b and 4k drives in the same pool will cause performance issues for sure.

He will lose any mirroring or redundancy for the VM data store though if done as above, though he can just take snapshots to the other 160gb or on the zfs pool.

Yes I could do that but I want redundancy, it's harder to recover from a snapshot.

 

[gea]

What I have in mind:
L3426
x8sil-f
32GB ECC RAM
USB drive for ESXi (I will clone the drive for fast recovery)
2x40GB Intel 320 SSD (mirror for OpenIndiana) => Is it wise to use the remaining space as ZIL log?
4x500GB 2,5" sata in RAID10 (ESXi VMs datastore) => Will 5400rpm drives be fast enough?
6x3TB 4k drives (RAIDZ2 conf)
6xXTB 4k drives (RAIDZ2 conf, X because this is for the future.)
2x IBM 1015
Norco RPC-3216

I would
- using 40GB SSD for OI is ok, ZFS mirror is an option as well as using one Intel SASUC8I as ESXi datastore mirror
(leaves the option to pass-through Sata in AHCI mode - quite fast), OI disk performance is not critical.
- never use a SSD as boot + ZIL + ARC
- not use 5400U/m laptop drives as datastore (too slow). I would think about a SSD mirror for VM's example 2 x 240 GB
- use the old disks for backups (try to keep bays free for one or better two removable backup pools for important data)

depending on VM's: i would disable sync and run it without ZIL, use a USV.

 

[Obscurax]

I would
- using 40GB SSD for OI is ok, ZFS mirror is an option as well as using one Intel SASUC8I as ESXi datastore mirror
(leaves the option to pass-through Sata in AHCI mode - quite fast), OI disk performance is not critical.
- never use a SSD as boot + ZIL + ARC
- not use 5400U/m laptop drives as datastore (too slow). I would think about a SSD mirror for VM's example 2 x 240 GB
- use the old disks for backups (try to keep bays free for one or better two removable backup pools for important data)

depending on VM's: i would disable sync and run it without ZIL, use a USV.

Considering you're input and the above posters input i came to this:
Thanks for the useful information.

And what is a USV?

L3426
x8sil-f
32GB ECC RAM
USB drive for ESXi
2x160GB 2,5" laptop drives (ZFS mirror for OpenIndiana)
2x256GB Crucial m4 (ZFS mirror for ESXi VMs datastore)
2x2TB 512b drives (ZFS mirror backup pool from old drives)
6x3TB 4k drives (RAIDZ2 conf)
6xXTB 4k drives (RAIDZ2 conf, X because this is for the future.)
2x IBM 1015
Norco RPC-3216

 

[gea]

And what is a USV?

USV = unterbrechungsfreie Stromversorgung (sorry, german) = UPS
Uninterruptible power supply - Wikipedia, the free encyclopedia