HDD Setup

n00bftw

Member
Jul 16, 2020
39
4
8
United Kingdom
Hi, I have 8x14TB drives, they get about 210MB/s R/W. What setup should I be going with if I am looking for maximum speed?

I will be running TrueNas, thanks in advance.
 

dandanio

Active Member
Oct 10, 2017
148
51
28
RAID 10. But, spinning rust is slow. So, HDDs and "maximum speed" is an oxymoron of sorts.
 

Spearfoot

Member
Apr 22, 2015
68
26
18
62
Hi, I have 8x14TB drives, they get about 210MB/s R/W. What setup should I be going with if I am looking for maximum speed?

I will be running TrueNas, thanks in advance.
For maximum speed use mirror vdevs. Pool IOPS scale by the number of vdevs, so 4 mirrored vdevs will give you 4 times the IOPS of a single RAIDZ1/2 vdev.
 

Rand__

Well-Known Member
Mar 6, 2014
4,644
922
113
Pool IOPS scale by the number of vdevs, so 4 mirrored vdevs will give you 4 times the IOPS of a single RAIDZ1/2 vdev.
Only if you have have sufficient IOPS consumers - i.e. with a single thread scaling will (basically) not occur.

@ OP - whats your use case? VMs, large file storage, small file storage ... few user/processes/vms or many?
 
Last edited:
  • Like
Reactions: n00bftw

i386

Well-Known Member
Mar 18, 2016
2,120
562
113
31
Germany
Raid 0/striping maximum "speed" aka throughput, most efficient use storage, highest chance for failure (chance grows with every devices added :D)
Raid 10/striped mirrors fastest layout with some sort of redundancy, 50% storage efficiency, can lose up to 50% of all devices in the best case (worst case: only 2 devices in a mirror pair)
Raid 5/6 aka z1/2/3 "slowest" layout, maximum storage efficiency with redundancy (87.5% z1, 75% z2), can tolerate 1 failing device (2 for z2)
 
  • Like
Reactions: n00bftw

n00bftw

Member
Jul 16, 2020
39
4
8
United Kingdom
Only if you have have sufficient IOPS consumers - i.e. with a single thread scaling will (basically) not occur.

@ OP - whats your use case? VMs, large file storage, small file storage ... few user/processes/vms or many?
This machine is being used for backing up mainly (Mostly large files, and some small), might throw a VM or two on there. Only a single user.
 

Rand__

Well-Known Member
Mar 6, 2014
4,644
922
113
You intend to use NFS or iscsi for the vms?
Sync or async?
I've been using mirrors exclusively for years but at this point with your use case I actually would say get an optane slog and then run z2
 

Spearfoot

Member
Apr 22, 2015
68
26
18
62
This machine is being used for backing up mainly (Mostly large files, and some small), might throw a VM or two on there. Only a single user.
Those two use-cases are quite different. :)

If speed is what you want (as you originally posted) then mirrors are the way to go. This is true if you want to run VMs too.

If you just want a backup target, RAIDZ2 will work fine. And you can run VMs on a RAIDZ2 pool, they just won't be as performant as they would be if they were based on mirrors.

Regarding a ZIL SLOG device... You can ignore the issue of synchronous writes if you're just playing around with VMs in your home lab. This is not the case if you're running enterprise-critical applications. For details see:

 
  • Like
Reactions: n00bftw

n00bftw

Member
Jul 16, 2020
39
4
8
United Kingdom
Got it sorted now, the problem was that I had to add 'tso lro' into the options field of the NIC. Beautiful. I also have it running in ESXi ;). Runs perfect.

Untitled.png