ZFS Z2 or Z3

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Albert Yang

Member
Oct 26, 2017
72
1
8
30
Hi,
I was wondering if someone could shed some light, currently going to buy Dell server R720XD
I was thinking to put H200 6Gbps SAS HBA LSI 9210-8i (=9211-8i) P20 IT Mode as the server has 24 disk slots 2.5 sas disks thinking to put each one 600gigs or 900gigs 10k rpm. But What i need is storage so i was thinking to go with Z2 but was reading that its safer to go with Z3?

Thank you
 

pricklypunter

Well-Known Member
Nov 10, 2015
1,708
515
113
Canada
ZFS RAIDZ2 is more than plenty capable at those sizes of disk. I would really only consider RAIDZ3 when your disk sizes get to a point where you are running scrubs/ resilvers that take weeks to complete, rather than hours/ days, as the potential for more disk failures during an array rebuild becomes greater. RAIDZ is no substitute for a back-up plan, like all raid storage implementations, it's purpose is to bring any downtime to a minimal level and afford a degree of protection to your live data while you fix whatever is wrong.

RAIDZ2 will allow your array to be kept intact and usable while you have 2 disk failures, and RAIDZ3 allows 3 disk failures. There are tradeoff's you should consider when using either level. If performance is your goal, you will want mirrors, rather than parity. If capacity is your goal, then you will want Z2 or Z3, depending on your disk sizes, acceptable read and write speed etc :)
 

dandanio

Active Member
Oct 10, 2017
182
70
28
Holy sh*tballs dude. :) Consider that any resilver will take AGES on 24 spinners in 1 pool. Split it in 3 pools, maybe 2, of Z1s, maybe Z2s, depending on how important that data is. Then scrub it weekly. Cron. Walk away. :)
 

Albert Yang

Member
Oct 26, 2017
72
1
8
30
Thanks for the reply, i guess ill go with the Z2 the scrubbing, i normally do weekly but changed it to monthly as it took days to finish but i guess it also depends on how often the data gets moved?
 

gea

Well-Known Member
Dec 31, 2010
3,141
1,184
113
DE
Since not only Oracle Solaris but also current Open-ZFS does sequential resilvering (Open-ZFS calls it sorted resilvering) resilvering/crubbing time is a fraction of the former time. No longer the main reason why you want or need z3 in large vdevs. Despite, the extra security over Z2 remains.
 
Last edited:

redeamon

Active Member
Jun 10, 2018
291
207
43
You should never go more than 8 disk wide via ZFS zvol. First you'll get terrible performance as any raid via ZFS is limited to the iOPS of 1 drive in the zvol. Instead you should do:

6 raidz1/z2 x4 (that's 6 disks per zvol) which will give roughly 4x single drive speed
OR
4x6 if performance is absolutely necessary

Also as others have said, dump those spinners and go SSDs- it's not worth the headache.
 
Last edited:
  • Like
Reactions: Albert Yang

Albert Yang

Member
Oct 26, 2017
72
1
8
30
Thanks for the reply, duly noted but lets say i have 8 disks i could go with raid 10 saying that i need speed rather then storage?
as for the spinner im pretty happy with 10krpm SAS disks the issue with SSD is the space very small unless i do a SLOG but not sure if it would help much
 

redeamon

Active Member
Jun 10, 2018
291
207
43
Raid 10 is by far the ideal ZFS setup aside from the 50% loss. Easy to add and remove disks. Performance is solid.
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,669
1,081
113
artofserver.com
Hi,
I was wondering if someone could shed some light, currently going to buy Dell server R720XD
I was thinking to put H200 6Gbps SAS HBA LSI 9210-8i (=9211-8i) P20 IT Mode as the server has 24 disk slots 2.5 sas disks thinking to put each one 600gigs or 900gigs 10k rpm. But What i need is storage so i was thinking to go with Z2 but was reading that its safer to go with Z3?

Thank you
Not directly answering your question, but I think the R720XD is better off using a H310 mini or H710 mini with IT mode firmware (for ZFS) instead of H200. You can re-use the existing SAS cable from the backplane to the motherboard, and save yourself a PCIe slot.

back to your question... i think raidz2 across 24xHDDs of that size is probably okay, but it depends on your tolerance for risk. another consideration is using multiple vdevs... so raidz2(12xHDDs)+raidz2(12xHDDs) as an example... will help increase your iops with more vdevs within the pool.

maybe if you describe your priorities (performance/fault tolerance/etc.) and expected I/O use cases (io sizes, sequential vs random, reads/writes), it would be easier for people to offer more specific configurations.
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,669
1,081
113
artofserver.com
Raid 10 is by far the ideal ZFS setup aside from the 50% loss. Easy to add and remove disks. Performance is solid.
fault tolerance is a bit compromised... if you have a pool made of mirror vdevs... if 2 disks within any single mirror set fail, the entire pool will fail since the records are striped across all vdevs.
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,669
1,081
113
artofserver.com
Holy sh*tballs dude. :) Consider that any resilver will take AGES on 24 spinners in 1 pool. Split it in 3 pools, maybe 2, of Z1s, maybe Z2s, depending on how important that data is. Then scrub it weekly. Cron. Walk away. :)
that's not always true... a 24xHDD raidz2 pool with 0 bytes of data will resilver in almost 0 seconds. ZFS doesn't resilver unused space. the resilver time has more to do with the data set size than just the raid topology.
 
  • Like
Reactions: dandanio

dandanio

Active Member
Oct 10, 2017
182
70
28
that's not always true... a 24xHDD raidz2 pool with 0 bytes of data will resilver in almost 0 seconds. ZFS doesn't resilver unused space. the resilver time has more to do with the data set size than just the raid topology.
Labelling that as "FUNNY". You imply that someone would actually build a 24xHDD raidz2 pool to hold exactly 0 bytes of data? Actually, one could...
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,669
1,081
113
artofserver.com
Labelling that as "FUNNY". You imply that someone would actually build a 24xHDD raidz2 pool to hold exactly 0 bytes of data? Actually, one could...
maybe you missed my point. it was just an extreme example to point out that resilver time is dependent on the size of the actual data on the pool, not the topology.