ZFS Design Principles

Discussion in 'Solaris, Nexenta, OpenIndiana, and napp-it' started by gea, Mar 6, 2018.

  1. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    1,804
    Likes Received:
    604
    #1
    jerrytsao and K D like this.
  2. TangoWhiskey9

    TangoWhiskey9 Active Member

    Joined:
    Jun 28, 2013
    Messages:
    390
    Likes Received:
    59
    You should talk to @Patrick about doing these as main site articles. You'd get more eyes on napp-it than PDFs
     
    #2
  3. BackupProphet

    BackupProphet Active Member

    Joined:
    Jul 2, 2014
    Messages:
    662
    Likes Received:
    230
    The common pitfall about ZFS information that many fails to tell correctly is memory requirements and requirement of ECC memory. You got the details about ECC memory correct, but memory requirements wrong. ZFS requires almost no memory. You will be just fine on a system with 1GB ram. The recommended is 4GB, as ZFS will then by default enable prefetching which will increase read iops for data on spinning drives. For SSD's prefetching may be unnecessary, you can check the zfs-stats if prefetching has a good effect
    Code:
    ARC Efficiency:                                 399.70m
            Cache Hit Ratio:                83.25%  332.75m
            Cache Miss Ratio:               16.75%  66.96m
            Actual Hit Ratio:               20.73%  82.87m
    
            Data Demand Efficiency:         86.35%  74.73m
            Data Prefetch Efficiency:       84.14%  308.71m
    
     
    #3
    Linda Kateley likes this.
  4. BackupProphet

    BackupProphet Active Member

    Joined:
    Jul 2, 2014
    Messages:
    662
    Likes Received:
    230
    You also get the part about Optane wrong, the only time you want ZIL is when you have spinning drives. For SSD's or Optane there is no need. With the exception of consumer level SSD's though, they will benefit greatly from having an enterprise SSD as ZIL.

    The difference between 900p and p4800x is not related to powerloss protection, Optane doesnt need powerloss protection by design. There is also no difference in performance.
    There real difference however is that p4800x support the Intel Memory Drive technology which let you use the p4800x as additional system memory.
     
    #4
  5. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,325
    Likes Received:
    302
    #5
    gustav likes this.
  6. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,327
    Likes Received:
    1,281
    Irrelevant. Who doesn't have a single 4GB DIMM in 2017.

    A ZIL is on pool until you add a SLOG device such as an Optane which will then house the ZIL
    Adding an optane to say a pool of 4x SSD will improve their performance.
     
    #6
  7. BackupProphet

    BackupProphet Active Member

    Joined:
    Jul 2, 2014
    Messages:
    662
    Likes Received:
    230
    There are plenty of systems with less than 4GB ram. Cloud servers, embedded systems and old systems.

    In 999 999 of 1 million cases you do not want to use a whole Optane 900p / p4800x as a standalone SLOG. The simplest and fastest is just using the Optane drive for storage and sync writes. SLOG was implemented to speed up spinning drives, which has awful sync write performance. Enterprise SSD's are easily 1000-10000 times faster and the benefit of having an additional SLOG is really never worth it.
     
    #7
  8. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,327
    Likes Received:
    1,281
    • It is not "SIMPLE" to use only optane drive for storage because not many people can load up 4 or 8 optane drives in their chassis at this time. Also, the drives currently have minimal capacity so it is not even a possibility to hit capacities of SSD currently and def. not as simple as a 24x SSD array.

    • An optane will increase performance of enterprise SSD on sync write. You believe what you want you are wrong.








     
    #8
  9. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    10,967
    Likes Received:
    3,922
    In the airport so detailed feedback later. @gea if you did want to do this as a guest post or series on STH, happy to help with that.
     
    #9
  10. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    1,804
    Likes Received:
    604
    Thanks all for your comments.
    I will care about as the pdf is ongoing work

    My main intention is not to replace ZFS manuals from Solaris or give a full overview but give some basic ideas why and how things work and give my opinion on controversal things like RAM/ ECC/ Slog/ number of disks per vdev and basic configuration ideas regarding snaps and backup.

    about RAM
    I thought to make it clear that there is no extra RAM need for ZFS over say ntfs.
    Oracle claims 2 GB as minimum for Solaris and that is the case for every 64bit OS

    But nobody has fun with a 1 -2 GB ZFS filer on such a 64bit system even when stable.
    Unless you use fast disks such a server is simply too slow and who would use expensive disks in a low memory setup as they are more expensive that enough RAM.


    about Optane
    While Optane can be a RAM replacement the main difference of a 900P to Flash is that you can adress data directly like with RAM and not blockwise like with Flash with the need of Trim or erase prior write. Its clear that a 900P is not RAM. Maybe I must clarify this.

    There have been some discussions about Optane and Powerloss Protection. Intel guarantees this only for the 4800. When I bought my first 900P this was also in the specs but removed later by Intel.

    Now you can speculate if it is removed due marketing or a technical reason. But PLP is not a guaranteed feature on smaller Optane models.


    @Patrick
    None of the content is really new and for a main site article propably not detailled enough. Intention is more to give some basic recommendations and collect some answers (and opinions) to all the beginners questions or in my opinion wrong statements like min RAM/ECC vs Scrub of Death, Slog vs write cache, golden number of disks in Raid-Z, performance considerations vs pool layout, first steps and similar.

    If you are under the impression that parts can be used for guest posts or a series, I am open.
     
    #10
  11. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    10,967
    Likes Received:
    3,922
    #11
  12. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    1,804
    Likes Received:
    604
    Yes, the Buyers Guide is not really up to date.
    While most ZFS suggestions are the same for BSD, Linux or Solaris,
    some are different due different add-ons, driver support or recommendations
    where I partly have a different opinion than the FreeNas Guys.

    If you like I would be glad to help to improve or add some do and don't for beginners.
    This is the essential of my pdf.
     
    #12
  13. Linda Kateley

    Linda Kateley New Member

    Joined:
    Apr 25, 2017
    Messages:
    21
    Likes Received:
    4
    I can't tell you how many times I have heard swags on memory requirements. The only one i know is freenas needs 8GB because the utility needs it, not zfs.
     
    #13
  14. Jeggs101

    Jeggs101 Well-Known Member

    Joined:
    Dec 29, 2010
    Messages:
    1,392
    Likes Received:
    196
    Everyone's always saying for 4x8TB you need 32GB RAM, but my FreeNAS never uses more than 12GB
     
    #14
  15. Linda Kateley

    Linda Kateley New Member

    Joined:
    Apr 25, 2017
    Messages:
    21
    Likes Received:
    4
    The Freenas community is the only place you will hear that.
     
    #15
    dswartz likes this.
  16. gea

    gea Well-Known Member

    Joined:
    Dec 31, 2010
    Messages:
    1,804
    Likes Received:
    604
    This is also my impression.

    For a Solarish based single user ZFS system with not too many active data, 4-8 GB RAM may be enough for a decent performance as you do not need a large readcache.

    If this is a multiuser mailserver where you want to read most data from ram, you may want 64-128 GB RAM. It is the use case that demands RAM, not ZFS.
     
    #16
Similar Threads: Design Principles
Forum Title Date
Solaris, Nexenta, OpenIndiana, and napp-it Trying to determine ZFS design of vdevs Sep 27, 2016
Solaris, Nexenta, OpenIndiana, and napp-it Best LSI HBA Designs for OmniOS / Napp-IT May 3, 2013
Solaris, Nexenta, OpenIndiana, and napp-it All-In-One Server design Apr 2, 2013

Share This Page