Question about ZFS L2ARC

Discussion in 'Linux Admins, Storage and Virtualization' started by qwrty, Dec 4, 2018.

  1. qwrty

    qwrty New Member

    Jul 18, 2018
    Likes Received:
    Hello everyone,

    I'm wondering if it should be interesting to set up a L2ARC cache with 32Gb of ram ?
    My actual system:

    32Gb RAM
    3*6To RAIDZ1
    (I will add another vdev of 3*6To soon)

    (ZFS volume is used to VM, content sharing and backup right now)

    I have got a 500Gb 970 EVO NVME SSD and I would like dedicated a 128/200 Gb partition of it to L2ARC.

    I can't find a real statement about L2ARC and amount of RAM in the system, and threads I could read was pretty old, so maybe it changed with time....

    Any thoughts are welcome :)

    Thank you !
  2. gea

    gea Well-Known Member

    Dec 31, 2010
    Likes Received:
    Arc in RAM and L2Arc, the ARC extensions to SSD work blockbased (not filebased) on a last read, most read strategy. This work best with small random reads and reads of metadata. With 32GB RAM and a VM workload I would not expect an essential improvement with L2Arc. You may enable read ahead on the L2Arc. This can help a little with more sequential workloads.

    For a sequential workload pool iops is the limiting factor. If you care about use a pool from multiple mirrors. For VM storage you also need sync write or on a crash several gigabytes of last writes can be lost in the rambased write cache. Best Slog is an Intel Optane up from 800P. These are the items you should care about, not L2Arc.
  3. zxv

    zxv The more I C, the less I see.

    Sep 10, 2017
    Likes Received:
    The amount of ram consumed by the l2arc is influenced by both the size of the l2arc and the recordsize.

    Last time I checked, it was 70 bytes per record, in which case, the formula is:
    L2ARC size * ( 70 bytes / recordsize in bytes ) = RAM consumed

    For a 4K record size, the ratio is (70/4096), or ~1:58.
    120G of L2ARC / 58 = 2G of RAM.

    For a 128K records size, the ratio is (70/131072) = 1:1872
    120G of L2ARC / 1872 = 64MB of RAM

    See: [zfs-discuss] Relation between L2ARC and RAM size

    An additional option one can consider if RAM is limited, is to use a smaller amount of L2ARC for metadata only, using:
    zfs set secondarycache=metadata POOLNAME
    BackupProphet likes this.
  4. qwrty

    qwrty New Member

    Jul 18, 2018
    Likes Received:
    Thnak you for feddback :)

    So in my case, it's not worth at all.

    Better choice, should be moving VM to the NVME disk I guess.
  5. BackupProphet

    BackupProphet Well-Known Member

    Jul 2, 2014
    Likes Received:
    Yeah you would be better off moving the VM to a NVME disk, though the EVO 970 will be bottlenecked as it has very slow sync writes. I have a 280GB Optane as a L2ARC and is currently using 30GB memory for the L2ARC. But my system has 320GB system memory.
Similar Threads: Question L2ARC
Forum Title Date
Linux Admins, Storage and Virtualization Setup ZFS quick question Jul 19, 2019
Linux Admins, Storage and Virtualization Proxmox Ceph hardware question Mar 12, 2018
Linux Admins, Storage and Virtualization proxmox questions Jan 17, 2018
Linux Admins, Storage and Virtualization Proxmox VE "noob" build Ceph question May 2, 2017
Linux Admins, Storage and Virtualization FreeNAS + 4 Proxmox Nodes Question Mar 28, 2017

Share This Page