Question about ZFS L2ARC

[qwrty]

Hello everyone,

I'm wondering if it should be interesting to set up a L2ARC cache with 32Gb of ram ?
My actual system:

32Gb RAM
3*6To RAIDZ1
(I will add another vdev of 3*6To soon)

(ZFS volume is used to VM, content sharing and backup right now)

I have got a 500Gb 970 EVO NVME SSD and I would like dedicated a 128/200 Gb partition of it to L2ARC.

I can't find a real statement about L2ARC and amount of RAM in the system, and threads I could read was pretty old, so maybe it changed with time....

Any thoughts are welcome

Thank you !

 

[gea]

Arc in RAM and L2Arc, the ARC extensions to SSD work blockbased (not filebased) on a last read, most read strategy. This work best with small random reads and reads of metadata. With 32GB RAM and a VM workload I would not expect an essential improvement with L2Arc. You may enable read ahead on the L2Arc. This can help a little with more sequential workloads.

For a sequential workload pool iops is the limiting factor. If you care about use a pool from multiple mirrors. For VM storage you also need sync write or on a crash several gigabytes of last writes can be lost in the rambased write cache. Best Slog is an Intel Optane up from 800P. These are the items you should care about, not L2Arc.

 

[zxv]

The amount of ram consumed by the l2arc is influenced by both the size of the l2arc and the recordsize.

Last time I checked, it was 70 bytes per record, in which case, the formula is:
L2ARC size * ( 70 bytes / recordsize in bytes ) = RAM consumed

For a 4K record size, the ratio is (70/4096), or ~1:58.
120G of L2ARC / 58 = 2G of RAM.

For a 128K records size, the ratio is (70/131072) = 1:1872
120G of L2ARC / 1872 = 64MB of RAM

See: [zfs-discuss] Relation between L2ARC and RAM size

An additional option one can consider if RAM is limited, is to use a smaller amount of L2ARC for metadata only, using:
zfs set secondarycache=metadata POOLNAME

 

[qwrty]

Thnak you for feddback

So in my case, it's not worth at all.

Better choice, should be moving VM to the NVME disk I guess.

 

[BackupProphet]

Yeah you would be better off moving the VM to a NVME disk, though the EVO 970 will be bottlenecked as it has very slow sync writes. I have a 280GB Optane as a L2ARC and is currently using 30GB memory for the L2ARC. But my system has 320GB system memory.