ZLOG and L2ARC - ZFS

11Blade

Member
Aug 8, 2016
31
6
8
50
Performance questions on a dual xeon 2620v3 X10 supermicro board with a SAS2 (16 drive) backplane. 128Gb ram - Running Proxmox - 1 VM running mySQL , 2nd VM for data manipulation.

The backplane has a single SAS connector to the controller in IT mode

right now on the backplane -
8 x 10 TB HGST drives for storage pool (probably will be 4 mirrored vdevs for total 40TB instead of RAIDZx
3TB WD and 2 15k 130Gb Seagates

2 Satadom 64Gb drives (1 for boot, 1 for storage)

which will provide better performance ?
1. Zlog on the PCIe bus with Nvme vs fast ssd on SAS2 backplane?
2. L2ARC - PCIe options or fast SSD on SAS2 backplane

Filesizes range from 20Mb to 100Mb - a few approach 500Mb.

I guess the question is where will the bottlenecks occur - at the PCI bus or controller/backplane

(the data is backed up on an older slower storage while I optimize the setup)

11Blade
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
6,992
1,568
113
CA
No question adding a PCIE NVME SLOG device will yield the best improvement in write performance.

Add more RAM will increase ARC (meta read cache) performance assuming your data should go in ARC (may need tuning). You can configure ZFS to cache sequential too, maybe need that and l2arc depending on your workload.

With a spinning pool (HDD) you may want to try an Intel S3710 200GB (or 400gb if you can get a deal) as the NVME may not yield much improvement over the SSD since you're going to be limited to write performance of the pool ultimately.

If you want to get better performance you're going to need to go to 4x (or more) SSD pools in mirrored VDEV. And then once you're going with more SSD you should swap out that backplane for a non-expander backplane. With only 16 drives that's 2 HBA and 4 cables to connect it to the non-expander backplane.
 
Last edited:
  • Like
Reactions: K D