ZFS on Linux - arc not used

Stril

Member
Sep 26, 2017
184
9
18
37
Hi!

I will test Solaris, but I think, there most be an issue with the hardware, as Freenas is showing such a different performance in the two systems...

19s vs over 3 minutes.

I hope, it's only one bad disk...
 

zxv

The more I C, the less I see.
Sep 10, 2017
153
51
28
What I tried now, is FreeNAS (because this was easy):

SMB with ZoL: 3:16 min
SMB with EXT4: 0:16 min
SMB with FreeNAS: 0:19 min

I would prefer ZoL, but as I do not havy any idea, yet....
Here is another option to narrow down the cause.
This might help identify whether it's local filesystem issue vs a transport/Samba integration issue.

On the file server local host, in the root of the ZFS, and EXT4 filesystems, run:

time find . -ctime 1 >/dev/null

That makes a stat() call on each and every file and directory.
Run it more than once and take the time of the last run.
That should reproduce in part the effect of robocopy, but without Samba.
If this also shows similar differences between filesystems, it may point to the filesystem itself and not a Samba tuning issue.
 
  • Like
Reactions: fossxplorer

Stril

Member
Sep 26, 2017
184
9
18
37
Hi!

It must be something completely different:

FreeNAS -Tests:


1. 32xSAS-Pool, 2x CPU E5-2643 v2, 128 GB RAM
2:24 min
2. NVMe-Pool (Intel 900P), 2x CPU E5-2643 v2, 128 GB RAM
2:23 min

3. VM on VMWare ESXi 6.7, 32GB RAM, CPU Intel Gold 6124, 4 Cores, iSCSI-storage as VHD
0:19 min


I am on a trip for the next 4 days and hope to get any idea on what the problem is and test Solaris, then.
 

zxv

The more I C, the less I see.
Sep 10, 2017
153
51
28
Phoronix did a much more thorough comparison recently:
FreeBSD ZFS vs. TrueOS ZoF vs. DragonFlyBSD HAMMER2 vs. ZFS On Linux Benchmarks - Phoronix

As just one data point, I did a quick comparison of stat() calls
ZFS vs EXT4 on Debian Buster alpha4, on a DL380p.
I chose Debian buster because their contrib repo has zfs 0.7.12.

With a directory tree holding 480,000 files:
time find . -ctime 1 >/dev/null
EXT4: real 0m1.930s user 0m0.431s sys 0m1.493s
ZFS: real 0m3.326s user 0m0.495s sys 0m2.825s

EXT4 and ZFS were each on a single 10K WD3000BLFS SATA drive on
a HP P420i controller, in raid mode, as a single drive raid0 volume.
Even though the newer hpsa driver handles the P420i, you can still
get smart data via "smartctl -a -d cciss,0 /dev/sdb".

I think my results are fairly consistent with what Phoronix shows.
 

Stril

Member
Sep 26, 2017
184
9
18
37
Hi!

I think, I found the problem: it must be something related to the ActiveDirectory-Permissions:

If I disable AD-permissions, the directory-scan speeds up from 2 minutes to 8s.