ZFS settings to improve Windows Search performance

[crazyj]

Just doing some searches of the data on my AIO napp-it, and I'm not sure it can get any slower. Read write speeds are fine, but Windows7 searching it is agonizingly slow. Can anything be done to index(?) it for quicker searches?

 

[pricklypunter]

Do you see any messages stating that it's not an indexed location?

 

[gea]

Regardless indexing, ZFS read performance depends on iops performance and caching.
With enough RAM, all random re-reads are delivered from the rambased ARC. You can extend the rambased ARC with an SSD based L2Arc where you can additionally enable "read ahead" to improve sequential readings.

So RAM is the prime matter. Solarish runs stable with 2 GB RAM regardless poolsize. If you want to cache currently active data only, 4-8GB RAM may be ok. If you want to cache all metadata of a pool (around 1% of data), you need up to 1GB RAM per TB active data (not poolsize). You can then add the RAM to cache datablocks. For an average high performance system 16-64GB RAM makes sense. If your workload reads more random data even more can be a good idea (64-512GB RAM). If your workload is more sequential, add an L2Arc with read ahead enabled, preferable up to max 5-10x of RAM size best from an NVMe.

 

[crazyj]

I've got 24GB allocated to the host. I haven't seen the guest use more than 2.4. Storage size is around 2.7TB total, but I really only do heavy searching in one of them having 0.8TB data.

 

[K D]

Regardless indexing, ZFS read performance depends on iops performance and caching.
With enough RAM, all random re-reads are delivered from the rambased ARC. You can extend the rambased ARC with an SSD based L2Arc where you can additionally enable "read ahead" to improve sequential readings.

So RAM is the prime matter. Solarish runs stable with 2 GB RAM regardless poolsize. If you want to cache currently active data only, 4-8GB RAM may be ok. If you want to cache all metadata of a pool (around 1% of data), you need up to 1GB RAM per TB active data (not poolsize). You can then add the RAM to cache datablocks. For an average high performance system 16-64GB RAM makes sense. If your workload reads more random data even more can be a good idea (64-512GB RAM). If your workload is more sequential, add an L2Arc with read ahead enabled, preferable up to max 5-10x of RAM size best from an NVMe.

Off topic but I really wanted to thank you for the responses you provide here. Usually more than what the question was about and making sure that the concept is clearly understood.

 

[crazyj]

Do you see any messages stating that it's not an indexed location?

So, after doing a bit more digging, it seems that ever since windows-7, M$ has depracated UNC path indexing, which pretty much confines my NAS to being non-indexed and super slow at locating JPEG information (GREAT - thanks Microsoft). They basically force you to run Windows Server, as it performs all the indexing on its own local drives, and feeds that information to the clients.

So, nothing to do with ZFS, and everything to do with Microsoft sucking...

 

[Evan]

What about netapp etc ?

 

[pricklypunter]

So, nothing to do with ZFS, and everything to do with Microsoft sucking...

That's kind of what I was getting at, W7 usually tells you in a wee message that it's not indexed, W10, I didn't test long enough to know if it does or not