ZFS and Lustre

[Patrick]

Sorry for the crazy question, but has anyone used this?
ZFS on Linux
https://www.nersc.gov/assets/Uploads/W01-ZFS-and-Lustre-June-1024.pdf

I received a request to test something that utilizes this setup and am trying to decide:

• Is it worth the time?
• What should I run on this/ benchmarks?
• What am I going to have to allocate client wise to test even a half rack system?

If anyone has input here, that would be great.

 

[unwind-protect]

You say client. Is that via NFS?

I have a custom version of Bonnie but it doesn't do any multi-client simulation either.

The entire presentation doesn't mention ZFS settings. Settings are critical. They turn on dedup on a 64 GB machine with a large ZFS and KABOOM.

Do they use one or more cache drives with the array?

Is their workload mostly readonly?

They also need to test resync time. Pull a drive in the middle of a write test (while observing throughput). Wipe drive. Put it back and see how long it takes to sync, while you put deliberate load on the system. That load on the system should be a benchmark so that you can see what is left of your throughput during resync, and you observe how the resync rate changes while you hammer thingie with a benchmark.

 

[Patrick]

I think that LNL runs what most would consider a large infrastructure (that presentation had almost 100,000 nodes listed in the setup.) I think at that scale when you have sustained writes in the 850GB/s range to multi PB systems they probably just use their own application and have a good idea around the operational challenges.

I am more thinking, should I engage on a project even if it is only a "small" few hundred TB setup (controller, disk nodes, switches). It is a lot of power and space so I want to figure out if it is worthwhile.