Intel Optane or S3700 for ZFS ZIL/SLOG?

[BNet]

After seeing some early reviews of the DC P4800X it looks like it would be ideal for a ZFS ZIL, with the only caveat being the price. My understanding is that Intel is also releasing some 16/32GB drives in M.2 form which they are targeting as consumer cache drives with much more palatable prices. With ZFS ZILs not needing much capacity, would running these 16/32GB M.2 drives just as regular block storage (i.e. independent of Intel's RST '2.0') be possible on a C612 chipset? Or would it make more sense to just pick up a 100GB S3700 and not have to wait?
For context, I am planning on this being an addition to my current 12x 7200 4tTB HDDs, 800GB Intel 750 L2ARC and 64GB of RAM for VM datastore purposes.

 

[Patrick]

I am less keen on the m.2 version btw.

The other option, of course, is NVMe SSD with PLP.

 

[BNet]

I just want to thank Cliff Robinson for providing a paragraph specifically answering my question in the STH Optane review. With such a low die count (only two on the 32GB), sequential write performance was always going to be Optane Memory's main weakness in write cache scenarios.

 

[bds1904]

Optane is designed to be read cache and performs as such. The write speed isn't exceptional but it's good for the price. The latency is nice and low. Overall I don't think it would be a bad SLOG for a basic lab. I personally will wait until Intel launches the "Optane SSD". That has the potential to be the cats meow when it comes to a cheap SLOG. Larger size optane module means faster write speeds while hopefully maintaining the low latency.

 

[Generic George]

Is there any reason a 16gb optane drive wouldn't make a good boot drive for a FreeNAS installation? It's not like you need to use the NVME connections for the speed, when you're accessing something over even a 10gbe link.

 

[Stux]

Halfway house between optane p4800x and s3700 would be the p3700, wouldn't it?

 

[i386]

Optane is designed to be read cache and performs as such. The write speed isn't exceptional but it's good for the price.

This might be true for the m.2 consumer version of 3d xpoint. The enterprise version (p4800x) outperforms almost all other ssds (except some dram based custom devices for hyperscalers and ssds with x8/x16 pcie links in sequential tests).
It's a killer device, especially for being the first generation of 3d xpoint.

 

[TedB]

@i386 did you manage to get your hands on one ?

 

[i386]

Sadly, no

 

[rune-san]

The 4800X is probably a great SLOG device, but I don't think I would use the M.2 drive in such a scenario. It's only rated at 100GB/writes per day, which depending on how much data you have in flight for your VMs, and how many VMs you have, may not be enough.

 

[funkywizard]

After seeing some early reviews of the DC P4800X it looks like it would be ideal for a ZFS ZIL, with the only caveat being the price. My understanding is that Intel is also releasing some 16/32GB drives in M.2 form which they are targeting as consumer cache drives with much more palatable prices. With ZFS ZILs not needing much capacity, would running these 16/32GB M.2 drives just as regular block storage (i.e. independent of Intel's RST '2.0') be possible on a C612 chipset? Or would it make more sense to just pick up a 100GB S3700 and not have to wait?
For context, I am planning on this being an addition to my current 12x 7200 4tTB HDDs, 800GB Intel 750 L2ARC and 64GB of RAM for VM datastore purposes.

I can answer one part of your question -- "would it make more sense to just pick up a 100GB S3700" -- no. The DC S3700 series of drives gets top performance from 400gb (or larger) drives. 200gb is slower, and 100gb is even slower. Even if I needed very little disk space, I would get a 400gb, not a 100gb. As a bonus, the lifetime wear scales with size -- useful for a write cache like this.

 

[funkywizard]

Halfway house between optane p4800x and s3700 would be the p3700, wouldn't it?

1.6TB P3605 is commonly available on ebay with favorable performance and lifetime wear. Varies from $700 - $1000. Lower sizes don't perform as well, sadly.

Personally, an option I like is a DC S3700 (or 3600/3610/3710) paired with a hardware raid card with power backup. Specifically LSI 9271-8i (or 4i) with cachevault accessory is going used in the $300 - $400 ballpark these days (sometimes less). With writeback enabled you can expect excellent speed low latency burst writes. I.E. anything that doesn't overflow the cache will go at 1.0 - 1.2GB/s (single threaded writes / i.e. queue depth 1).

With a single drive "in raid 0" (or 2 in raid1), you should be able to flush cache ram at 400MB/s while the cache handles bursts in excess of 1GB/s.

One other benefit to the writeback cache is you'll see higher sequential write speeds even for extended writes that are well beyond the size of the ram cache. This is because many programs wait for a write commitment before writing more data. The writeback cache allows the porgram to receive that commit success much faster, allowing the software more "round trips" per second. In this common scenario, a writeback cache will allow you to use a much higher percentage of the drive's maximim performance. This also lets the SSD be written to with larger writes at any given time -- the raid card will buffer a good amount of data before sending it to the drive, and most SSD controllers work much faster writing a lot of data at once instead of a little at a time.

 

[SlickNetAaron]

Personally, an option I like is a DC S3700 (or 3600/3610/3710) paired with a hardware raid card with power backup. Specifically LSI 9271-8i (or 4i) with cachevault accessory is .

Have you used this setup yourself? I've been trying to figure out something like that for a while. Haven't played with it for a long time though.

I have heard murmurs that having the raid card with caching in front of an ssd was slower

 

[funkywizard]

I haven't used it for SLOG / ZIL. I have used hardware and software raid, with and without writeback cache, with ssds and hard drives, as nornal storage (EXF4, etc).

For that use the hardware raid with writeback cache does greatly improve write speeds in any tests I could come up with.

 

[vrod]

Never use hardware raid for a ZIL. You will be shooting your own foot in the end. I don’t know your exact use case but depending on how much it needs to write per day, you could get rid of the L2ARC (which actually is not ideal since your memory is low), then overprovision your 750 NVMe to 10GB and use that partition for ZIL. The 750 has battery backup so your data would be safer in the event of s power outage.

I personally do not recommend any ssd without battery backup or decent write endurance for ZIL, as that’s all it will be doing - writing. The write performance of your pool (with sync) will be “limited” to the write performance of the ZIL device, so if getting the S3700, get higher capacity than the 100gb one.