CacheCade - using RAID0 for mostly read-only application?

[chrispitude]

Hi all,

I'm building a Linux-based home fileserver. I have an LSI 9267-8i on the way, along with six 3TB WD Red drives in RAID6. The server will be primarily read-only (Plex server), and will idle most of the time. However, the array will be occasionally written to for OS upgrades, installing new software to play with, browser cache, and so on.

I'd like to run two 250GB SSDs in RAID0. I see that LSI doesn't recommend this because if either drive dies, the array goes down until the drive is replaced.

How does CacheCade work? When there's a burst of write activity to the SSD, how long does that data remain ONLY on the SSDs without also being copied to the HDDs? If the controller flushes to the HDDs as soon as usage permits (regardless of whether it remains on the SSDs or not), then the exposure to SSD failure will be quite low for my application.

Then there's the question of, is it worth it? I was planning on using two 256GB Samsung 850 Pro drives, which are regarded as being decently reliable. Will 2 x 256GB drives in RAID0 provide tangible benefits versus a single 512GB drive?

 

[i386]

The last time I read the LSI/Broadcom documentations & manuals cachecade supported raid 0/single drive (read cache only) and raid 1/10 (read and write cache).

CacheCade read caching works by keeping track of the requested blocks. Most frequent accessed blocks are copied to the cachecade layer. Whenever this data is requested from the host the controller reads it from the cache and returns it way faster than the underlying hdds could do.
For writing the cachecade algorithm stores the data in the cachecade layer and flushes it to the hdds every x seconds (I know only from adaptec controllers that you can change this value). Before flushing it to the hdds the controller orders the data blocks by address and turns small, random writes into larger, sequential (= faster) writes. (This is how the activity leeds on a supermicro chassis with a cachecade cache looks like:

serveron the right side is using cachecade).

"Is it worth it" is hard to anwer. If you have enough threads and high queue depths (ex enterprise workloads like vdi or huge databases) then it's abolutely crazy and totally worth it. For a homeserver serving mostly large files it's suboptimal because you don't have enough concurrent acceess.

Two ssds in raid 0 for read caching will be faster than one 512gb ssd.

I would stay away from the 850 pros (or other consumer ssd) as the caching is a sustained workload and the performance would degrade drastically. Use write optimized ssds like the intel s3700 family or hgst husmm ( the last m stand for mixed/write intesive workloads) ssds instead.

 

[msg7086]

Worth ... what? 6 disks RAID 6 won't be too slow. Maybe not a direct comparison, but we have a 12 disks 4TB/ea Z3 array that writes at 600MB+/s. You need to tell us what specific heavy-lifting pressure you want to put on those disks. We have bunch of VMDK living on an NFS share so we do use SSD for sync write cache, but for daily async write I don't see much demand on SSD caching.

 

[chrispitude]

First, thanks to both of you for replying. I'm coming from an aging 3ware SE9650-4LPML with 4x2TB WD Red drives in RAID5. I have no experience whatsoever with the LSI controllers, so everything you say is valuable to me.

The last time I read the LSI/Broadcom documentations & manuals cachecade supported raid 0/single drive (read cache only) and raid 1/10 (read and write cache).

This information is frustratingly hard to find, even in LSI's own documentation! I looked through a bunch of PDFs, including both their controller and CacheCade sales sheets and manuals, and I could not find anything about supported RAID levels for read and read/write cache. Do you remember where you read this?

"Is it worth it" is hard to anwer. If you have enough threads and high queue depths (ex enterprise workloads like vdi or huge databases) then it's abolutely crazy and totally worth it. For a homeserver serving mostly large files it's suboptimal because you don't have enough concurrent acceess.

Plex certainly won't benefit at all. What I'd like to speed up is all OTHER use of this machine, such as occasionally compiling software (using multithreaded make), running my Windows 7 VM for odds and ends, and generally just puttering about in Linux. In particular, compiling software would benefit from a read/write cache; this is much slower on my 3ware RAID 5 array versus a single HDD of the same kind. I imagine the Windows virtual machine would benefit too.

I would stay away from the 850 pros (or other consumer ssd) as the caching is a sustained workload and the performance would degrade drastically. Use write optimized ssds like the intel s3700 family or hgst husmm ( the last m stand for mixed/write intesive workloads) ssds instead.

Thank you! I originally thought CacheCade flushed writes to the HDDs as soon as possible. Now that I see the data lives (only!) in the cache longer than I thought, so I will investigate the drives you suggested.

Worth ... what? 6 disks RAID 6 won't be too slow. Maybe not a direct comparison, but we have a 12 disks 4TB/ea Z3 array that writes at 600MB+/s. You need to tell us what specific heavy-lifting pressure you want to put on those disks. We have bunch of VMDK living on an NFS share so we do use SSD for sync write cache, but for daily async write I don't see much demand on SSD caching.

My current 3ware RAID 5 array does well with I/O transfer rates, but it lacks for random access operations like software compiling and virtual machine vdisk access. But again, my experience is all with 3ware; I know nothing about the LSI stuff.

 

[msg7086]

Thanks for clarification.

SSD cache is generally considered as a non-volatile fast storage to be used to compensate the slow HDD I/Os. If, as you might think, the controller writes data almost immediately, then the caching effects basically goes to ~zero.

Random access sync writes on RAID 5 or 6 is pretty much the same as single disk, or worse; and async writes put your data on risk (however allow access optimization happening before physical write). Using a non-volatile SSD turns sync writes to async writes safely (similar to DRAM+BBU). If the SSD cannot be trusted, the effects are voided.

Given your workload, I'd recommend you just put your other stuff on SSD alone, while leaving your RAID 6 for media storage only.

(Or get a better card with large RAM and a BBU.)

 

[ServerSemi]

So if you use ssd cache and the ssd dies you lose your full raid data or just whats inside the ssd at the moment?

 

[i386]

Maxcache & cachecade don't have an option in the (web) management software to use raid 0 on the ssds for write caching*. It should lose only the data in the ssd cache in the worst case scenario.

*Don't know if the cli tools can be used to setup the ssds in raid 0 for write caching.