SSD Caching for NVR

[Wader]

Currently using zfs as storage for a dedicated Network Video Recorder system. This means I have a lot of drives spinning 24/7. My thought -- mirror 2x 1TB SSDs as the l2arc. Can it be configured to somehow spin down the array most of the day, recording to SSD, and then dump everything to the disk array either on a fixed time based schedule or %full schedule? Video incoming at 600GB per day would fit in the 1TB cache to be flushed to disk once a day.

Can zfs be configured to do this, or can another file system?
The NVR application saves files in the structure below:
storage\camera-name\date1\h264\files.mkv
\date2\h264\file.mkv
... 30 days saved
\camera-name2\date1\h264\files.mkv
\date2\h264\files.mkv
... 30 days saved
\camera-name ... currently 10 cameras and growing

 

[Evan]

Why not just setup daily write to SSD and copy to your array once a day ?
I assume the video recording software should be able to handle that?

 

[T_Minus]

L2ARC is a read cache.

 

[dandanio]

First of all, you wanted to say SLOG instead of L2ARC.
Second of all, your logic is right, but your tools and terminology is wrong: You can create a 1TB STRIPED VDEV (RAID0 EQUIVALENT) and dump your data there. At midnight, snap the zvol, import the "cool" zpool with the spinners, send the zvol with zfs send, unmount the spinners and remove the snapshot. Done. Makes sense?

 

[Wader]

Thanks for understanding what I meant, and not exactly what I said
Evan, dandanio: I don't think that will work as I left out a detail of viewers need to be able to see and search through the full 30 days of footage.The NVR software is managing the archive to maintain 30 days recording for each camera. If my write is set up to the SSD (Evan), then the NVR software doesn't see or manage the videos on the hdds. Also, in the case someone remote wants to review and search footage using the NVR software's Android or iPhone application, if I've set them up to see the SSD directories they can only see the current day one footage. If I set them up to see the hdd directories they see the previous days/weeks footage, but not the current day. I think the same problem exists for dandanio's solution.
I was thinking mirror of the two SSDs so that when one SSD fails it's still recording with the other SSD.

 

[bateau]

Your NVR would need some kind of transparent cache if I understand it correctly, where the dual tier system you need would be hidden from the NVR. You want HSM (Hierarchical storage management - Wikipedia). Not having any details of how your NVR talks to the storage, you MIGHT get lucky with openarchive.
But (and I know that this might not help you), there are NVR systems out there that have dual tier storage handling built in. You may want to look at another NVR also. Even BlueIris has some form of it.

 

[Wader]

Thank you bateau, I'll look into HSM and openarchive. I'm using Xeoma for NVR. I should read up on BlueIris too though.

 

[bateau]

Not sure if you thought a bout it, but please take a disk with a high DWPD/TBW (you need about 1100TBW if you want to hold on to the disk for 5 years). Most consumer disks will not last long with that kind of load.
And with regards to power consumption, there are spinners out there that use lower power than SSDs, but they also have write endurance limitations.
Some comparisons:
* Intel DC S3610 1.6TB (see ebay, 10700TBW, 7W)
* a striped set with 2 WD RE disks (any capacity), (cheap, each 540TBW, 8W total)
* Samsung 860EVO 2TB ($, 1200TBW, 4W)
* Samsung 970Pro 1TB ($$, 1200TBW, 6W)