Could ZFS copies=2 be the answer for this build?

mulot

New Member
Mar 23, 2022
4
0
1
Hello,

I'd like to replace my current setup with a low-power ZFS homeserver.

The setup right now is very modest with Debian on a Odroid HC4 with:
  • 1x 1TB SATA SSD containing boot, data, containers
  • 1x 8TB HDD (mirror/backup of SATA, some more backups, some highly expendable data like a steam cache).
The advantage of this is typical ~3W idle power consumption (disk is usually spun down) for a reasonably responsive gigabit nextcloud server.
The disadvantages are low single core performance, no redundancy (but I'm good at 3-2-1 backups and test/monitor them), no checksumming, no ECC, no room to grow and certainly no extra resources for VMs.

I have a mothballed Dell T20 and intend to run Debian with ZFS on it. This machine runs at ~13W idle as listed with disks spun down:

Intel E3-1225v3
16GB ECC
4x SATA with older 4TB CMR WD Reds
Mellanox ConnectX-3 10GbE
+one unused 16x PCIe 3.0 slot without bifurication

That last line leads me to my question:

I need some reasonably large and energy efficient SSD storage without spending a fortune on hardware that is about to become obsolete. The plan is to replace the T20 once DDR5 ECC setups become affordable.
So no dual M.2 add-in card with a PCIe switch, no power hungry HBA and preferably no SATA SSDs that have no hope of keeping up with 10GbE in the future.

The solution I came up with is a single, large, possibly lightly used NVMe enterprise SSD in a direct PCIe to M.2 or U.2 adapter.

I'd split this into two ZFS datasets. One (25% of disk size) with copies=2 to protect important data against bitrot, the other (remaining 50% of disk size) as a simple dataset for expendable data. The first dataset would get snapshotted to mirrored WD Reds and backed up offsite.

cons:
  • I will have to restore from a snapshot/backup if this one SSD fails catastrophically.
  • Write performance of the first dataset (which isn't write heavy to begin with) will be reduced by 50% but likely still better than SATA for 10GbE.
  • Any data written to the first dataset would count twice towards the TBW of the SSD, but I'm expecting <10TB per year on a drive rated for at least 2500 TBW.
pros:
  • I'd get fast storage with checksumming using a single device.
  • I could reuse the SSD in a future server and buy a second one as a mirror then.
  • Idle power consumption would hardly go up.
Is there a flaw in my plan?
 
Last edited:

ttabbal

Active Member
Mar 10, 2016
776
209
43
45
Even a good SSD is unlikely to keep up with 10Gbe. If you trust your backups and don't mind a little down time, there's nothing wrong with the idea. I don't know that I'd bother with copies=2 in that case though. You have checksums and verification, along with good backups. It doesn't hurt anything really, other than slightly increasing the data written and slowing performance a bit though.