Backup/restore of ZFS filesystems with S3 (lan or internet) and restic

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

gea

Well-Known Member
Dec 31, 2010
3,677
1,452
113
DE
Currently ZFS replication and rsync with a ZFS snap as source are the classic methods for ZFS backup/restore.
The main alternative beside a local replication to preserve ACL is more and more S3 object storage with S3 based backup tools like restic because

- access performance is quite the same with a 10 TB and a 10 PB storage.
- any file is a link, accessible via authenticated https url

- S3 server can be installed in a basic single node/ single disk mode for a normal backup/restore
- S3 server can run as a cluster in x node/ y disk mode. Up from x=4, such a cluster offers high availability
(a node can fail, basically in a very simplified view something like a raid over nodes
with 4 nodes like Raid-Z1, 5 nodes like Z2 and 6 nodes like Z3)
with a load balancer, S3 remains accessible for clients under original ip when any node fails
without a load balancer, you can access another node with its ip
- S3 servers can include Site Replication, basically like a async mirror setup of two S3 servers

S3 object storage is available from Amazon AWS. As the S3 api is well documented, S3 api based storage is the standard for Cloud based backup/restore, not only from Amazon but from many Cloud or ISP providers at low costs. Many of them used the S3 compatible MinIO in the past as well as many NAS systems.

MinIO as an OpenSource S3 server is no longer an option, so we need an alternative. Most promising is RustFS, intended to be a 1:1 direct replacement of MinIO. Current state of RustFS is a 1.0.0 beta for first tests, A stable is anounced in summer 2026 (not finally confirmed). RustFS is based on Rust so it can be built on nearly any OS. Prebuild binaries are available for Linux, OSX and Windows at Releases · rustfs/rustfs (click on release for downloads).

On other systems you must build yourself. This can be tricky due to some dependencies. On problems ask AI (I used Claude) for help. To build RustFS on OmniOS, see (with a documented shell script to build) Topicbox
 
Last edited:
  • Like
Reactions: gb00s

gea

Well-Known Member
Dec 31, 2010
3,677
1,452
113
DE
ZFS vs. Object Storage

ZFS is perfect as a foundation for collaborative document editing within a LAN. It features snapshots, supports ACLs with SMB, multi-user file locking, and all the great features surrounding hybrid pools, deduplication, compression, synchronous writes, and encryption—plus flawless software RAID handling, soon to include the sensational "Any RAID" approach.

What it lacks is internet access, clustering, network replication, or a simple snapshot- and file-comparison-based sync to complement the stream-based zfs send (which relies on shared snapshots). This is where S3 Object Storage comes into play. In S3, every file is an authenticated HTTPS URL. It offers cluster modes (where one or more nodes can fail) as well as site replication—essentially a near-real-time mirror between two S3 servers.

It becomes perfect when you combine both storage technologies: ZFS locally for RAID with snapshots and as an SMB share, combined with S3 for backup/restore/migration, internet access, and site replication (high availability). ACL backup/restore can be mapped in S3, ZFS snapshots can be used as the source, and Restic provides backup snapshots, deduplication, and encryption on top of that.

My tools of choice are RustFS (S3 server), Rclone ("rsync for the cloud"), and Restic (backup with dedup, snapshots, and encryption). They are available for almost every OS and also solve the backup/migration issue between Solaris ZFS and Illumos/OpenZFS.


 
Last edited:

gea

Well-Known Member
Dec 31, 2010
3,677
1,452
113
DE
Assisted by Claude, I could finally create a build script for RustFS 1.0 beta8 on OmniOS
(took me days due ~1h build time) that reflects needed modifications to run current RustFS on Unix/Illumos

RustFS 1.0 beta 8: ok
RustFS webconsole: ok (tested login)
rc management tool: ok
rclone: ok (rsync for cloud)
restic: ok (backup restore with encryption, snaps and dedup)
acl save/restore: ok
any OS save/restore: ok (ZFS snap based, also Solaris <-> Illumos/OpenZFS)
Console v0.1.7 embedded: ok -> http(s)://ip:9001