Napp-it cs web-gui for any OS server and cluster

[gea]

Not everyone has a cluster with a dozen servers. Regardless of the OS, whether it’s a 1 GB mini IoT device or a petabyte system, whether using Storage Spaces, S3 object storage, or ZFS—not everyone wants to manage everything centrally with simple, fast, encrypted file sync or any-to-any ZFS replication (e.g., via GitHub - guenther-alka/cs-stream: Encrypted TCP stream transport for ZFS replication (replaces netcat)).

However, that is precisely our target audience with napp-it cs. S3 integration, in particular, is a hot topic right now. It allows you to publish local files to the internet easily and securely (e.g., setting up a private cloud with RustFS). RustFS can also run independently in a "set-and-forget" mode to keep two servers or buckets in bidirectional sync. High availability and real-time backups over the internet or LAN, plain and simple.

What’s currently missing is the bridge between "shared multi-user SMB with ACLs" and object storage, which inherently lacks multi-user locking with file ACLs. This is where I'm stepping in with the cs-sync module.

With cs-sync, my goal is to keep two folders—such as an SMB share on a NAS and an S3 share—in sync in real time using event triggers (reacting instantly on change, without needing full file comparisons), including SMB ACLs. Regardless of the OS, this is now possible thanks to Go, with the option for real-time automated backups of changes over the network to a backup system (see GitHub - guenther-alka/cs-sync: Realtime bidirectional folder sync for ZFS hosts (napp-it CS)).

I would love to get your feedback or suggestions on this concept! A first preview featuring napp-it cs and real-time sync is coming soon. Anyone interested in serious testing should consider getting Claude Pro along with the Filesystem and Desktop Commander extensions/agents (on Windows or other OSs via Plink/SCP SSH access for Claude to inspect scripts, logs, and members) for analysis, stress testing, audits, etc.

 

[gea]

3 object storage features eg RustFS (current 1.0 rc11 beta) offers

- Internet Access for files on a ZFS fileserver
- webconsole for remote file access
- backup target with encryption, snaps and dedup (any OS backup/restore)
- set and forget Site/Bucket bidirectional replication between RustFS hosts

S3 lacks:
- multiuser access with file locking and ACL

So the conclusion is:
To use SMB/ZFS and S3 with same data on either service, you need a bidir sync service
that mirrors data and at least folder ACL with inheritance between S3 and SMB shares

This is what I included in napp-it cs 26.06 07.29 rc
Update: Menu About > Frontend Update

-missing private or osx menus
-osx setup: https://www.napp-it.org/pdf/apple_osx_en.pdf

Go based multi os cs-stream job

Releases · guenther-alka/cs-stream

Encrypted TCP stream transport for ZFS replication (replaces netcat) - guenther-alka/cs-stream

github.com

-ZFS replication or filebased sync (via rclone)
-encrypted, buffered or io limited tunnel any to any

Go based multi os cs-sync background service

Releases · guenther-alka/cs-sync

Realtime bidirectional folder sync for ZFS hosts (napp-it CS) - guenther-alka/cs-sync

github.com

-event driven local uni/bi service with folder ACL support for realtime sync (eg SMB and S3 shares)
-event driven remote uni sync service with folder ACL support for realtime backup (encrypted)

Status ample (current and last 10m/h/day/week)
Pool, Cap, Disk, IO, RAM, Jobs, Sync, Last

Main menu "Servergroup" shows running sync services on members

Bidir realtime sync (optionally with an additional realtime backup primary filesystem > backupserver)

 

[gea]

The new event driven (on file modifications) realtime cs-sync in napp-it cs is not only a perfect method to keep local S3 object storage and local SMB shares bidirectionally in sync but a new approach for realtime sync between data and backup, best paired with ZFS snaps on destination to have backup versions eg snaps hourly-keep 24, daily-keep 24, ..

remains an important aspect:

Behavior on slow or unstable connections

Handling disconnects and slow links is a core design goal, not an afterthought -- local-only 1.x never had to deal with this at all. See cs-sync-2.0-design.info section 3 for the full rationale.

• Disconnects (flaky WLAN, link flaps) are normal operation, not errors. No timeout, no alarm, just a log line. Every change that can't reach the remote right now goes into a persisted pending queue that survives both process restart and host reboot.
• Coalesced per path. A file that changes 100 times while offline is transferred once, latest state only -- the queue stores "this path needs sync," not an event log.
• Reconnect uses exponential backoff (1s -> 2s -> ... capped at 5min) before a retry counts against a file's failure count, so a flapping link can't burn through the retry budget in seconds.
• Backpressure, not blocking. The filesystem watcher never blocks on the network -- it can produce changes faster than a slow link can carry them indefinitely; the sender just drains the queue at whatever speed the link allows.
• Queue depth is unbounded by design. A deep queue just means "slow link + lots of data," which has to keep working. The retry limit (10 attempts, see backoff above) applies only to individual files that keep failing for their own reasons (permission errors, unreadable source, etc.) -- such a file is quarantined and logged so it stops blocking the rest of the queue, everything else keeps syncing.
• Atomic writes + end-to-end hash. Every transfer goes to a temp file, is hash-verified, then renamed into place -- a drop mid-transfer never leaves a half-written file that size/mtime could later mistake for current.
• Torn-copy detection. If the source changes while a slow transfer is still reading it, a before/after size+mtime mismatch discards the copy and re-queues it -- no locking needed.
• --bwlimit throttles a remote leg so an initial full sync over a narrow/shared link doesn't saturate it for everything else using the same connection.

Not yet implemented: delta transfer for large partially-changed files (a changed file is always sent in full -- see Known gaps above) and a queue disk-space warning threshold for very long outages. (an option for a next release for very large files)

 

[gea]

Coming from ESXi, there are two things I'm used to / that matter to me:

1. When I take a ZFS snap of a VM storage, it shouldn't just contain the virtual disks but also the VM configuration, so I can restore a VM from a ZFS snap (creating a snap, unlike PBS, is instant with nothing to copy, and even zfs send over LAN is much faster). On ESXi that was a .vmx file in the VM folder. On Proxmox, VM configurations live in /etc/pve.

Solution: I've added a snap option "include regular folder" to the snap job. This lets me capture the state of /etc/pve at snap time along with the snap.

Another use case would be Windows, to have c:/Users/"me" with personal documents on NTFS included in the ZFS snap.

1. Consistent VM snaps
   A ZFS snap is always consistent for the host managing the ZFS pool, thanks to copy-on-write and ZFS's internal snapshot handling. It looks completely different for VMs and their guest filesystems on ZFS. A ZFS snapshot on the host doesn't care whether their filesystems stay consistent. They end up in the snap in the same state as after an unexpected power outage. The risk of corrupted data can be reduced by making sure no write operations happen during the snap (VM freeze). Proxmox has various methods for that. I've now integrated zfs snapshot into a Go-based tool that automatically provides various freeze options to reduce the likelihood of a corrupted VM in the snap. The only way to avoid it entirely would be to snap while offline, or to also save RAM state — but with current RAM sizes that's very slow.

Go tool (included in current napp-it CS in autosnap and replication jobs)
GitHub - guenther-alka/cs-freeze4snap: Consistent ZFS snapshots for Proxmox VM/LXC guests - freeze if possible, snapshot regardless

 

[kapone]

@gea - Once again, my friend, you're trying to do things in your product that can be solved...well...in other ways. Like Ansible.

Proxmox (it is a major target for ZFS based systems, please don't get me started on the hot mess that's Truenas...) just...doesn't have enough to be comparable to vSphere/ESX. Long way off. As far as Solaris/OpenIndiana/OmniOS/whatever are concerned, they're dead and buried.

So. And So. That's where we stand. Anybody trying to play with Proxmox at scale isn't trying to use napp-it (or anything like it). They simply build their automation pipeline with tools that can do it (and there's many, choose your poison).

I'm a prime example of what I just said. I run Proxmox...at scale...is debatable, if two sites, each with ~28 nodes and 3 different clusters don't count. I had to build my own tooling, because Proxmox...well, just doesn't have it. It's a good product, but way off from the likes of ESXi.

 

[gea]

There's more than one way to skin a cat.

 

[kapone]

There's more than one way to skin a cat.

And?

Edit: p.s. VMs are fairly dead. Containers are where it's at as we speak. So, this may be a moot point.

 

[gea]

The only real VM you need in Proxmox could be a storage vm eg TN, OmniOS or Windows (napp-it cs is there to avoid this) or a Windows client VM.