Amazon S3 compatible ZFS cloud with minIO

gea

Well-Known Member
Dec 31, 2010
2,520
852
113
DE
I have put some work into my "Cloud-Filer" concept.
This is an approach to combine a regular filer use of documents that are edited via SMB locally with a Cloud S3 access of same files in a multiuser environment from the Internet.

Problem:
If you allow a cloudsync while files are edited/open or try to open locally while a file is uploaded via S3, a corrupted file may be the result.

Solution:
-Use two filesystems, one for local access, one for S3. Keep needed files or folders in sync (one way or two way based on last edited stamp) via rclone or rsync or other sync tools on demand or based on a timetable and snaps.

-Use ZFS dedup to save datablock only once
- Add a NVMe mirror as a special vdev to hold the dedup table

Problem:
SMB allows authorisation and authentication. You need this on S3 as well based on same users

Solution:
- S3 user/group management that is in sync with SMB user/groups
see MinIO | Learn how to configure MinIO for multiple long term users

I have added this to napp-it 20.dev

s3user.png

Problem
You need policies on S3

I have added policy management:
policies.png

Problem:
you need to create policies per bucket (none,read,write,readwrite)

create_policy.png

Problem:
You need to edit or add these policies later
edit.png

 
Last edited:
  • Like
Reactions: gb00s, sth and Lix