Self-hosted file syncing with selective sync without database. Am I asking too much?

[Simplex]

I've been looking for a solution for this for quite some time. I think that what I'm looking for doesn't exist but I thought I should check with you bright chaps before I throw in the towel.

I'm looking for a self hosted file serving solution a la Dropbox which fulfills the following requirements.

• Uses existing local directory structure *not* an internal database.
• FOSS
• Allows "selective sync" or whatever you want to call it. Where the files can be browsed on the local filesystem but are only actually transferred from the remote when needed (like Dropbox smart sync).

Anyone have any suggestions?

 

[i386]

(s)ftp(s)?

 

[Sean Ho]

I'm not too familiar with DropBox smart sync, but the use case of being able to see filesystem metadata (e.g., browse directory listings) quickly but only download file contents on demand is pretty much what network filesystems like NFS and SMB do, yes?

I'm also not entirely clear what you mean by #1 -- do you mean preferring a filesystem interface rather than object storage a la S3? FS metadata like directory structure is essentially an indexed table lookup just like a DB, and some fs do exactly that, e.g. ReFS, btrfs. Perhaps you're referring to the index DB used by file sync software, e.g., syncthing? This is necessary in order to determine quickly which files have changed. You might consider filesystem synchronization using snapshots, e.g., zfs or btrfs.

 

[amalurk]

I've been looking for a solution for this for quite some time. I think that what I'm looking for doesn't exist but I thought I should check with you bright chaps before I throw in the towel.

I'm looking for a self hosted file serving solution a la Dropbox which fulfills the following requirements.

• Uses existing local directory structure *not* an internal database.
• FOSS
• Allows "selective sync" or whatever you want to call it. Where the files can be browsed on the local filesystem but are only actually transferred from the remote when needed (like Dropbox smart sync).

Anyone have any suggestions?

NextCloud Virtual Files feature? From the website " The new Virtual Files, by contrast, always shows all files, mostly manages itself and works on a per-file level. When you go into the Nextcloud folder, you can work with the files like normal – but the first time you open a file, it will have to be downloaded. Afterward, it will remain available offline. This initial download comes with a short delay that depends on network speed and file size, but of course the files take no space until you need them! "

I have not tried this feature however. I only use NextCloud on a remote device that I rsync my home NAS to for an offsite backup.

 

[Simplex]

Thanks for the replies everyone.
I didn't realise that ownCloud and NextCloud both offer the virtual filesystem I was after and you can mount an "external" source for storage so I can use my local filesystem structure rather than a db.
I am playing around with this now but it seems like this is exactly the solution I was looking for.