Hey guys,
I have currently a problem. My Internet in my Home-Office is slow. Here I get max 100 MBit down and 10MBit up. But most of my work, I have in the Datacenter on a TrueNAS ZFS. I tried to solve the problem with a Home-Server. To have some kind of files at home (for example static data like Intros, Footage) and to sync the working Data with a sync service.
To Sync files I tried Syncthing, it didn't work for me very well. Then I tried Resilio. It worked with normal files very well. With large files, it had some problems, but the most issues I got with a large number of small files. Like npm / node Packages. Resilio often deleted some of the files right under my butt.
Another problem was Unraid for me, with the one "cache" drive / nvme. As long as I have the Data on one fast NVME drive, when the drive dies, the data is goneski. When I run the mover hourly, and write large files to the Fileserver, the written files are moved away, as well as the directory. All of this is Crap.
What I'm looking is some kind of cache for my fileserver. Something like the ZIL / SLOG layer for my Fileserver. I want to see everything, that is on my Server in the Datacenter. But if I want to get a file from there, if it is on my cache Server, the Data will be transferred from there. If not, then from the Server in the Datacenter. Something like IPFS is doing already.
Does anyone has a solution?
I have currently a problem. My Internet in my Home-Office is slow. Here I get max 100 MBit down and 10MBit up. But most of my work, I have in the Datacenter on a TrueNAS ZFS. I tried to solve the problem with a Home-Server. To have some kind of files at home (for example static data like Intros, Footage) and to sync the working Data with a sync service.
To Sync files I tried Syncthing, it didn't work for me very well. Then I tried Resilio. It worked with normal files very well. With large files, it had some problems, but the most issues I got with a large number of small files. Like npm / node Packages. Resilio often deleted some of the files right under my butt.
Another problem was Unraid for me, with the one "cache" drive / nvme. As long as I have the Data on one fast NVME drive, when the drive dies, the data is goneski. When I run the mover hourly, and write large files to the Fileserver, the written files are moved away, as well as the directory. All of this is Crap.
What I'm looking is some kind of cache for my fileserver. Something like the ZIL / SLOG layer for my Fileserver. I want to see everything, that is on my Server in the Datacenter. But if I want to get a file from there, if it is on my cache Server, the Data will be transferred from there. If not, then from the Server in the Datacenter. Something like IPFS is doing already.
Does anyone has a solution?