What problem your new feature would solve
Storing Data in S3/Object Store backend to utilize redundancy and cheaper storage costs when hosting the server in the cloud.
And using the untrusted node feature the cloud instance can serve as a secure centralized reliable distribution point that can also serve as backup and provide increased bandwidth.
How or why you think it is generally useful (i.e., not just for you)
This would allow:
- To more efficiently integrate syncing in a hybrid cloud environment where dynamically cloud nodes can be added without unnecessarily duplicating the data within the cloud provider.
- To reduce costs (block devices are more costly than S3 buckets) when one or more of the nodes are hosted within cloud environments like Amazon AWS, Scaleway, ...
What alternatives or workarounds you considered
- Using S3fs as an overlay for the S3 to mount it as block storage so that Syncthing can write into S3 using this as a glue layer in between. => But this causes issues with caching as well as the possibility for timeouts and data loss. Esp. When a lot of data is accessed and the local disk is way too small for the data stored in the S3. It also has issues with multiple instances (like when the syncthing web-ui is deployed in on an HA environment or k8s cluster) accessing the same S3 bucket simultaneously...
- Using s3backer kinda works, it solves a lot of the issues of s3fs, but doesn't allow concurrent access similar to s3fs.
- Using Nextcloud with S3 backend instead. => Currently quite buggy and unusable esp. for very large files. Also, costs are extremely high because the S3 is never cleaned up and old data is still present.
- Using Seafile => Currently bugged when used together with Backblaze...