The servers' apps backups archives are about 120GB. The files are about 700GB. The backups for the apps are synced daily with my local machine. So I download 120GB daily. Then the files are synced incrementally to the same machine and now they are about 2.2 MILLION files.....takes some 10 hours to sync since it has to read the timestamp of each file in order to know what files need to be updated. Then the backup archives and the files, are mirrored to another external HDD.
All in all these backups take a long time and a lot of storage space. The greatest offenders are Peertube for file storage, although the core app is only 2GB in size, but the files are about 600GB. However Friendica and Matrix are by far the greatest offenders in terms of the number of files, over 1.5 million I'd guess. These are thumbnails, media files, and such. The Matrix server has a 10GB database already, same as Friendica.
Is this scalable? I guess it is. If this gets too crazy I'll stop backup the cached thumbnails for Matrix and Friendica since these are the ones that take up so long to sync due to being so so so many of them. Maybe will do so for thumbnails for Peertube as well. After all these backups are just in case of a fatal failure of the server. And if that happens we can deal with generating the thumbnails again for Peertube and the like....
clacke: exhausted pixie dream boy 🇸🇪🇭🇰💙💛 likes this.
Tio
Unknown parent • •Tio
Unknown parent • •