If a folder to be archived has a particularly large collection of contents, the machine can easily run out of memory whilst archiving the folder (he says, having watched his test backups fail 😞 ). Let's find a way of limiting the archive size.
Do we do this by either:
a) Limiting the amount of files in a given archive (e.g. max. 100 files per archive, then create a new one for the next 100, etc.)
b) Splitting the archives by file size
a) is the easiest way, and requires less modification to the existing workflow in order to implement. However, it is only ever a best-guess system. Even if the limit were as low as 5 files/archive, 5 20GB files would still break a machine somewhere...
b) is more reliable in terms of not breaking any machines, as the file size would always be consistent, and perhaps user-configurable. However, it does present the issue that in order for a folder to be recovered, the entire collection of archives that make up a given folder would have to be downloaded. Is this an issue? We'd also have to find a way of keeping track of how many archives make up a subfolder, and keep them grouped together.
Still, would all be worth having...
If a folder to be archived has a particularly large collection of contents, the machine can easily run out of memory whilst archiving the folder (he says, having watched his test backups fail 😞 ). Let's find a way of limiting the archive size.
Do we do this by either:
a) Limiting the amount of files in a given archive (e.g. max. 100 files per archive, then create a new one for the next 100, etc.)
b) Splitting the archives by file size
a) is the easiest way, and requires less modification to the existing workflow in order to implement. However, it is only ever a best-guess system. Even if the limit were as low as 5 files/archive, 5 20GB files would still break a machine somewhere...
b) is more reliable in terms of not breaking any machines, as the file size would always be consistent, and perhaps user-configurable. However, it does present the issue that in order for a folder to be recovered, the entire collection of archives that make up a given folder would have to be downloaded. Is this an issue? We'd also have to find a way of keeping track of how many archives make up a subfolder, and keep them grouped together.
Still, would all be worth having...