Article ID: kb00235Last Modified: 22-Aug-2024

Small-sized files uploaded to 'S3 Standard' instead of a storage class specified in the backup plan settings

Situation

A backup plan that is set up to upload files to a specific storage class in Amazon S3...

… uploads certain files to S3 Standard storage class instead, seemingly against the specified rule:

Explanation

The situation described above is expected behavior of the MSP360 Backup software.

It is a result of a cost-saving measure implemented with per-object billing properties of different storage classes in Amazon S3 in mind.

Amazon S3 has several storage classes beyond the "S3 Standard". Each storage class offers a cheaper per-object storage cost at the expense of limitations in data retrieval time, speed, minimum storage duration charges, and minimum storage capacity per-object charges.

However, while the upfront per-object storage costs are lower in other storage classes (that are not "S3 Standard"), when all the cost-related factors are considered together (especially, the minimum storage capacity per-object charges) - there exists a minimum file size threshold beyond which it would be more cost-effective to store a small-sized file in "S3 Standard" storage class than in another storage class that was initially intended for backup purposes.

This is relevant for the new backup format plans but especially so for the file-level backups made in the current backup format. Small-sized files could be numerous in the file-level backup plan's source and it's reasonable to store such files in a way that would be least impactful for the monthly AWS storage services billing. It's a cost-saving measure.

By design, the MSP360 Backup software uploads any given small-sized file to "S3 Standard" storage class if size of such file is smaller than:

  • 71234 bytes (~71.2 KB) for a file initially intended by the backup plan settings for the "Standard-IA" storage class;

  • 56987 bytes (~56.9 KB) for a file initially intended by the backup plan settings for the "One Zone-IA" storage class;

  • 13890 bytes (~13.8 KB) for a file initially intended by the backup plan settings for the "Glacier" storage class;

  • 131072 bytes (~131 KB) for a file initially intended by the backup plan settings for the "Glacier Instant Retrieval" storage class;

  • 9602 bytes (~9.6 KB) for a file initially intended by the backup plan settings for the "Glacier Deep Archive" storage class.

This rule is applicable only to small-sized files that might be present in the backup plan's source dataset. Any source files with sizes equal to or above the aforementioned ones are to be uploaded with no caveats to the storage class specified in the backup plan settings.

https://git.cloudberrylab.com/egor.m/doc-help-kb.git