Removing Incomplete Multipart Uploads/chunks Using Lifecycle Policy

Situation

Any backups can be interrupted for various reasons: network issues, electricity power outages, etc. In these cases, incomplete uploads will be kept on backup storage, so this can become subject to additional charges.

Solution

For S3 storage accounts, you can configure a bucket lifecycle policy that removes uncompleted multipart uploads automatically.

Create Lifecycle Policy

1. Sign in to the AWS Management Console and open the Amazon S3 console.

2. In the Buckets list, select the name of the bucket that you want to create a lifecycle rule for.

3. Switch to the Management tab. !

4. Click Create lifecycle rule.

5. In Lifecycle rule name, enter a name for your rule.
6. Choose the scope of the lifecycle rule:
   • To apply this lifecycle rule to all objects with a specific prefix or tag, select Limit the scope to specific prefixes or tags
   • To limit the scope by prefix, in Prefix, enter the prefix
   • To limit the scope by tag, select Add tag, then enter the tag key and value
   • To apply this lifecycle rule to all objects in the bucket, choose This rule applies to all objects in the bucket, then select I acknowledge that this rule applies to all objects in the bucket
7. To filter a rule by object size, you can enable Specify minimum object size, Specify maximum object size, or both options.
   • If you specify the minimum object size or maximum object size, the value must be between 0 bytes and 5TB. You can specify this value in bytes, KB, MB, or GB
   • If you specify both options, the maximum object size must be larger than the minimum object size
8. Under Lifecycle rule actions, select the actions you want the lifecycle rule to perform:
   • Delete expired delete markers or incomplete multipart uploads

Depending on the actions you select, different options appear.

9. Under Delete expired delete markers or incomplete multipart uploads, select Delete incomplete multipart uploads. Then, enter the number of days after the multipart upload initiation that you want to end and clean up incomplete multipart uploads.
10. Select Create rule.

If the lifecycle rule does not contain any errors, Amazon S3 enables it, so you can see it on the Management tab under Lifecycle rules.