S3 Retention Policy Configuration and Enforcement

Retention policy on S3 is two separate systems, Lifecycle rules and Object Lock, and they solve two different problems. It's two separate systems, Lifecycle rules and Object Lock, and they solve two different problems. Lifecycle rules decide how long you pay for data and at what storage tier. Object Lock decides whether anyone, including an admin having a bad day, can delete or edit that data before its time is up.

Mix those two up and gaps show. A lifecycle rule that quietly expires audit logs at 90 days isn't a retention policy, it's a shredder on a timer. And an Object Lock config with no expiration date isn't a cost strategy, it's a bill that never stops growing.

The regulations don't leave much wiggle room here either. GDPR sets a ceiling: you can't hold personal data longer than the reason you collected it justifies, and any minimum retention floor usually comes from some other sector-specific law, not GDPR itself. PCI DSS punishes you for holding onto cardholder data past the transaction window, since every extra day it sits there widens your audit scope. SEC Rule 17a-4, FINRA, and CFTC rules want tamper-proof retention that a normal backup simply can't provide. And the FCA flips the problem around: delete records too early and you're just as non-compliant as if you'd kept them forever. Two directions, one deadline, no room for guessing.

How Lifecycle rules work: automated tiering and expiration

A Lifecycle configuration is a set of rules attached to a bucket. Each rule gets scoped by a prefix or a tag, and fires automatically once an object hits a certain age. There are two kinds of actions here, and they do very different jobs.

Transition actions move objects to cheaper storage as they age. Expiration actions delete objects permanently once their window closes. Simple in concept, but AWS puts hard floors on how fast you can move objects down the tier ladder: Standard to Standard-IA needs a minimum of 30 days after creation, Standard-IA to Glacier Flexible Retrieval needs 60 total days, and Glacier to Glacier Deep Archive needs 180 days. S3 won't let you skip ahead of those tiers.

That looks like this for compliance logs, based on the pattern in critical-cloud.ai's June 2026 guide. Days 0 through 90, logs sit in Standard or Standard-IA, fully queryable, cheap enough to retrieve on demand. Days 90 through 365, they move to Glacier Instant Retrieval, still available within milliseconds but at a fraction of the cost. Past day 365, they drop into Glacier Deep Archive, retrievable within hours, the cheapest tier AWS offers. Then at day 2,555 (seven years), the objects expire entirely, which satisfies FCA log retention for most record categories.

Prefix scoping is the decision that actually matters here. A single bucket-wide rule almost never fits, because different data categories carry wildly different retention needs. Application logs might only need 30 to 90 days. Security logs like CloudTrail or VPC Flow Logs usually need 90 days live before archiving. Access logs from an ALB or API Gateway follow a similar pattern, 30 to 90 days online, archived only if compliance demands more. Debug and trace logs are the odd one out: high volume, short shelf life, often gone within 7 to 14 days.

Versioned buckets add a wrinkle. Without a non-current version expiration rule, every overwritten object version just sits there racking up storage cost forever, silently. And every bucket, versioned or not, should include an AbortIncompleteMultipartUpload rule. Failed uploads leave orphaned parts behind, and those parts cost money with zero benefit (the Ceph RGW guide from March 2026 makes the same point, and it holds just as true for S3 proper).

One more practical detail: Lifecycle rules aren't instant. According to GeeksforGeeks documentation, it can take 24 to 48 hours for a new rule to kick in and start migrating older objects. And to be clear about the boundary here: Lifecycle rules move and delete data on schedule, but they don't stop anyone from deleting or editing that data early. That's a different job entirely, and it belongs to Object Lock.

How Object Lock works: WORM protection and mode selection

Object Lock stops objects from being deleted or overwritten for a set period. It's the right tool for audit trails, financial records, anything that has to survive untouched no matter who has admin access.

Object Lock has to be enabled at the bucket level before it can protect any objects.

The mode decision is the one that shapes everything downstream. Governance mode blocks deletion and edits unless a principal holds the s3:BypassGovernanceRetention permission. That flexibility is handy when admins need a way to fix genuine mistakes, but it comes with a real weakness: one identity holding that permission can release a hold, shorten the retention date, and delete the object, all in a single sequence. For anything that needs a second set of eyes before deletion, that's a gap, not a feature.

Compliance mode closes that gap completely. No one, not even the root account, can touch a locked object before its retention period runs out. If the requirement is true immutability, this is the mode that actually delivers it.

Retention itself comes in two flavors. Fixed retain-until-date sets a calendar date on an object, useful when the clock starts the moment the object lands in the bucket. Legal holds work differently: they're a binary switch, on or off, no expiration built in, and protection ends the instant the hold is lifted. Don't confuse the two, they behave nothing alike.

For the regulatory heavy-hitters, FINRA Rule 4511, SEC Rule 17a-4(f), CFTC Regulation 1.31, Object Lock in compliance mode paired with solid audit logging is the approach assessed as suitable for those requirements. A standard backup, however frequent, doesn't cut it.

Setting up a seven-year compliance bucket is fairly direct. Create the bucket with Object Lock enabled, then apply a default retention policy with mode set to COMPLIANCE and a seven-year retention period. From that point on, every object uploaded locks for seven years, no exceptions.

A bucket default is a suggestion, not a guarantee. If someone uploads an object with its own Object Lock parameters specified, that overrides the bucket default entirely. To actually force every object into compliance, you need a bucket policy or a resource control policy behind it, which is covered further down.

Variable retention with event holds: the September 2026 addition to Object Lock

Fixed retain-until-date assumes the clock starts the day an object gets uploaded. That's fine for plenty of use cases, but plenty of real retention rules don't start at upload at all, they start when a contract closes, when an audit wraps, when litigation ends. Event holds, announced by AWS on September 8, 2026, solve exactly that mismatch.

You place a hold on an object along with a retention duration. While the hold is active, the object is fully protected. Once the hold gets released, S3 keeps the object locked for the specified duration counting from that release point. That's the whole trick, and it's a meaningful one.

Don't confuse this with a legal hold. A legal hold's protection ends the second it's removed. An event hold's protection actually kicks off a new countdown once removed, so the object stays covered for exactly as long as required, no more, no less.

You can apply event holds three ways. Directly on a single object through the API. As a bucket default, so every object inherits it automatically. Or at scale, through S3 Batch Operations, where a single job can process a manifest with up to 20 billion objects.

Two new condition keys, s3:object-lock-event-hold and s3:object-lock-event-hold-duration-days, come along with this. Both plug into IAM and bucket policies, letting teams set minimum or maximum hold durations and control exactly who can set or release a hold in the first place.

None of this matters much without a paper trail, so CloudTrail logs hold operations when data-event logging is turned on, and S3 Inventory reports hold status per bucket, giving teams a structured view of lock state across their objects. AWS's announcement states that Cohasset Associates has assessed this event hold capability for use in environments governed by SEC Rule 17a-4(f), FINRA Rule 4511, and CFTC Regulation 1.31. It's available across every AWS Region, GovCloud and China Regions included, at no extra charge.

Picture a financial services team ingesting signed contracts. The retention clock is supposed to start at contract close, not at the moment the file lands in S3. Event holds make that distinction precise, so nothing gets deleted too soon and nothing sits around costing money longer than it has to.

Turning rules into enforcement: bucket policies, RCPs, and AWS Config

Bucket defaults and lifecycle rules are instructions. They are not guarantees, and that distinction is where most retention policies quietly fail. An upload with its own Object Lock settings overrides the bucket default without so much as a warning. A bucket with no lifecycle rule at all just keeps growing, forever, at whatever cost that implies.

Bucket policies are the first line of defense. Configure one to deny any request that skips the required Object Lock parameters, using the s3:object-lock-event-hold and s3:object-lock-event-hold-duration-days condition keys. That closes off the most obvious bypass.

Resource control policies go a step further. An RCP can't be overridden by a bucket policy, and it can't be overridden by an account administrator either. For anything regulated, a bucket policy standing alone doesn't matter; the RCP layer is what actually matters.

Detection is its own separate job. AWS Config's s3-lifecycle-policy-check managed rule flags any bucket missing a lifecycle policy entirely, and it needs periodic review, since retention enforcement isn't a configure-once-and-forget kind of control. Layer CloudTrail on top to log any modification to the policy itself. That's the audit trail proving nobody quietly loosened the rules.

For scale, none of this should be clicked together by hand. Define retention policies as code, CloudFormation, Terraform, the AWS CLI, so they're applied consistently the moment a bucket is created rather than patched on afterward. Pair that with Lambda or Systems Manager to auto-remediate the moment Config flags drift.

CloudWatch Logs deserves its own mention here, since log groups have no retention limit by default and will accumulate indefinitely if left alone. A practical fix is to identify log groups with no retentionInDays value set and apply a consistent default across the board.

A single identity holding s3:BypassGovernanceRetention can unlock and delete an object in the same breath, which is the governance mode risk. A single identity holding s3:BypassGovernanceRetention can unlock and delete an object in the same breath. Wherever a policy genuinely needs two separate people to sign off before deletion, that's not a permissions tweak to fix, it's an architecture decision made at the mode-selection stage.

Prefix-scoped rule design: mapping data categories to retention requirements

One rule for an entire bucket is almost always the wrong call, because different data categories carry different retention windows, different cost profiles, and different immutability needs. Prefixes are how you map real categories onto real rules, and this holds just as true on S3-compatible stores like Ceph RGW as it does on AWS proper (per the oneuptime.com guide).

A worked example makes this concrete. For files under audit/, a seven-year expiration (2,555 days) paired with Object Lock in compliance mode applies, since this is regulatory evidence that must never be altered. Under app-logs/, a 30-day expiration with no Object Lock at all, since GDPR actively argues against holding this data any longer than needed. Under tmp/, a 7-day expiration, scratch data with no long-term purpose. Under security-logs/, 90 days on Standard or Standard-IA before transitioning to Glacier for whatever's left of a longer compliance window.

Every one of those prefix-scoped rules, if the bucket is versioned, needs a NoncurrentVersionExpiration clause attached. If it's skipped, historical versions pile up quietly in the background, eating away at whatever savings the tiering rules were supposed to deliver. And AbortIncompleteMultipartUpload belongs in every bucket, versioned or not, since failed large uploads leave partial data behind with no way to recover it and no reason to keep paying for it.

Tags offer an alternative when prefixes don't cleanly separate categories, say, multiple data types landing in the same folder. Tag objects at upload time and scope the rule to that tag instead of the path. It works well, but only if whatever's doing the ingesting actually tags things correctly and consistently, since a mistagged object just falls through the cracks.

On Ceph RGW, the same S3 API calls apply against the RGW endpoint without modification. Just make sure rgw_enable_lc_threads is enabled, and check the schedule with radosgw-admin lc list before assuming anything's running on time. To force a rule through immediately rather than waiting on the schedule, radosgw-admin lc process does the job (per oneuptime.com).

Layering the two mechanisms

Three questions settle which mechanism applies to any given category of data. Does this data need to survive until a specific date no matter who tries to delete it? That's Object Lock, compliance mode for anything regulatory, governance mode for internal operational needs. Does cost efficiency require moving the data through cheaper tiers as it ages? That's a Lifecycle transition rule, and it runs independently of whatever Object Lock setting is also in place. Does the retention clock start at some future event rather than at the moment of upload? That's an event hold, the addition AWS shipped in September 2026.

None of this is either-or. Object Lock and Lifecycle rules aren't competing options, they're built to run side by side on the same prefix. Audit logs are the clearest case: lock them in compliance mode for seven years so nobody can touch them, and let a transition rule move them into cold storage after 90 days so the bucket isn't burning frequent-access pricing on data nobody's actively querying.

That's the whole model, really. Lifecycle rules answer what you're paying for and where it lives. Object Lock answers who's allowed to touch it and when. When run together, scoped tightly by prefix, a retention policy stops being a checkbox on a compliance form. It starts actually doing what it says on the label.