S3 Request Rate Limits and Prefix Sharding

If you take only one thing from everything that follows, make it this: S3's rate limits are a per-prefix constraint, which means you can multiply your throughput ceiling almost arbitrarily by spreading your keys across more prefixes. That's the whole trick. The rest of this is explaining why it works, what it actually costs you, and when you need to reach for something else entirely.

What S3's per-prefix rate limits actually guarantee

AWS publishes a clear number: at least 3,500 PUT, COPY, POST, or DELETE requests per second, and at least 5,500 GET or HEAD requests per second. Per prefix. Not per bucket.

That distinction matters more than it sounds.

There is no documented cap on the number of prefixes you can have in a bucket. So the ceiling is not a hard wall on your bucket. It's a per-lane speed limit on a highway that you can add lanes to.

A few things worth understanding clearly before moving on:

• A "prefix" in this context is just any leading substring of an object key. It does not have to end at a slash. S3 does not care about your folder metaphors.
• Write headroom (3,500/s) is meaningfully lower than read headroom (5,500/s). If your workload is write-heavy, that gap matters. Plan for the tighter number.
• Rate limits are the ceiling, but latency is the floor. Even when you are well under the rate limit, each request carries 10 to 100ms of round-trip overhead from HTTP handling, authentication, and the multi-AZ replication pipeline. Sharding does not help with that. Key design alone cannot help with that.

Think of these numbers as design parameters rather than arbitrary restrictions. S3 is a distributed key-value store operating over HTTP at global scale. Partitioning is how any distributed system absorbs load without melting. The published numbers are just where AWS has landed after running that system for a very long time.

How S3 partitions keys internally and where partition boundaries form

This is the part that explains everything else, so it's worth slowing down here.

S3 is, under the hood, a massive distributed key-value database. When load on a particular range of keys gets high enough, S3 automatically redistributes those keys across more internal nodes. This happens in the background, without you doing anything.

The partitioning logic is lexicographic. That means S3 is literally sorting keys alphabetically and then deciding where to draw dividing lines. If a prefix is under too much load, S3 looks at the actual characters in those key names and cuts the keyspace roughly at the midpoint of whatever spread it finds. Something like: keys starting with "a" through "m" go here, keys starting with "n" through "z" go there.

A few things follow directly from this:

• The slash character is not special. S3 does not know or care that you use it as a folder separator. It is just another character in the sort order.
• Splits happen anywhere in the key string, including in the middle of a word.
• Scaling is not instantaneous. While S3 is repartitioning in the background, you will see HTTP 503 "Slow Down" responses. This is not a failure. It is S3 telling you it is catching up.
• There is no documented way to pre-warm shards. You cannot force S3 to create partitions ahead of a traffic spike.

Here is the important implication, the one that makes key design matter: S3 can only draw partition boundaries where there is lexicographic spread. If all your keys share a long common prefix, the partitioner has nothing to work with. It cannot split "logs/2025-01-15/event-0000001" from "logs/2025-01-15/event-0000002" in any useful way because they are nearly identical for the first twenty characters.

That is not a quirk or a bug. It is just how lexicographic partitioning works. And it is the root reason why key design determines your throughput ceiling.

Why the 2018 S3 update changed key-design advice, but not the underlying problem

Before 2018, the standard advice was blunt: prepend a random hash to every object key. Something like the first few characters of an MD5. AWS explicitly recommended this because shared prefixes were a hard throughput ceiling and the only way around it was to force lexicographic spread manually.

In 2018, AWS updated the documentation and explicitly stated that the update "removes any previous guidance to randomize object prefixes to achieve faster performance." Auto-partitioning had improved to the point where manual hashing was no longer the default recommendation.

That is good news. It means most workloads today do not need any special key design to stay under the rate limits. For the average S3 user, 3,500 or 5,500 requests per second per prefix is plenty.

But. And this is a meaningful but.

The underlying constraint did not change. S3 still needs lexicographic spread to partition. What changed is that auto-partitioning now handles enough spread on its own that manual intervention is less often necessary.

Sequential keys are still a problem. A key like "00000001" and "00000002" sit almost on top of each other in sort order. A time-series log file named by timestamp has the same issue. Auto-partitioning struggles to find useful split points in a narrow lexicographic band.

The 2018 shift is what makes "put a high-cardinality business dimension first in your key" the modern recommendation instead of "prepend MD5 to everything." You are still solving the same problem. You are just solving it with meaning instead of noise.

The conditions that actually produce throttling in production

Knowing the limit exists is different from knowing when you will actually hit it. Here is what actually causes 503s in real systems.

Single-prefix concentration. All your concurrent writers or readers are pointing at the same leading key segment. This is extremely common in time-series data, per-day partitions, and single-tenant buckets where every key starts with something like "data/2025-01-15/."

Traffic spikes faster than S3 can repartition. A prefix that was quiet yesterday and suddenly gets hammered today will throw 503s while S3 scales in the background. The bucket's traffic history matters. A freshly created bucket or a newly active prefix has no partition head start.

KMS-encrypted buckets. This one catches people off guard. If your bucket uses KMS encryption, every GET and PUT triggers a KMS API call. Generate Data Key on writes. Decrypt on reads. KMS has its own request rate limits that are region-specific and entirely independent of your S3 prefix design. No amount of key sharding fixes a KMS bottleneck. It is a separate ceiling.

Multi-GPU training clusters. Hundreds of concurrent processes reading from the same dataset prefix simultaneously is one of the most reliable ways to saturate a prefix. One team running experiments with 200 machines sustained 230,000 reads per second, but that throughput required the internal shards to already be populated from prior traffic. An empty bucket hitting that load from a cold start would not hold it.

The observable signal for all of this is the HTTP 503 "Slow Down" response. Not a hard failure. Not data loss. It is S3 saying "I am repartitioning, please retry with backoff." The AWS SDK handles this correctly with exponential backoff by default. That does not raise your ceiling. It just manages the ramp gracefully so you do not flood a prefix that is still catching up.

Prefix sharding strategies and what each one costs

The arithmetic is simple once you accept the premise. Ten prefixes, each with a 5,500 GET/s limit, gives you 55,000 GET/s. Writes scale proportionally. The trick is spreading your keys across those prefixes in a way that is both effective and something you can live with operationally.

Three approaches, in order of increasing aggressiveness:

Natural high-cardinality prefix. Take an existing business dimension that already has lots of distinct values and put it first in your key. Tenant ID. Region code. User ID. If you already have a dimension with thousands of distinct values, you are done. Zero overhead, keys still mean something, and S3 gets plenty of spread to partition on. This is the right default when the dimension exists.

Numeric shard IDs. Create fixed shard buckets, something like "00/" through "99/", and assign objects via round-robin or hash. Predictable, easy to reason about. The downside is that it's a little artificial and you are now maintaining a shard assignment scheme.

Hash prefix. Prepend a short hash of the object name to the key. First four characters of an MD5, for example. Maximizes lexicographic spread. Works extremely well for throughput. Breaks lexicographic ordering entirely. LIST operations become painful, range scans stop working, and partition-based access policies get messy.

The operational costs of sharding are often understated:

• LIST operations return results per prefix. A sharded namespace means you have to fan out LIST calls across all shard prefixes and merge results in your application. This is extra code, extra requests, extra latency.
• Lifecycle rules match on prefixes. If your keys are sharded across 100 prefixes, you either write 100 lifecycle rules or you accept that some lifecycle logic cannot be applied uniformly.
• Access policies in multi-tenant architectures that rely on prefix matching get complicated when a hash prefix displaces the tenant ID from the leading position.

The modern compromise for data lake workloads is to put a high-cardinality business dimension first, then a Hive-style time partition. This gives query engines like Athena a usable partition structure for pruning while giving S3 enough spread to auto-partition under load. Two problems, one key schema.

One more thing: sharding is not a one-time decision. A prefix that is cold today can become hot when a new pipeline goes live. Design for the hottest anticipated access pattern, not just today's.

Where prefix sharding stops being a sufficient answer

Sharding is a good solution to a specific problem. It is not a general solution to all S3 performance problems. Here is where it runs out of road.

Latency is not a sharding problem. Distributing keys across more prefixes does nothing about the 10 to 100ms per-request round-trip. If your workload is latency-sensitive and you are hitting S3 for every request, sharding will not help you.

AI training at scale. Clusters of 256 or more GPUs generating hundreds of gigabytes per second of aggregate reads can exceed what prefix sharding alone can deliver from standard S3, even with well-distributed keys. The scale just exceeds the model.

LLM checkpoint writes. A training run writes hundreds of gigabytes every few hours in a sharp burst. Even a well-sharded bucket will spike a prefix transiently during that ramp, triggering 503s while S3 catches up.

KMS. Said it before, worth saying again. No key schema change touches the KMS API limit. If KMS is your binding constraint, you need a different solution entirely (custom KMS key limits, envelope encryption strategies, or switching encryption approaches).

Operational overhead. More prefixes means more lifecycle rules, more access policy definitions, more LIST fan-out, more monitoring surface. Manageable. But real. There is a point where the complexity of maintaining a heavily sharded namespace outweighs the benefit.

A well-sharded deployment shows linear throughput scaling once the key design is right and the bucket has enough traffic history for S3 to have completed its partitioning. Getting to that steady state, though, requires both conditions to be true simultaneously.

S3 Express One Zone as an alternative to sharding for latency-sensitive workloads

AWS launched S3 Express One Zone at re:Invent 2023 specifically for workloads where standard S3's latency profile is the problem. The headline numbers: single-digit millisecond access, up to 10x faster data access than S3 Standard, up to 80% lower request costs per AWS documentation.

The rate limits are in a completely different category. We are talking up to 2,000,000 GET and 200,000 PUT transactions per second per directory bucket by default. Prefix sharding simply stops being a conversation at those numbers for most workloads.

There are also no 503 ramp-up windows. Express One Zone scales instantaneously. The traffic spike problem that makes cold-start standard S3 buckets dangerous simply does not exist here.

AWS has published concrete performance benchmarks: up to 5.8x faster loading for SageMaker, roughly 2.1x quicker query times for Athena, up to 4x performance improvement for EMR. For the right workload, the difference is not marginal.

The trade-off is durability. This is not a footnote. S3 Express One Zone stores data in a single availability zone. If that AZ has a failure, your data is unavailable until it recovers. Standard S3 replicates across a minimum of three AZs. That is a real difference and it should drive the decision.

Where Express One Zone fits:

• Latency-sensitive inference serving
• Active training dataset access
• Hot query caches
• Anything where the durability trade-off is acceptable and the workload justifies the higher per-request cost

Where it does not fit:

• Archival storage
• Compliance-sensitive data with explicit cross-AZ durability requirements
• Anything that needs the standard S3 durability guarantee

Think of it as a different product that happens to share an API. Not an upgrade. A trade.

Caching and prefetching as the layer that absorbs what S3 rate limits can't

Here is something that should be obvious but often gets missed: if you are reading the same objects dozens of times, most of those reads do not need to hit S3 at all.

Multi-epoch training is the canonical case. You cycle through the same dataset repeatedly across training epochs. After the first pass, everything you need has already come down from S3. A caching layer stores those objects on NVMe, EBS, or memory. The second epoch reads from local storage. The S3 rate limit is never approached again for that data.

Tools like Mountpoint for Amazon S3 make this straightforward. They handle the local caching of dataset objects so repeated reads convert to local filesystem reads after the first pass.

Prefetching is the complementary technique. Load the next batch asynchronously while the GPU is processing the current one. A producer-consumer pattern. The data is ready when the compute cycle needs it. I/O latency disappears from the critical path.

Tool selection matters independently of any key design decision. s5cmd significantly outperforms the AWS CLI for bulk S3 downloads. The same prefix layout and key schema will behave differently depending on which client is doing the work.

The scale of the problem when this is neglected: one analysis found that 56% of GPU cycles were stalled waiting for training data. More than half of expensive compute time spent waiting on storage. That is not a key design problem. That is a pipeline architecture problem.

The relationship between caching and sharding is complementary, not competitive. Sharding raises the S3 throughput ceiling. Caching reduces how often that ceiling is approached in workloads that re-read the same data. For write-heavy workloads like checkpoints, buffering writes locally and flushing asynchronously to S3 absorbs the burst pattern that causes transient 503s even in well-sharded buckets.

How key design decisions interact with the rest of the data stack

Key design is not just a storage concern. It propagates into everything that touches your data.

Lifecycle rules operate on prefix matches. A sharded key schema means lifecycle rules have to be defined per shard prefix. One rule at the top level that used to cover everything now needs to be replicated or generalized across all your shard prefixes. More rules to maintain, but manageable.

Multi-tenant SaaS. Putting tenant ID as the leading key segment solves the sharding problem and aligns with IAM prefix-based access policies at the same time. One key design decision does two jobs. Tenant isolation in access policies and natural lexicographic spread for partitioning. That kind of alignment is worth seeking out.

Query engines. Athena, Spark, and EMR rely on Hive-style partition layout for partition pruning. If you prepend a hash to maximize S3 spread and that hash displaces your time or entity dimension from the leading position, the query engine can no longer discover your partitions correctly. Your throughput ceiling goes up and your query performance goes down. Getting both right requires thinking about which dimension serves the query engine versus which dimension serves S3's partitioner.

Large objects. AWS added a 50 TB maximum object size (late 2025). For very large AI training datasets or high-resolution video, consolidating data into fewer, larger objects means fewer requests. Fewer requests means less pressure on prefix rate limits, sometimes eliminating the problem without any key redesign at all.

The most important practical point: key design decisions made early are expensive to reverse. Moving existing data to new key names means copying every object, updating every reference, rewriting every lifecycle rule, and auditing every access policy. Getting the leading dimension right at schema design time costs almost nothing. Retrofitting it costs a lot.

A reasonable mental model for structuring keys:

• First segment: a throughput and access-control primitive. High cardinality. Aligns with your IAM boundary.
• Second segment: the query-engine partition dimension. Time, entity type, whatever your analytics layer expects.
• Remainder: whatever is natural for the application.

That ordering is not arbitrary. It reflects the priority order of the constraints that are hardest to fix later.