Stop Syncing and Start Mounting: A Smarter Alternative to aws s3 sync
If you still rely on aws s3 sync to move data between a local directory and Amazon S3, you have plenty of company. It's among the first tools engineers reach for in the AWS Command Line Interface, familiar and straightforward enough for basic file transfers. As datasets grow, however, the cracks in that approach become impossible to ignore. Repetitive transfers, mounting latency, and fragile CLI workarounds turn what should be a settled problem into a recurring bottleneck.
Why Everyone Starts with aws s3 sync
For most engineers, aws s3 sync is the default choice when files need to move to or from Amazon S3. Built into the AWS CLI toolkit, it feels natural, particularly for anyone comfortable with Unix-style sync or recursive copy patterns. The command delivers exactly what it advertises: it syncs a local directory with an S3 bucket or the reverse, copying new and updated files while skipping everything else.
Here's a simple example:
bash
CopyEdit
aws s3 sync ./data s3://my-bucket/data
This command syncs the contents of your current local directory to the specified bucket. Behavioral flags give you additional control, such as --delete to remove destination files that no longer exist locally, or --exclude to skip specific files. In theory, that combination gives you a lightweight, reproducible way to manage object storage.
For small projects, quick backups, or one-off transfers, aws s3 sync gets the job done. It supports flags like --exact-timestamps and --acl bucket-owner-full-control, and it lets you override the command's default URL or adjust output formats through the cli binary format setting. When your workflow involves only a handful of files and minimal metadata, syncing can genuinely feel like the simplest path forward.
But as the next section shows, simplicity carries a price.
But Syncing Has Serious Limitations
On the surface, aws s3 sync looks capable, yet the further your data infrastructure reaches, the more its shortcomings emerge. What handles simple file uploads or basic backups without issue tends to fall apart when datasets are constantly changing, directories are large, or workflows are data-intensive.
Performance Bottlenecks for Large Datasets
Each time you run the sync command, the AWS CLI compares your local directory against the target bucket's contents. With a dozen files, that comparison is trivial. With thousands or millions of objects, the overhead compounds quickly. Because the sync operation is batch-based, there's no concept of continuous access or low-latency interaction; the process is always check, then copy.
Sync also evaluates differences through metadata comparisons, so edge cases involving object metadata, server side encryption, or metadata default settings can introduce inconsistencies or false positives.
"Syncing can quietly become a performance tax, especially when it starts competing with compute jobs for bandwidth or time."
-Archil Engineering Lead, Harrison Leath
Operational Complexity Grows Quickly
The moment your project outgrows its first bucket, flag management takes over. You're specifying file types to exclude (--exclude '*.tmp'), adding ACLs (--acl bucket-owner-full-control), controlling delete behavior (--delete), and making sure the cli binary format setting is properly configured. That's before pagination controls like --no-paginate or disable cli pager even enter the picture, and those are often mandatory in automated environments.
In larger teams, someone inevitably runs a sync that deletes the wrong files or fails without any visible indication, particularly when the tooling doesn't surface command output clearly. The fallout: created files disappear, backups overwrite production data, or permissions are mishandled.
Risk of Data Loss or Duplication
One of the most dangerous pitfalls in sync usage is the --delete flag. Applied carelessly, it can remove destination files that don't exist in the source, even files placed there intentionally by another service or team. There's also no native way to preview or dry-run that behavior for verification without reaching for flags like --exact-timestamps or scripting around cli input parameters.
Because aws s3 sync provides no transactional guarantees, interruptions from timeouts, SSL verification errors, or maximum socket connect time issues can leave your storage layer in a half-finished state.
Common Sync Pain Points Devs Deal With
Beyond performance and reliability, aws s3 sync introduces a surprising amount of low-level friction, especially once usage goes past basic file uploads. These aren't edge cases; they're the daily reality for developers managing data pipelines and multi-environment deployments.
Here are the frustrations that surface again and again:
Lack of visibility before execution: Without wrapper scripts or carefully constructed filters, previewing what the sync operation will actually do is difficult. The sync command syncs objects in bulk, often with minimal feedback in the command output.
Fragile filtering and exclusions: Getting syncing to cover only the files you need can require a patchwork of --exclude, --include, and --exact-timestamps flags. Miss one, and you risk accidentally uploading files that shouldn't be there or deleting ones that should remain.
Inconsistent metadata handling: Sync doesn't reliably preserve object metadata, caching behavior, or server side encryption settings unless you explicitly specify flags like --metadata-directive or --sse. Subtle bugs and downstream failures can result.
Error-prone CLI ergonomics: Misconfigured defaults such as the cli binary format setting, or automatic behaviors like disable automatically prompt and disable cli pager, can trip up automation scripts. Vague errors like cancel save preferences unable and overly verbose logs are common unless engineers carefully tune the only show errors flag.
Lack of symbolic link support: If your local file structure uses symlinks, a common pattern in ML projects, aws s3 sync ignores them entirely. This can silently break assumptions about file presence or path dependencies.
When a sync operation requires repeated overrides or excessive debug logging, you're almost certainly using the wrong abstraction for the job.
When Syncing Makes Sense
Despite its drawbacks, aws s3 sync still has a legitimate place in modern cloud workflows, particularly where simplicity and portability matter more than speed or scale. Certain use cases are a genuinely good fit.
Here's where the sync command holds up well:
One-time file transfers: Moving static assets, such as .txt upload files or archived logs, from a local directory to a bucket.
Basic backups: Generating point-in-time copies of the current local directory, especially inside CI pipelines or developer environments.
Cold storage pushes: Sending infrequently accessed files to S3 storage tiers via the AWS Command Line Interface without needing real-time access.
Air-gapped workflows: When source and target systems can't stay continuously connected, sync provides a transactional model that's straightforward to reason about.
Team-wide scripting: When you want a single, portable command that works consistently across machines with aws cli installed.
For small-scale workloads, a few thousand files, or situations where you simply need to dump files matching certain patterns to a specified bucket, syncing may be entirely sufficient.
Once you move into dynamic workflows, however, particularly where you're repeatedly syncing files that already exist, overriding metadata, or working with large datasets, the operational tax accumulates fast.
What Mounting Offers That Syncing Doesn't
If syncing feels increasingly brittle as workloads scale, that's because it was never designed to behave like a file system. It's a copy tool and nothing more. Mounting fundamentally changes how you interact with cloud storage by treating S3 as a live, navigable, and responsive file system.
Here's what mounting unlocks:
Real-Time Access to S3 Files
Mounted volumes let you access files on demand without re-transferring, re-syncing, or tracking version state. There are no more "sync or recursive copy" loops or scripts built around which new and updated files changed. Files are simply there, just as they would be in a traditional local filesystem.
This approach eliminates sync drift entirely. If a file exists in the bucket, it's accessible immediately from your local current directory without triggering a background transfer job or reconciling timestamps.
POSIX Compatibility
Archil's mounted volumes behave like a standard Linux-compatible file system. In practice, that means:
You can use standard Unix commands (cat, cp, ls) without worrying about sync flags
Your code and pipelines don't need to change
You get proper file semantics, including support for symbolic links, permission flags, and predictable behavior with large file trees
For teams working with legacy tools or batch-processing systems that assume files are on disk rather than in a remote object store, this is especially powerful.
Zero Data Movement
Unlike sync, which always copies data between endpoints, a mounted volume streams files directly from S3. The practical result:
No duplicate storage usage from created files sitting in both source and destination
No re-uploading the same files because of minor metadata changes
No confusion around delete output, sync operation, or whether the specified command used the correct flags
The mount behaves like a window into the bucket, not a mirror of it.
"Once you stop thinking in terms of file transfer and start thinking in terms of file presence, everything gets faster and simpler."
- Archil Founder, Hunter Leath
Archil in Action: Mounting S3 the Right Way
Archil replaces the repetitive, error-prone nature of aws s3 sync with a single declarative step: mount your S3 bucket as if it were a local file system. No data gets copied unless it's accessed, no scripting gymnastics, no sync loops.
Here's what it looks like in practice:
bash
CopyEdit
archil mount s3://my-bucket ~/data
This command creates a live, POSIX-compliant view of your bucket at ~/data. Files appear in your local directory structure instantly. There's no sync command constantly rechecking for new and updated files, no concern about cli binary format setting, and no risk of conflicting states from multiple tools trying to "fix" the same folder.
Archil also respects S3's native configurations, whether you're using access point aliases, server side encryption, or bucket owner full control ACLs. Under the hood, a custom caching and streaming protocol delivers sub-second access to S3 objects even in large-scale environments.
Because Archil is built for Unix-style environments, it works seamlessly with common shell tools, data pipelines, and AI/ML workflows without requiring you to rewire everything just to avoid syncing.
Better Performance, Lower Costs
Running aws s3 sync means more than moving data; you're also paying for it. Syncing large directories drives up storage usage, multiplies API calls, and duplicates data transfer, particularly across multiple environments. Even small mistakes, like resyncing files because of changed object metadata or a forgotten flag, can generate real costs.
Archil sidesteps this problem entirely. By mounting the bucket as a live file system, it fetches only what you need and only when you need it, with no unnecessary file copying. Storage duplication disappears, transfer bandwidth isn't wasted, and no extra compute is spent scanning the current local directory for differences.
No More Paying to Re-Sync the Same Files
The savings extend beyond dollars:
You avoid delays caused by excessive sync command syncs and S3 rate limits
You don't need to script around maximum socket connect time or retry logic
You reduce the risk of incurring charges from re-uploading the same txt upload or deleting files with the wrong flag
For teams working with large models, real-time analytics, or high-churn datasets, sync becomes an unnecessary tax. Mounting replaces all of that with something faster and cheaper.
Better Performance, Lower Costs
Running aws s3 sync is never just a data movement operation; it's also a billing event. Syncing large directories leads to increased storage usage, more API calls, and duplicated data transfer, especially across multiple S3 environments. Even small missteps, like resyncing files because object metadata changed or a flag was forgotten, can rack up real costs.
Archil avoids this entirely. Mounting the bucket as a live file system means fetching only what you need, when you need it, without copying files unnecessarily. There's no storage duplication, no wasted transfer bandwidth, and no extra compute spent scanning the current local directory for differences.
The savings go beyond just dollars:
You avoid delays caused by excessive sync command syncs and S3 rate limits
You don't need to script around maximum socket connect time or retry logic
You reduce the risk of incurring charges from re-uploading the same txt upload or deleting files with the wrong flag
"One of our customers reduced their monthly S3 I/O costs by 78% just by swapping out their daily sync jobs for a single Archil mount."
- Archil Founder, Hunter Leath
For teams working with large models, real-time analytics, or high-churn datasets, sync is an unnecessary tax. Mounting replaces all of that with something faster and cheaper.
When to Still Use aws s3 sync
Its limitations aside, aws s3 sync isn't disappearing anytime soon, and it still makes sense in a handful of specific scenarios.
The Right Tool for the Right Moment
Here's when sync might be the better option:
One-time data pushes from a source directory during setup or migration
Backup jobs that run overnight and require simple diffs of uploaded files
Air-gapped workflows where mounting isn't possible or supported
Static archiving of folders using txt delete, exclude files, or simple inclusion patterns
If your use case doesn't require low latency or live access, and if you're comfortable working through the aws cli user guide and fine-tuning configured cli binary format flags, sync can still serve you well.
In those cases, the following sync command may be all you need:
bash
CopyEdit
aws s3 sync ./backup s3://my-archive-bucket --delete --exclude "*.tmp"
But It Doesn't Scale
The more frequently you run syncs, the more fragility you absorb: inconsistent metadata handling, unpredictable file states, and CLI flags that break silently when a default value or only valid value is misapplied. Sync was never designed to deliver fast iteration, real-time access, or high availability.
For teams handling fast-moving data or pipelines that must function reliably at scale, mounting simply removes the guesswork.
Final Thoughts: Don't Sync When You Can Mount
The aws s3 sync command served its purpose, giving developers a straightforward way to push files to S3 from a local machine. Cloud workloads have since moved on. Teams now need real-time file access, predictable performance, and far fewer brittle scripts. Mounting S3 directly provides a stronger foundation.
With Archil, you step away from the complexity, fragility, and cost of traditional sync-based workflows. Here's what you no longer have to worry about:
Overriding the command's default URL or tweaking the aws configure command
Troubleshooting unclear file transfer progress or broken automation flows
Managing default format, credential file, access key id, or secret access key
Handling permissions and ownership via account's canonical id and granted permissions
Remembering to verify SSL certificates or disable automatic pagination in CI/CD pipelines
Accidentally breaking encryption settings like specifies server side encryption or customer provided encryption key
Losing or duplicating metadata from misused metadata directive value or specifies caching behavior
Breaking pipelines due to mishandled symlinks (support symbolic links) or unix like quotation rules
Introducing noise by forgetting to store essential cookies, collect anonymous statistics, or save cookie preferences
Digging through the AWS Console footer to debug a silent failure in your sync job
"We built Archil so you can interact with S3 like it's part of your machine, not a remote object you have to constantly chase down."
- Archil Founder, Hunter Leath
The takeaway is clear: if your infrastructure still depends on syncing, it's probably time to move to something built for scale, speed, and sanity.
Try It Yourself
Ready to stop syncing and start mounting?
Archil lets you turn any S3 bucket into a high-performance, POSIX-compliant file system without changing your application code, cloud environment, or folder structure.
Get instant access to your S3 data with a single mount command
Reduce AWS costs by eliminating unnecessary sync jobs
Streamline dev, ML, and analytics workflows with real-time, on-demand access
👉 **Get started with Archil now,** and make your data infrastructure simpler, faster, and future-proof.