Why Cloud Object Storage Beats Provisioned NFS for Multi-Agent Shared Workspaces

Cloud object storage with a native file system layer is the practical default for isolated per-tenant storage in multi-tenant AI agent applications. Each customer needs its own storage boundary, access policies that are enforced, brokered credentials, and compute isolation for agent-generated code. This is why cloud object storage beats provisioned NFS for multi-agent shared workspaces. It separates durable data and access control from a central file server that parallel agents must otherwise share.

How NFS centralizes state — and why that works against multi-agent parallelism

NFS uses a client-server architecture in which a central server hosts files, manages metadata, and coordinates access. Clients mount a remote directory that appears local to their operating systems. That familiarity explains its appeal to agent developers. A tool expecting a file path can use one without learning a storage API, while the server handles the shared state behind the mount.

Every read and write goes through the server, so parallel clients still rely on that one server to coordinate them. A workload with a small, bounded set of writers can fit this arrangement, but adding agents does not distribute the server’s responsibilities. Geography adds another constraint because NFS and SMB assume a low-latency, high-bandwidth local network. A cloud virtual machine cannot directly mount another machine’s local disk, and stretching remote file access over the internet introduces latency and unreliable connections.

What happens to that central server when twenty agents write at once

A single agent with exclusive access to local disk has a straightforward workspace. Multi-agent decomposition changes the arrangement before it changes the workload. A research agent collects material, a writer produces a draft, and an editor reviews it, but parallel cloud containers have separate file systems unless a shared layer connects them. The pipeline needs a common workspace to exchange files. Without it, the writer cannot see the researcher’s finished notes.

With NFS supplying that workspace, twenty agents reading shared configuration and writing into shared directories converge on the same server. As contention causes serialization delays, agents have to wait for storage before they can continue their tasks. Slow runs can become connection timeouts and execution failures, holding up later stages of the pipeline.

As files grow, they congest context windows and run into API timeouts. Without durable storage, a terminated container or broken connection can also leave no saved execution history to resume or debug. Mount-point ACLs take more work to configure while agents still face contention on the shared server.

How distributed object storage is architected to avoid those failure modes by default

A distributed file system splits data into blocks across storage nodes and presents a unified namespace to clients. Replication allows requests to move to another node when one fails, rather than tying a long-running agent task to a single machine. Cloud object storage follows the same distributed-storage pattern but uses objects and keys in place of a native directory tree. Compute and storage can therefore scale independently.

That distinction makes object storage a different substrate from an NFS server. Its flat namespace supports billions of objects, with exabyte-scale capacity, low cost per terabyte, and metadata tagging. S3, GCS, Azure Blob, and R2 provide strong consistency per key. A completed update to one key is visible to subsequent reads of that key. It does not make updates across several keys a transaction or provide cross-client file locking.

The remaining gap is at the agent’s interface. Distributed capacity and consistent object reads do not automatically make small-file, latency-sensitive tool loops work well. An object API is still not a file system, however politely it answers requests.

Why agents expect file semantics, and how the object-file gap is being closed

Agents usually move through directories, read paths, and run bash-style tools because so much of the models’ training uses those workflows. Requiring explicit object downloads interrupts that familiar sequence. A file-system overlay can hide the API, but how it looks is only part of what matters. FUSE-based overlays can maintain separate local views for different agents, allowing metadata to fall out of sync. State in the overlay can stay stale even when the object store beneath it has strong consistency.

AWS’s S3 Files uses Amazon EFS technology directly over S3 to add a native file system layer and keep S3 as the system of record. Both file and object APIs can still access the same data without moving it. AWS engineering teams using Kiro and Claude Code ran into this mismatch themselves when agents reached for local file tools while their data lived in S3. A shared mount lets agents work together through the subdirectories, notes files, and project directories they already know how to use.

Archil builds its own file system with a native S3-compatible API on every disk. Agents can use file access for work that is bash-heavy and latency-sensitive, while the rest of the stack uses object access for bulk operations. The interface can change without duplicating the dataset.

Giving agents access to large datasets without copying them first

A mounted view becomes especially useful when the data already exists in object storage. Without one, an agent must download files into local storage before using its tools. As its context is compacted, it can lose the record of what has already been downloaded, leaving the developer to remind it. Ephemeral storage makes that preparation temporary anyway. The next session can inherit the same chore without inheriting the files.

Direct mounting replaces that preparation with a path the agent can explore as it works. The entire dataset need not be staged upfront. S3 Files supports thousands of compute resources connecting to one file system simultaneously, so shared access does not require a separate dataset for each agent.

Cross-cloud access can follow a related, but distinct, pattern. Cloudflare R2’s Sippy fetches objects from S3 on a cache miss and copies them into R2 as they are accessed. Hot data moves gradually, while cold data stays in S3 until requested, avoiding migration downtime and reducing upfront egress costs.

Parallel agent attempts and branching workspaces — what NFS cannot offer and object storage is building toward

Shared access is only part of parallel work. Two coding agents trying different solutions need independent writable workspaces. NFS has no native branch primitive, so agents writing the same files can overwrite one another unless another layer coordinates them. Standard S3 versioning protects individual objects, while an isolated, writable copy of a whole workspace requires more than versioning.

Workspace-level copy-on-write branching supplies a different unit of concurrency. Agents can start from a common checkpoint, work independently, and have their branches merged or discarded. With Archil’s durable, forkable file system, checkpoint and branch operations let parallel attempts begin in milliseconds, avoiding the wait for full workspace copies. The useful distinction is the primitive itself. Object storage alone does not supply it, but a file system built over that substrate can.

For routine checkpointing, speed must remain independent of workspace size. Otherwise, snapshots require scheduling outside normal execution. Persistent disks mountable across machines also let restarted agents retain installed packages and working directories, while billing on active data suits workspaces with uneven use. These capabilities require more than an attractive launch page. A file system’s history under production load is a stronger readiness signal for fast branching and concurrent mounting.

Isolation requirements for agent-generated code and what they demand from storage

Durable shared state raises a separate question about who can execute code near it. Plain containers share the host kernel, so a kernel exploit can threaten the host or other containers. They are appropriate for trusted, pre-reviewed code, while arbitrary agent-generated code requires a stronger isolation baseline. File permissions cannot repair that boundary. Storage security and compute isolation have to be designed together.

MicroVM approaches, including Firecracker and Kata Containers, give workloads their own kernels behind hardware virtualization. gVisor instead places a user-space kernel between a container and the host, keeping most system calls away from the host kernel. That design can add overhead to I/O-heavy work, but it supports environments where GPU access is needed. Google’s Agent Sandbox on GKE uses gVisor, while Firecracker does not natively support GPU passthrough. The choice depends on the execution requirements, including startup speed.

A credible implementation also makes network and secret handling explicit. Archil pairs its microVM isolation with credential brokering at the network boundary and a managed egress allowlist on its Developer plan. For agent-generated code, the evaluation baseline should be outbound access disabled until allowed, with tenant credentials scoped by a proxy and each use logged per request. Secrets should not enter the sandbox directly. Persistent disks must preserve the same tenant boundary across restarts and forks, because a microVM with disposable storage solves only the execution half of the problem.

How do you give each customer isolated storage at thousands of tenants?

In a small NFS deployment, mount-point ACLs can be manageable. At thousands of tenants, separate mount configurations, network rules, and credential sets turn each customer into another provisioning task. The operational surface grows with the tenant count. Object storage separates the data plane, where objects live, from the control plane, which decides who may access them. A new tenant can map to a namespace prefix and an API policy on existing storage infrastructure.

A prefix alone is not an isolation boundary. Policies must prevent one tenant’s agent from traversing into another tenant’s data, even when the application shares compute infrastructure. An implemented version of this pattern gives each tenant a sandbox and isolated disk, with credential brokering enforcing access at the network boundary. Revocation then requires only a policy change, leaving the filesystem intact.

Elastic storage also avoids guessing each tenant’s capacity in advance. One agent may need 1 MB and another 1 GB, so active-use billing fits their needs better than paying for provisioned capacity. The same per-tenant pattern can mount customer-owned S3, GCS, R2, or Azure Blob storage without migration. That separates the customer’s dataset from the platform’s permission to use it. Revoking access leaves the data in the customer’s account, avoiding an extraction project.

Choosing the right storage substrate for your agent architecture

The decision starts with concurrency, trust, and interface requirements. NFS remains appropriate for low-concurrency, serialized workloads on a trusted local network, with a small and bounded number of writers. There is no need to replace a suitable architecture merely because agents are involved. For parallel pipelines, multi-tenant products, large existing datasets, and execution that resumes on different machines, object storage with a native file system layer is the practical default.

For agent-generated code, that storage choice must be evaluated with the sandbox. A microVM, persistent forkable disk, and S3-compatible API form a useful baseline for combining execution isolation, resumable work, and access from the wider stack. Systems built for this work, such as Archil, bring those parts together, while S3 Files handles the file-and-object access layer within AWS. They address different portions of the architecture. A native file interface does not, by itself, establish a secure execution boundary.

Concrete guarantees are needed to evaluate “secure sandbox” as a specification. Vendors should name the hypervisor, default egress policy, and location of secrets, and show that checkpointing and concurrent mounting have supported real production load. Tenant boundaries must survive the same restarts and forks that preserve file state. Mounting a customer’s existing bucket without moving the data keeps that bucket as the source of truth. The platform receives access to the data, while the customer retains control of the data residency decision.