Unstructured Data Access Patterns for AI

Most storage systems were built on a bet: that reads would be sequential, requests would be bounded, and traffic would arrive one request at a time. AI breaks that bet on all three counts at once. A training job reads the same dataset over and over, in random order, as fast as the accelerators can chew through it. A checkpoint write dumps a massive burst of data to disk in one shot and needs it done now, not eventually. An inference request wants one small answer back in milliseconds, no matter how many thousand other requests are asking the same thing at the same moment. None of this is new behavior stacked on old behavior. These three demands collide on the same storage system, at the same time, each one incompatible with the assumptions behind the others.

Unstructured data makes this worse before it makes it better. It now accounts for up to 90% of the enterprise data estate, and it doesn't arrive in tidy batches. It streams in constantly from cloud platforms, edge devices, and distributed systems scattered across a company's footprint, and legacy storage was never built to absorb that kind of input at speed.

The AI pipeline itself keeps the problem alive on a loop. Data gets ingested, parsed, chunked, turned into embeddings, retrieved, fed into a model, and evaluated, and every one of those steps produces new material (embeddings, logs, checkpoints) that becomes an input for the next run. Each cycle adds more reads and writes to a system that was already straining.

Splitting the workload into its three sub-patterns helps explain why no single fix covers the whole problem. Training reads hammer the full dataset repeatedly and in random order, at whatever rate the accelerators can absorb. Checkpoint writes arrive as sudden, massive bursts, and they have to land without stalling the cluster behind them. Inference reads ask for small amounts of data, fast, under heavy concurrent load, where the real constraint is latency rather than throughput. Tail latency is the worst-case response time, and it hides behind an otherwise healthy average.

How unstructured data growth compounds access pressures

Three separate forces are feeding this growth at the same time, and none of them is slowing down. AI pipelines generate their own fuel: every training run produces new datasets, embeddings, logs, and model artifacts that then need storing, managing, and often feeding back into the next run. Application modernization adds a second stream, and you get machine logs, telemetry, and event data thrown off around the clock. Surveillance footage, healthcare imaging, and sensor data pile on a third, and regulatory retention rules mean none of it can simply be deleted once it goes cold, even when nobody looks at it again for years.

Cloud sprawl compounds all three. AI pipeline runs leave behind duplicate copies, snapshots, and backups, and these multiply cost and governance risk on their own, even apart from whatever legitimate growth was already happening.

Most of what companies store can't be found by anyone who needs it. Only 15% to 20% of unstructured data gets actively used for insights. The vast majority of what companies store sits unlabeled, unclassified, and effectively invisible to the pipelines that might use it. Storage keeps growing. Discoverability doesn't grow with it. That widening gap is what turns a volume problem into an access problem, and it gets harder to close with every passing quarter, not easier.

Four failure modes that slow AI pipelines

This mismatch between how AI reads and writes data and how storage was designed to handle it appears in four specific places. Each has its own cause, and fixing one rarely touches the other three.

The first is I/O starvation during training. Training reads the same data repeatedly, in random order, as fast as the accelerators want it, and if storage can't keep pace, the GPUs sit there doing nothing while they wait for bytes to arrive. On a modern GPU cluster, idle GPU time burns real capital and real power, and the cost adds up fast. The scale is mismatched, because storage protocols built for human-speed requests now have to serve machine-speed consumption. A survey of 504 enterprises, run by Freeform Dynamics for Scality, found that a majority of enterprises now rank storage performance as a top priority for avoiding AI bottlenecks, putting storage on equal footing with (or ahead of) compute and network as the thing most likely to choke a production AI system.

The second is a metadata bottleneck on small-file datasets. Picture a computer vision job built on millions of individual image files. The storage system's advertised throughput numbers might look fine on a spec sheet, but the real holdup is just finding and opening each file before a single byte gets read. Bandwidth was never the problem, so tuning for faster sequential reads does nothing, because the slowdown happens before you even start the read. The same Freeform Dynamics/Scality survey found that two in five enterprises name metadata handling at scale as a genuine risk to their pipelines.

The third is the checkpoint write storm. As models grow larger, saving a checkpoint can freeze an entire training cluster for minutes at a stretch, because the storage system has to absorb a massive, nearly simultaneous burst of writes without pausing the compute sitting on top of it. This is a separate failure from training-read throughput. A system tuned to handle sustained sequential reads beautifully can still grind to a halt the moment it has to flush a checkpoint.

The fourth is inference latency under concurrency. Inference needs small, fast, random reads served to a lot of users at once, and storage systems built around maximizing throughput tend to introduce exactly the kind of tail latency that makes an inference service feel sluggish and unreliable. Object storage on its own doesn't solve this. Where millisecond response times and more expressive querying matter most, its throughput-first design and limited query interfaces become the obstacle rather than the answer.

Why object storage dominates at scale

Object storage has become the default home for AI data at scale: its flat namespace scales to billions of objects without the slowdown that hierarchical file systems hit as directories get crowded, and it supports rich custom metadata that AI pipelines rely on to discover and label content. The Freeform Dynamics/Scality survey backs this up directly: the large majority of enterprises running AI in production report meaningful use of object storage, with most using it extensively. That makes it the baseline architecture rather than a niche option sitting off to the side.

The design choices that make object storage good at scale also set its limits. It was built around throughput-first, latency-tolerant assumptions, and aggregate throughput scales beautifully under that design. Per-object latency does not scale the same way, and tail latency under concurrent load is what training and inference workloads expose first. The same optimization works in reverse: a namespace built to handle billions of objects adds lookup overhead that punishes any workload built from millions of small files.

Google announced Cloud Storage Rapid at Google Cloud Next '26, building on Rapid Storage previewed the year before at Google Cloud Next '25, and it shows what it takes to push object storage into AI-grade performance. Google reports that a single zonal bucket under Rapid Bucket delivers more than 15 TB/s of bandwidth, 20 million requests per second, and sub-millisecond latency. In production, Google reported cutting blocked GPU time by roughly half and speeding up data loading for multi-modal training runs by up to 2.5x, with meaningful gains in checkpoint restore and write times as well. Thinking Machines Lab runs multi-model training alongside its Tinker fine-tuning API for open-source models, and when it integrated Rapid Cache, stable read throughput peaked above 1.8 TB/s, with lower tail-end latency and fewer errors.

Treat that as a case study in what closing the gap costs; the gap itself isn't closed for good. Object storage didn't suddenly solve inference. Even the throughput side of the problem took dedicated engineering investment well beyond a standard object storage configuration, and inference latency is still its own separate fight, with its own separate answers.

Eliminating data movement: why GPU-direct and RDMA change the pipeline architecture

The traditional model moved data to compute: stage it somewhere, copy it to local scratch space, then hand it to the GPU. That model falls apart once data volumes grow past what can be staged in time to meet a training or inference deadline. The fix flips the old order around. Instead of hauling data toward the GPU, the architecture lets compute reach directly into storage.

NVIDIA's cuObject and NIXL are the engineering answer to that shift at the GPU-storage boundary. cuObject moves data directly between GPU memory and S3-compatible object storage over RDMA, skipping CPU kernel processing for the data payload entirely and removing the old bottleneck of staging everything through a local scratch file system first. NVIDIA released general availability of the cuObject client and server libraries together with the SCADA Server SDK, and cuObject introduces a shared RDMA wire protocol. A companion initiative called xio-sig gives any server-side implementation that follows the protocol a path to stay interoperable with the cuObject client.

Dell Technologies, working with NVIDIA, put this into production with a concrete example. The companies contributed an accelerated engine for the NIXL OBJ plugin, now merged upstream, so that vLLM, LMCache, and NIXL can offload KV cache to Dell ObjectScale over RDMA and land it directly in GPU memory through cuObject. The measured difference: rebuilding context from scratch took 11,223 milliseconds of GPU time. Pulling it instead from storage took 837 milliseconds. An 11-second tax on the GPU shrinks to under a second, and that freed-up GPU time goes toward the one job nothing else can do, running the model.

A second, quieter bottleneck sits upstream of storage entirely: the CPU transformation layer. Training runs can consume terabytes of preprocessed data an hour, and if the CPU can't transform data that fast, it becomes the limiting factor before you ever get to blame storage. NVIDIA's RAPIDS ecosystem, and GPU-accelerated DataFrame operations through cuDF, move that transformation work onto the same memory hierarchy the training process already runs on, cutting out the PCIe transfer overhead of shuttling data back and forth between CPU and GPU memory.

Tiered architectures balancing throughput, latency, and cost

Most organizations end up running a hybrid, tiered storage setup, and that's a deliberate engineering decision, not a shortcut taken for lack of a better option. No single storage tier can deliver training-grade throughput, inference-grade latency, and archive-grade cost all at once, so you split the job across tiers built for each.

The pattern that emerges looks consistent across most deployments. Cold and archival data lives in object storage, where it scales cheaply, holds up durably, and tolerates the slower access latency that comes with data nobody touches often. Active training datasets get staged close to the GPU cluster on fast local or network-attached storage, built to absorb the random-read, high-concurrency pattern training throws at it. You call in cloud capacity to absorb temporary demand spikes; provisioning permanently for a peak load that only shows up occasionally would waste capacity the rest of the time.

The hard part is moving data between tiers without creating a new bottleneck in the process. Manual staging pipelines ask engineers to guess which data a training run will need before it even starts, and that guess falls apart the moment the scope of the experiment changes mid-run. Building and maintaining custom ETL pipelines to shuffle data between tiers is pure overhead: it keeps the model fed, but it does nothing to make the model better. Moving data across cloud regions to rebalance tiers brings latency and egress costs that can wipe out whatever savings the tiering was supposed to deliver.

Two production examples show what it looks like when this is done well. Spice.ai's work with Twilio staged critical control-plane datasets in object storage, then accelerated them locally for sub-second queries, pushing P99 query times under 5 milliseconds, with automated multi-tenancy controls that propagate updates in minutes instead of hours. Barracuda ran a similar playbook with a datalake accelerator, bringing P99 query times under 200 milliseconds by moving audit logs into Parquet files on S3, cutting the load on Cassandra and removing the need for costly Databricks Spark queries.

Staging and local acceleration are deliberate design choices, built on the condition that the movement between tiers has to be automated and continuous, not something an engineer has to guess at by hand before every training run.