Cross-Cloud Data Access Without ETL

Data doesn't sit in one cloud anymore, and pretending otherwise is how companies pay twice for the same terabyte. Four architectural approaches actually get rid of ETL for cross-cloud data access, instead of just hiding it behind a nicer dashboard. Federated query engines and filesystem-level abstraction are the two worth building around here. Private interconnects and Iceberg matter too, but mostly as plumbing that makes the other two work better, not as the main event.

Nobody wakes up and decides to spread data across three clouds for fun. A company buys another company that happened to build on Azure. The fraud team needs data to stay in the EU for legal reasons while product runs everything on AWS. A vendor deal locked in GCP for one workload years ago and nobody's touched it since. Multi-cloud is a byproduct of how companies grow, the way a junk drawer accumulates without ever becoming a filing system. It's just where things ended up.

The traditional fix, building ETL pipelines to copy and stage everything in one place, sounds tidy on a whiteboard. In practice it stacks up three costs that compound on each other. There's the egress fee every time data crosses a cloud boundary. There's the maintenance debt of keeping pipelines running and patched. And there's the slow drift where the copy stops matching the source, so someone downstream makes a call off stale numbers without knowing it.

The egress math alone should make anyone pause. AWS charges $0.09 per gigabyte to move data to the public internet, and $0.02 per gigabyte for cross-region transfers. Per Sedai, these charges eat up 15 to 20 percent of total cloud spend, on average. Gartner projects cloud spending will hit $723.4 billion in 2025. Run that percentage and you land on $110 to $145 billion leaving companies purely in transfer fees, a good chunk of it from architectures nobody ever sat down to design on purpose.

That asymmetry isn't an accident of engineering, either. Ingress is free. Egress costs money. Hyperscalers built it that way because it's a retention mechanism wearing a pricing model as a costume: once your data's in, leaving costs money.

Then there's data gravity, a term that's aged well. As datasets grow, moving compute to the data gets cheaper than moving the data to the compute, the same logic as buying new furniture instead of shipping the old couch across the country. The economics eventually force the rethink, whether anyone in the org wants one or not.

Compliance adds a wall on top of the cost problem, and it's a hard wall, not a soft one. Some data legally can't leave its jurisdiction, full stop. "Just copy everything into one lake" is cheap in that scenario. It's also illegal.

Consultants at cloudatler.com cite an 80 percent egress reduction figure for topology-aware routing. Treat that as directional, a sense of what's recoverable, not a number to stamp on every migration plan.

The real question is how to minimize data movement, not whether to eliminate it entirely. It's which of the four architectures below eliminates it for the specific workload sitting in front of you, because picking the wrong one just moves the pain somewhere else on the org chart.

Where the pain is sharpest: AI training pipelines waiting on data that never arrives fast enough

Nowhere does slow data access cost more, in dollars anyone can point to, than AI training. A GPU cluster sitting idle is a concrete, measurable cost. It's a rented resource burning money while it waits for bytes to show up, like paying for a taxi to sit outside your house while you finish getting dressed.

Per Hyperbolic, reading training data from remote object storage like S3 or Azure Blob can eat up to 70 percent of total training time. Not compute time. Waiting time. The GPU sits there, fully paid for, doing nothing at all.

The utilization numbers back this up, and they're rough reading. Poor I/O configuration averages around 45 percent GPU utilization, according to arxiv research, while a well-tuned I/O setup gets that number to a consistent 95 percent. That gap, roughly half the cluster's capacity, is the difference between a training run that finishes on schedule and one that quietly blows through its budget.

Meta ran into this at scale. Reports show 56 percent of GPU cycles stalled waiting on training data, even with Tectonic, Meta's own distributed filesystem, holding exabytes of storage. Storage capacity wasn't the bottleneck here. Tectonic couldn't keep petabyte-scale datasets local to the training hardware fast enough, so Meta built a dedicated Data PreProcessing Service just to stop the bleeding.

The performance bar isn't modest, either. Large language model training can call for 10-plus gigabytes per second of sustained read throughput. Image training pipelines working with high-resolution datasets can demand 4 gigabytes per second, per GPU. Multiply that across a cluster and "good enough" storage stops being good enough, fast.

And the clusters keep growing. Frontier AI training runs have already reached 100,000 GPUs, with clusters of 300,000-plus GPUs reportedly in planning for 2025. At that scale, a data pipeline hiccup doesn't stay a footnote in the postmortem. It becomes the headline.

Multi-cloud AI training is a structural necessity. It's a constraint forced by physics and permits. Per SemiAnalysis, Google and OpenAI are spreading large model training across multiple datacenter campuses, because power availability, permitting timelines, and construction schedules won't let a single site scale fast enough. Microsoft has committed roughly $80 billion to AI-enabled datacenter buildout. Google Cloud has committed $25 billion over two years to data center and AI infrastructure expansion, while separate estimates put total 2025 capex between $75 and $85 billion. Nobody spends that kind of money because multi-cloud is trendy. They spend it because the alternative doesn't fit inside one campus, full stop.

The gap between what object storage delivers over a network connection and what a GPU can actually chew through is the defining infrastructure problem of the current AI buildout. Whatever closes that gap without forcing a data copy has a structural advantage that compounds with every training run.

The four architectural approaches that eliminate ETL rather than replace it with something equally fragile

Four approaches solve this, and none of them wins outright, no matter what a vendor's slide deck claims. Federated query engines, open table formats, private cross-cloud interconnects, and filesystem-level abstraction each chip away at a different slice of the same problem.

Fit matters more than preference here. Latency tolerance matters. So does the governance regime, whether the data is structured or a pile of unlabeled files, and how often the queries actually run. A team defaulting to whatever tool they already know is a common reason these migrations go sideways.

All four share one design principle: leave the source data where it sits, and move the compute or the metadata instead. Compare that to the old playbook. Data replication produces copies that can fall out of sync with the original. ETL pipelines carry maintenance overhead and create staleness risk whenever jobs fail or fall behind. Full data lake consolidation means an expensive migration and a fresh set of compliance headaches nobody asked for.

The next four sections take each approach in turn, covering the mechanics, the tradeoffs, where each one earns its keep, and where it falls apart.

Federated query engines: running analytics where the data lives without touching it

The mechanism is simple to describe even if it's not simple to build. A query engine sends execution to wherever the remote cloud lives, reads only the slice of data needed to answer the question, and hands back a result. No staging table. No full copy of anything.

Google Cloud's borderless Lakehouse (announced at Google Cloud Next, with more updates at Next Tokyo) is the clearest production example. It's built on open Apache Iceberg, using the Iceberg REST catalog to federate metadata across clouds, so BigQuery, Managed Service for Apache Spark, or any Iceberg-compatible engine can query data sitting in another cloud without moving it first.

Metadata discovery connects to remote Iceberg REST catalog endpoints, Databricks Unity Catalog on AWS and GCP, AWS Glue, Snowflake's Horizon Catalog, and finds the data through those endpoints without ever copying the underlying files. Queries then read straight from the remote cloud, and Lakehouse caches segments of that data locally within Google Cloud, so a second query on the same slice doesn't pay the egress cost twice.

Per Google's documentation, this setup supports reduced data movement, unified analytics across clouds, and what Google calls borderless AI and ML, meaning agents run directly against remote data without a migration step first. A Conversational Analytics feature lets someone build an agent grounded in tables that span multiple clouds, so one conversation pulls from data sitting in different places. A Knowledge Catalog feature handles data profiling across those federated sources, syncing metadata automatically from AWS Glue, Unity Catalog, and Horizon Catalog.

Snowflake takes a related but distinct approach with Cross-Cloud Snowgrid, Snowflake's own layer stretched across clouds and regions. It handles instant discovery, access, and sharing of data, services, and apps across cloud boundaries with no ETL step in between. Governance policies, tags, and lineage travel with the data and get enforced from one place, no matter which cloud the data physically sits in. There's a business continuity angle too: replication and failover across clouds, plus regional locality controls that matter for regulated industries. HD Supply has used Snowgrid for business continuity and cost savings. Pfizer has used it for cross-region collaboration.

None of this erases network physics, and it shouldn't pretend to. Federated query performance is bounded by the latency and bandwidth of whatever path connects the two engines. Cross-region queries over the public internet add somewhere around 50 to 100 milliseconds per round-trip on small files, and object storage's REST overhead piles on top of that.

Pick this approach for analytical work where queries run at a moderate pace, data volume per query stays bounded, and the team already lives inside one platform ecosystem: Snowflake talking to Snowflake, or BigQuery reaching into AWS. Skip it for real-time or streaming pipelines, anything needing sub-millisecond reads, and unstructured data with no catalog schema to hang onto in the first place. This caveat is not minor. It rules out most of the AI training use case from the section above entirely.

Apache Iceberg as the format layer that makes data readable across engines without a shared platform

Iceberg is an open-source table format built for large analytic datasets. It defines how files sitting in object storage, S3, ADLS, GCS, get organized into something that behaves like a proper table: schema, partition layout, consistent point-in-time snapshots, the works.

What makes it matter for cross-cloud work: the same Iceberg table gets read and written by Spark, Flink, Trino, Dremio, DuckDB, Snowflake, BigQuery, and a growing list of others. Picking an engine and picking a cloud stop being the same decision.

The Iceberg REST Catalog Specification is what makes this practical instead of theoretical. It defines an HTTP API for managing tables, so any engine that can make an HTTP call, basically anything written in Python, Rust, Go, or JavaScript, can create, list, read, and commit to Iceberg tables without a Java SDK. That's what makes the format language-agnostic and cloud-agnostic at once.

There's a real performance story underneath, too. Scan planning in Iceberg runs at roughly constant time against the metadata, instead of scaling linearly with partition directories the way older formats did. The metadata sits in a hierarchy, table metadata pointing to manifest lists pointing to manifest files, and that structure holds up even at billions of files without choking query planning.

The market has more or less settled the format wars, and Iceberg won them. Databricks has made major moves to align with Iceberg's ecosystem. Snowflake launched Polaris, its own Iceberg-based catalog. Major cloud providers have standardized around Iceberg as the default answer. Reports suggest well over half of Fortune 500 companies are testing it, though that figure deserves a primary source before anyone repeats it as settled fact.

Databricks built a compatibility layer called UniForm that generates Iceberg-compatible metadata for Delta Lake tables, so non-Databricks engines read Delta tables without converting the underlying data. Dremio Cloud, for instance, connects to Unity Catalog through UniForm and queries Delta Lake tables right alongside everything else.

Nobody has to standardize on one vendor's platform to get cross-cloud interoperability anymore, which is the whole point of the last five years of catalog wars. Iceberg's open catalog API means data sitting in S3 is queryable from BigQuery or Snowflake with no migration project attached.

One limit deserves stating directly, though, because vendors tend to skip past it. Iceberg handles structured and semi-structured tabular data. It has nothing to say about unstructured files, documents, images, model weights, log files, the stuff that doesn't fit into rows and columns to begin with. Treat it as a format for tables, not a format for everything.

Private cross-cloud interconnects: what changes when the network path itself is the control variable

Most cross-cloud setups default to routing over the public internet, and that default drags baggage with it: unpredictable latency, higher egress rates, no service-level guarantee on bandwidth. It's the data equivalent of mailing a package instead of driving it over yourself, at the mercy of whatever route it happens to take that day.

Google Cloud's Cross-Cloud Interconnect offers a different path, literally. It's a private, dedicated link between Google Cloud and AWS, Azure, Oracle Cloud Infrastructure, or Alibaba Cloud. Per Google's documentation, data moves across a private network connection that never touches the public internet. That routing brings lower egress charges out of AWS compared to standard internet egress, especially paired with private interconnect capacity, and it delivers more predictable latency and bandwidth than the public internet will ever offer.

There's a Partner Cross-Cloud Interconnect option too, priced hourly and usage-based, for flexible bandwidth below the 10G or 100G tier, with an SLA-backed connection and zero variable egress cost pulling data from AWS, according to Google Cloud's own blog. Under the hood, a BigQuery request routes through a Service Directory service, into an Internal Load Balancer, out through Hybrid Connectivity Network Endpoint Groups, and over AWS PrivateLink into S3. The traffic never touches the open internet anywhere in that chain, start to finish.

That matters beyond raw speed. A private, known, auditable path is something a compliance officer can actually sign off on, which is a real consideration for healthcare, finance, or government workloads where "the data went out over the internet somewhere" doesn't fly as an answer.

There's a cost-model shift buried in here too. Moving from per-gigabyte egress billing to flat-rate interconnect capacity changes the math for any workload doing high-frequency, high-volume cross-cloud queries. A predictable monthly bill beats a variable charge that swings depending on how busy last month happened to be.

Setting this up costs real money before it pays anything back, and that upfront cost is the part sales decks tend to gloss over. Private interconnects need provisioning, VPC configuration, PrivateLink setup, load balancers, and a capacity commitment before any of it earns its keep. Skip it for occasional or exploratory queries; it only pays off once cross-cloud access becomes a sustained, high-volume habit rather than a once-in-a-while need.

Google's borderless Lakehouse pairs a caching layer on top of all this, storing fragments of remote data temporarily inside Google Cloud so repeated queries don't re-cross the network at all. Combined with the interconnect, that cuts the actual bytes moving over even the private link, a detail that sounds minor until the bill shows up at the end of the month.

Filesystem-level abstraction: what changes when unstructured data is accessible without a catalog or schema

Everything covered so far shares one blind spot. Iceberg and federated query engines work great on tabular data that lives in a catalog. Private interconnects cut the cost of moving that data around. None of it touches the pile of documents, images, audio files, model checkpoints, and logs that make up most of what healthcare, finance, and government organizations actually generate. And that's most of the data most companies hold, not some side case nobody needs to worry about.

That gap is bigger than it sounds. Most of that data is unstructured by nature, and forcing it into a database schema before it becomes useful is exactly the bottleneck that's kept data-rich fields functionally data-poor for years. A hospital doesn't lack data. It lacks a way to query the scans and notes sitting in its file systems without a six-month schema-design project standing in the way first.

The filesystem interface turns out to be the most universal abstraction available, because POSIX semantics (read, write, seek, rename, lock) are understood by every programming language, every framework, and honestly every piece of software ever written to work with a command line. No SDK required. No schema to define up front. No ETL step standing between the file and whatever wants to read it.

A cloud filesystem mounted directly over object storage works roughly like this. It points at an existing S3, GCS, Azure Blob, or R2 bucket and presents it as a real POSIX filesystem, no migration, no ETL, no rewriting the application that already expects a normal file path. Reads that hit a local NVMe cache come back at sub-millisecond speed. On a cache miss, the system fetches from the source bucket and caches it for next time. Writes replicate before the write call even returns, then flush asynchronously back to the bucket, which stays the single source of truth the whole time.

Capacity scales elastically and gets billed on what's actually cached at any given moment, which matters for AI workloads where the working data set for one training run might be a hundredth the size of the next one. And because it supports full POSIX semantics (atomic rename, file locking, memory-mapped I/O), existing applications don't need to know or care that the "disk" they're writing to is actually an object store three regions away.

That's the piece the other three approaches can't offer. A way to make unstructured, schema-less data accessible across clouds without forcing it through a catalog built for a different job entirely, which is exactly why the AI training bottleneck from earlier gets solved here and nowhere else on this list.