How to Test What Another Reader Sees After fsync

Write files with known checksums, sync them, and then read them through a new mount, snapshot or fork at increasing delays. A successful fsync and a successful read by another client show different things, so a storage test must check both.

How to test what another reader sees after fsync

Once fsync returns, another reader tries to read the file without going through the writer. That reader might be a different sandbox, a snapshot or a fork. Its job is not merely to find a file with the right name and size, but to read the expected bytes. A file can look good in a directory listing and still fail its audition.

Timing gives that check its meaning. A read taken immediately after fsync asks whether the data is available at the writer’s commit point, while a later read asks whether visibility catches up. Repeating those reads across different paths separates an immediate handoff from an eventual one. Neither should be mistaken for the other. If a file is missing or corrupt after fsync returned, that read path did not see data the writer had already made durable by POSIX’s definition.

That finding needs a carefully drawn boundary. It establishes what the tested reader could see, but does not, by itself, establish whether every copy of the write was lost. The data may be durable somewhere even if that path cannot see it. A read that works after a clean shutdown also does not show that the write would survive a host failure.

What fsync promises

POSIX fsync asks the system to transfer a file’s data to the storage device so it survives a crash. Applications such as databases rely on it as their commit point. That makes the return from the call a meaningful boundary. A fast acknowledgment is valuable only in the context of what it acknowledges.

Durability and visibility are different properties. Durability means the write survives and is on the storage, while visibility means another running client sees it. That second property depends on the reader’s caches. As a result, “the writer finished” and “the reader has the new bytes” describe different ends of the handoff.

For testing, the lesson is to preserve that distinction in the result. A missing file through a snapshot is a finding about that snapshot’s view after fsync. A matching checksum is equally specific evidence about the path that returned it. Precision here prevents a visibility test from putting on a crash-test costume.

Why sandbox and cloud storage can differ

Some sandbox storage acknowledges writes on local NVMe and uploads them to remote storage later. That can make fsync fast even when another path has to wait to see the data. A snapshot, fork or other host may read from the remote copy before the upload has reached it. The writer and reader are then consulting different copies. The acknowledgment has arrived, but the bytes have not necessarily reached the reader’s source.

Vercel’s Drives docs describe NVMe-speed writes and say a snapshot mount reflects the Drive as it was at the moment it was mounted. A view fixed at mounting answers a question about that moment. It does not automatically answer what a later mount would see. The test therefore needs to name the read path and measure the delay.

Reading back through the writer does not exercise the other paths that an application may depend on. Likewise, a successful new mount cannot stand in for a successful fork simply because both involve another reader. Each path needs its own result. Otherwise, the benchmark checks one entrance from top to bottom and calls the whole building open.

The test, step by step

The procedure below tests the handoff directly. It uses known content to establish correctness, several read paths to locate differences, and delays to measure when those differences disappear. Repetition helps distinguish a consistent outcome from one that varies between trials. A busy-writer run tests a separate condition while keeping the baseline the same.

1. Write known content and sync it. Write a small file of 4 KiB and a large file of tens of MiB, both with deterministic content. Call fsync on each file and on the directory. Record their sha256 checksums. Those checksums are the reference for later reads. The small and large files provide separate observations, so success for one should not stand in for success for the other.
2. Exercise every relevant read path. Read through a new mount on a different sandbox, a snapshot, a fork and a mount after stopping the writer. Keep each path distinct in the results. A pass through one path says nothing conclusive about an untested path.
3. Sweep the delay and repeat the trials. Use delays of 0, 0.1, 1, 5, 30, 60 and 120 s between fsync returning and the read. Repeat each cell several times, then repeat with the writer busy writing other data. Keeping that boundary consistent makes the delays comparable across the different paths and conditions.
4. For corrupt files, record which byte ranges differ and re-read later with the page cache dropped. File size alone is insufficient. Recording the damaged ranges distinguishes an absent file from a file that exists but contains incorrect bytes, while the later read tests whether the observed result changes.
5. State the limit of the shutdown test. Stopping the writer cleanly lets the provider flush first, which is not the same as a host failure. Report that result as a clean-stop result. It can show that a later mount receives the file under that condition, but it cannot establish what happens when a host disappears before an upload finishes.

The matrix needs to keep the conditions for each result. A result such as “ok after 30 s” is incomplete without the read path and whether the writer was busy. One reader found no file, while another found the wrong content. Both fail the checksum-based test, but they describe different views of the storage.

If a file becomes readable later, the test has found a visibility interval under those conditions. It has not established that the same wait will always suffice, or that acknowledged data survives a failure during that interval. That is why the repeated trials and the failure-boundary questions belong in the same investigation.

A worked example: Vercel Drives

“Vercel Drive fsync durability” publishes a vendor-run test with its script. The setup used region pdx1 with 10 trials per cell; a few cells ran 8 or 11. The script uses only the Vercel Python SDK, so others can rerun it. Those details make the test inspectable without making it independent. The results should be read as published observations from that setup.

The test reported that Drive snapshots taken immediately after fsync returned were missing the file in 10 of 10 trials. At 30 s, the file was still missing in 7 of 10 trials. Forks were missing it in 11 of 11 immediate trials and 9 of 10 trials at 30 s. From 60 s, the files were intact. These results distinguish an immediate read from a delayed one rather than treating “eventually readable” as an immediate pass.

The busy-writer results exposed a different failure shape. At 30 s, 4 of 10 snapshots held a full-size copy whose last 12 to 28 MiB read as zeros. A file-size check would have missed that discrepancy. The file was present, but presence was not correctness.

Stopping the writer and mounting the Drive read-write in a new sandbox found the file intact in 10 of 10 trials, about 3 s after fsync. That observation belongs beside the snapshot and fork results. Because a clean stop permits flushing, it does not resolve what would happen during an abrupt host failure.

Archil, which sells a disk that mounts in Vercel sandboxes, published the comparison and reports that its disk had both files intact in a new sandbox immediately after fsync in 20 of 20 trials. Archil’s snapshots and forks were not part of that test. Its docs say that fsync returns once the write is stored across Availability Zones, so its disk is another option to check with the same read-path and failure-boundary questions.

Questions to ask a storage provider

The practical follow-up is to ask what sits behind the acknowledgment. Speed alone cannot explain which storage copy a reader will consult or which failure the acknowledged write can survive. These questions tie what the test found to what the provider says its storage does. They also keep a discussion about fsync from drifting into a general conversation about how fast the disk feels.

• When fsync returns, is the data on local disk, on one remote server or across several failure domains?
• Are snapshots and forks taken from the same storage that fsync wrote to?
• How long until another client sees a write, and is there a way to force a refresh?
• What happens to acknowledged writes if the host fails before upload?

What a provider says about where it stores data covers the durability boundary, while its refresh rules cover what the reader sees. Neither substitutes for the other. The useful evidence is a match between those stated rules and the paths tested, with clean shutdown kept separate from host failure. That gives an application a defined handoff to rely on, rather than a pause whose main engineering credential is hope.