Designing Dropbox: File Sync with Chunking, Delta Sync, and Conflicts

A user edits one paragraph in a 2GB PowerPoint on a hotel Wi-Fi connection, and three seconds later the change is on their laptop at home. Uploading 2GB obviously did not happen. The entire design of a file sync service falls out of one decision: files are not the unit of storage — chunks are.

Requirements

Functional: sync a folder across devices, share folders between users, version history, offline edits that reconcile later. Non-functional: handle files from 1KB to 50GB, minimize bandwidth (users on metered connections), sync latency of a few seconds, never lose a byte — durability beats availability here.

Split metadata from blocks

The first structural move: two planes with completely different access patterns.

• Block storage — immutable, content-addressed chunks in object storage (S3). Huge volume, dumb operations: put chunk, get chunk.
• Metadata service — the file tree: which files exist, which chunk list makes up each version, sharing ACLs, version history. Small data, strongly consistent, transactional (Dropbox runs this on sharded MySQL).

Clients talk to both: negotiate state with the metadata service, then move only missing chunks to/from block storage.

Chunking: why content-defined beats fixed-size

Naive: split every file into fixed 4MB chunks, hash each (SHA-256), store by hash. Content addressing gives you deduplication for free — a chunk shared by a thousand users is stored once — and integrity checking falls out of the name.

The subtle failure: insert one byte at the front of a file and every fixed-size chunk boundary shifts, so every hash changes, so you re-upload the whole file. The fix is content-defined chunking: slide a rolling hash (Rabin fingerprint) over the bytes and cut a chunk wherever the hash matches a pattern. Boundaries are determined by content, not offsets, so an insertion only changes the chunk it lands in and its immediate neighbors.

def chunks(data, mask=0xFFF):        # ~4KB average chunk
    start = 0
    h = RollingHash()
    for i, byte in enumerate(data):
        h.roll(byte)
        if h.value & mask == 0 or i - start >= MAX_CHUNK:
            yield sha256(data[start:i+1]), data[start:i+1]
            start = i + 1
    if start < len(data):
        yield sha256(data[start:]), data[start:]

Delta sync is then just set arithmetic: client computes chunk hashes, asks the metadata service "which of these do you not have," and uploads only those. The 2GB PowerPoint edit becomes a few hundred KB.

The sync protocol

A file version is a manifest: an ordered list of chunk hashes plus metadata. Committing an upload is a metadata transaction: "create version N+1 of file F with this chunk list, parent version N."

That parent version field is the concurrency control. It makes commits compare-and-swap: if two devices both commit against parent N, the first wins and the second gets a conflict — the server never silently overwrites.

1. Client watches FS events, chunks changed file, hashes
2. POST /commit {file_id, parent_version: N, chunks: [h1..hk]}
3. Server: missing chunks? -> 412 + list -> client uploads to block store -> retry
4. Server: parent stale?   -> 409 conflict
5. Success: version N+1; notify other devices (long-poll/WebSocket)
6. Other devices pull manifest, fetch missing chunks, rebuild file

Notification is a long-poll or WebSocket channel carrying only "namespace changed, cursor X" — devices then pull the delta since their cursor. Push the signal, pull the data: it keeps fan-out cheap and clients correct after sleep/offline gaps.

Conflicts: do not merge, fork

Two devices edit the same file offline. On reconnect, both commit against parent N; one gets a 409. For arbitrary binary files, automatic merging is impossible — and last-writer-wins destroys someone's work, which for a file product is unforgivable.

The right answer is the boring one Dropbox ships: keep both. The losing commit becomes report (Rishi's conflicted copy 2026-07-13).docx. Both versions preserved, human resolves. In the interview, saying "LWW" here is the fastest way to fail; data loss is the one non-negotiable.

Version history is nearly free: manifests are tiny, chunks are immutable and shared across versions. Old versions cost only the chunks that changed. Garbage collection — deleting chunks no manifest references — needs care: reference-count asynchronously, delete lazily, never inline with user actions.

Scale notes

Metadata shards by namespace (user or shared folder) so one folder's commits serialize on one shard — which is exactly the CAS semantics you want. Block traffic goes straight to object storage with pre-signed URLs so bytes never flow through your API tier. Cold chunks tier down to cheaper storage classes.

Interview probe	Answer sketch
Rename a 10GB file?	Metadata-only op — chunk list unchanged, zero bytes move
Dedupe across users?	Content addressing; add per-user encryption keys and you lose it (privacy trade-off, say it explicitly)
Millions of tiny files?	Batch commits; pack small chunks; metadata QPS is the bottleneck, not bandwidth
Sync loop storm?	Cursor-based deltas + jittered backoff; idempotent commits by content hash

The one-line summary: content-addressed chunks + CAS commits on a strongly consistent metadata service + conflict copies instead of merges.