What the measurements say

A 9B model with a q4_0 KV cache, snapshots cut into content-defined chunks with desync.

A snapshot at 12.3K tokens is 315 MiB. The next turn adds about 600 tokens and writes 416 MiB, of which 158 MiB (38%) is new. Most of that new data is the model's recurrent state and new context checkpoints, not the new tokens' KV, which is about 8 MB.

A second conversation with the same 7.1K-token prefix shares 36% of its chunks with the first. That holds whether the prefix was reused from RAM or computed again: recomputation is byte-identical, so dedup across conversations is reliable.

zstd saves only 4–5% on snapshots, so chunks are stored uncompressed. 16 KB and 64 KB average chunks give the same dedup. Chunking a 415 MiB snapshot takes 1.3 s.

• 38% new data per turn
• 36% shared across conversations
• Byte-identical recomputation
• No compression needed

Why not an existing tool

desync can already mount chunks as a file. Reading them back is the problem.

llama-server restores a snapshot with one sequential, single-threaded read. A 415 MiB restore takes 0.46 s cold from a plain file and 0.27–0.42 s warm. Through desync mount-index it takes 4.9 s cold, and 3.2–3.7 s even with every chunk in the page cache.

A single reader gets 89–93 MiB/s at any block size, while 8 parallel readers reach 681 MiB/s. The causes: no read-ahead, requests handled one at a time, and chunks read and decoded again for every request.

The design

A FUSE filesystem in Go: ordinary files on top, content-addressed chunks underneath.

Files stay files. Each snapshot is a small stub in a plain directory: its content is the chunk index, an extended attribute holds the real size. Rename (the proxy's .tmp → final name), unlink and the proxy's LRU timestamp touch are ordinary operations on the stub, so they stay atomic.

Writes stream. Bytes that arrive in order are queued and cut into chunks while llama-server is still writing; only chunks the store doesn't have are written, and the new index replaces the stub when the file is closed. Anything out of order falls back to a staging copy.

Reads prefetch. On sequential access, the next 32 MiB of chunks are fetched by 8 workers into a 256 MiB cache of decoded chunks, so each chunk is read from disk once per restore.

Garbage collection runs hourly and at start. It deletes chunks that no snapshot and no open file needs, and never runs while a save is adding chunks, so a chunk about to be reused can't disappear.

llama-server / proxy→ordinary files→kv-chunk-fs (FUSE)→chunk index per file→chunk store

• Streaming chunked writes
• Parallel read-ahead
• Decoded-chunk cache
• Atomic index updates
• Garbage collection and crash recovery

Build it, or patch desync?

Adding read-ahead and a cache to desync's own mount would have fixed reads in about 100–150 lines. But every save would have stayed a full plain-file write plus a 1.3 s chunking pass, losing the per-turn write saving. Building the filesystem took about the same code for reads, and the write path built on it. desync stays in as a library for the chunker and its index format.

Results

Measured on an NVMe SSD (Sabrent Rocket 4.0, running at PCIe 3.0 ×4).

Restore, cold, 415 MiB snapshot: plain file 0.43–0.44 s, kv-chunk-fs 0.49–0.51 s, desync's mount 4.9 s. Every chunk was read from disk exactly once.

Live, behind the real proxy and llama-server: turn 2 of an agent conversation saved 435 MB (415 MiB), of which 157 MiB was new (38%). A second conversation with the same system prompt wrote 202 MiB of new data out of 315 MiB, so 36% was already stored. After restarting llama-server and the proxy, the next turn was restored from the chunks: 29 tokens processed, 12,336 from the cache.

Plain-attention models dedup far better: a Qwen3-1.7B turn writes about 6% new data, and two identical conversations share everything. The hybrid 9B model's recurrent state and checkpoints change completely every turn, which is where its 36% comes from.

The cost: in the read-write mount, a 541 MB snapshot saves in 1.16 s against 0.51 s on plain ext4, and restores in 0.62 s against 0.37 s. The ceiling is the serial chunker. Packing chunks into larger files would close most of the restore gap.

• 0.62 s restore of 516 MiB (plain: 0.37 s)
• 38% new data per turn (9B hybrid)
• ~6% per turn (Qwen3-1.7B)
• Restores survive restarts

What the kernel and llama-server taught

llama-server saves in two passes: the state first, then a reopen in append mode for the context checkpoints. The first version took the reopen for a rewrite and staged a full copy; appends now continue the existing chunk list.

A rewrite came out wrong: saving over an existing snapshot produced 910 MB instead of 541 MB. go-fuse doesn't enable the kernel's atomic O_TRUNC, so the kernel sent the open without truncation and a separate size-0 truncate without a file handle, and the old content was written back. Fixed three ways, with regression tests for each.

Saves were twice as slow as they are now. desync's default chunk hash ran at 503 MB/s on one core; SHA-256 with the CPU's SHA instructions runs at 1,702 MB/s. With SHA-256 IDs and a queue between the writer and the chunker, a 541 MB save went from 2.26 s to 1.16 s.

A container's bind of the mount doesn't survive a restart of the filesystem. Binding the plain parent directory instead, with rslave propagation, lets running models see the new mount at once.

Operations

Built so whatever does eviction can see what each snapshot costs.

Settings come from defaults, a config file, the environment or flags, and kv-chunk-fs config shows where each value came from. Each snapshot carries its use count, last use, creation date and labels as extended attributes, kept across rewrites.

An optional HTTP API lists snapshots with their unique bytes (what deleting one frees), shared bytes and a configurable score, plus Prometheus metrics, a log buffer, deletes and garbage collection on demand. A token-prefix tree shows which conversations share which prefix. kv-chunk-fs itself never evicts.