The network is the first thing you blame. When a coding agent's session jumps from one GPU host to another and the next turn takes forever, the link between the machines looks like the obvious culprit. Put in a faster cable, fix the problem.
viiwork measured it on two RTX 5090 hosts, and the faster cable barely moved the number that mattered.
The claim
A model does not reread your conversation on every turn. As it processes the prompt, each layer produces key and value tensors for every token — the KV cache. Keep that cache warm and the model skips straight to generating. Lose it and the receiving host has to run the whole prompt again before it can emit a single token. That is prefill, and on a long session it is pure repeated work: about 41 seconds for 50,000 tokens on an older RTX A4000, minutes at 200,000 tokens on slower hardware.
Keeping a session on the host that holds its cache avoids this. But hosts fill up, and when a turn spills, kv_share copies a replica of the cache to the host the session would spill to next, so it can import instead of recompute.
The copy is incremental. After a turn of at least 8,192 tokens, the host saves the session and ships only the delta — roughly 240 MB of a 1 GB cache per 50k-token turn, because most of the conversation went across last time.
So: how much does the cable matter?
Three numbers
247 MiB was the typical replica delta. Sending it took 3.24 seconds at 1 GbE, 0.81 seconds over 10 GbE through Tailscale, and 0.55 seconds on the direct cable. The raw link carried 9.41 Gb/s; Tailscale managed 3.0–3.4 Gb/s, which is why the first jump was about fourfold rather than tenfold.
93.8% → 96.8% was the change in warm host switches, in the 16-session runs, going from 1 GbE to 10 GbE with Tailscale. Partly-warm switches fell from 5.4% to 2.2%. Direct 10 GbE was faster on transfer but landed at 96.2% warm, inside the run-to-run noise.
2.8 seconds was the import into the receiving engine. Every time. At every link speed.
That last number is the story. Once the replica arrives, there is a fixed cost the network cannot touch, and it is larger than the best-case transfer.
The bug
The most useful finding was not a number. Traces showed that viiwork remembered importing a replica, but the engine could later evict that conversation; on a return visit it skipped the import even though the cache was gone. The "import-once" bug was fixed in v2.11.1 by asking the engine what it still holds before skipping.
That bug made the whole mechanism look weaker than it was. Some of what looked like a bandwidth limit was a state machine being optimistic.
What to do with this
Measure end to end before you buy hardware. The send time improved sixfold; the warm-switch rate improved three points. Optimizing the transfer and stopping there would have produced a confident, expensive, wrong conclusion.
Then profile the import path. Fixed per-move costs set a floor that bandwidth cannot lower, and at this workload 1 GbE was already workable — average kv_share traffic was 320–450 Mb/s in each direction, and most switches were warm anyway. The caveat is real: these are single 90-minute runs, not statistical trials, and the author says so.
Buy the 10 GbE if it is cheap. Fix the state machine first.
Source: Two RTX 5090 hosts sharing a KV cache over Ethernet — viiwork.