Commit History
-
Keep multi-turn conversations fast: snapshot the conversation's KV before the follow-up-suggestion pass evicts it, and restore it before the next turn so only the new turn prefills instead of re-processing the whole thread. Measured 20x faster re-prefill on a 459-token thread (12s to 0.6s, same GPU back-to-back), byte-identical by determinism, gated to long threads on capable desktops.
humuhumu33 +1: Claude Fable 5Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>a593567
-
Bonsai kappa-stream: self-contained showcase (app + engine closure)
Humuhumu33a88a5cf