Post
87
Cutting an agent's context window from 256K to 16K decodes 3.6x to 7x more tokens per second on identical H100s (GLM 5.2 3.6x, Kimi K2.6 5.6x, Qwen 3 Coder Next 7x), and in the paper's tests accuracy barely moved.
"You don't need long context" (arXiv 2611.05306) argues that long-horizon agents mostly need memory management, not bigger windows. Frequent compaction into a compact state, plus selective reading of inputs with bash tools, kept accuracy close to full-history baselines while contexts stayed short.
📊 The numbers
• Compaction over 6 benchmarks: Opus 4.5 -3.7 to +5.6 ppts, GPT 5.6 Sol -3.3 to +4.4 versus full history
• One action per context: SWE-bench Verified 70.2 for Opus 4.5 (baseline 70.0) and 82.4 for GPT 5.6 Sol (baseline 84.4), median peak context 7K to 9K tokens
• Long inputs stored as files and read selectively: Opus 4.5 -0.4 to +40.0 ppts, estimated cost savings 4.4x to 233.3x across both models
🛠️ Try it
• Set a compaction threshold near 12K to 36K tokens
• Add a tool that reads files or history on demand
• Compare accuracy, total tokens and cost against full history
⚠️ Caveats
• Speedups come from open models, accuracy results from closed ones (Opus 4.5, GPT 5.6 Sol)
• Cost savings are estimates, the harness is bash-only, and the preprint has no independent replication yet
You don't need long context (2611.05306)
https://arxiv.org/abs/2611.05306
What would your agent need to carry in its handoff state, and what could it safely fetch later?
📚 Read it elsewhere
• The deep dive on Medium: https://medium.com/p/41f69cdbe3b3
• The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/qwen-3-coder-next-decodes-7x-faster
• The thread on X: https://x.com/saavoji/status/2118402118927438127
• Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#claude-opus-4
"You don't need long context" (arXiv 2611.05306) argues that long-horizon agents mostly need memory management, not bigger windows. Frequent compaction into a compact state, plus selective reading of inputs with bash tools, kept accuracy close to full-history baselines while contexts stayed short.
📊 The numbers
• Compaction over 6 benchmarks: Opus 4.5 -3.7 to +5.6 ppts, GPT 5.6 Sol -3.3 to +4.4 versus full history
• One action per context: SWE-bench Verified 70.2 for Opus 4.5 (baseline 70.0) and 82.4 for GPT 5.6 Sol (baseline 84.4), median peak context 7K to 9K tokens
• Long inputs stored as files and read selectively: Opus 4.5 -0.4 to +40.0 ppts, estimated cost savings 4.4x to 233.3x across both models
🛠️ Try it
• Set a compaction threshold near 12K to 36K tokens
• Add a tool that reads files or history on demand
• Compare accuracy, total tokens and cost against full history
⚠️ Caveats
• Speedups come from open models, accuracy results from closed ones (Opus 4.5, GPT 5.6 Sol)
• Cost savings are estimates, the harness is bash-only, and the preprint has no independent replication yet
You don't need long context (2611.05306)
https://arxiv.org/abs/2611.05306
What would your agent need to carry in its handoff state, and what could it safely fetch later?
📚 Read it elsewhere
• The deep dive on Medium: https://medium.com/p/41f69cdbe3b3
• The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/qwen-3-coder-next-decodes-7x-faster
• The thread on X: https://x.com/saavoji/status/2118402118927438127
• Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#claude-opus-4