Sketchpad: certified exact arithmetic with a small vision-language model
Abstract
Language models are accurate on most long-arithmetic problems, but their accuracy is estimated on samples and carries no guarantee for an unseen input; tool use is exact but moves the computation outside the network. We show that exactness at all input lengths becomes a finite, checkable property of a network when each of its steps reads only a bounded view of an external working memory. In Sketchpad, Qwen3.5-0.8B with a low-rank adapter executes arithmetic procedures on a rendered sheet. Each forward pass reads a three-cell view and emits one step: a digit, a carry or borrow, and the next operation. A fixed runtime moves the view and writes no digit of a result. We prove in Lean that the model is then exact on all inputs of any length if and only if it is exact on the finite set of views the runtime presents, that no finite test suffices when steps also read earlier steps, and that the guarantee tolerates bounded numerical perturbation and extends to compositions of procedures. All 2,446 views pass on the deployment device, a laptop GPU running the model in a web browser. The certified model performs signed addition and subtraction, long multiplication and division, square roots and greatest common divisors at arbitrary length. It solved 200 of 200 pre-registered problems in the browser, and 220 of 220 sums of up to 1,000 digits and products of up to 20 digits, of which GPT-6.1 Sol, the strongest of ten other models, solved 217. A symbolic runtime built on these procedures solves, expands and factors polynomials with every number computed by the model, and a multiplicative gate on the adapter recovers the base model exactly when the sheet is not in use.
Community
Author here. Can a small model be proven exact at arithmetic on inputs of every length?
🧮 The theory
If each step of a model reads only a bounded view of an external memory, exactness at all lengths reduces to exactness on a finite set of views. Proved in Lean, along with:
- Necessity: once steps can also read earlier steps, no finite test suffices.
- Robustness: the guarantee survives bounded numerical error.
- Composition: certified procedures stay certified when combined.
⚙️ In practice
Qwen3.5-0.8B + a small adapter passes all 2,446 views on a laptop GPU, in the browser. It is certified for + − × ÷ √ gcd at any length.
| 220 pre-registered problems | Score |
|---|---|
| Sketchpad (0.8B) | 220 |
| GPT-6.1 Sol · Claude Opus 5.5 · Gemini 4 Argon | 217 · 216 · 207 |
| Five open-weight models | 149–199 |
| Qwen3.5-27B · Qwen3.5-0.8B (base) | 121 · 23 |
On 1,000-digit sums: Sketchpad 20/20; frontier models 19, 17, 9; every open-weight model ≤ 2.
Get this paper in your agent:
hf papers read 2611.04434 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 2
mohsenvand/sketchpad-0.8b-runs
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper