Attention at absolute zero: cold attention on CLRS-30
The trained models of cold attention. An attention layer sums evidence over its witnesses and reads the sum at a temperature; as the temperature falls to zero, the sum keeps two exact quantities, the score of the best witness and the number of witnesses that reach it. Cold attention computes them and passes the rest of the network only their comparisons.
One small block of it (about 150,000 parameters, one configuration) trained separately on each of the 30 algorithms of DeepMind's CLRS-30 benchmark, from inputs of at most 16 items, and tested at 64. This repository holds every trained model: 152 models, 30 tasks, five blocks, three seeds each (five for compare-first quickselect).
Paper: Attention at Absolute Zero (arXiv:2611.11937). Code, training and the demo: see the companion repository, github.com/NeoVand/cold-attention; its REPRODUCE.md says how each model was checked. The demo page runs these models in the browser.
The block in one paragraph
Every ordered pair of items (u, v) carries a vector of 128 numbers. In each loop the block asks two learned yes-or-no questions of every middle item k: a test on the pair (u, k) and a test on the pair (k, v). An item that passes both is a witness for (u, v). Half of the 128 channels count their witnesses exactly and pass on only comparisons between two counts ("more", "same"); the other half keep the best witness. The block is looped, and its answer is read by a fixed rule. The block runs in two settings. In the general setting the input's numbers enter through the learned embedding; in the cold setting they stay in exact registers and only comparisons of them enter the learned state.
| Block | Setting | Numbers enter | Read |
|---|---|---|---|
census |
general | through the learned input embedding | after 8 loops |
cold |
cold | none (the task has no input numbers); only bits enter the state | when its bits stop changing (at most n loops) |
compare_first |
cold | never: input numbers sit in registers and the state receives only comparisons of them (on depth-first search, the positions; its bits are not kept) | when its bits stop changing (at most n loops) |
pair_numbers |
cold | as above, plus one number per pair the block builds (a best path sum or bottleneck); Floyd-Warshall adds via pairs, Prim a second built number (the hops) | when its bits and numbers stop changing (at most 4n loops) |
census_bilinear |
general | as census; its first 16 count channels count a witness only if it also passes a bilinear test of the two legs' states (rank 2), trained 320,000 steps (segments intersect) |
after 8 loops |
Results
Best block per task
Average over the 30 tasks: 98.47 (FloydNet 96.64); at or above FloydNet on 25 of 30. A selection over blocks per task, so optimistic; the census block alone (one configuration on all 30) averages 97.04. Where blocks tie, the table names the block the task's symmetry calls for (best.json lists the others).
| Task | FloydNet | Best block | Mean test (seeds) |
|---|---|---|---|
| BFS | 100.00 | census |
100.00 (3) |
| Bellman-Ford | 100.00 | pair_numbers |
100.00 (3) |
| Dijkstra | 97.67 | pair_numbers |
100.00 (3) |
| DAG Shortest Paths | 100.00 | pair_numbers |
100.00 (3) |
| Floyd-Warshall | 96.20 | pair_numbers |
100.00 (3) |
| MST-Prim | 99.67 | pair_numbers |
99.98 (3) |
| Topological Sort | 100.00 | cold |
100.00 (3) |
| DFS | 100.00 | compare_first |
100.00 (3) |
| SCC | 90.07 | census |
90.61 (3) |
| Articulation Points | 100.00 | census |
100.00 (3) |
| Bridges | 100.00 | census |
100.00 (3) |
| MST-Kruskal | 99.63 | pair_numbers |
100.00 (3) |
| Insertion Sort | 100.00 | compare_first |
100.00 (3) |
| Bubble Sort | 100.00 | compare_first |
100.00 (3) |
| Heapsort | 100.00 | compare_first |
100.00 (3) |
| Quicksort | 100.00 | compare_first |
100.00 (3) |
| Minimum | 100.00 | compare_first |
100.00 (3) |
| Quickselect | 80.33 | compare_first |
100.00 (5) |
| Binary Search | 94.47 | compare_first |
100.00 (3) |
| Find Max. Subarray | 86.20 | census |
87.86 (3) |
| Matrix Chain Order | 91.50 | census |
99.80 (3) |
| Optimal BST | 82.87 | census |
82.71 (3) |
| LCS Length | 95.70 | census |
95.66 (3) |
| Activity Selector | 95.30 | compare_first |
99.94 (3) |
| Task Scheduling | 91.05 | compare_first |
99.86 (3) |
| Naive String Match | 100.00 | census |
99.07 (3) |
| KMP Matcher | 99.43 | census |
99.15 (3) |
| Segments Intersect | 99.90 | census_bilinear |
99.79 (3) |
| Graham Scan | 99.63 | census |
99.80 (3) |
| Jarvis' March | 99.50 | census |
99.92 (3) |
Every block
| FloydNet | Census block (one configuration, 3 seeds) | |
|---|---|---|
| Average accuracy over the 30 tasks, 64 items | 96.64 | 97.04 (seeds 97.14, 97.07, 96.91) |
| Tasks above / equal / below FloydNet | 10 / 7 / 13 |
| Task | FloydNet | Census block | Bilinear tests | Cold block (n loops) | Compare-first | Exact pair numbers |
|---|---|---|---|---|---|---|
| Activity Selector | 95.30 | 99.40 | 99.94 | |||
| Articulation Points | 100.00 | 100.00 | ||||
| Bellman-Ford | 100.00 | 99.67 | 100.00 | |||
| BFS | 100.00 | 100.00 | ||||
| Binary Search | 94.47 | 98.89 | 100.00 | |||
| Bridges | 100.00 | 100.00 | ||||
| Bubble Sort | 100.00 | 100.00 | 100.00 | |||
| DAG Shortest Paths | 100.00 | 99.84 | 100.00 | |||
| DFS | 100.00 | 72.97 | 100.00 | 100.00 | ||
| Dijkstra | 97.67 | 99.63 | 100.00 | |||
| Find Max. Subarray | 86.20 | 87.86 | ||||
| Floyd-Warshall | 96.20 | 99.71 | 100.00 | |||
| Graham Scan | 99.63 | 99.80 | ||||
| Heapsort | 100.00 | 100.00 | 100.00 | |||
| Insertion Sort | 100.00 | 100.00 | 100.00 | |||
| Jarvis' March | 99.50 | 99.92 | ||||
| KMP Matcher | 99.43 | 99.15 | ||||
| LCS Length | 95.70 | 95.66 | ||||
| Matrix Chain Order | 91.50 | 99.80 | ||||
| Minimum | 100.00 | 100.00 | 100.00 | |||
| MST-Kruskal | 99.63 | 99.49 | 100.00 | |||
| MST-Prim | 99.67 | 98.93 | 99.98 | |||
| Naive String Match | 100.00 | 99.07 | ||||
| Optimal BST | 82.87 | 82.71 | ||||
| Quickselect | 80.33 | 99.77 | 100.00 (5) | |||
| Quicksort | 100.00 | 99.98 | 100.00 | |||
| SCC | 90.07 | 90.61 | 87.79 | |||
| Segments Intersect | 99.90 | 99.33 | 99.79 | |||
| Task Scheduling | 91.05 | 89.00 | 99.86 | |||
| Topological Sort | 100.00 | 99.99 | 100.00 |
Default block per task (the one the task's symmetry calls for; the demo opens on it): compare-first for the sorts, minimum, quickselect, binary search, activity selector, task scheduling and depth-first search (positions in its register); exact pair numbers for Bellman-Ford, Dijkstra, DAG shortest paths, Floyd-Warshall, Kruskal and Prim; the cold block for topological sort and strongly connected components; the census block for the rest.
Test accuracy (%) on the official CLRS-30 test split (64 items; 80 for the string matchers), the mean over seeds (3 unless a count is given in brackets), read by each block's registered rule and scored with the library's scorer. FloydNet: arXiv 2601.19094, Table 9 (best of runs with and without hints).
Files
<block>/<task>/seed<k>/
model.safetensors parameters (float32), names like blk/a, in30/e1, one30/min
config.json task, block, model and training configuration, test and val scores, parameter count
reference_predictions.npz the model's predictions on the official test and val splits (the registered read and
every loop), made when it was trained; the code reproduces them (check_release.py)
index.json every model with its scores
best.json per task, the block with the best mean test score over its seeds (ties: the block the
task's symmetry calls for)
Use
from witness_clrs import load_hub, registered_read, score, OUTPUTS
from witness_clrs.data import load_split
params, cfg = load_hub("mohsenvand/cold-attention-clrs30", "census/bfs/seed0")
d = load_split("data/bfs/test.npz") # official split, written by witness_clrs.generate
outs = [(o["name"], o["kind"]) for o in cfg["outputs"]]
pred, loops = registered_read(params, cfg["model"], outs, d)
print(score(pred, d, outs)) # 1.0
Training data and evaluation
Data from the library itself (dm-clrs 2.0.3): the official test (64 items, 80 for the string matchers) and val (16)
splits, and training pools drawn as the library's own training script draws them (sizes 4 to 16). No hints
(intermediate steps) are used. Scores use the library's scorer. Each model's training configuration is in its
config.json.
Limits
Tasks whose answers depend on sums of numbers that are not comparisons (optimal BST, the maximum subarray, LCS) remain below 100%. Depth-first search reaches 100% only with the cold block read after n loops (the census block, read after 8, scores 73%). On strongly connected components the cold block (86 to 90%) is below the census block (about 91%). The cold-setting blocks cover only the tasks listed.
License
Apache 2.0.
Citation
@misc{mohsenvand2026absolutezero,
title = {Attention at Absolute Zero},
author = {Mohsenvand, Mostafa},
year = {2026},
eprint = {2611.11937},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2611.11937}
}