README.md

---
license: mit
tags:
- world-model
- kv-cache
---

# Context Compaction for World Models: weights

Trained weights for the world models of
[context-compaction-world-models](https://github.com/jadidbourbaki/context-compaction-world-models). Each folder
is one run and holds `weights.pt` and the run's `summary.json`.

`weights.pt` holds the model state dict, the full training config, the training and evaluation
recording hashes, and the step count. The optimizer, scheduler, and sampler state are left out,
so these files cannot resume training. The full checkpoints live in the project bucket.

Every model has width 384, six heads, and eight blocks, and trained for 20,000 updates on
128-transition windows at seeds 42, 43, and 44.

| Folder prefix | Model | Paper name |
|---|---|---|
| `roomnav_transition_v1_per_layer_h128` | a cache in every layer | Decoder-only |
| `roomnav_transition_v1_ced_full_h128` | one pair per upper layer over full lower blocks | CED |
| `roomnav_transition_v1_ced_h128` | one pair per upper layer over windowed lower blocks | CED-SWA |
| `roomnav_transition_v1_shared_cache_h128` | one pair shared by every upper layer | DD |
| `roomnav_equal_memory_v1_per_layer_w16_h128` | Decoder-only with a 16-transition window | Decoder-only, equal cache |
| `roomnav_equal_memory_v1_ced_g32_h128` | CED-SWA with a 32-transition global window | CED-SWA, equal cache |
| `roomnav_equal_memory_v1_ced_full_w16_g16_h128` | CED with a 16-transition window in every block | CED, equal cache |

Download with `just fetch-weights` from the code repository, then pass a `weights.pt` to the
probe as `--checkpoint`.