| --- |
| license: mit |
| tags: |
| - world-model |
| - kv-cache |
| --- |
| |
| # shared-cache-world-model weights |
|
|
| Trained weights for the world models of |
| [shared-cache-world-model](https://github.com/jadidbourbaki/shared-cache-world-model). Each folder |
| is one run and holds `weights.pt` and the run's `summary.json`. |
|
|
| `weights.pt` holds the model state dict, the full training config, the training and evaluation |
| recording hashes, and the step count. The optimizer, scheduler, and sampler state are left out, |
| so these files cannot resume training. The full checkpoints live in the project bucket. |
|
|
| Every model has width 384, six heads, and eight blocks, and trained for 20,000 updates on |
| 128-transition windows at seeds 42, 43, and 44. |
|
|
| | Folder prefix | Model | Paper name | |
| |---|---|---| |
| | `roomnav_transition_v1_per_layer_h128` | a cache in every layer | Decoder-only | |
| | `roomnav_transition_v1_ced_full_h128` | one pair per upper layer over full lower blocks | CED | |
| | `roomnav_transition_v1_ced_h128` | one pair per upper layer over windowed lower blocks | CED-SWA | |
| | `roomnav_transition_v1_shared_cache_h128` | one pair shared by every upper layer | DD | |
| | `roomnav_equal_memory_v1_per_layer_w16_h128` | Decoder-only with a 16-transition window | Decoder-only, equal cache | |
| | `roomnav_equal_memory_v1_ced_g32_h128` | CED-SWA with a 32-transition global window | CED-SWA, equal cache | |
| | `roomnav_equal_memory_v1_ced_full_w16_g16_h128` | CED with a 16-transition window in every block | CED, equal cache | |
|
|
| Download with `just fetch-weights` from the code repository, then pass a `weights.pt` to the |
| probe as `--checkpoint`. |
|
|
