metadata
license: mit
tags:
- world-model
- kv-cache
shared-cache-world-model weights
Trained weights for the world models of
shared-cache-world-model. Each folder
is one run and holds weights.pt and the run's summary.json.
weights.pt holds the model state dict, the full training config, the training and evaluation
recording hashes, and the step count. The optimizer, scheduler, and sampler state are left out,
so these files cannot resume training. The full checkpoints live in the project bucket.
Every model has width 384, six heads, and eight blocks, and trained for 20,000 updates on 128-transition windows at seeds 42, 43, and 44.
| Folder prefix | Model | Paper name |
|---|---|---|
roomnav_transition_v1_per_layer_h128 |
a cache in every layer | Decoder-only |
roomnav_transition_v1_ced_full_h128 |
one pair per upper layer over full lower blocks | CED |
roomnav_transition_v1_ced_h128 |
one pair per upper layer over windowed lower blocks | CED-SWA |
roomnav_transition_v1_shared_cache_h128 |
one pair shared by every upper layer | DD |
roomnav_equal_memory_v1_per_layer_w16_h128 |
Decoder-only with a 16-transition window | Decoder-only, equal cache |
roomnav_equal_memory_v1_ced_g32_h128 |
CED-SWA with a 32-transition global window | CED-SWA, equal cache |
roomnav_equal_memory_v1_ced_full_w16_g16_h128 |
CED with a 16-transition window in every block | CED, equal cache |
Download with just fetch-weights from the code repository, then pass a weights.pt to the
probe as --checkpoint.