Blame for README.md

---
license: mit
language:
- en
tags:
- reinforcement-learning
- embodied-agent
- video-game
- pokemon
- self-improvement
- inverse-dynamics-model
base_model:
- Qwen/Qwen3-1.7B
- google/siglip-base-patch16-224
- facebook/dinov2-base
---

# ASH: Agents that Self-Hone in Long-Horizon Worlds

[Paper](https://arxiv.org/abs/2605.14211) · [Project page](https://toolbox-hq.github.io/ASH/) · [Code](https://github.com/Toolbox-HQ/ASH)

[Benjamin Schneider](https://benjaminschneider.ca/), [Xavier Schneider](https://www.linkedin.com/in/xavier-schneider-360860317/), [Victor Zhong](https://www.victorzhong.com/), [Sun Sun](https://sunsunyyl.github.io/) · NeurIPS 2026

ASH is an agentic system that learns a long-horizon policy from unlabeled, noisy internet video, without reward shaping or expert annotation. It plays *Pokémon Emerald* for hours at a time. When it gets stuck, it trains an Inverse Dynamics Model (IDM) on its own trajectories, retrieves visually similar gameplay video from the internet, labels that video with the IDM, and updates its policy. Key moments discovered in the retrieved video by unsupervised clustering form a long-term memory that lets the agent make decisions that depend on events many steps in the past.

This repository holds everything needed to run the released ASH agent in Pokémon Emerald: the base models the policy is built from, ten trained checkpoints from a single self-improvement run, and the start-of-game save state. The game ROM is not included.

## Repository contents

```
assets/
├── Qwen3-1.7B/                 # language backbone of the policy
├── siglip-base-patch16-224/    # vision encoder of the policy
├── dinov2-base/                # retrieval and key-moment embeddings
├── rnd_idm_model.pt            # IDM initialization (trained on random-policy data)
└── truck_start.state           # Pokémon Emerald save state at the start of the game
checkpoints/
├── checkpoint-1/               # earliest bootstrap
│   ├── agent.safetensors       # policy weights
│   ├── idm.pt                  # IDM weights
│   └── objective_manager.pkl   # key-moment classifier (HDBSCAN) and discovered key moments
├── ...
└── checkpoint-10/              # latest bootstrap
rom/                            # place your own Pokémon Emerald ROM here as rom.gba
```

The three base models under `assets/` are unmodified copies of the upstream Hugging Face releases, included so the agent runs fully offline. They keep their own licenses (Apache 2.0 for all three).

## Checkpoints

The ten checkpoints are successive bootstraps of one Pokémon Emerald run of four parallel agents sharing a single policy. Every time the agents got stuck, ASH retrieved internet video, retrained the IDM and the policy, and extended its set of known key moments. Later checkpoints have seen more of the game and know more key moments.

Each checkpoint is packaged as a **start-of-game** checkpoint. The policy, IDM, and key-moment classifier are exactly what the run had at that bootstrap, but the long-term memory is empty, so the agent can be launched from the beginning of the game with any of them.

| Checkpoint | Bootstraps completed | Key-moment clusters | `objective_manager.pkl` |
|---|---|---|---|
| checkpoint-1 | 1 | 467 | 2.5 GB |
| checkpoint-2 | 2 | 581 | 3.0 GB |
| checkpoint-3 | 3 | 604 | 3.1 GB |
| checkpoint-4 | 4 | 662 | 3.4 GB |
| checkpoint-5 | 5 | 702 | 3.7 GB |
| checkpoint-6 | 6 | 767 | 4.1 GB |
| checkpoint-7 | 7 | 826 | 5.3 GB |
| checkpoint-8 | 8 | 874 | 5.5 GB |
| checkpoint-9 | 9 | 909 | 6.0 GB |
| checkpoint-10 | 10 | 923 | 6.1 GB |

`agent.safetensors` is 3.0 GB and `idm.pt` is 1.8 GB in every checkpoint. The pickle grows with the checkpoint because the HDBSCAN clusterer stores the embeddings it was fit on.

### File formats

- `agent.safetensors`: a `safetensors` state dict for the ASH policy (`LMObjectiveAgent` in the code repository), which combines the SigLIP vision encoder and the Qwen3-1.7B language model and conditions on the current frames, recent actions, and up to ten key-moment images from long-term memory.
- `idm.pt`: a PyTorch state dict for the IDM, which predicts one of nine Game Boy Advance inputs from a window of frames.
- `objective_manager.pkl`: a Python pickle of the key-moment manager, containing a fitted cuML HDBSCAN clusterer and the set of valid clusters. Loading it requires the code repository and a GPU with cuML installed.

## Usage

The agent is run through the code repository at https://github.com/Toolbox-HQ/ASH, which drives the mGBA emulator, the policy, the IDM, and the self-improvement loop. Point its inference configuration at this repository's folders: the three model directories under `assets/`, the start state, one of the `checkpoints/checkpoint-N/` folders, and your ROM at `rom/rom.gba`. Detailed run instructions live in the code repository.

To download the repository:

```bash
git lfs install
git clone https://huggingface.co/Toolbox-HQ/ASH
```

To download a single checkpoint together with the assets:

```bash
hf download Toolbox-HQ/ASH --include "assets/*" --include "checkpoints/checkpoint-5/*" --local-dir ASH
```

The full repository is about 100 GB.

## Intended use and limitations

- ASH is a research artifact for studying self-improving agents in long-horizon environments. The checkpoints play Pokémon Emerald only; the Zelda results in the paper come from a separate run that is not released here.
- The checkpoints are not an end-to-end "solver". In the paper's 8-hour evaluation ASH reaches an average of 11.2 of 12 human-defined milestones, and runs are stochastic, with temperature sampling over actions.
- The agent needs a GPU with enough memory for the 1.7B-parameter policy, the IDM, DINOv2, and the clusterer, and it needs a copy of the Pokémon Emerald ROM, which you must supply yourself.

## Citation

```bibtex
@misc{schneider2026ashagentsselfhonelonghorizon,
      title={ASH: Agents that Self-Hone in Long-Horizon Worlds}, 
      author={Benjamin Schneider and Xavier Schneider and Victor Zhong and Sun Sun},
      year={2026},
      eprint={2605.14211},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2605.14211}, 
}
```