|
Download README.md from Bobliu/qwenagent-workers: direct link, hf CLI and curl.
- Browser
- Download file 2.9 kB
-
https://huggingface.co/Bobliu/qwenagent-workers/resolve/main/README.md
- Command line
-
hf download hf://Bobliu/qwenagent-workers/README.md
-
curl -L -o README.md https://huggingface.co/Bobliu/qwenagent-workers/resolve/main/README.md
2.9 kB
| license: apache-2.0 | |
| base_model: Qwen/Qwen3-0.6B | |
| library_name: pytorch | |
| tags: | |
| - qwen3 | |
| - interpretability | |
| - recurrent-workers | |
| - research | |
| - negative-result | |
| # qwenagent worker checkpoints (Qwen3-0.6B, layer 12) | |
| Small trainable "worker" modules (private GRU slot states, a learned router, a return channel) that are injected into a **frozen** Qwen3-0.6B at the output of layer 12. | |
| These files contain **only the worker parameters (about 1.8 MB each)**; they do not contain, and are not a re-upload of, the Qwen3-0.6B weights, which you load from `Qwen/Qwen3-0.6B`. | |
| - Code, task generator, results and explainer video: https://github.com/Bobliuuu/qwenagent | |
| - The license shown is the base model's (Apache-2.0); the GitHub repo does not state a separate license. | |
| ## What is here | |
| | file | mode | what it is | | |
| |---|---|---| | |
| | `checkpoints/q7_adapter_s0.pt` | adapter | stateless per-token MLP at layer 12, parameter-matched | | |
| | `checkpoints/q7_single_s0.pt` | single | ONE persistent GRU state, parameter-matched | | |
| | `checkpoints/q7_uniform_s0.pt` | uniform | K private states, every token updates every slot, no routing | | |
| | `checkpoints/q7_multi_s0.pt` | multi | K=4 private states with a learned router (the "subagent" claim) | | |
| | `checkpoints/q7_oracle_s0.pt` | oracle | K private states, router replaced by true register ids | | |
| All are the **seed-0** run of `q7`: query task on interleaved register programs, trained on 2-3 registers with random register names, 8000 steps, K=4, layer 12, occupancy gating on. | |
| Each `.pt` is `{"state": state_dict, "args": training args}`; the matching per-run result JSON is in `results/`. | |
| ## Results of these exact checkpoints (accuracy %, chance 10%, 200 programs per size, from `results/q7_*_s0.json`) | |
| | mode | P=2 | P=3 | P=4 | P=6 | P=8 | P=12 | P=16 | | |
| |---|---|---|---|---|---|---|---| | |
| | adapter | 60.5 | 59.0 | 49.5 | 47.5 | 41.0 | 33.5 | 27.5 | | |
| | single | 61.0 | 58.5 | 55.5 | 46.0 | 41.0 | 39.0 | 25.0 | | |
| | uniform | 57.0 | 55.5 | 54.5 | 24.0 | 12.5 | 11.0 | 10.0 | | |
| | multi | 53.5 | 51.0 | 53.5 | 43.5 | 38.0 | 33.5 | 26.5 | | |
| | oracle | 48.0 | 48.5 | 41.5 | 27.0 | 13.5 | 11.0 | 12.5 | | |
| Single seed. Over 5 seeds in the GitHub repo, **private slots do not beat a stateless adapter or a single shared state**, and the slot version is less stable across seeds. | |
| This is a research artifact of a mostly negative result, not a model for deployment. | |
| ## How to load / run | |
| ```bash | |
| git clone https://github.com/Bobliuuu/qwenagent && cd qwenagent | |
| pip install torch transformers | |
| huggingface-cli download Bobliu/qwenagent-workers --local-dir hf | |
| python eval_ckpt.py hf/checkpoints/q7_multi_s0.pt --out /tmp/probe_q7_multi_s0.json # reloads Qwen/Qwen3-0.6B plus the workers | |
| ``` | |
| `eval_ckpt.py` rebuilds the model and the layer-12 hook from `train.py`, so the frozen base model is downloaded from `Qwen/Qwen3-0.6B` on first use. Training needs a CUDA GPU. | |