qwenagent worker checkpoints (Qwen3-0.6B, layer 12)
Small trainable "worker" modules (private GRU slot states, a learned router, a return channel) that are injected into a frozen Qwen3-0.6B at the output of layer 12.
These files contain only the worker parameters (about 1.8 MB each); they do not contain, and are not a re-upload of, the Qwen3-0.6B weights, which you load from Qwen/Qwen3-0.6B.
- Code, task generator, results and explainer video: https://github.com/Bobliuuu/qwenagent
- The license shown is the base model's (Apache-2.0); the GitHub repo does not state a separate license.
What is here
| file | mode | what it is |
|---|---|---|
checkpoints/q7_adapter_s0.pt |
adapter | stateless per-token MLP at layer 12, parameter-matched |
checkpoints/q7_single_s0.pt |
single | ONE persistent GRU state, parameter-matched |
checkpoints/q7_uniform_s0.pt |
uniform | K private states, every token updates every slot, no routing |
checkpoints/q7_multi_s0.pt |
multi | K=4 private states with a learned router (the "subagent" claim) |
checkpoints/q7_oracle_s0.pt |
oracle | K private states, router replaced by true register ids |
All are the seed-0 run of q7: query task on interleaved register programs, trained on 2-3 registers with random register names, 8000 steps, K=4, layer 12, occupancy gating on.
Each .pt is {"state": state_dict, "args": training args}; the matching per-run result JSON is in results/.
Results of these exact checkpoints (accuracy %, chance 10%, 200 programs per size, from results/q7_*_s0.json)
| mode | P=2 | P=3 | P=4 | P=6 | P=8 | P=12 | P=16 |
|---|---|---|---|---|---|---|---|
| adapter | 60.5 | 59.0 | 49.5 | 47.5 | 41.0 | 33.5 | 27.5 |
| single | 61.0 | 58.5 | 55.5 | 46.0 | 41.0 | 39.0 | 25.0 |
| uniform | 57.0 | 55.5 | 54.5 | 24.0 | 12.5 | 11.0 | 10.0 |
| multi | 53.5 | 51.0 | 53.5 | 43.5 | 38.0 | 33.5 | 26.5 |
| oracle | 48.0 | 48.5 | 41.5 | 27.0 | 13.5 | 11.0 | 12.5 |
Single seed. Over 5 seeds in the GitHub repo, private slots do not beat a stateless adapter or a single shared state, and the slot version is less stable across seeds. This is a research artifact of a mostly negative result, not a model for deployment.
How to load / run
git clone https://github.com/Bobliuuu/qwenagent && cd qwenagent
pip install torch transformers
huggingface-cli download Bobliu/qwenagent-workers --local-dir hf
python eval_ckpt.py hf/checkpoints/q7_multi_s0.pt --out /tmp/probe_q7_multi_s0.json # reloads Qwen/Qwen3-0.6B plus the workers
eval_ckpt.py rebuilds the model and the layer-12 hook from train.py, so the frozen base model is downloaded from Qwen/Qwen3-0.6B on first use. Training needs a CUDA GPU.