CoEvoX · who_when model-harness pair

arXiv

Model-harness pair for the who_when domain from CoEvo-RSI: When Harnesses Evolve, Which Models Should Follow?

Domain who_when
Model repo root (Qwen3.5-9B + full-parameter SFT)
Harness harness/
Paper arXiv:2610.25078

Reported scores

Failure attribution (agent / step accuracy, %), evaluated with the harness/ in this repo. The test benchmarks are held out from both training and evolutionary selection.

Benchmark Who (agent) When (step)
Who&When Hand-Crafted 68.39 29.31
Who&When Algorithm-Generated 62.17 44.71
Who&When Pro 69.43 57.39
TraceElephant 70.78 28.92

Layout

.            # model weights (config.json, *.safetensors, tokenizer, ...)
harness/     # harness source for this pair

Inference

Serve the model with vLLM. The tool-call and reasoning parsers are required, because the harness talks to the model through tool calls.

huggingface-cli download nullptr04/CoEvo-RSI-who-when --local-dir ckpts/CoEvo-RSI-who-when

CUDA_VISIBLE_DEVICES=0 vllm serve ckpts/CoEvo-RSI-who-when \
  --host 127.0.0.1 --port 8000 \
  --served-model-name CoEvo-RSI-who-when \
  --max-model-len 65536 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3

Then, from the CoEvoX repository, run the evaluation with the evolved harness (generated, the same as harness/ here). The script reads the model from POLICY_BASE_URL (default http://127.0.0.1:8000/v1).

bash scripts/run_whowhen_full_test.sh generated

Citation

If you use this model or harness, please cite:

@misc{su2026coevorsiharnessesevolvemodels,
      title={CoEvo-RSI: When Harnesses Evolve, Which Models Should Follow?},
      author={Jiaqi Su and Guibin Zhang and Jie-Jing Shao and Cong Pang and Tiankuo Yao and Ivor Tsang and Haiyan Yin and Shuicheng Yan},
      year={2026},
      eprint={2610.25078},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2610.25078},
}
Downloads last month
81
Safetensors
Model size
1.47M params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including nullptr04/CoEvo-RSI-who-when

Paper for nullptr04/CoEvo-RSI-who-when