mfzzzzzz commited on
Commit
98f38ad
·
verified ·
1 Parent(s): aa51569

Add public AutoMaster model card, benchmark results and project figures

Browse files
.gitattributes CHANGED
@@ -34,3 +34,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
 
 
 
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
+ assets/automaster-logo.png filter=lfs diff=lfs merge=lfs -text
38
+ assets/automaster-framework.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -1,6 +1,107 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # AutoMaster-bc-9b
2
 
3
- This repository contains an AutoMaster checkpoint based on Qwen3.5-9B. AutoMaster uses Skill-Guided Fine-Tuning (Skill-FT) to train agent models on skill-guided teacher demonstrations.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
 
5
- - [Paper: AutoMaster: Autonomous Post-Training by Making the Same Model a Better Teacher](https://arxiv.org/abs/2611.26510)
6
- - [AutoMaster model collection](https://huggingface.co/collections/RUC-AIBOX/automaster)
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen/Qwen3.5-9B
3
+ library_name: transformers
4
+ pipeline_tag: text-generation
5
+ language:
6
+ - en
7
+ tags:
8
+ - automaster
9
+ - skill-ft
10
+ - llm-agents
11
+ - tool-use
12
+ - arxiv:2611.26510
13
+ ---
14
+
15
  # AutoMaster-bc-9b
16
 
17
+ <p align="center">
18
+ <img src="assets/automaster-logo.png" alt="AutoMaster: a teacher robot shares a skill with a student robot" width="760">
19
+ </p>
20
+
21
+ **AutoMaster-bc-9b is the 9B AutoMaster checkpoint for multi-step web search and evidence-based question answering.** Based on **Qwen3.5-9B**, it is trained with **Skill-Guided Fine-Tuning (Skill-FT)** on demonstrations produced by a skill-guided teacher.
22
+
23
+ <p align="center">
24
+ <a href="https://arxiv.org/abs/2611.26510"><img src="https://img.shields.io/badge/Paper-arXiv-B31B1B?logo=arxiv&amp;logoColor=white" alt="Paper"></a>
25
+ <a href="https://aweai-team.github.io/AutoMaster/"><img src="https://img.shields.io/badge/Project-Homepage-606f2e" alt="Project homepage"></a>
26
+ <a href="https://huggingface.co/collections/RUC-AIBOX/automaster"><img src="https://img.shields.io/badge/🤗_Hugging_Face-Collection-FFD21E" alt="AutoMaster model collection"></a>
27
+ <a href="https://github.com/RUCAIBox/AutoMaster"><img src="https://img.shields.io/badge/Code-GitHub-181717?logo=github&amp;logoColor=white" alt="Code on GitHub"></a>
28
+ </p>
29
+
30
+ ## Model Overview
31
+
32
+ | Property | Description |
33
+ | --- | --- |
34
+ | Base model | [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) |
35
+ | Domain | Search |
36
+ | Benchmark | BrowseComp (Lite-185) |
37
+ | Training | Full-parameter supervised fine-tuning with Skill-FT |
38
+ | Release format | Full model weights in Hugging Face Safetensors format |
39
+ | Inference framework | [AweAgent](https://github.com/RUCAIBox/AutoMaster/tree/main/recipes/inference) |
40
+
41
+ This repository contains the **fine-tuned student model**. Generated skills shape the teacher's training demonstrations and are absent during student evaluation.
42
+
43
+ ## Method: Skill-Guided Fine-Tuning
44
+
45
+ Successful trajectories can still demonstrate weak habits, such as unsupported assumptions or superficial verification. AutoMaster uses an LLM agent to study trajectories, develop natural-language skills, and select demonstrations that teach more effective behavior.
46
+
47
+ - **Inner loop:** analyze behavior, design and test skills, and curate teacher demonstrations.
48
+ - **Outer loop:** fine-tune the student and use evaluation feedback to guide the next round.
49
+
50
+ <p align="center">
51
+ <img src="assets/automaster-framework.png" alt="AutoMaster inner loop for skill design and demonstration selection, and outer loop for student training and evaluation" width="100%">
52
+ </p>
53
+
54
+ *Framework figure from the [AutoMaster paper](https://arxiv.org/abs/2611.26510), distributed with the [public project repository](https://github.com/RUCAIBox/AutoMaster).*
55
+
56
+ ## Training
57
+
58
+ The paper uses **512 training trajectories** and **three epochs of full-parameter SFT** per run. Skill instructions remain in the training context. The Vanilla SFT comparison uses the same teacher, task pool, training set size, and SFT recipe, with successful unguided teacher trajectories.
59
+
60
+ ## Evaluation
61
+
62
+ Results on **BrowseComp (Lite-185)**, as reported in [Table 1 of the paper](https://arxiv.org/html/2611.26510v1#S4.T1):
63
+
64
+ | Model | Accuracy (%) ↑ (mean ± SD) |
65
+ | --- | ---: |
66
+ | Qwen3.5-9B | 35.50 ± 2.72 |
67
+ | Qwen3.5-9B + Vanilla SFT | 46.85 ± 2.98 |
68
+ | **AutoMaster-bc-9b** | **51.53 ± 1.25** |
69
+
70
+ Scores are the mean ± sample standard deviation over **three evaluation runs per checkpoint**. Evaluation uses the domain-specific AweAgent configuration, without generated skill prompts. See the [paper](https://arxiv.org/html/2611.26510v1) and [inference guide](https://github.com/RUCAIBox/AutoMaster/blob/main/recipes/inference/README.md) for task assets, tools, and evaluation settings.
71
+
72
+ ## Using the Model
73
+
74
+ Use a Qwen3.5-compatible serving backend and keep the checkpoint's tokenizer and chat template with its weights. The public repository provides a serving script and benchmark configurations:
75
+
76
+ ```bash
77
+ # Run from the AutoMaster repository after setting up its inference environment.
78
+ export MODEL_PATH="RUC-AIBOX/AutoMaster-bc-9b"
79
+ export SERVED_MODEL_NAME=automaster
80
+ bash recipes/inference/serve_qwen35.sh
81
+ ```
82
+
83
+ In a separate terminal with AweAgent installed and the benchmark assets configured:
84
+
85
+ ```bash
86
+ awe-agent run --config recipes/inference/browsecomp.yaml --max-instances 1
87
+ ```
88
+
89
+ Follow the [inference guide](https://github.com/RUCAIBox/AutoMaster/blob/main/recipes/inference/README.md) to configure the endpoint and required tools or sandbox.
90
+
91
+ ## Related Models
92
+
93
+ [bc-4b](https://huggingface.co/RUC-AIBOX/AutoMaster-bc-4b) · [bc-9b](https://huggingface.co/RUC-AIBOX/AutoMaster-bc-9b) · [doc2repo-4b](https://huggingface.co/RUC-AIBOX/AutoMaster-doc2repo-4b) · [doc2repo-9b](https://huggingface.co/RUC-AIBOX/AutoMaster-doc2repo-9b) · [tb21-4b](https://huggingface.co/RUC-AIBOX/AutoMaster-tb21-4b) · [tb21-9b](https://huggingface.co/RUC-AIBOX/AutoMaster-tb21-9b)
94
+
95
+ ## Citation
96
 
97
+ ```bibtex
98
+ @misc{chen2026automasterautonomousposttrainingmaking,
99
+ title={AutoMaster: Autonomous Post-Training by Making the Same Model a Better Teacher},
100
+ author={Guoxin Chen and Shuang Sun and Fanzhe Meng and Xinyu Tang and Ruihua Song and Wayne Xin Zhao},
101
+ year={2026},
102
+ eprint={2611.26510},
103
+ archivePrefix={arXiv},
104
+ primaryClass={cs.CL},
105
+ url={https://arxiv.org/abs/2611.26510}
106
+ }
107
+ ```
assets/automaster-framework.png ADDED

Git LFS Details

  • SHA256: a7a32a33ef0dad548b8742dc2316e5e726c6dafb07202fbc6b1e1d5618039ba4
  • Pointer size: 132 Bytes
  • Size of remote file: 4.23 MB
assets/automaster-logo.png ADDED

Git LFS Details

  • SHA256: f3704337d967c9080b6776aa8377501e90ca09fdd7f6bcecfd08cd3596988764
  • Pointer size: 131 Bytes
  • Size of remote file: 466 kB