Hugging Face's logo

InternScience
/
MLExpert

Text Generation
Transformers
Safetensors
qwen3_5_moe
image-text-to-text
Mixture of Experts
agentic
machine-learning-engineering
code
tool-use
reinforcement-learning
conversational
Model card Files
xet
 Community
YoungDubby commited on
Commit
d8a08ec
·
verified ·
1 Parent(s): d2467ab

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +22 -21
README.md CHANGED
@@ -10,16 +10,17 @@ tags:
10
  - tool-use
11
  - reinforcement-learning
12
  ---
13
- # M1: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering
14
 
15
- [💻 GitHub](https://github.com/InternScience/M1) · [🤗 Hugging Face](https://huggingface.co/InternScience/M1)
16
 
17
- **M1** is a 35B Mixture-of-Experts agentic model for autonomous machine learning engineering. It organizes experiments, writes and revises code, and improves solutions through sustained interaction with execution environments. Hierarchical agentic SFT develops general tool-use and domain-specific optimization skills, while **Windowed Interaction Reinforcement Learning (WIRL)** trains multi-turn decisions using feedback from completed experiments.
 
 
18
 
19
  ## Highlights
20
 
21
  - **Long-horizon optimization:** Plans experiments, interprets execution feedback, revisits previous solutions, and selects submission candidates within a persistent workspace.
22
- - **Strong MLE performance:** Achieves a **72.7% medal rate on MLE-Bench Lite**, matching GLM-5.3 and Kimi-K3 under the M1 Harness.
23
  - **General-purpose harness compatibility:** Achieves a **69.7% medal rate with Claude Code**, compared with 42.4% for its base model.
24
  - **Scientific task optimization:** Matches published scientific results on seven of ten NatureBench Lite tasks and surpasses them on five.
25
 
@@ -29,13 +30,13 @@ tags:
29
 
30
  We primarily evaluate our trained models on MLE-Bench Lite, a 22-task subset of MLE-Bench consisting of historical Kaggle competitions for end-to-end machine learning engineering. We report the percentage of tasks awarded each medal tier (Gold, Silver, and Bronze), the percentage receiving any medal (Any Medal), and the percentage exceeding the median human score (Med+), with Any Medal as the primary metric. All five metrics are computed over the 22 tasks and reported as percentages.
31
 
32
- The first five metric columns use the M1 Harness; the last two columns report the general-purpose harness and its medal rate. All values are percentages.
33
 
34
  ![MLE-Bench Lite main results (Table 1 from the paper)](./figures/mle-bench-lite-table.png)
35
 
36
- **M1 achieves the highest medal rate among same-scale models and reaches the performance level of larger frontier models.** Under the unified M1 Harness, M1 raises the medal rate of its Qwen3.6-35B-A3B base model from 37.9% to 72.7%, achieving the highest rate among the same-scale models evaluated and performance comparable to GLM-5.3 and Kimi-K3. These results show that long-horizon agentic post-training substantially improves the base model's MLE capability, bringing a 35B model to a performance level comparable to larger frontier models.
37
 
38
- **M1 retains its advantage over the base model under a general-purpose coding harness.** With Claude Code, M1 achieves a 69.7% medal rate, compared with 42.4% for its base model, and exceeds the strongest evaluated same-scale baseline by 22.7 percentage points. The consistent advantage over the base model across both harnesses suggests that the improvement from post-training is not specific to the M1-Harness and remains effective in a general-purpose agentic coding environment.
39
 
40
  ### NatureBench Lite
41
 
@@ -43,11 +44,11 @@ We also evaluate on NatureBench, following the ten-task Lite subset used in Fron
43
 
44
  ![NatureBench Lite main results (Table 2 from the paper)](./figures/naturebench-lite-table.png)
45
 
46
- **On NatureBench Lite, M1 improves scientific task performance over the base model.** With Claude Code, M1 raises the Surpass-SOTA rate from 0.0% to 50.0%, exceeding Nex-N2.5-mini at 20.0% and matching GLM-5.3 at 50.0%. Its Match-SOTA rate also improves from 40.0% to 70.0%. These results show that post-training improves scientific task optimization, enabling M1 to match or exceed published reference results on more tasks.
47
 
48
  ## Usage
49
 
50
- The commands below load `InternScience/M1` and serve an API endpoint at `http://localhost:8000/v1`.
51
 
52
  ### SGLang
53
 
@@ -63,7 +64,7 @@ uv pip install sglang
63
 
64
  ```shell
65
  python -m sglang.launch_server \
66
- --model-path InternScience/M1 \
67
  --host 127.0.0.1 \
68
  --port 8000 \
69
  --tp-size 2 \
@@ -76,7 +77,7 @@ python -m sglang.launch_server \
76
 
77
  ```shell
78
  python -m sglang.launch_server \
79
- --model-path InternScience/M1 \
80
  --host 127.0.0.1 \
81
  --port 8000 \
82
  --tp-size 2 \
@@ -99,7 +100,7 @@ uv pip install vllm --torch-backend=auto
99
  **Text Generation**
100
 
101
  ```shell
102
- vllm serve InternScience/M1 \
103
  --host 127.0.0.1 \
104
  --port 8000 \
105
  --tensor-parallel-size 2 \
@@ -111,7 +112,7 @@ vllm serve InternScience/M1 \
111
  **Tool Calling**
112
 
113
  ```shell
114
- vllm serve InternScience/M1 \
115
  --host 127.0.0.1 \
116
  --port 8000 \
117
  --tensor-parallel-size 2 \
@@ -124,13 +125,13 @@ vllm serve InternScience/M1 \
124
 
125
  ### Recommended Sampling Parameters
126
 
127
- | Parameter | Value |
128
- | :--------------------- | ----: |
129
- | `temperature` | 1.0 |
130
- | `top_p` | 0.95 |
131
- | `top_k` | 20 |
132
- | `min_p` | 0.0 |
133
- | `presence_penalty` | 1.5 |
134
- | `repetition_penalty` | 1.0 |
135
 
136
  These parameters apply to both thinking and non-thinking modes. Set `chat_template_kwargs.enable_thinking` to `true` to enable thinking or `false` to disable it. Machine-readable profiles are provided in [inference_profiles.json](./inference_profiles.json).
 
10
  - tool-use
11
  - reinforcement-learning
12
  ---
 
13
 
14
+ # MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering
15
 
16
+ [💻 GitHub](https://github.com/InternScience/MLExpert) · [🤗 Hugging Face](https://huggingface.co/InternScience/MLExpert)
17
+
18
+ **MLExpert** is a 35B Mixture-of-Experts agentic model for autonomous machine learning engineering. It organizes experiments, writes and revises code, and improves solutions through sustained interaction with execution environments. Hierarchical agentic SFT develops general tool-use and domain-specific optimization skills, while **Windowed Interaction Reinforcement Learning (WIRL)** trains multi-turn decisions using feedback from completed experiments.
19
 
20
  ## Highlights
21
 
22
  - **Long-horizon optimization:** Plans experiments, interprets execution feedback, revisits previous solutions, and selects submission candidates within a persistent workspace.
23
+ - **Strong MLE performance:** Achieves a **72.7% medal rate on MLE-Bench Lite**, matching GLM-5.3 and Kimi-K3 under the MLExpert Harness.
24
  - **General-purpose harness compatibility:** Achieves a **69.7% medal rate with Claude Code**, compared with 42.4% for its base model.
25
  - **Scientific task optimization:** Matches published scientific results on seven of ten NatureBench Lite tasks and surpasses them on five.
26
 
 
30
 
31
  We primarily evaluate our trained models on MLE-Bench Lite, a 22-task subset of MLE-Bench consisting of historical Kaggle competitions for end-to-end machine learning engineering. We report the percentage of tasks awarded each medal tier (Gold, Silver, and Bronze), the percentage receiving any medal (Any Medal), and the percentage exceeding the median human score (Med+), with Any Medal as the primary metric. All five metrics are computed over the 22 tasks and reported as percentages.
32
 
33
+ The first five metric columns use the MLExpert Harness; the last two columns report the general-purpose harness and its medal rate. All values are percentages.
34
 
35
  ![MLE-Bench Lite main results (Table 1 from the paper)](./figures/mle-bench-lite-table.png)
36
 
37
+ **MLExpert achieves the highest medal rate among same-scale models and reaches the performance level of larger frontier models.** Under the unified MLExpert Harness, MLExpert raises the medal rate of its Qwen3.6-35B-A3B base model from 37.9% to 72.7%, achieving the highest rate among the same-scale models evaluated and performance comparable to GLM-5.3 and Kimi-K3. These results show that long-horizon agentic post-training substantially improves the base model's MLE capability, bringing a 35B model to a performance level comparable to larger frontier models.
38
 
39
+ **MLExpert retains its advantage over the base model under a general-purpose coding harness.** With Claude Code, MLExpert achieves a 69.7% medal rate, compared with 42.4% for its base model, and exceeds the strongest evaluated same-scale baseline by 22.7 percentage points. The consistent advantage over the base model across both harnesses suggests that the improvement from post-training is not specific to the MLExpert-Harness and remains effective in a general-purpose agentic coding environment.
40
 
41
  ### NatureBench Lite
42
 
 
44
 
45
  ![NatureBench Lite main results (Table 2 from the paper)](./figures/naturebench-lite-table.png)
46
 
47
+ **On NatureBench Lite, MLExpert improves scientific task performance over the base model.** With Claude Code, MLExpert raises the Surpass-SOTA rate from 0.0% to 50.0%, exceeding Nex-N2.5-mini at 20.0% and matching GLM-5.3 at 50.0%. Its Match-SOTA rate also improves from 40.0% to 70.0%. These results show that post-training improves scientific task optimization, enabling MLExpert to match or exceed published reference results on more tasks.
48
 
49
  ## Usage
50
 
51
+ The commands below load `InternScience/MLExpert` and serve an API endpoint at `http://localhost:8000/v1`.
52
 
53
  ### SGLang
54
 
 
64
 
65
  ```shell
66
  python -m sglang.launch_server \
67
+ --model-path InternScience/MLExpert \
68
  --host 127.0.0.1 \
69
  --port 8000 \
70
  --tp-size 2 \
 
77
 
78
  ```shell
79
  python -m sglang.launch_server \
80
+ --model-path InternScience/MLExpert \
81
  --host 127.0.0.1 \
82
  --port 8000 \
83
  --tp-size 2 \
 
100
  **Text Generation**
101
 
102
  ```shell
103
+ vllm serve InternScience/MLExpert \
104
  --host 127.0.0.1 \
105
  --port 8000 \
106
  --tensor-parallel-size 2 \
 
112
  **Tool Calling**
113
 
114
  ```shell
115
+ vllm serve InternScience/MLExpert \
116
  --host 127.0.0.1 \
117
  --port 8000 \
118
  --tensor-parallel-size 2 \
 
125
 
126
  ### Recommended Sampling Parameters
127
 
128
+ | Parameter | Value |
129
+ | :--- | ---: |
130
+ | `temperature` | 1.0 |
131
+ | `top_p` | 0.95 |
132
+ | `top_k` | 20 |
133
+ | `min_p` | 0.0 |
134
+ | `presence_penalty` | 1.5 |
135
+ | `repetition_penalty` | 1.0 |
136
 
137
  These parameters apply to both thinking and non-thinking modes. Set `chat_template_kwargs.enable_thinking` to `true` to enable thinking or `false` to disable it. Machine-readable profiles are provided in [inference_profiles.json](./inference_profiles.json).