SeaWolf-AI commited on
Commit
ff75693
·
verified ·
1 Parent(s): a43d65f

card: link POCKET-Darwin-180B-GGUF (4-bit, laptop / CPU-only)

Browse files
Files changed (1) hide show
  1. README.md +95 -91
README.md CHANGED
@@ -1,91 +1,95 @@
1
- ---
2
- license: other
3
- license_name: qwen-community-1.0
4
- license_link: LICENSE
5
- language: [en, ko, zh, ja, multilingual]
6
- library_name: transformers
7
- pipeline_tag: image-text-to-text
8
- base_model: FINAL-Bench/Darwin-180B-RSI
9
- tags:
10
- - darwin
11
- - darwin-rsi
12
- - model-level-rsi
13
- - recursive-self-improvement
14
- - self-improvement
15
- - reasoning
16
- - moe
17
- - ztc
18
- ---
19
-
20
- # Darwin-180B-RSI-R3
21
-
22
- **The second round of model-level self-improvement on top of [Darwin-180B-RSI](https://huggingface.co/FINAL-Bench/Darwin-180B-RSI).**
23
-
24
- Darwin-180B-RSI (R1) holds first place on seven Hugging Face official leaderboards. R3 continues training from R1 with the same recipe: the model solves verifiable problems, keeps only its own solutions that check out as correct, and trains on them. No human-written solutions or reasoning traces are used.
25
-
26
- R3 is released so anyone can download it, run it, and check the numbers below.
27
-
28
- ## What changed from R1
29
-
30
- - **Starting point:** R1 weights (not the parent). R3 is a true second round.
31
- - **Practice problems:** 3,000 SuperGPQA questions (middle and hard difficulty) that were never used in R1 training. R1 solved each one 8 times.
32
- - **What it learned from:** only the "boundary" problems, where R1 was right on 2 to 6 of 8 attempts (462 problems). From those, up to 2 of R1's own correct, untruncated solutions per problem (714 solutions in total).
33
- - **What was trained:** the same components as R1 (attention paths and shared experts) via LoRA, then merged. All 512 routed experts, the router and the vision encoder are unchanged.
34
- - **No benchmark data:** GPQA Diamond and the held-out set below were never used for training or selection.
35
-
36
- ## Results (same settings for every model)
37
-
38
- ### Held-out SuperGPQA, 1,000 questions never used in training or selection
39
- 4 samples per question, 16K thinking budget, temperature 1.0.
40
-
41
- | Model | Single sample | Mean of 4 | Majority of 4 |
42
- |---|---:|---:|---:|
43
- | R1 (Darwin-180B-RSI) | 65.30 | 65.67 | 68.30 |
44
- | **R3 (this model)** | **66.30** | **66.70** | **69.00** |
45
-
46
- Paired per-question difference, R1 → R3 (mean of 4): **+1.03 points, 95% CI [+0.05, +2.00]**. The gain is small but statistically significant.
47
-
48
- ### GPQA Diamond, 198 questions
49
- 8 samples per question, 32K thinking budget, temperature 1.0.
50
-
51
- | Model | Single sample | Mean of 8 | Majority of 8 |
52
- |---|---:|---:|---:|
53
- | R0 (parent, Qwen3.8-Flash-Next) | 84.85 | 85.35 | 90.91 |
54
- | R1 (Darwin-180B-RSI) | 84.85 | 85.80 | 89.90 |
55
- | **R3 (this model)** | **85.86** | **86.05** | 90.40 |
56
-
57
- Paired differences on GPQA (mean of 8): R0 → R3 +0.69 [−0.71, +2.10], R1 → R3 +0.25 [−1.18, +1.69]. With 198 questions these are within noise; we report them as measured.
58
-
59
- **How to read this:** on a large held-out set, the second round of self-improvement produced a measurable gain over R1. On GPQA the model was already near its ceiling and the differences are not significant.
60
-
61
- ## Leaderboard status
62
-
63
- The seven Hugging Face leaderboard #1 results belong to **R1** ([Darwin-180B-RSI](https://huggingface.co/FINAL-Bench/Darwin-180B-RSI)). R3 has not been submitted to any leaderboard. We are asking independent evaluators to measure it directly.
64
-
65
- ## Quickstart
66
-
67
- Serving is the same as Darwin-180B-RSI (vLLM, tensor parallel + expert parallel). This is a reasoning model; give it a long generation budget.
68
-
69
- ```bash
70
- vllm serve FINAL-Bench/Darwin-180B-RSI-R3 \
71
- --tensor-parallel-size 8 --enable-expert-parallel \
72
- --max-model-len 139264
73
- ```
74
-
75
- Recommended sampling: temperature 1.0, top_p 0.95, top_k 20, up to 131,072 generated tokens.
76
-
77
- ## ZTC
78
-
79
- The ZTC probe published with Darwin-180B-RSI was fitted on R1's hidden states. A probe fitted on R3 will be added to this repository; until then, use R1's probe only as a rough signal.
80
-
81
- ## Model-level RSI vs. harness-level RSI
82
-
83
- Darwin-180B-RSI-R3 is **Model-level RSI**: the model itself (its weights) improves by learning only from its own solutions. No human-written solutions or reasoning traces are used; correctness is checked automatically. Harness-level RSI (e.g., Google's RRSI) improves the prompts, tools and workflow around a fixed model. The two are complementary.
84
-
85
- ## License
86
-
87
- Qwen Community License 1.0 (inherited from Qwen3.8-Flash-Next). See [LICENSE](LICENSE).
88
-
89
- ## About
90
-
91
- Built by [VIDRAFT](https://vidraft.net). Darwin family paper: [arXiv 2605.14386](https://arxiv.org/abs/2605.14386).
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: qwen-community-1.0
4
+ license_link: LICENSE
5
+ language: [en, ko, zh, ja, multilingual]
6
+ library_name: transformers
7
+ pipeline_tag: image-text-to-text
8
+ base_model: FINAL-Bench/Darwin-180B-RSI
9
+ tags:
10
+ - darwin
11
+ - darwin-rsi
12
+ - model-level-rsi
13
+ - recursive-self-improvement
14
+ - self-improvement
15
+ - reasoning
16
+ - moe
17
+ - ztc
18
+ ---
19
+
20
+ # Darwin-180B-RSI-R3
21
+
22
+ **The second round of model-level self-improvement on top of [Darwin-180B-RSI](https://huggingface.co/FINAL-Bench/Darwin-180B-RSI).**
23
+
24
+ Darwin-180B-RSI (R1) holds first place on seven Hugging Face official leaderboards. R3 continues training from R1 with the same recipe: the model solves verifiable problems, keeps only its own solutions that check out as correct, and trains on them. No human-written solutions or reasoning traces are used.
25
+
26
+ R3 is released so anyone can download it, run it, and check the numbers below.
27
+
28
+ <a href="https://huggingface.co/FINAL-Bench/POCKET-Darwin-180B-GGUF"><img src="https://img.shields.io/badge/💻_POCKET_4--bit-Laptop_·_CPU_only_·_DGX_Spark-0f766e?style=for-the-badge"></a>
29
+
30
+ > 💻 **Run it on your own machine — [POCKET-Darwin-180B-GGUF](https://huggingface.co/FINAL-Bench/POCKET-Darwin-180B-GGUF)**: the 4-bit GGUF of R3 (111 GB) runs on a **laptop with an 8 GB GPU and 32 GB RAM**, **CPU only at 18–21 tok/s**, a 128 GB mini PC or one DGX Spark — MMLU-Pro **identical to BF16 (87.65%)**.
31
+
32
+ ## What changed from R1
33
+
34
+ - **Starting point:** R1 weights (not the parent). R3 is a true second round.
35
+ - **Practice problems:** 3,000 SuperGPQA questions (middle and hard difficulty) that were never used in R1 training. R1 solved each one 8 times.
36
+ - **What it learned from:** only the "boundary" problems, where R1 was right on 2 to 6 of 8 attempts (462 problems). From those, up to 2 of R1's own correct, untruncated solutions per problem (714 solutions in total).
37
+ - **What was trained:** the same components as R1 (attention paths and shared experts) via LoRA, then merged. All 512 routed experts, the router and the vision encoder are unchanged.
38
+ - **No benchmark data:** GPQA Diamond and the held-out set below were never used for training or selection.
39
+
40
+ ## Results (same settings for every model)
41
+
42
+ ### Held-out SuperGPQA, 1,000 questions never used in training or selection
43
+ 4 samples per question, 16K thinking budget, temperature 1.0.
44
+
45
+ | Model | Single sample | Mean of 4 | Majority of 4 |
46
+ |---|---:|---:|---:|
47
+ | R1 (Darwin-180B-RSI) | 65.30 | 65.67 | 68.30 |
48
+ | **R3 (this model)** | **66.30** | **66.70** | **69.00** |
49
+
50
+ Paired per-question difference, R1 → R3 (mean of 4): **+1.03 points, 95% CI [+0.05, +2.00]**. The gain is small but statistically significant.
51
+
52
+ ### GPQA Diamond, 198 questions
53
+ 8 samples per question, 32K thinking budget, temperature 1.0.
54
+
55
+ | Model | Single sample | Mean of 8 | Majority of 8 |
56
+ |---|---:|---:|---:|
57
+ | R0 (parent, Qwen3.8-Flash-Next) | 84.85 | 85.35 | 90.91 |
58
+ | R1 (Darwin-180B-RSI) | 84.85 | 85.80 | 89.90 |
59
+ | **R3 (this model)** | **85.86** | **86.05** | 90.40 |
60
+
61
+ Paired differences on GPQA (mean of 8): R0 → R3 +0.69 [−0.71, +2.10], R1 → R3 +0.25 [−1.18, +1.69]. With 198 questions these are within noise; we report them as measured.
62
+
63
+ **How to read this:** on a large held-out set, the second round of self-improvement produced a measurable gain over R1. On GPQA the model was already near its ceiling and the differences are not significant.
64
+
65
+ ## Leaderboard status
66
+
67
+ The seven Hugging Face leaderboard #1 results belong to **R1** ([Darwin-180B-RSI](https://huggingface.co/FINAL-Bench/Darwin-180B-RSI)). R3 has not been submitted to any leaderboard. We are asking independent evaluators to measure it directly.
68
+
69
+ ## Quickstart
70
+
71
+ Serving is the same as Darwin-180B-RSI (vLLM, tensor parallel + expert parallel). This is a reasoning model; give it a long generation budget.
72
+
73
+ ```bash
74
+ vllm serve FINAL-Bench/Darwin-180B-RSI-R3 \
75
+ --tensor-parallel-size 8 --enable-expert-parallel \
76
+ --max-model-len 139264
77
+ ```
78
+
79
+ Recommended sampling: temperature 1.0, top_p 0.95, top_k 20, up to 131,072 generated tokens.
80
+
81
+ ## ZTC
82
+
83
+ The ZTC probe published with Darwin-180B-RSI was fitted on R1's hidden states. A probe fitted on R3 will be added to this repository; until then, use R1's probe only as a rough signal.
84
+
85
+ ## Model-level RSI vs. harness-level RSI
86
+
87
+ Darwin-180B-RSI-R3 is **Model-level RSI**: the model itself (its weights) improves by learning only from its own solutions. No human-written solutions or reasoning traces are used; correctness is checked automatically. Harness-level RSI (e.g., Google's RRSI) improves the prompts, tools and workflow around a fixed model. The two are complementary.
88
+
89
+ ## License
90
+
91
+ Qwen Community License 1.0 (inherited from Qwen3.8-Flash-Next). See [LICENSE](LICENSE).
92
+
93
+ ## About
94
+
95
+ Built by [VIDRAFT](https://vidraft.net). Darwin family paper: [arXiv 2605.14386](https://arxiv.org/abs/2605.14386).