card
Browse files
README.md
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
language: [ko, en]
|
| 4 |
+
tags: [max, ablation, pretraining]
|
| 5 |
+
---
|
| 6 |
+
# max-0.1b method-01b ablation arms
|
| 7 |
+
|
| 8 |
+
Research artifacts: a 0.1B-parameter methodology ablation for the max model family. Each arm was trained from scratch on
|
| 9 |
+
1.0B tokens; files are weights only (bf16), meant as `--init` for fine-tuning or for evaluation, not as finished models.
|
| 10 |
+
|
| 11 |
+
| arm | change vs. baseline |
|
| 12 |
+
|---|---|
|
| 13 |
+
| A | baseline |
|
| 14 |
+
| B | no LR decay + snapshot averaging (`B_avg_uniform20.pt` = uniform average of 20 snapshots) |
|
| 15 |
+
| C | deep-narrow shape |
|
| 16 |
+
| D | document-heavy data mix |
|
| 17 |
+
| E | code data in the cooldown |
|
| 18 |
+
| F2 | follow-up arm |
|
| 19 |
+
|
| 20 |
+
Verify downloads with `SHA256SUMS`. See [max-0.1b-v1](https://huggingface.co/quantumaikr/max-0.1b-v1) for the released model.
|
| 21 |
+
|
| 22 |
+
---
|
| 23 |
+
0.1B 방법론 비교 실험(각 1.0B 토큰) 가중치입니다. 완성 모델이 아닌 연구용 산출물입니다.
|