quantumaikr commited on
Commit
85d5675
·
verified ·
1 Parent(s): d41d7b4
Files changed (1) hide show
  1. README.md +23 -0
README.md ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language: [ko, en]
4
+ tags: [max, ablation, pretraining]
5
+ ---
6
+ # max-0.1b method-01b ablation arms
7
+
8
+ Research artifacts: a 0.1B-parameter methodology ablation for the max model family. Each arm was trained from scratch on
9
+ 1.0B tokens; files are weights only (bf16), meant as `--init` for fine-tuning or for evaluation, not as finished models.
10
+
11
+ | arm | change vs. baseline |
12
+ |---|---|
13
+ | A | baseline |
14
+ | B | no LR decay + snapshot averaging (`B_avg_uniform20.pt` = uniform average of 20 snapshots) |
15
+ | C | deep-narrow shape |
16
+ | D | document-heavy data mix |
17
+ | E | code data in the cooldown |
18
+ | F2 | follow-up arm |
19
+
20
+ Verify downloads with `SHA256SUMS`. See [max-0.1b-v1](https://huggingface.co/quantumaikr/max-0.1b-v1) for the released model.
21
+
22
+ ---
23
+ 0.1B 방법론 비교 실험(각 1.0B 토큰) 가중치입니다. 완성 모델이 아닌 연구용 산출물입니다.