max-0.1b-method-01b / README.md
quantumaikr's picture
card
85d5675 verified
|
Raw History Blame Contribute Delete
884 Bytes
metadata
license: apache-2.0
language:
  - ko
  - en
tags:
  - max
  - ablation
  - pretraining

max-0.1b method-01b ablation arms

Research artifacts: a 0.1B-parameter methodology ablation for the max model family. Each arm was trained from scratch on 1.0B tokens; files are weights only (bf16), meant as --init for fine-tuning or for evaluation, not as finished models.

arm change vs. baseline
A baseline
B no LR decay + snapshot averaging (B_avg_uniform20.pt = uniform average of 20 snapshots)
C deep-narrow shape
D document-heavy data mix
E code data in the cooldown
F2 follow-up arm

Verify downloads with SHA256SUMS. See max-0.1b-v1 for the released model.


0.1B 방법론 비교 실험(각 1.0B 토큰) 가중치입니다. 완성 모델이 아닌 연구용 산출물입니다.