TightX commited on
Commit
2eeecae
·
verified ·
1 Parent(s): 04cb8da

fix links: GitHub URL + base/sft cross-links

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -14,7 +14,7 @@ library_name: pytorch
14
  # code-llm-435m · sft — the instruction-tuned variant
15
 
16
  The instruction-tuned version of the 435M from-scratch code model whose pretrained weights are in
17
- [`../base`](../base). Same architecture, same 32,000-token byte-level BPE vocabulary, same 1,024-token
18
  context — only the weights differ. It reads a plain-English request (*"write a function that checks
19
  whether a number is prime"*) or continues a bare `def` signature, and writes Python.
20
 
@@ -24,7 +24,7 @@ that did.
24
 
25
  ## Architecture
26
 
27
- Identical to [`../base`](../base) — see that card for the full table. In short: 434,680,832
28
  parameters, 22 layers, d_model 1,024 / 16 heads, SwiGLU (d_ff 4,096), RoPE (theta 500,000), RMSNorm,
29
  causal SDPA, untied embeddings, context 1,024, released file in bf16.
30
 
@@ -44,7 +44,7 @@ causal SDPA, untied embeddings, context 1,024, released file in bf16.
44
  Roughly 30 post-training runs were evaluated to reach this one. The approaches that did **not** work —
45
  instruction-free training, teaching the two objectives in two stages, self-distillation, weight
46
  averaging ("soups"), ratio sweeps — are documented with their numbers, including the mistakes, in the
47
- GitHub repository.
48
 
49
  ## Results
50
 
@@ -98,5 +98,5 @@ from the training corpus despite filtering and deduplication.
98
  ## Links
99
 
100
  `DESIGN.md` (decisions and why), `CHALLENGES.md` (every bug and contaminant), the ablation report, the
101
- post-training log with its corrections, and the evaluation harness are in the GitHub repository. MIT
102
- licensed.
 
14
  # code-llm-435m · sft — the instruction-tuned variant
15
 
16
  The instruction-tuned version of the 435M from-scratch code model whose pretrained weights are in
17
+ [`TightX/code-llm-435m-base`](https://huggingface.co/TightX/code-llm-435m-base). Same architecture, same 32,000-token byte-level BPE vocabulary, same 1,024-token
18
  context — only the weights differ. It reads a plain-English request (*"write a function that checks
19
  whether a number is prime"*) or continues a bare `def` signature, and writes Python.
20
 
 
24
 
25
  ## Architecture
26
 
27
+ Identical to [`TightX/code-llm-435m-base`](https://huggingface.co/TightX/code-llm-435m-base) — see that card for the full table. In short: 434,680,832
28
  parameters, 22 layers, d_model 1,024 / 16 heads, SwiGLU (d_ff 4,096), RoPE (theta 500,000), RMSNorm,
29
  causal SDPA, untied embeddings, context 1,024, released file in bf16.
30
 
 
44
  Roughly 30 post-training runs were evaluated to reach this one. The approaches that did **not** work —
45
  instruction-free training, teaching the two objectives in two stages, self-distillation, weight
46
  averaging ("soups"), ratio sweeps — are documented with their numbers, including the mistakes, in the
47
+ GitHub repository at https://github.com/TightXu/Simple_Code_LLM.
48
 
49
  ## Results
50
 
 
98
  ## Links
99
 
100
  `DESIGN.md` (decisions and why), `CHALLENGES.md` (every bug and contaminant), the ablation report, the
101
+ post-training log with its corrections, and the evaluation harness are in the GitHub repository at
102
+ https://github.com/TightXu/Simple_Code_LLM. MIT licensed.