Upload README.md with huggingface_hub

README.md

@@ -1,37 +1,64 @@
----
-license: apache-2.0
-base_model: CyanoAI/Prochlo-190M-Base
-tags:
-- sft
-- small-language-model
-- english-only
----
-
-# Prochlo-190M-SFT
-
-Supervised fine-tuned version of [Prochlo-190M-Base](https://huggingface.co/CyanoAI/Prochlo-190M-Base), a 190M-parameter English dense SLM trained from scratch on a DCU cluster.
-
-## Training
-- **Base**: CyanoAI/Prochlo-190M-Base (pre-trained 20B tokens, 20-layer LLaMA-style, vocab 50261)
-- **Data**: HuggingFaceTB/smol-smoltalk (~460k conversations, English-only, chat-formatted with `<|user|>/<|assistant|>` tokens)
-- **Method**: Full-parameter SFT, 2 epochs, lr=3e-4 cosine, batch 216, max_len 1024, fp16 with loss masking on assistant tokens
-- **Duration**: ~27h on single 16GB DCU accelerator
-
-## Known Limitations
-- Behavioral eval (26-item rubric): **61.5% pass** pre-DPO. Strong on code instructions and factual QA; weaker on concise replies, Chinese (by design), and long multi-turn continuity.
-- English only; Chinese inputs may produce garbage by design.
-- This is a *base SFT* — not yet preference-aligned. DPO/RLAIF variants will follow as `Prochlo-190M` proper.
-
-## Usage
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-tok = AutoTokenizer.from_pretrained("CyanoAI/Prochlo-190M-SFT")
-m = AutoModelForCausalLM.from_pretrained("CyanoAI/Prochlo-190M-SFT", dtype="auto").cuda()
-msgs = "<|user|>Hello<|end|><|assistant|>"
-ids = tok(msgs, return_tensors="pt").input_ids.cuda()
-out = m.generate(ids, max_new_tokens=128, do_sample=False, eos_token_id=tok.convert_tokens_to_ids("<|end|>"))
-print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
-```
-
-## License
-Apache-2.0
+---
+license: apache-2.0
+base_model: CyanoAI/Prochlo-190M-Base
+tags:
+- sft
+- small-language-model
+- english-only
+language:
+- en
+---
+
+# Prochlo-190M-SFT
+
+Supervised fine-tuned version of [Prochlo-190M-Base](https://huggingface.co/CyanoAI/Prochlo-190M-Base) — a 190.5M-parameter dense English SLM trained from scratch on a DCU cluster.
+
+## Model family
+
+| Repo | Stage |
+|---|---|
+| [`CyanoAI/Prochlo-190M-Base`](https://huggingface.co/CyanoAI/Prochlo-190M-Base) | Pre-trained base |
+| [`CyanoAI/Prochlo-190M-SFT`](https://huggingface.co/CyanoAI/Prochlo-190M-SFT) | Supervised fine-tune (this repo) |
+| `CyanoAI/Prochlo-190M` | Preference-aligned (DPO + RLAIF) — in training |
+
+All variants share architecture, tokenizer, and vocabulary.
+
+## Training
+
+| | |
+|---|---|
+| Base | CyanoAI/Prochlo-190M-Base (~7.7B pre-training tokens, 12 layers, vocab 50,261) |
+| Data | HuggingFaceTB/smol-smoltalk, ~460k English conversations, chat-formatted with `<\|user\|>` / `<\|assistant\|>` |
+| Method | Full-parameter SFT, 2 epochs, lr 3e-4 cosine, batch 216, max_len 1024, fp16, loss masked to assistant tokens |
+| Steps | 96,135 |
+| Final loss | 1.9308 (train), ~1.94 (eval) |
+| Duration | ~27 h on a single 16 GB DCU accelerator |
+
+## Evaluation
+
+26-item behavioral rubric (format adherence, anti-sycophancy, code, concision): **61.5% pass**.
+
+Strong on code instructions and factual QA. Weak on concise replies, long multi-turn coherence, and arithmetic. English only — Chinese input produces garbage by design.
+
+## Usage
+
+The chat format is plain text with control tokens, not a chat template:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+tok = AutoTokenizer.from_pretrained("CyanoAI/Prochlo-190M-SFT")
+model = AutoModelForCausalLM.from_pretrained("CyanoAI/Prochlo-190M-SFT", dtype="float16").cuda()
+
+prompt = "<|user|>Explain what a closure is.<|end|><|assistant|>"
+ids = tok(prompt, return_tensors="pt").input_ids.cuda()
+out = model.generate(ids, max_new_tokens=256, do_sample=False,
+                     eos_token_id=tok.convert_tokens_to_ids("<|end|>"))
+print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
+```
+
+Note: weights are sharded across 32 safetensors files with a `model.safetensors.index.json`; `from_pretrained` handles this transparently.
+
+## License
+
+Apache-2.0