Mention SVF experts
Browse files
README.md
CHANGED
|
@@ -19,7 +19,8 @@ self-modeling auxiliary objective. It targets from-scratch training on free Kagg
|
|
| 19 |
## Usage
|
| 20 |
The architecture is custom, so load it with the MORPH code (github.com/dhairya-pandya/MoRPH).
|
| 21 |
Tokenizer: `HuggingFaceTB/SmolLM2-135M`. Files: `model.safetensors` (base), `sft/model.safetensors`
|
| 22 |
-
(instruction-tuned, ChatML), `config.json`, `eval/` (HellaSwag, sample replies)
|
|
|
|
| 23 |
|
| 24 |
```bash
|
| 25 |
git clone https://github.com/dhairya-pandya/MoRPH && cd MoRPH && pip install -r requirements.txt
|
|
|
|
| 19 |
## Usage
|
| 20 |
The architecture is custom, so load it with the MORPH code (github.com/dhairya-pandya/MoRPH).
|
| 21 |
Tokenizer: `HuggingFaceTB/SmolLM2-135M`. Files: `model.safetensors` (base), `sft/model.safetensors`
|
| 22 |
+
(instruction-tuned, ChatML), `config.json`, `eval/` (HellaSwag, sample replies), `svf/` (SVF experts for math, code and stories
|
| 23 |
+
plus the two-pass router, used with `morph/train/svf_train.py`; −6 to −20% domain loss).
|
| 24 |
|
| 25 |
```bash
|
| 26 |
git clone https://github.com/dhairya-pandya/MoRPH && cd MoRPH && pip install -r requirements.txt
|