Add training dataset metadata to the model card
Browse files
README.md
CHANGED
|
@@ -17,6 +17,20 @@ tags:
|
|
| 17 |
- tercet
|
| 18 |
pipeline_tag: text-generation
|
| 19 |
library_name: tiny_gdn
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
base_model: kerzgrr/Tercet-base
|
| 21 |
---
|
| 22 |
|
|
@@ -130,9 +144,9 @@ Same TinyGDN hybrid as [`Tercet-base`](https://huggingface.co/kerzgrr/Tercet-bas
|
|
| 130 |
|
| 131 |
| Stage | Details |
|
| 132 |
|-------|---------|
|
| 133 |
-
| **Base** |
|
| 134 |
-
| **Mid-SFT** |
|
| 135 |
-
| **Instruct SFT** |
|
| 136 |
| **Checkpoint** | optimizer step 4,500 (latest complete instruct snapshot) |
|
| 137 |
| **Weights** | EMA (this repo's `model.safetensors`) |
|
| 138 |
| **Val loss (EMA)** | 1.744 (ppl 5.72) |
|
|
|
|
| 17 |
- tercet
|
| 18 |
pipeline_tag: text-generation
|
| 19 |
library_name: tiny_gdn
|
| 20 |
+
datasets:
|
| 21 |
+
- HuggingFaceFW/fineweb-edu
|
| 22 |
+
- HuggingFaceTB/smoltalk2
|
| 23 |
+
- HuggingFaceTB/smoltalk
|
| 24 |
+
- nvidia/Llama-Nemotron-Post-Training-Dataset
|
| 25 |
+
- open-thoughts/OpenThoughts3-1.2M
|
| 26 |
+
- teknium/OpenHermes-2.5
|
| 27 |
+
- open-r1/Mixture-of-Thoughts
|
| 28 |
+
- CohereLabs/aya_dataset
|
| 29 |
+
- NousResearch/hermes-function-calling-v1
|
| 30 |
+
- simplescaling/s1K-1.1
|
| 31 |
+
- allenai/tulu-3-sft-personas-instruction-following
|
| 32 |
+
- Salesforce/xlam-function-calling-60k
|
| 33 |
+
- THUDM/LongAlign-10k
|
| 34 |
base_model: kerzgrr/Tercet-base
|
| 35 |
---
|
| 36 |
|
|
|
|
| 144 |
|
| 145 |
| Stage | Details |
|
| 146 |
|-------|---------|
|
| 147 |
+
| **Base** | [HuggingFaceFW/fineweb-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) → [`Tercet-base`](https://huggingface.co/kerzgrr/Tercet-base) |
|
| 148 |
+
| **Mid-SFT** | [HuggingFaceTB/smoltalk2](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) Mid: [Llama-Nemotron-Post-Training-Dataset](https://huggingface.co/datasets/nvidia/Llama-Nemotron-Post-Training-Dataset) + [OpenThoughts3-1.2M](https://huggingface.co/datasets/open-thoughts/OpenThoughts3-1.2M) |
|
| 149 |
+
| **Instruct SFT** | [HuggingFaceTB/smoltalk2](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) SFT (SmolTalk, OpenHermes-2.5, OpenThoughts3, Aya, Hermes function calling, s1K, Tulu-3 personas IF, xLAM, LongAlign, Mixture-of-Thoughts, …), seq 16,384, AdamW 5×10⁻⁵, 61.1 hours |
|
| 150 |
| **Checkpoint** | optimizer step 4,500 (latest complete instruct snapshot) |
|
| 151 |
| **Weights** | EMA (this repo's `model.safetensors`) |
|
| 152 |
| **Val loss (EMA)** | 1.744 (ppl 5.72) |
|