kerzgrr commited on
Commit
6dc5607
·
verified ·
1 Parent(s): 48eb149

Add training dataset metadata to the model card

Browse files
Files changed (1) hide show
  1. README.md +17 -3
README.md CHANGED
@@ -17,6 +17,20 @@ tags:
17
  - tercet
18
  pipeline_tag: text-generation
19
  library_name: tiny_gdn
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
  base_model: kerzgrr/Tercet-base
21
  ---
22
 
@@ -130,9 +144,9 @@ Same TinyGDN hybrid as [`Tercet-base`](https://huggingface.co/kerzgrr/Tercet-bas
130
 
131
  | Stage | Details |
132
  |-------|---------|
133
- | **Base** | FineWeb-Edu pretrain → [`Tercet-base`](https://huggingface.co/kerzgrr/Tercet-base) |
134
- | **Mid-SFT** | SmolTalk2 mid mix (`runs/tercet-r-sft`) |
135
- | **Instruct SFT** | SmolTalk2 instruct mix, seq 16,384, AdamW 5×10⁻⁵, 61.1 hours |
136
  | **Checkpoint** | optimizer step 4,500 (latest complete instruct snapshot) |
137
  | **Weights** | EMA (this repo's `model.safetensors`) |
138
  | **Val loss (EMA)** | 1.744 (ppl 5.72) |
 
17
  - tercet
18
  pipeline_tag: text-generation
19
  library_name: tiny_gdn
20
+ datasets:
21
+ - HuggingFaceFW/fineweb-edu
22
+ - HuggingFaceTB/smoltalk2
23
+ - HuggingFaceTB/smoltalk
24
+ - nvidia/Llama-Nemotron-Post-Training-Dataset
25
+ - open-thoughts/OpenThoughts3-1.2M
26
+ - teknium/OpenHermes-2.5
27
+ - open-r1/Mixture-of-Thoughts
28
+ - CohereLabs/aya_dataset
29
+ - NousResearch/hermes-function-calling-v1
30
+ - simplescaling/s1K-1.1
31
+ - allenai/tulu-3-sft-personas-instruction-following
32
+ - Salesforce/xlam-function-calling-60k
33
+ - THUDM/LongAlign-10k
34
  base_model: kerzgrr/Tercet-base
35
  ---
36
 
 
144
 
145
  | Stage | Details |
146
  |-------|---------|
147
+ | **Base** | [HuggingFaceFW/fineweb-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) → [`Tercet-base`](https://huggingface.co/kerzgrr/Tercet-base) |
148
+ | **Mid-SFT** | [HuggingFaceTB/smoltalk2](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) Mid: [Llama-Nemotron-Post-Training-Dataset](https://huggingface.co/datasets/nvidia/Llama-Nemotron-Post-Training-Dataset) + [OpenThoughts3-1.2M](https://huggingface.co/datasets/open-thoughts/OpenThoughts3-1.2M) |
149
+ | **Instruct SFT** | [HuggingFaceTB/smoltalk2](https://huggingface.co/datasets/HuggingFaceTB/smoltalk2) SFT (SmolTalk, OpenHermes-2.5, OpenThoughts3, Aya, Hermes function calling, s1K, Tulu-3 personas IF, xLAM, LongAlign, Mixture-of-Thoughts, …), seq 16,384, AdamW 5×10⁻⁵, 61.1 hours |
150
  | **Checkpoint** | optimizer step 4,500 (latest complete instruct snapshot) |
151
  | **Weights** | EMA (this repo's `model.safetensors`) |
152
  | **Val loss (EMA)** | 1.744 (ppl 5.72) |