Hugging Face's logo

techdotus
/
Tokle-3M

Text Generation
Transformers
Safetensors
English
tokle
causal-lm
custom-architecture
custom_code
slm
small-language-model
Model card Files
xet
 Community
anandselvadurai-techus commited on
Commit
ad27408
·
verified ·
1 Parent(s): ff59975

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -68,7 +68,7 @@ All scores are 0-shot acc_norm, using the Open SLM Leaderboard methodology.
68
 
69
  | HellaSwag | ARC-Easy | ARC-Challenge | PIQA | ArithMark-3 |
70
  |---|---|---|---|---|
71
- | 27.20% | 34.89% | 23.98% | 55.01% | 41.00% |
72
 
73
  ### Ablation: SPAB vs. Distilled
74
 
@@ -77,7 +77,7 @@ Stage 1 model (SPAB active) vs. the released Tokle-3M (SPAB removed and distille
77
  | Model | Params | Int Index | HellaSwag | ARC-Easy | ARC-Chal | PIQA | ArithMark-3 |
78
  |---|---|---|---|---|---|---|---|
79
  | Tokle-SPAB-11.3M (with SPAB) | 11.3M (2.91M trainable + 8.39M frozen) | 9.16 | 27.22% | 34.68% | 24.49% | 54.95% | 41.70% |
80
- | Tokle-3M (SPAB distilled) | 2.91M | 8.96 | 27.20% | 34.89% | 23.98% | 55.01% | 41.00% |
81
 
82
  ## Comparison Results
83
 
@@ -85,7 +85,7 @@ All scores are 0-shot acc_norm, using the Open SLM Leaderboard methodology. Scor
85
 
86
  | Model | Params | Int Index | HellaSwag | ARC-Easy | ARC-Chal | PIQA | ArithMark-3 |
87
  |---|---|---|---|---|---|---|---|
88
- | Tokle-3M (Tech.us) | 2.91M | **8.96** | 27.20% | **34.89%** | **23.98%** | 55.01% | **41.00%** |
89
  | Ember-2 (SurjoLabs) | 2.96M×2 | 7.21 | 27.28% | 33.42% | 22.01% | 55.11% | 35.90% |
90
  | BananaMind-2-Micro (BananaMind) | 2.9M | 6.01 | **28.27%** | 33.12% | 21.93% | 53.21% | 34.00% |
91
  | GPT-S-1.4M (Axiomic Labs) | 1.4M | 5.40 | 26.89% | 31.57% | 21.93% | **55.17%** | 30.20% |
 
68
 
69
  | HellaSwag | ARC-Easy | ARC-Challenge | PIQA | ArithMark-3 |
70
  |---|---|---|---|---|
71
+ | 27.20% | 34.85% | 23.98% | 55.01% | 40.80% |
72
 
73
  ### Ablation: SPAB vs. Distilled
74
 
 
77
  | Model | Params | Int Index | HellaSwag | ARC-Easy | ARC-Chal | PIQA | ArithMark-3 |
78
  |---|---|---|---|---|---|---|---|
79
  | Tokle-SPAB-11.3M (with SPAB) | 11.3M (2.91M trainable + 8.39M frozen) | 9.16 | 27.22% | 34.68% | 24.49% | 54.95% | 41.70% |
80
+ | Tokle-3M (SPAB distilled) | 2.91M | 8.92 | 27.20% | 34.85% | 23.98% | 55.01% | 40.80% |
81
 
82
  ## Comparison Results
83
 
 
85
 
86
  | Model | Params | Int Index | HellaSwag | ARC-Easy | ARC-Chal | PIQA | ArithMark-3 |
87
  |---|---|---|---|---|---|---|---|
88
+ | Tokle-3M (Tech.us) | 2.91M | **8.92** | 27.20% | **34.85%** | **23.98%** | 55.01% | **40.80%** |
89
  | Ember-2 (SurjoLabs) | 2.96M×2 | 7.21 | 27.28% | 33.42% | 22.01% | 55.11% | 35.90% |
90
  | BananaMind-2-Micro (BananaMind) | 2.9M | 6.01 | **28.27%** | 33.12% | 21.93% | 53.21% | 34.00% |
91
  | GPT-S-1.4M (Axiomic Labs) | 1.4M | 5.40 | 26.89% | 31.57% | 21.93% | **55.17%** | 30.20% |