Card restructure: branded metrics block and colored banner
Browse files
README.md
CHANGED
|
@@ -15,11 +15,26 @@ tags:
|
|
| 15 |
---
|
| 16 |
<div align="center">
|
| 17 |
<img src="fabryka-banner.svg" alt="Fabryka AI" width="480">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
</div>
|
| 19 |
|
| 20 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
-
|
| 23 |
|
| 24 |
> Intelligence, in production.
|
| 25 |
|
|
@@ -31,23 +46,16 @@ tags:
|
|
| 31 |
|
| 32 |
## Measured output
|
| 33 |
|
| 34 |
-
| Benchmark | Result |
|
| 35 |
-
|---|---:|
|
| 36 |
-
| ARC-Easy (raw accuracy) | **66.58%** |
|
| 37 |
-
| BLiMP (67-subtask mean) | 78.38% |
|
| 38 |
-
| WikiText-2 byte perplexity | 1.907 |
|
| 39 |
-
| **Leaderboard raw overall** | **81.51** |
|
| 40 |
-
|
| 41 |
Evaluation protocol: lm-evaluation-harness 0.4.13, document-level WikiText, rolling likelihood at 1024-token context, ARC-Easy zero-shot, BLiMP sentence-pair likelihood. No test or validation data was used in training; the fine-tuning set is the clean ARC-Easy train split only.
|
| 42 |
|
| 43 |
## Architecture
|
| 44 |
|
| 45 |
| Property | Value |
|
| 46 |
-
|---|---|
|
| 47 |
| Parameters | 149,333,081 |
|
| 48 |
| Layers | 26 |
|
| 49 |
| Hidden size | 576 |
|
| 50 |
-
| Attention | GQA (9 query
|
| 51 |
| Vocabulary | 24,576 BPE tokens |
|
| 52 |
| Context | 2,048 tokens |
|
| 53 |
| Embeddings | tied |
|
|
|
|
| 15 |
---
|
| 16 |
<div align="center">
|
| 17 |
<img src="fabryka-banner.svg" alt="Fabryka AI" width="480">
|
| 18 |
+
|
| 19 |
+
<h1>Slayer149-ARC</h1>
|
| 20 |
+
|
| 21 |
+
<p><strong>Open models. Measured output.</strong></p>
|
| 22 |
+
<p>A 149M-parameter language model from <a href="https://fabryka.ai">Fabryka AI</a> — fine-tuned from the <a href="https://huggingface.co/SlayerLab/Slayer149-balanced">Slayer149-balanced</a> base for strong multiple-choice question answering, without losing the general language ability that makes a model actually useful.</p>
|
| 23 |
</div>
|
| 24 |
|
| 25 |
+
---
|
| 26 |
+
|
| 27 |
+
<div align="center">
|
| 28 |
+
|
| 29 |
+
| | |
|
| 30 |
+
|:---|:---|
|
| 31 |
+
| **ARC-Easy** | `66.58%` — best on the board |
|
| 32 |
+
| **BLiMP** | `78.38%` |
|
| 33 |
+
| **WikiText-2 byte PPL** | `1.907` |
|
| 34 |
+
| **Raw overall** | `81.51` — **#1** |
|
| 35 |
+
| **Efficiency** | `81.52` — **#1** |
|
| 36 |
|
| 37 |
+
</div>
|
| 38 |
|
| 39 |
> Intelligence, in production.
|
| 40 |
|
|
|
|
| 46 |
|
| 47 |
## Measured output
|
| 48 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
Evaluation protocol: lm-evaluation-harness 0.4.13, document-level WikiText, rolling likelihood at 1024-token context, ARC-Easy zero-shot, BLiMP sentence-pair likelihood. No test or validation data was used in training; the fine-tuning set is the clean ARC-Easy train split only.
|
| 50 |
|
| 51 |
## Architecture
|
| 52 |
|
| 53 |
| Property | Value |
|
| 54 |
+
|---|---:|
|
| 55 |
| Parameters | 149,333,081 |
|
| 56 |
| Layers | 26 |
|
| 57 |
| Hidden size | 576 |
|
| 58 |
+
| Attention | GQA (9 query / 3 KV heads, 64-dim) |
|
| 59 |
| Vocabulary | 24,576 BPE tokens |
|
| 60 |
| Context | 2,048 tokens |
|
| 61 |
| Embeddings | tied |
|