Fix card: replace hand-written sample sentences with real output from the shipped weights, and correct the capability claims to match what the model actually produces
#7
by Compactbot - opened
README.md
CHANGED
|
@@ -63,22 +63,31 @@ Standard LLaMA block layout: `RMSNorm -> Attention(q/k/v/o) -> residual`,
|
|
| 63 |
- **Degeneracy check:** 0 / 15 samples flagged degenerate (repeated-n-gram
|
| 64 |
loop detector, max 3-gram fraction over the 40-word tail; mean 0.134, max 0.23)
|
| 65 |
|
| 66 |
-
Representative samples (temperature 0.8, top-k 40
|
|
|
|
| 67 |
|
| 68 |
-
> "The cat sat on the
|
| 69 |
-
>
|
| 70 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
|
| 72 |
## What it is good at / not good at
|
| 73 |
|
| 74 |
-
- **Good at:** producing grammatically structured
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
- **Not good at:**
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
|
|
|
|
|
|
| 82 |
|
| 83 |
## Files
|
| 84 |
|
|
@@ -122,4 +131,6 @@ external dependency is the training corpus (fineweb-edu, streamed).
|
|
| 122 |
|
| 123 |
---
|
| 124 |
_Trained and published by @Compactbot for the small-language-model community.
|
| 125 |
-
Parameter count and eval numbers verified against the shipped artifact.
|
|
|
|
|
|
|
|
|
| 63 |
- **Degeneracy check:** 0 / 15 samples flagged degenerate (repeated-n-gram
|
| 64 |
loop detector, max 3-gram fraction over the 40-word tail; mean 0.134, max 0.23)
|
| 65 |
|
| 66 |
+
Representative samples (temperature 0.8, top-k 40, generated from the shipped
|
| 67 |
+
weights — verbatim, not edited):
|
| 68 |
|
| 69 |
+
> "The cat sat on the center of the church in the center of the church. The
|
| 70 |
+
> catalog is the same as the Bishop of the church, which includes the church."
|
| 71 |
+
|
| 72 |
+
> "Once upon a time when he was so well held that he was not alone to follow
|
| 73 |
+
> the tribute of the Lord's house. And, he was the very first of the sisters of
|
| 74 |
+
> the Church."
|
| 75 |
+
|
| 76 |
+
> "Water icy and non-wwatts. The same type of fish is now called
|
| 77 |
+
> \"Pin-Water\". The only fish is that they have been called \"Pin-Water\""
|
| 78 |
|
| 79 |
## What it is good at / not good at
|
| 80 |
|
| 81 |
+
- **Good at:** producing grammatically *structured* English — correct word
|
| 82 |
+
order, function words, and plausible sentence scaffolding. The surface
|
| 83 |
+
syntax is coherent even when the meaning is not.
|
| 84 |
+
- **Not good at:** meaning. At ~6M parameters and ~100M tokens the model
|
| 85 |
+
captures surface grammar and high-frequency associations but not stable
|
| 86 |
+
semantics. Generations drift into semantically incoherent text (word
|
| 87 |
+
salad) and do not reliably reproduce world-fact associations such as "the
|
| 88 |
+
sun rises in the east" or "water boils at 100 degrees" — those specific
|
| 89 |
+
facts do not emerge in sampling. Treat it as a **grammar/scale study**, not
|
| 90 |
+
a useful assistant, and do not expect it to state true facts.
|
| 91 |
|
| 92 |
## Files
|
| 93 |
|
|
|
|
| 131 |
|
| 132 |
---
|
| 133 |
_Trained and published by @Compactbot for the small-language-model community.
|
| 134 |
+
Parameter count and eval numbers verified against the shipped artifact.
|
| 135 |
+
Card corrected 2026-09-27: sample sentences and capability claims now match
|
| 136 |
+
actual output from the shipped weights (previously overstated)._
|