Download README.md from mithraindir/macrostories-micro-2layer-d48: direct link, hf CLI and curl.
- Browser
- Download file 3.34 kB
-
https://huggingface.co/mithraindir/macrostories-micro-2layer-d48/resolve/main/README.md
- Command line
-
hf download hf://mithraindir/macrostories-micro-2layer-d48/README.md
-
curl -L -o README.md https://huggingface.co/mithraindir/macrostories-micro-2layer-d48/resolve/main/README.md
language:
- nl
license: mit
tags:
- text-generation
- recurrent-transformer
- micro-models
- dutch
- children-stories
datasets:
- mithraindir/macrostories-stories
pipeline_tag: text-generation
MacroStories Model 2 (2-Layer Recurrent Micro-Transformer)
A 71,616-parameter (279 KB) recurrent language model capable of generating coherent, 4-beat Dutch children's stories directly inside a web browser or on micro-edge devices with 0 ms network latency.
This model is the champion checkpoint of the MacroStories research project by Pjotter Bolte.
Key Specifications
| Parameter | Value |
|---|---|
| Parameters | 71,616 |
| Model Size (FP32) | 279 KB (L2 Cache resident) |
| Architecture | Recurrent Transformer (2 physical layers $\times$ 2 recurrent loops per layer) |
| Hidden Dimension ($) | 48 |
| FFN Dimension ({ff}$) | 72 (SwiGLU) |
| Attention Heads | 3 Query heads, 1 Key-Value head (GQA, {head}=16$) |
| Positional Embedding | Rotary Positional Embeddings (RoPE) |
| Vocabulary Size | 800 tokens (Byte-Level BPE, 0.00% OOV on Dutch morphology) |
| Validation Loss | 1.2426 nats/token (Perplexity: 3.46) |
| Training Steps | 4,000 steps on 1x Nvidia T4 GPU (72 seconds total training time) |
How the Architecture Works
Unlike conventional multi-billion parameter LLMs that stack 64–128 physical layers (the "skyscraper" design), MacroStories uses a two-room workshop with an elevator:
- Layer 1: Specializes in Dutch syntax, morphology, spelling, and BPE subwords.
- Layer 2: Specializes in causal narrative state transitions across the 4-beat story arc.
- Loops: Tokens loop through each physical layer twice (=2$) with Value-Residual Mixing (hBmix = 0.5) to anchor memory across recurrent iterations without state drift.
Narrative Schema
The model expects a scenario header conditioning prompt and follows a strict 4-beat causal structure: 5137\text{Inleiding / Doel} \longrightarrow \text{Uitdaging} \longrightarrow \text{Handeling} \longrightarrow \text{Oplossing}5137
Prompt Format
Special structural markers:
- : Introduction of the character, setting, and object.
- : The obstacle preventing immediate success.
- : The physical action resolving the hurdle.
- : The happy, stable resolution.
- : Story terminator.
Generated Sample
"Daan was in de schuur met een oude hamer die van opa was geweest. De houten plank van het bankje lag scheef en de spijkers staken omhoog, zodat Daan er niet op kon zitten. Daan pakte de plank recht en sloeg voorzichtig met de oude hamer op de losse spijkers, tik, tik, tik. De plank lag nu stevig en Daan ging blij op het bankje zitten."
Recommended Inference Configuration
For optimal coherence without over-generation or chatter:
- Temperature: –
- 3-Gram Repetition Penalty: Ban repeated 3-grams during autoregressive decoding to prevent stuttering.
- FSM Termination: Once the model outputs the section and produces at least 6 tokens, terminate generation immediately upon encountering the first terminal punctuation ( or ).
Training Dataset
Trained on , consisting of certified consistent Dutch story arcs generated via compatible scenario pairings and distilled 4-beat schemas.