README.md

---
license: cc-by-nc-sa-4.0
language: en
pipeline_tag: text-generation
base_model: AbstractPhil/mini-beatrix-2s
tags:
- byte-level
- linear-attention
- splat-attention
- aleph
- adapters
- custom_code
---

# mini-beatrix-2.5s

**The mini-beatrix-2s core, packaged with its arms.** The 237M-parameter
full-splat byte model that completed its 16.101B-token mission on
2026-08-31, shipped here with a library of **detachable arms** — small
aleph-addressed patch heads trained on this exact frozen core — and the
runtime that mounts them.

The core's weights are **bit-identical** to
[mini-beatrix-2s](https://huggingface.co/AbstractPhil/mini-beatrix-2s).
Nothing was retrained for 2.5s: an arm is a guest on the core, never a
migration of it, and the package proves that on every load (see
*Detach is exact*).

```python
import torch
from transformers import AutoModelForCausalLM

m = AutoModelForCausalLM.from_pretrained(
        "AbstractPhil/mini-beatrix-2.5s", trust_remote_code=True).eval()

m.say("Who are you?")                     # the bare core, its own frame
# 'I am Beatrix, a small byte-level language model. I read raw bytes ...'

m.mount_arm("chain")                      # one of the packaged arms
m.say("If someone is sook, then they are quen. If someone is quen, then "
      "they are harl. If someone is harl, then they are torv. If someone "
      "is torv, then they are prin. If someone is prin, then they are "
      "mund. Wren is sook. What follows?")
# 'So Wren is quen. So Wren is harl. So Wren is torv. So Wren is prin. So Wren is mund.'

m.detach_arm()                            # bit-exact core restored
```

The model reads **raw UTF-8 bytes**: `input_ids` are byte values 0–255.
There is no tokenizer to download.

## The arms

`m.arms()` returns this table as data; each row carries the frame it was
trained in, its decode settings, its measured score and how settled the
result is.

| arm | what it does | type | measured | cell | status |
|---|---|---|---|---|---|
| `rules` | reads five if-then rules about made-up words and answers with only the final word | arm | unseen words .640 · the original eight .960 | L1 (8 shared-prefix pairs, 64-word pool, quiet weight 2) | settled (2 seeds) |
| `rules-seed1` | the same arm trained from a different seed — the pair that makes the result two-seeded | arm | unseen words .673 · the original eight .907 | L1 | settled (2 seeds) |
| `rules-minted` | the same answer-only rule reading, trained on the 216-word lexicon minted for the v3 curriculum (96 easy + 96 trap + 12 shared-onset pairs) at the ruled dial: half the rows pairs | arm | minted words .707 · the original eight .947 · unseen .560 | DIAL (pair rate .50, quiet weight 2) | settled (2 seeds, 2026-09-22) |
| `rules-minted-seed1` | the second seed of the ruled dial; the quietest arm in the package on general text | arm | minted words .647 · unseen .593 · general-text cost .0066 bpb | DIAL | settled (2 seeds, 2026-09-22) |
| `chain` | reads five if-then rules and writes out every step: 'So Wren is …' five times | arm | chain .867 · general-text cost .003 bpb | E-I (trained with an abstention term, so it composes ungated) | settled (2 seeds) |
| `chain-plain` | the same step-by-step chain writing, trained without the abstention term: the best chain number in the library, but it writes on off-domain text too | arm | chain .887 | C1 | settled (2 seeds) |
| `eco-chain` | a loaded chain arm carried through a second training stage beside a fresh turn-end arm, and still detachable | arm | alone .833 · in the pair .853 · quiet .0054 bpb (the loaded arm read .867 before this stage) | ECO / L4 | settled (2 seeds) |
| `stop` | ends its turn cleanly with a blank line instead of rambling on | arm | clean stop .83 (second seed 1.00) | E-I | settled (2 seeds) |
| `stop-seed1` | the second seed: every probe ends its turn | arm | clean stop 1.00 | E-I | settled (2 seeds) |
| `eco-stop` | the fresh turn-end arm trained in that second stage, with its own abstention term | arm | turn end .953 in the pair · chat 1.00 | ECO / L4 | settled (2 seeds) |
| `routed-gain` | both arms carried by a dense signed dispatch: every member contributes at every byte, weighted by a signed read and a bounded gain that follows each member's recent usage | routed | chain .813 (seed 1 .787) | E-D0 | candidate (2 seeds; the conserved form is seed-unstable) |
| `pair` | both arms applied always-on with no mixer between them — the composition the regime law says should destroy a non-selective pair, and does not destroy these two | stack | chain .860 · turn end .973 · stop rate 1.00 | E-I | settled (2 seeds) |
| `eco-pair` | the staged co-training pair applied together | stack | chain .853 · turn end .953 | ECO / L4 | settled (2 seeds) |

*13 rows over 11 weight files; the full library of ~80 trained arms — including the refuted and control rows this package leaves out — lives on the training repo.*

**Status** is the research grade, not a rating: *settled (2 seeds)* means
the result reproduced on a second seed under the same recipe; *candidate*
means one axis is still open. **Cell** names the training recipe the row
came from. **Measured** is what the training campaign's own harness
scored on the full exam — `m.arms()` returns the complete per-arm
measurement dictionary behind each of these summaries.

Those numbers carry over to this package rather than being re-measured
here, and the reason is checkable: every packaged row produces logits
**bit-identical** to the runtime the arms were trained with (see *Detach
is exact*). Same weights and same logits under a greedy decode means the
same bytes, so re-scoring could only reproduce the stored number.

### Mounting

```python
m.mount_arm("pair")        # a stack: chain + turn end, both always-on
m.mount_arm("routed-gain") # a signed dispatch over both, with a usage gain
h = m.arm                  # the live handle
with h.all_off():          # every member masked: the bare core's logits
    m.say("Hello there.")
m.detach_arm(verify=True)  # raises if the restored core is not bit-exact
```

Mounting a second arm detaches the first. `verify=True` (the default)
re-runs a fixed probe through the restored core and asserts the logits
are bit-identical to the pre-mount fingerprint.

### What an arm is

Each arm places one module after every decoder block. At a site it
projects the block's output into `n_slots` slots of dimension `D`, reads
them through a `K`-atom **aleph address** — a closed-form signed read,
never a selector — and adds a gated patch back:

```
slots = proj(x).view(B, n, n_slots, D)
m_hat = sum_k sinh(u_k) A_k / sum_k cosh(u_k),   u = (x_hat · A) / tau
x     = x + sigmoid(gate) * consume(m_hat)
```

8.6M parameters over 20 sites (3.6% of the core). There is no argmax, no
top-k and no softmax over atoms anywhere in the path.

A **stack** applies several arms always-on with no mixer between them. A
**routed** recipe carries its members inside one dense signed dispatch:

```
out = x + sum_a (sinh u_a / sum_all cosh u) (arm_a(x) - x)
```

The denominator sums over **every** member, masked or not, and masking a
member removes its contribution without renormalizing — masked is not
solo. The `routed-gain` row adds a detached bounded gain per member,
driven by a causal average of that member's own share.

### Selective and non-selective arms

Arms marked *selective* were trained with an abstention term — a KL to
the bare core on rows outside their domain — and cost ~0.003 bpb on
general text. That is what lets two of them run together ungated
(`pair`): the same composition of two non-selective arms destroys both.
`chain-plain` is the counter-example kept in the package on purpose: the
best chain score in the library, and it writes on off-domain text too.

## Frames

| frame | used by | form |
|---|---|---|
| `specials` | the bare core | `[SYS] … [END] [USER] … [END] [MODEL]` on the 13 byte values UTF-8 can never produce — the frame is unforgeable, not by convention but by arithmetic |
| `text` | every packaged arm | `A conversation with Beatrix…\nUser: …\nBeatrix:` with a blank line ending the turn |

`say()` picks the frame the live arm was trained in and cuts the reply at
that frame's stop. The arms predate the specials anneal, so they speak the
text frame; the core's own chat frame is the specials one.

## The core

237.1M parameters · d_model 1024 · 20 layers · context 4096 · byte-trigram
embedding. A governed multi-constellation **CausalSplatHUB** — signed
address linear attention over learned codebook blackboards — in **every**
block (4 constellations × 64 anchors @ D=128, min-separation governed from
birth at 45°), three anchored bank experts per block under signed dispatch,
and a dual head (linear readout + a 256@256 signed aleph read revived
mid-mission after the born-null path self-buried).

Each layer *encodes* the sequence onto a fixed-width addressed blackboard
instead of caching it, so the decode state is **constant size** in context
length. Final held-out reading: **1.1097 bpb** on fineweb-edu.

Trained 16.101B bytes in 61,422 steps on one RTX 6000 Pro Blackwell:
wikitext warmup → fineweb-edu → a nine-stage early-life curriculum →
a two-phase anneal that taught the conversation frame. Zero loss spikes
across the run; the pre-registered softmax twin of the same recipe
destabilized and ended at 2.8846 bpb (the full comparison is on the 2s
card).

## Detach is exact

The claim this package exists to make checkable:

```python
import torch
before = m.say("Hello there.", frame="specials")
h = m.mount_arm("rules"); m.detach_arm(verify=True)
assert m.say("Hello there.", frame="specials") == before
```

At release every packaged row was mounted through this package **and**
through [amoe-lora](https://github.com/AbstractEyes/amoe-lora), the
runtime the arms were trained with, and the two paths produced
bit-identical logits (max |Δ| = 0.0 on all 13 rows, fp32). The core's
state dict is unchanged by any number of mount/detach cycles.

## Limitations

- Small and early: conversational in shape, thin on knowledge, and
  confidently wrong at times. Short exchanges suit it best.
- **Arms are trunk-bound.** They were trained on this exact core at step
  61,422 and carry its id in their metadata; `mount_arm` refuses a
  mismatch. A 31% gain was lost when an arm was moved across trunks.
- The arms' numbers were measured **greedy in fp32**. bf16 serving works
  and is not what the table measured.
- The rule-chain arms are a controlled capability on made-up words, not
  general reasoning: they read five if-then rules and either write the
  steps out or answer with the final word.
- Every packaged arm is the **wide** adapter shape: 32 slots of dimension
  4 read through a 64-atom address, 8.57M parameters. The certified twins
  of these same recipes at the narrower 16-atom, dimension-8 shape (a
  near-identical 8.56M parameters) live on the training repo and are
  **not** packaged here, so nothing in this table is a comparison
  between the two shapes.
- `rules-minted` was trained on the lexicon minted for the **next** craft's
  curriculum; it is packaged here because it is the newest settled arm and
  the same recipe, not because this core saw those words in pretraining.
- Saving the model with an arm mounted is refused: detach first, or the
  arm's parameters would be written into the core's weight file.

## Files

```
config.json  modeling_minibeatrix.py  arms.py        the package
alephlm.py address.py attention.py bank.py embedding.py
governor.py head.py relay.py presets.py              vendored model code
model.safetensors                                    the core (bf16, 476 MB)
arms/index.json                                      the arm table as data
arms/*.safetensors                                   the arms
```

`arms.py` is self-contained (torch + safetensors): the arm runtime is
vendored so the package has no git dependency, and it is verified against
amoe-lora at release rather than trusted.

The vendored model files are verbatim copies of `geolip/alephllm/model/*.py`
and `presets.py` from
[alephllm](https://github.com/AbstractEyes/alephllm) 0.10.2, with two
marked deviations, both inert for this craft and both there so the package
loads under `trust_remote_code`: the relative imports of the `data` and
`fusion` modules (a curriculum builder and the v3 weak-token fusion, neither
reachable at inference here) are deferred and resolved by name instead of
as flat sibling files. Every deviation is commented `VENDORED` in place.

## Lineage

Code: [alephllm](https://github.com/AbstractEyes/alephllm) ·
adapters: [amoe-lora](https://github.com/AbstractEyes/amoe-lora) ·
core weights, the full ~80-arm library, checkpoints and the boundary
reports: [alephllm-mini-beatrix-training](https://huggingface.co/AbstractPhil/alephllm-mini-beatrix-training) ·
predecessor: [mini-beatrix-1](https://huggingface.co/AbstractPhil/mini-beatrix-1)
(112M, 3-hub hybrid) · the core alone:
[mini-beatrix-2s](https://huggingface.co/AbstractPhil/mini-beatrix-2s).

The research memory behind every design decision is maintained privately;
the laws the code cites (supply ≤ 2D, min-separation governance, budget
composition, born-null weight-zero, the damping law) are measured, not
aesthetic.

## Licence

The weights are **non-commercial**: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/), because the training text includes CHILDES child-directed speech (CC BY-NC-SA, via TalkBank) and Wikipedia / WikiText (CC BY-SA). The code files, the model design and the full training process are MIT ([AbstractEyes/alephllm](https://github.com/AbstractEyes/alephllm)); `relay.py` and `arms.py` are adapted from amoe-lora under Apache-2.0 ([NOTICE](NOTICE)). The full terms and the list of training sources: [LICENSE.md](LICENSE.md).