---
license: apache-2.0
library_name: pytorch
tags: [dynamic-chunking, tokenization, hnet, byte-level, interpretability]
---

# H-Net dynamic-chunking pilot models

31 byte-level H-Net checkpoints (22.5M params, 6000 steps) trained to study **what a
learned chunker discovers when it is free to choose its own computational granularity**.
Every run shares data, seed schedule and FLOP budget; only the chunker's objective varies.

## Runs

| family | n | what varies |
|---|---|---|
| `pilot_A_s{0,1,2}` | 3 | no parity loss — the natural chunker (baseline) |
| `pilot_B_s{0,1,2}` | 3 | parity loss, per-content-matched-sentence beta |
| `pilot_C_s{0,1,2}` | 3 | parity applied then removed (reversion arm) |
| `pilot_D_s{0,1,2}` | 3 | stronger equalisation |
| `pilot_E_s{0,1,2}` | 3 | entropy-matched variant |
| `pilot_Apc_s{0,1,2}` | 3 | per-codepoint control |
| `pilot_Bpc_s{0,1,2}` | 3 | parity loss, per-codepoint beta |
| `emactl_noema_s{0,1,2}` | 3 | EMA ablation control (built from A) |
| `emactl_genreg_s{0,1,2}` | 3 | generic-regularisation control, entropy-matched to B |
| `jitter_s{0,1,2}` | 3 | threshold-jitter training (tau ~ U[0.30, 0.75]) |
| `baseline_en` | 1 | English-only baseline |

## Format

Weights are `model.safetensors`; `config.json` carries `model_config`, the training config,
and a `tied_weights` map. `emb.weight` and `head.weight` are tied, so only `emb.weight` is
stored — restore with:

```python
from safetensors.torch import load_file
import json
sd  = load_file("model.safetensors")
cfg = json.load(open("config.json"))
for dst, src in cfg["tied_weights"].items():
    sd[dst] = sd[src]          # head.weight <- emb.weight
```

Verified bit-exact against the original training checkpoints (`max|Δ| = 0.0`, 111 tensors).
`final.json` and `log.jsonl` are the trainer's own records.

## What these were used to find

- Segmentation crystallises **after** capability: the router lags BPB descent in 36/36
  language-seed pairs at half of each clock's own change.
- Learned units are **not linguistic**. Morpheme F1 sits at or below a rate-matched random
  baseline; gold word+morph features add ≤0.001 held-out AUROC over computational features
  in 9/9 languages.
- Chunks are **not functionally localised**: aligned-minus-misaligned activation patching is
  at the construction rate (0/28 cells above 2 across-seed SD), while killing the whole chunk
  pathway costs +0.34 nats/byte. Information is additive and diffuse across chunks.
- The chunk-level main network is worth only **~0.07 BPB** at this scale.

Negative results are reported as results. See the companion
[results dataset](https://huggingface.co/datasets/AdaptiveChunking/hnet-chunking-results)
and [probe dataset](https://huggingface.co/datasets/AdaptiveChunking/hnet-chunking-probes).

## Limitations

16.2M/22.5M params, 6000 steps, one hierarchy stage, 3 seeds. Only the final step was
retained (`keep_ckpt: false`), so no intermediate weights exist. Third-party models used for
comparison (`allenai/Bolmo-1B`, BLT, ByT5) are **not** redistributed here.