File size: 11,387 Bytes
05f5056 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 | # Architecture
Five layers, strictly one-directional. Nothing below a layer imports from above it.
```
βββββββββββββββββββββββββββββββ
β scripts/ β smoke.py, e2e.py
ββββββββββββββββ¬βββββββββββββββ
β
βββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββ
β β β
βββΌβββββββββββββββ βββββββββΌβββββββββ βββββββββββΌββββββββββ
β rsi.py β β train.py β β eval.py β
β accept/revert ββββββββββββββ€ loop, ckpt βββββββββββββββββΊβ score, verify, β
ββββββββββββββββββ scores β resume, CLI β model+tokenizerβ gate, repair β
βββββββββ¬βββββββββ βββββββββββ¬ββββββββββ
β β
ββββββββββββββΌββββββββββββββ β
β model.py βββββββββββββββββββββ
β GQA, RoPE, RMSNorm, β
β SwiGLU, MiniTransformerβ
ββββββββββββββ¬ββββββββββββββ
β
ββββββββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββ
β data.py β config.py β
β tokenizer, manifest, streams β frozen config objects β
ββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββ
tools.py β orthogonal: called by eval (tool scoring) and by any agent runtime
```
`tools.py` is deliberately a leaf. The model never imports it; only the evaluation loop and external callers do. That keeps the sandbox auditable in isolation.
---
## Module contracts
### `config.py` β validation at the boundary
Four frozen dataclasses: `ModelConfig`, `TrainingConfig`, `EvalConfig`, `SafetyConfig`. Every one validates itself in `__post_init__`, so an invalid configuration cannot be constructed β it raises at construction, not deep inside a training step.
Cross-field invariants that are enforced (and would otherwise be silent bugs):
- `num_query_heads % num_kv_heads == 0` β GQA cannot broadcast otherwise.
- `d_model % num_query_heads == 0` β head_dim must be integral.
- `head_dim % 2 == 0` β RoPE rotates pairs of dimensions.
- `max_steps > warmup_steps` β the LR schedule would divide by zero.
- `precision == "fp32"` and `gradient_checkpointing is True` β the CPU contract is not negotiable.
- `schedule == "cosine"` β the only implemented schedule.
`ModelConfig` also owns the **parameter accounting** (`parameter_count_tied` / `parameter_count_untied`) as a closed-form formula, so the number in the README is derived, not measured. `scripts/smoke.py` asserts it against the constructed model.
### `model.py` β the network
`RMSNorm` β `GroupedQueryAttention` β `SwiGLU` β `DecoderBlock` β `MiniTransformer`.
`MiniTransformer.forward` returns logits directly: with tied embeddings the output projection is `hidden @ E.T` and `self.output_projection is None`. There is no bias anywhere in the projections.
Attention is computed in **query blocks of 128** so the score matrix is never materialised for the full sequence. This is the single most important memory decision in the model and the reason a 8,192-token context fits the inference budget.
Two guards matter for numerical safety:
- rows where **no** key is allowed (fully padded) would make `softmax` produce `NaN`; they are zeroed before the softmax and re-masked after it.
- masked scores are filled with `torch.finfo(dtype).min`, not `-inf`, so a `NaN` in one block cannot propagate.
`generate` is a straightforward autoregressive loop with a sliding window of `config.context_length`; it extends the attention mask by one position per step and stops as soon as any sequence in the batch emits `eos_id`. There is no KV cache β see [THEORY.md](THEORY.md#inference-memory) for what that means for the memory estimate.
`estimate_inference_memory_bytes()` is an **analytic** upper bound: weights + KV + hidden + one attention block + next-token logits. It is a budget-checking tool, not a measurement. See [OPERATIONS.md](OPERATIONS.md#verifying-the-memory-budget).
### `data.py` β bytes in, token streams out
`SimpleTokenizer` is deliberately a byte-level tokenizer: 6 reserved ids (`pad`, `eos`, `unk`, `tool`, `python`, `shell`) and one id per byte. It is not a good tokenizer β it is a *reproducible* one, which matters more at this stage, and it makes the whole pipeline Unicode-safe with no vocab file to ship.
`build_train_stream` produces fixed-length token windows from a list of examples, cycling the source, and accepts `start_token` so training can resume at an exact token offset. `build_eval_stream` pads to a fixed length instead.
`validate_manifest` enforces the provenance contract: a source may only be `enabled: true` if it has a 64-hex SHA-256 and a positive token budget; a disabled source must have `token_budget: 0`. URLs must be http(s).
### `train.py` β the loop
`train_model` owns the whole lifecycle:
1. move the model to the device, enable gradient checkpointing, wrap in `nn.DataParallel` when more than one CUDA device is visible;
2. build the optimizer;
3. optionally restore a checkpoint (weights, optimizer, step, token cursor, RNG states);
4. iterate the token stream, accumulating 8 sequences of 1024 tokens into one optimizer step;
5. checkpoint on the token grid, emit structured progress events, return `{step, tokens_seen, last_loss}`.
All telemetry is JSON Lines on **stderr** (`train_start`, `gpu_ready`, `train_progress`, `checkpoint_saved`, `train_done`), leaving stdout for the final result object. That split is what makes the run greppable in a Kaggle/Colab log.
### `eval.py` β score, verify, gate
`evaluate_model` moves the model to CPU, switches to eval mode, and per example:
- **math / coding** β generate, then run `VerifierRepairLoop(max_repairs=1)`: verify the answer, and on failure feed a feedback continuation back through the model once. A score counts only if the logits were finite *and* the verifier passed.
- **tool** β hand the parsed tool call to the executor; the score is the executor's `ok`.
The training/eval mode flag of the model is restored in a `finally` block, so an exception mid-eval cannot leave the model stuck in eval mode.
`strict_gate` returns an `EvalGateResult` that is truthy, so `if strict_gate(scores):` reads naturally.
### `rsi.py` β the policy
`rsi_accept` is a pure function over two `EvalScores`. It is the only place that decides whether a change is allowed to survive. `RSIController` adds apply/accept/revert state; `run_rsi_cycle` wraps a full experiment and **always reverts in a `finally`**, so a crashed experiment cannot leave a rejected candidate installed.
### `tools.py` β protocol and sandbox
`build_envelope` / `parse_envelope` define a canonical JSON envelope (`sort_keys`, compact separators) with protected `tool` / `request_id` fields, so a tool can never spoof the envelope's own metadata. `parse_tool_call` normalises a call into `{"kind", "payload"}` with `kind β {python, shell}`.
Execution is described in [SECURITY.md](SECURITY.md).
---
## Checkpoint format
A single `torch.save` payload:
```python
{
"model": state_dict, # device-agnostic, always saved from the unwrapped module
"optimizer": optimizer.state_dict(),
"tokens_seen": int,
"step": int,
"rng_state": {"python": ..., "torch": ..., "cuda": [...]},
"extra": {"last_loss": float, "next_checkpoint_at": int},
"data_state": {"cursor": int}, # must equal tokens_seen, or loading raises
}
```
Properties that matter operationally:
- **Atomic.** Written to `.{name}.{pid}.tmp` and moved into place with `os.replace`, so a kill mid-write cannot corrupt the previous checkpoint.
- **Device-agnostic.** `load_checkpoint` always maps to CPU and transparently adds or strips the `module.` prefix that `nn.DataParallel` introduces, so a CPU checkpoint resumes on a GPU run and vice versa.
- **Verifiable cursor.** A checkpoint whose `data_state.cursor` disagrees with `tokens_seen` is rejected rather than silently resuming at the wrong position in the corpus.
## Data flow
```
manifest.json ββvalidate_manifestβββΊ allowed sources
β
examples ββencodeβββΊ token ids ββbuild_train_stream(start_token=β¦)βββΊ [1024] windows
β
ββββββββββββββββββββββββ
βΌ
[1,1023] inputs + [1,1023] labels βββΊ forward βββΊ cross_entropy βββΊ backward
β
accumulate until 8192 tokens βββ€
βΌ
set LR ββΊ unscale grads ββΊ clip(1.0) ββΊ AdamW.step
β
every 5M tokens ββΊ save_checkpoint (atomic)
```
## Extension points
| To add⦠| Do this | Not this |
| --- | --- | --- |
| a corpus | add a pinned entry to `data/manifest.json` with sha256 + budget, set `enabled: true` | dropping files into the repo |
| a tokenizer | implement `encode(text, add_eos=True) -> list[int]` and `decode(ids) -> str` | editing the training loop |
| a model change | subclass or edit `model.py`; re-run `scripts/smoke.py` so the parameter-count assertion still holds | bypassing `ModelConfig` |
| a new domain | add a branch in `evaluate_model` and a field on `EvalScores` + `EvalConfig` | adding an unvalidated score |
| a new tool | extend `parse_tool_call` + `execute_tool_call` and add it to the allowlists | calling `subprocess` directly from eval |
| an RSI experiment | call `run_rsi_cycle` with apply/evaluate/revert callbacks | mutating the baseline in place |
|