README.md

---
license: gpl-3.0
language:
- en
pipeline_tag: text-generation
tags:
- verifiable-training
- reproducibility
- gpt2
- gguf
datasets:
- AOSSIE/openverifiable-enwiki-20260901-20260918-r1-evidence
---

# OpenVerifiableLLM 23M

A 23.1M-parameter GPT trained on exactly one pass over all of English Wikipedia (the 2026-09-01 dump),
then a short conversation phase. Its training can be recomputed bit for bit: all 37 training segments
were recomputed on a second RTX 5090 and matched. The model is small and often wrong about facts. The
point is that you can check how it was made.

- Code and docs: <https://github.com/AOSSIE-Org/OpenVerifiableLLM>
- Training data and the training record (chain, checkpoints, timestamped anchor):
  [`AOSSIE/openverifiable-enwiki-20260901-20260918-r1-evidence`](https://huggingface.co/datasets/AOSSIE/openverifiable-enwiki-20260901-20260918-r1-evidence)
- How it works: <https://github.com/AOSSIE-Org/OpenVerifiableLLM/blob/main/docs/how-it-works.md>

## Files

| File | What it is |
|---|---|
| `conversation.safetensors` | The chat model (chain entry 37) |
| `base.safetensors` | The Wikipedia-only model, before the conversation phase (chain entry 31) |
| `ovl-23m-chat-F32.gguf` | The chat model for llama.cpp and Ollama, all weights F32 |
| `Modelfile` | The Ollama build file, with the training chat format |
| `recipe.json` | The training recipe: model size, schedule, seed |
| `tokenizer.json` | The byte-level BPE tokenizer (31,995 tokens after 5 control tokens) |

SHA-256 of each file (check a download with `sha256sum -c`):

```text
00c1c450acb2619f616af973d283fe5ef5cf1baaae5f16ff80bca97688f25168  conversation.safetensors
5404ce5c7583e2e299d6e9fa44ed48e98bc5b5e11bc976905e8466a6fc5c36c2  base.safetensors
541d49c97b87cd4ed0acb90b4327535ee20b8c11c452e2583d460154e9d980de  ovl-23m-chat-F32.gguf
30631c28d3c5ce44a2ff4cdd5c74c415dad12f40df87a0b118d0ffa1c4d4b5b8  Modelfile
28a2fb407a5582ae95c9e9b0bba2c14df946e87bba037800399c6059256b6e6d  recipe.json
6eab0d9bcd5736cee484992b24e7a84459bee1cc3024c0e958a76842a57728c0  tokenizer.json
```

The GGUF has the same bytes as `ovl-23m-chat.gguf` in the GitHub releases.

The safetensors files hold exactly the weights of the released `.pt` files, including both copies of
the tied embedding and output weights. The metadata key `ovl` records the `.pt` file's SHA-256 and the
model digest. `simple/export_safetensors.py` rebuilds these files byte for byte from the `.pt` files.

## Use it

```bash
ollama run ryoari/openverifiablellm "Tell me about the Moon."
```

The chat format is `<|user|>` question `<|eos|>` `<|assistant|>` answer `<|eos|>`. llama.cpp's outputs
are close to PyTorch's but not bit-identical, so use the safetensors files to check the model.

| | |
|---|---|
| Architecture | GPT-2-style: 6 layers, 6 heads, 384 dimensions, 512-token context, tied embeddings |
| Training | 560,177 Wikipedia updates and 441 conversation updates, batch 32 × 512 tokens, seed 1234 |
| Data | 7,232,582 Wikipedia articles (7,118,121,139 training targets) and 3,481 OpenAssistant conversations |

## Check it

On a laptop, check that a weights file is exactly the state in the timestamped training record. This
needs `ovl.py` from v1.1.0 or newer, Python 3.12 or newer, `git`, `openssl` and network access:

```bash
pip install huggingface_hub torch numpy safetensors
hf download AOSSIE/OpenVerifiableLLM-23M conversation.safetensors --local-dir model
hf download AOSSIE/openverifiable-enwiki-20260901-20260918-r1-evidence --repo-type dataset \
  --include "release/simple-run/*" --exclude "*/ckpt-*" --exclude "*.pt" --local-dir ovl
git clone -q --depth 1 --branch v1.1.0 https://github.com/AOSSIE-Org/OpenVerifiableLLM ovl-code
OVL_DEVICE=cpu python ovl-code/simple/ovl.py verify --run ovl/release/simple-run --weights model/conversation.safetensors
# anchor ...: chain head MATCH (38 entries)
# anchor timestamp freetsa: OK (Oct 20 14:38:43 2026 GMT)
# anchor timestamp digicert: OK (Oct 20 14:38:43 2026 GMT)
# weights model/conversation.safetensors: MATCH chain entry 37
```

This is a consistency check: it recomputes no training. To verify the training itself, recompute a
segment on an RTX 5090 (about an hour, about US$1.50):
<https://github.com/AOSSIE-Org/OpenVerifiableLLM/blob/main/docs/verify.md>.

## Limits

- The model is small. It writes fluent, encyclopedia-style text, often gets facts wrong, and holds a
  basic conversation. "Verified" is about how it was made, not about what it says.
- Bits match only on the training GPU type (RTX 5090). An RTX 4090, an H100 and a CPU give different
  bits.
- So far every segment check was run by the project. No independent third party has checked one yet.
- The timestamps prove that the record existed by 2026-10-20 14:38:43 UTC, not when or how training
  happened.

## License

GPL-3.0, like the code. The training data keeps its own terms: English Wikipedia text (generally
CC BY-SA 4.0, with documented exceptions) and OpenAssistant `oasst1` (Apache 2.0). See the dataset's
`LICENSES.md`.