OpenVerifiableLLM 23M: safetensors, GGUF, tokenizer, recipe and model card

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+ovl-23m-chat.gguf filter=lfs diff=lfs merge=lfs -text

Modelfile

@@ -0,0 +1,13 @@
+# Ollama build file for the released chat model. Build the GGUF first:
+#   python simple/export_gguf.py --run RUN --tokenizer PREP/tokenizer/tokenizer.json --out ovl-23m-chat.gguf
+#   ollama create openverifiablellm -f simple/Modelfile
+# Run `ollama create` from the directory that holds ovl-23m-chat.gguf, or edit the FROM path.
+FROM ./ovl-23m-chat.gguf
+
+# The training format: <|user|> text <|eos|> <|assistant|> text <|eos|> ...
+TEMPLATE """{{- range .Messages }}{{ if eq .Role "user" }}<|user|>{{ .Content }}<|eos|>{{ else if eq .Role "assistant" }}<|assistant|>{{ .Content }}<|eos|>{{ end }}{{ end }}<|assistant|>"""
+
+PARAMETER stop "<|eos|>"
+PARAMETER stop "<|user|>"
+PARAMETER num_ctx 512
+PARAMETER temperature 0.8

README.md

@@ -0,0 +1,93 @@
+---
+license: gpl-3.0
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- verifiable-training
+- reproducibility
+- gpt2
+- gguf
+datasets:
+- AOSSIE/openverifiable-enwiki-20260901-20260918-r1-evidence
+---
+
+# OpenVerifiableLLM 23M
+
+A 23.1M-parameter GPT trained on exactly one pass over all of English Wikipedia (the 2026-09-01 dump),
+then a short conversation phase. Its training can be recomputed bit for bit: all 37 training segments
+were recomputed on a second RTX 5090 and matched. The model is small and often wrong about facts. The
+point is that you can check how it was made.
+
+- Code and docs: <https://github.com/AOSSIE-Org/OpenVerifiableLLM>
+- Training data and the training record (chain, checkpoints, timestamped anchor):
+  [`AOSSIE/openverifiable-enwiki-20260901-20260918-r1-evidence`](https://huggingface.co/datasets/AOSSIE/openverifiable-enwiki-20260901-20260918-r1-evidence)
+- How it works: <https://github.com/AOSSIE-Org/OpenVerifiableLLM/blob/main/docs/how-it-works.md>
+
+## Files
+
+| File | What it is | SHA-256 |
+|---|---|---|
+| `conversation.safetensors` | The chat model (chain entry 37) | `00c1c450acb2619f616af973d283fe5ef5cf1baaae5f16ff80bca97688f25168` |
+| `base.safetensors` | The Wikipedia-only model, before the conversation phase (chain entry 31) | `5404ce5c7583e2e299d6e9fa44ed48e98bc5b5e11bc976905e8466a6fc5c36c2` |
+| `ovl-23m-chat.gguf` | The chat model for llama.cpp and Ollama | `541d49c97b87cd4ed0acb90b4327535ee20b8c11c452e2583d460154e9d980de` |
+| `Modelfile` | The Ollama build file, with the training chat format | `df4dacfc3191e56b1dc1195764ea66ca1a7695856be04ecec17d8dde4c66fc8b` |
+| `recipe.json` | The training recipe: model size, schedule, seed | `28a2fb407a5582ae95c9e9b0bba2c14df946e87bba037800399c6059256b6e6d` |
+| `tokenizer.json` | The byte-level BPE tokenizer (31,995 tokens after 5 control tokens) | `6eab0d9bcd5736cee484992b24e7a84459bee1cc3024c0e958a76842a57728c0` |
+
+The safetensors files hold exactly the weights of the released `.pt` files, including both copies of
+the tied embedding and output weights. The metadata key `ovl` records the `.pt` file's SHA-256 and the
+model digest. `simple/export_safetensors.py` rebuilds these files byte for byte from the `.pt` files.
+
+## Use it
+
+```bash
+ollama run ryoari/openverifiablellm "Tell me about the Moon."
+```
+
+The chat format is `<|user|>` question `<|eos|>` `<|assistant|>` answer `<|eos|>`. llama.cpp's outputs
+are close to PyTorch's but not bit-identical, so use the safetensors files to check the model.
+
+| | |
+|---|---|
+| Architecture | GPT-2-style: 6 layers, 6 heads, 384 dimensions, 512-token context, tied embeddings |
+| Training | 560,177 Wikipedia updates and 441 conversation updates, batch 32 × 512 tokens, seed 1234 |
+| Data | 7,232,582 Wikipedia articles (7,118,121,139 training targets) and 3,481 OpenAssistant conversations |
+
+## Check it
+
+On a laptop, check that a weights file is exactly the state in the timestamped training record. This
+needs `ovl.py` from v1.1.0 or newer, Python 3.10 or newer, `git`, `openssl` and network access:
+
+```bash
+pip install huggingface_hub torch numpy safetensors
+hf download AOSSIE/OpenVerifiableLLM-23M conversation.safetensors --local-dir model
+hf download AOSSIE/openverifiable-enwiki-20260901-20260918-r1-evidence --repo-type dataset \
+  --include "release/simple-run/*" --exclude "*/ckpt-*" --exclude "*.pt" --local-dir ovl
+git clone -q --depth 1 --branch v1.1.0 https://github.com/AOSSIE-Org/OpenVerifiableLLM ovl-code
+OVL_DEVICE=cpu python ovl-code/simple/ovl.py verify --run ovl/release/simple-run --weights model/conversation.safetensors
+# anchor ...: chain head MATCH (38 entries)
+# anchor timestamp freetsa: OK (Oct 20 14:38:43 2026 GMT)
+# anchor timestamp digicert: OK (Oct 20 14:38:43 2026 GMT)
+# weights model/conversation.safetensors: MATCH chain entry 37
+```
+
+This is a consistency check: it recomputes no training. To verify the training itself, recompute a
+segment on an RTX 5090 (about an hour, about US$1.50):
+<https://github.com/AOSSIE-Org/OpenVerifiableLLM/blob/main/docs/verify.md>.
+
+## Limits
+
+- The model is small. It writes fluent, encyclopedia-style text, often gets facts wrong, and holds a
+  basic conversation. "Verified" is about how it was made, not about what it says.
+- Bits match only on the training GPU type (RTX 5090). An RTX 4090, an H100 and a CPU give different
+  bits.
+- So far every segment check was run by the project. No independent third party has checked one yet.
+- The timestamps prove that the record existed by 2026-10-20 14:38:43 UTC, not when or how training
+  happened.
+
+## License
+
+GPL-3.0, like the code. The training data keeps its own terms: English Wikipedia text (generally
+CC BY-SA 4.0, with documented exceptions) and OpenAssistant `oasst1` (Apache 2.0). See the dataset's
+`LICENSES.md`.

base.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5404ce5c7583e2e299d6e9fa44ed48e98bc5b5e11bc976905e8466a6fc5c36c2
+size 147980488

conversation.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:00c1c450acb2619f616af973d283fe5ef5cf1baaae5f16ff80bca97688f25168
+size 147980496

ovl-23m-chat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:541d49c97b87cd4ed0acb90b4327535ee20b8c11c452e2583d460154e9d980de
+size 142815808

recipe.json

@@ -0,0 +1,25 @@
+{
+ "batch_size": 32,
+ "boundary_every": 18080,
+ "context": 512,
+ "conversation_epochs": 3,
+ "init": "normal-0.02-tied-v1",
+ "model": {
+  "attn_impl": "manual",
+  "dropout": 0,
+  "embed_dim": 384,
+  "max_seq_len": 512,
+  "num_heads": 6,
+  "num_layers": 6,
+  "vocab_size": 32000
+ },
+ "schedule": {
+  "conversation": "0.0001",
+  "floor": "0.00006",
+  "peak": "0.0006",
+  "warmup": 2000,
+  "wikipedia_updates": 560177
+ },
+ "seed": 1234,
+ "weight_decay": "0.01"
+}