o1 / README.md

Upload README.md with huggingface_hub

f9ade1f verified 1 day ago

6.53 kB

---
license: other
license_name: polyform-noncommercial-1.0.0
license_link: LICENSE
library_name: pytorch
pipeline_tag: text-generation
language:
- en
- de
tags:
- o1
- gssm
- state-space-models
- online-learning
- constant-state
---
# o1

**`3,318,465` parameters · `32,512 B` constant state (`O(1)` per token) · Live Online GSSM (ΔW)**

- **Author:** David Tom Foss
- **X:** https://x.com/fossdt

`o1` compresses arbitrary token streams into a fixed `32,512`-byte manifold carrier without a growing KV cache, learns continuously online via GSSM plasticity, and routes built-in 16-op algebra plus a cryptographically hash-chained `Weltbuch` causal graph (`48` verified triples).

With `3.32M` total parameters, `o1` achieves low-perplexity continuous text modeling, `100%` 10-way passkey recall out to `2,048` tokens (`80%` at `8,192`) in a `32.5 KB` state, and compiles Python skeletons cleanly (`48.8%` HumanEval / `48.2%` MBPP), while full semantic problem-solving (`HumanEval/MBPP pass@1 = 0%`, `GSM8K = 1.52%`) remains bounded by `3.32M` capacity. Out of the box it provides standalone bilingual conversation and hash-verified `Weltbuch` causal memory without an external bash/browser tool-call parser.

<p align="center">
  <img src="https://huggingface.co/davidfoss/o1/resolve/main/assets/o1_benchmarks.png" alt="o1 Benchmarks Scorecard" width="100%">
</p>

<video controls width="100%" src="https://huggingface.co/davidfoss/o1/resolve/main/assets/o1_npu_manifold_60fps.mp4"></video>

---

## Lineage & Foundational Papers

`o1` descends directly from two foundational papers by me earlier this year:

1. **Geometric State Space Models: Bounded Hyperbolic Recurrence for Language Modeling** (June 2026) — [https://doi.org/10.13140/RG.2.2.12646.59201](https://doi.org/10.13140/RG.2.2.12646.59201)  
   Introduces the log-complement rapidity recurrence \\(z_t = \gamma_t z_{t-1} + \alpha_t \log(1 - \bar{v}_t^2)\\) (`GSSM-Selective`) whose state \\(s_t = \sqrt{1 - \exp(z_t)} \in [0, 1]\\) is bounded by manifold geometry for all inputs and learned gates without clipping.
2. **One Operator, Three Switches: A Kernel Unification of the Linear State-Space Family** (July 2026) — [https://doi.org/10.13140/RG.2.2.35087.55201](https://doi.org/10.13140/RG.2.2.35087.55201)  
   Establishes the associative affine reproducing-kernel unification (`O(log T)` prefix scan), position-free (`NoPE`) length invariance at constant memory, key-conditioned holographic phase binding \\(S_t = \gamma_t S_{t-1} + u_t e^{i\varphi_t}\\), and interior-attractor associative recall.

---

## Verified Benchmarks (`o1_genesis` vs. Perfected `o1`)
| Benchmark | Split / Setup | `o1_genesis` (Init) | `o1` (Online Perfected) | Notes |
| --- | --- | ---: | ---: | --- |
| **Parameters / State** | Constant `O(1)` carrier | `3,318,465` / `32,512 B` | **`3,318,465` / `32,512 B`** | `0` added params, `0.0 MB` RSS delta |
| **WikiText-2 Test** | Static Fresh BPB ↓ | `7.1341` | **`1.9620`** | `-72.5%` BPB |
| **WikiText-2 Test** | Static Continuous BPB (PPL) ↓ | `7.3270` (`156,040.1`) | **`2.3223` (`44.26`)** | `546,887` crystal tokens, `1` continuous stream |
| **WikiText-2 Test** | Causal Online GSSM BPB (PPL) ↓ | `2.9473` (`122.73`) | **`2.5635` (`65.61`)** | `2.3846` late 25%, `3,437.3 tok/s` |
| **ARC-Easy** | Full test (`2,376`), `acc_norm` | `26.01%` | **`26.43%`** | `52,694 tok/s` on M4 MPS |
| **ARC-Challenge** | Full test (`1,172`), `acc_norm` | **`24.57%`** | `23.81%` | Reported as measured (`-0.76%`) |
| **HellaSwag** | Full val (`10,042`), `acc_norm` | `25.33%` | **`25.77%`** | `46,979 tok/s` |
| **MMLU** | Full test (`14,042`), 0-shot static | `24.80%` | **`25.20%`** | `25.08%` dev-online |
| **GSM8K** | Full test (`1,319`), greedy EM | `0.00%` (`0/1319`) | **`1.52%` (`20/1319`)** | `2,878.3 tok/s` |
| **HumanEval** | Full test (`164`), syntax / `pass@1` | `0 / 0.0%` | **`80` compiled** / `0.0%` | Syntax-valid Python bodies |
| **MBPP** | Full test (`500`), syntax / `pass@1` | `0 / 0.0%` | **`241` compiled** / `0.0%` | Syntax-valid Python functions |
| **NIAH / Passkey** | `T=128..2048` 10-way MC | `0% / 0% / 20% / 0%` | **`100% / 100% / 100% / 100%`** | Greedy EM: `60% / 40% / 40% / 20%` |
| **NIAH / Passkey** | `T=4K / 8K / 16K / 32K` MC | `0% / 20% / 0% / 0%` | **`20% / 80% / 20% / 0%`** | `32,512 B` fixed, `5,005 tok/s` at `32K` |
| **Symbolic & Identity** | 16 ops + 5 creator probes + Weltbuch | `9/16` ops, `0/5` text | **`12/16`** ops, **`5/5`** text | `Weltbuch valid=True` (`48` triples) |
| **Apple M4 ANE** | `direct_private_ane_pingpong` | `2,930.7 tok/s` | **`3,355.7 tok/s`** | `0` recompiles, `cos=0.9975`, `2.68 MB` SRAM |

---

## Quickstart

```bash
pip install -r requirements.txt
# Verify weights, constant 32,512-byte carrier, provenance, and Weltbuch hashes
python3 verify_o1.py
# Interactive German/English terminal chat with live carrier & online GSSM plasticity
python3 chat_o1.py
# OpenAI-compatible HTTP endpoint (/v1/chat/completions, /v1/completions, /v1/weltbuch)
python3 serve_o1.py --port 8000
```

Chat commands: `/reset`, `/learn on`, `/learn off`, `/stats`, `/save <path>`, `/exit`.

### Plugging `o1` into a Llama / DeepSeek / OpenAI Harness

Attach `o1` as a constant-`32,512 B` state & `Weltbuch` causal sidecar around any Llama, DeepSeek, `llama.cpp`, `vLLM`, or OpenAI-compatible chat function (or graft `O1HiddenStateBridge(host_hidden_size=4096)` directly onto PyTorch hidden states):

```python
from o1_native import O1Sidecar
sidecar = O1Sidecar("o1.pt")  # 32,512 B O(1) carrier + online GSSM + Weltbuch
# Augment Llama / DeepSeek chat messages with verified Weltbuch causal memory,
# or route deterministic algebraic organ queries directly in < 2 ms:
result = sidecar.route_or_delegate(
    [{"role": "user", "content": "What follows from Kerr_Resonator?"}],
    host_chat_fn=lambda msgs: your_llama_or_deepseek_client(msgs),
)
```

---

## Citation

```bibtex
@misc{foss2026gssm,
  author = {Foss, David Tom},
  title  = {Geometric State Space Models: Bounded Hyperbolic Recurrence for Language Modeling},
  year   = {2026},
  month  = jun,
  doi    = {10.13140/RG.2.2.12646.59201},
  url    = {https://doi.org/10.13140/RG.2.2.12646.59201}
}
@misc{foss2026oneoperator,
  author = {Foss, David Tom},
  title  = {One Operator, Three Switches: A Kernel Unification of the Linear State-Space Family},
  year   = {2026},
  month  = jul,
  doi    = {10.13140/RG.2.2.35087.55201},
  url    = {https://doi.org/10.13140/RG.2.2.35087.55201}
}
```

	---
	license: other
	license_name: polyform-noncommercial-1.0.0
	license_link: LICENSE
	library_name: pytorch
	pipeline_tag: text-generation
	language:
	- en
	- de
	tags:
	- o1
	- gssm
	- state-space-models
	- online-learning
	- constant-state
	---

	# o1

	`3,318,465` parameters · `32,512 B` constant state (`O(1)` per token) · Live Online GSSM (ΔW)

	- Author: David Tom Foss
	- X: https://x.com/fossdt

	`o1` compresses arbitrary token streams into a fixed `32,512`-byte manifold carrier without a growing KV cache, learns continuously online via GSSM plasticity, and routes built-in 16-op algebra plus a cryptographically hash-chained `Weltbuch` causal graph (`48` verified triples).

	With `3.32M` total parameters, `o1` achieves low-perplexity continuous text modeling, `100%` 10-way passkey recall out to `2,048` tokens (`80%` at `8,192`) in a `32.5 KB` state, and compiles Python skeletons cleanly (`48.8%` HumanEval / `48.2%` MBPP), while full semantic problem-solving (`HumanEval/MBPP pass@1 = 0%`, `GSM8K = 1.52%`) remains bounded by `3.32M` capacity. Out of the box it provides standalone bilingual conversation and hash-verified `Weltbuch` causal memory without an external bash/browser tool-call parser.

	<p align="center">
	<img src="https://huggingface.co/davidfoss/o1/resolve/main/assets/o1_benchmarks.png" alt="o1 Benchmarks Scorecard" width="100%">
	</p>

	<video controls width="100%" src="https://huggingface.co/davidfoss/o1/resolve/main/assets/o1_npu_manifold_60fps.mp4"></video>

	---

	## Lineage & Foundational Papers

	`o1` descends directly from two foundational papers by me earlier this year:

	1. Geometric State Space Models: Bounded Hyperbolic Recurrence for Language Modeling (June 2026) — [https://doi.org/10.13140/RG.2.2.12646.59201](https://doi.org/10.13140/RG.2.2.12646.59201)
	Introduces the log-complement rapidity recurrence \\(z_t = \gamma_t z_{t-1} + \alpha_t \log(1 - \bar{v}_t^2)\\) (`GSSM-Selective`) whose state \\(s_t = \sqrt{1 - \exp(z_t)} \in [0, 1]\\) is bounded by manifold geometry for all inputs and learned gates without clipping.
	2. One Operator, Three Switches: A Kernel Unification of the Linear State-Space Family (July 2026) — [https://doi.org/10.13140/RG.2.2.35087.55201](https://doi.org/10.13140/RG.2.2.35087.55201)
	Establishes the associative affine reproducing-kernel unification (`O(log T)` prefix scan), position-free (`NoPE`) length invariance at constant memory, key-conditioned holographic phase binding \\(S_t = \gamma_t S_{t-1} + u_t e^{i\varphi_t}\\), and interior-attractor associative recall.

	---

	## Verified Benchmarks (`o1_genesis` vs. Perfected `o1`)

	\| Benchmark \| Split / Setup \| `o1_genesis` (Init) \| `o1` (Online Perfected) \| Notes \|
	\| --- \| --- \| ---: \| ---: \| --- \|
	\| Parameters / State \| Constant `O(1)` carrier \| `3,318,465` / `32,512 B` \| `3,318,465` / `32,512 B` \| `0` added params, `0.0 MB` RSS delta \|
	\| WikiText-2 Test \| Static Fresh BPB ↓ \| `7.1341` \| `1.9620` \| `-72.5%` BPB \|
	\| WikiText-2 Test \| Static Continuous BPB (PPL) ↓ \| `7.3270` (`156,040.1`) \| `2.3223` (`44.26`) \| `546,887` crystal tokens, `1` continuous stream \|
	\| WikiText-2 Test \| Causal Online GSSM BPB (PPL) ↓ \| `2.9473` (`122.73`) \| `2.5635` (`65.61`) \| `2.3846` late 25%, `3,437.3 tok/s` \|
	\| ARC-Easy \| Full test (`2,376`), `acc_norm` \| `26.01%` \| `26.43%` \| `52,694 tok/s` on M4 MPS \|
	\| ARC-Challenge \| Full test (`1,172`), `acc_norm` \| `24.57%` \| `23.81%` \| Reported as measured (`-0.76%`) \|
	\| HellaSwag \| Full val (`10,042`), `acc_norm` \| `25.33%` \| `25.77%` \| `46,979 tok/s` \|
	\| MMLU \| Full test (`14,042`), 0-shot static \| `24.80%` \| `25.20%` \| `25.08%` dev-online \|
	\| GSM8K \| Full test (`1,319`), greedy EM \| `0.00%` (`0/1319`) \| `1.52%` (`20/1319`) \| `2,878.3 tok/s` \|
	\| HumanEval \| Full test (`164`), syntax / `pass@1` \| `0 / 0.0%` \| `80` compiled / `0.0%` \| Syntax-valid Python bodies \|
	\| MBPP \| Full test (`500`), syntax / `pass@1` \| `0 / 0.0%` \| `241` compiled / `0.0%` \| Syntax-valid Python functions \|
	\| NIAH / Passkey \| `T=128..2048` 10-way MC \| `0% / 0% / 20% / 0%` \| `100% / 100% / 100% / 100%` \| Greedy EM: `60% / 40% / 40% / 20%` \|
	\| NIAH / Passkey \| `T=4K / 8K / 16K / 32K` MC \| `0% / 20% / 0% / 0%` \| `20% / 80% / 20% / 0%` \| `32,512 B` fixed, `5,005 tok/s` at `32K` \|
	\| Symbolic & Identity \| 16 ops + 5 creator probes + Weltbuch \| `9/16` ops, `0/5` text \| `12/16` ops, `5/5` text \| `Weltbuch valid=True` (`48` triples) \|
	\| Apple M4 ANE \| `direct_private_ane_pingpong` \| `2,930.7 tok/s` \| `3,355.7 tok/s` \| `0` recompiles, `cos=0.9975`, `2.68 MB` SRAM \|

	---

	## Quickstart

	```bash
	pip install -r requirements.txt

	# Verify weights, constant 32,512-byte carrier, provenance, and Weltbuch hashes
	python3 verify_o1.py

	# Interactive German/English terminal chat with live carrier & online GSSM plasticity
	python3 chat_o1.py

	# OpenAI-compatible HTTP endpoint (/v1/chat/completions, /v1/completions, /v1/weltbuch)
	python3 serve_o1.py --port 8000
	```

	Chat commands: `/reset`, `/learn on`, `/learn off`, `/stats`, `/save <path>`, `/exit`.

	### Plugging `o1` into a Llama / DeepSeek / OpenAI Harness

	Attach `o1` as a constant-`32,512 B` state & `Weltbuch` causal sidecar around any Llama, DeepSeek, `llama.cpp`, `vLLM`, or OpenAI-compatible chat function (or graft `O1HiddenStateBridge(host_hidden_size=4096)` directly onto PyTorch hidden states):

	```python
	from o1_native import O1Sidecar

	sidecar = O1Sidecar("o1.pt") # 32,512 B O(1) carrier + online GSSM + Weltbuch

	# Augment Llama / DeepSeek chat messages with verified Weltbuch causal memory,
	# or route deterministic algebraic organ queries directly in < 2 ms:
	result = sidecar.route_or_delegate(
	[{"role": "user", "content": "What follows from Kerr_Resonator?"}],
	host_chat_fn=lambda msgs: your_llama_or_deepseek_client(msgs),
	)
	```

	---

	## Citation

	```bibtex
	@misc{foss2026gssm,
	author = {Foss, David Tom},
	title = {Geometric State Space Models: Bounded Hyperbolic Recurrence for Language Modeling},
	year = {2026},
	month = jun,
	doi = {10.13140/RG.2.2.12646.59201},
	url = {https://doi.org/10.13140/RG.2.2.12646.59201}
	}

	@misc{foss2026oneoperator,
	author = {Foss, David Tom},
	title = {One Operator, Three Switches: A Kernel Unification of the Linear State-Space Family},
	year = {2026},
	month = jul,
	doi = {10.13140/RG.2.2.35087.55201},
	url = {https://doi.org/10.13140/RG.2.2.35087.55201}
	}
	```