Text Generation
Transformers
Safetensors
olmo_hybrid
risc-v
compiled-weights
olmo-hybrid
gated-deltanet
tool-use
Instructions to use jgalego/olmputer-tiny with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use jgalego/olmputer-tiny with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="jgalego/olmputer-tiny")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("jgalego/olmputer-tiny") model = AutoModelForCausalLM.from_pretrained("jgalego/olmputer-tiny", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use jgalego/olmputer-tiny with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "jgalego/olmputer-tiny" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jgalego/olmputer-tiny", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/jgalego/olmputer-tiny
- SGLang
How to use jgalego/olmputer-tiny with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "jgalego/olmputer-tiny" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jgalego/olmputer-tiny", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "jgalego/olmputer-tiny" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jgalego/olmputer-tiny", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use jgalego/olmputer-tiny with Docker Model Runner:
docker model run hf.co/jgalego/olmputer-tiny
Upload folder using huggingface_hub
Browse files- README.md +40 -2
- assets/checks.png +0 -0
- assets/layers.png +0 -0
- assets/logo.svg +1 -0
README.md
CHANGED
|
@@ -12,6 +12,8 @@ tags:
|
|
| 12 |
|
| 13 |
# olmputer-tiny
|
| 14 |
|
|
|
|
|
|
|
| 15 |
A RISC-V computer (RV32IM + Zba) compiled into the layers of a stock `OlmoHybridForCausalLM`. No weight is trained: a compiler computes all of them. The code ROM, data ROM, RAM and registers live in the recurrent state of the Gated DeltaNet layers, and the model runs a program by greedy decoding: every token it generates is the next token of the CPU's execution trace.
|
| 16 |
|
| 17 |
This is the standalone half of [Olmputer](https://huggingface.co/jgalego/olmputer), which puts the same layers in front of [allenai/Olmo-Hybrid-Instruct-DPO-7B](https://huggingface.co/allenai/Olmo-Hybrid-Instruct-DPO-7B) so that OLMo's `calc` tool calls run on the CPU inside its weights. It ports [Qwenputer](https://github.com/divagr18/qwenputer), which does this with Qwen3.5-4B, to OLMo-Hybrid's head shapes. The compiler is the `chipformer` package of [JGalego/olmputer](https://github.com/JGalego/olmputer).
|
|
@@ -29,6 +31,23 @@ This is the standalone half of [Olmputer](https://huggingface.co/jgalego/olmpute
|
|
| 29 |
| Parameters | 3.13B, almost all zero |
|
| 30 |
| Vocabulary | the trace tokens (values in 15-bit chunks, jumps, stores, putchar, ...) plus PAD |
|
| 31 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
Qwen3.5-4B's heads are 128 × 128, two value heads per key head; OLMo's are 96 × 192, one each. The changes that keep every read exact:
|
| 33 |
|
| 34 |
- A state row's ballast sits in 3 of its 192 columns (Qwen: 2 of 128), so the gated output norm divides by the same power of 2.
|
|
@@ -43,12 +62,31 @@ For each program, the machine's trace from a reference RISC-V emulator is the ta
|
|
| 43 |
- the residual stream after the chip layers equals the circuit's IR evaluator on every dimension at every position (bf16; traces up to 60k tokens);
|
| 44 |
- free-running greedy `generate` from the program's start reproduces the reference trace token for token.
|
| 45 |
|
| 46 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
## Usage
|
| 49 |
|
| 50 |
```bash
|
| 51 |
-
pip install "chipformer @ https://github.com/JGalego/olmputer/archive/
|
| 52 |
```
|
| 53 |
|
| 54 |
```python
|
|
|
|
| 12 |
|
| 13 |
# olmputer-tiny
|
| 14 |
|
| 15 |
+
<img src="assets/logo.svg" alt="Olmputer logo: a pixel computer" width="384">
|
| 16 |
+
|
| 17 |
A RISC-V computer (RV32IM + Zba) compiled into the layers of a stock `OlmoHybridForCausalLM`. No weight is trained: a compiler computes all of them. The code ROM, data ROM, RAM and registers live in the recurrent state of the Gated DeltaNet layers, and the model runs a program by greedy decoding: every token it generates is the next token of the CPU's execution trace.
|
| 18 |
|
| 19 |
This is the standalone half of [Olmputer](https://huggingface.co/jgalego/olmputer), which puts the same layers in front of [allenai/Olmo-Hybrid-Instruct-DPO-7B](https://huggingface.co/allenai/Olmo-Hybrid-Instruct-DPO-7B) so that OLMo's `calc` tool calls run on the CPU inside its weights. It ports [Qwenputer](https://github.com/divagr18/qwenputer), which does this with Qwen3.5-4B, to OLMo-Hybrid's head shapes. The compiler is the `chipformer` package of [JGalego/olmputer](https://github.com/JGalego/olmputer).
|
|
|
|
| 31 |
| Parameters | 3.13B, almost all zero |
|
| 32 |
| Vocabulary | the trace tokens (values in 15-bit chunks, jumps, stores, putchar, ...) plus PAD |
|
| 33 |
|
| 34 |
+
Each generated token runs once through the 11 layers, and each layer does one part of executing the instruction:
|
| 35 |
+
|
| 36 |
+
```mermaid
|
| 37 |
+
flowchart TB
|
| 38 |
+
IN["a trace token in"] --> L1["L1: the PC anchor and counters (record position, commits since the last jump)"]
|
| 39 |
+
L1 --> L2["L2: the PC word and the link address"]
|
| 40 |
+
L2 --> L3["L3: fetch: the code ROM, 23 heads of 2,944 words each"]
|
| 41 |
+
L3 --> L4["L4: decode; read 3 register ports"]
|
| 42 |
+
L4 --> L5["L5: the bignum digit file; adder and multiplier stages"]
|
| 43 |
+
L5 --> L6["L6 to L8: the RAM, 86 heads of 96 words; loads, stores, compares"]
|
| 44 |
+
L6 --> L9["L9: the data ROM; divider digits"]
|
| 45 |
+
L9 --> L10["L10 to L11: pick the result and write the next token's code"]
|
| 46 |
+
L10 --> OUT["the next trace token"]
|
| 47 |
+
```
|
| 48 |
+
|
| 49 |
+

|
| 50 |
+
|
| 51 |
Qwen3.5-4B's heads are 128 × 128, two value heads per key head; OLMo's are 96 × 192, one each. The changes that keep every read exact:
|
| 52 |
|
| 53 |
- A state row's ballast sits in 3 of its 192 columns (Qwen: 2 of 128), so the gated output norm divides by the same power of 2.
|
|
|
|
| 62 |
- the residual stream after the chip layers equals the circuit's IR evaluator on every dimension at every position (bf16; traces up to 60k tokens);
|
| 63 |
- free-running greedy `generate` from the program's start reproduces the reference trace token for token.
|
| 64 |
|
| 65 |
+

|
| 66 |
+
|
| 67 |
+
| Run | Kernels | Programs | Exact | Trace tokens |
|
| 68 |
+
|---|---|---|---|---|
|
| 69 |
+
| c-sieve-gmpmul | torch | 2 | 2 | 66,825 |
|
| 70 |
+
| short-fla | fla | 55 | 55 | 26,677 |
|
| 71 |
+
| short-torch | torch | 55 | 55 | 26,677 |
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
A run, from an ELF to the HALT token:
|
| 75 |
+
|
| 76 |
+
```mermaid
|
| 77 |
+
flowchart LR
|
| 78 |
+
ELF["program.elf"] --> SNAP["snapshot: code ROM, data ROM, RAM, registers, PC"]
|
| 79 |
+
SNAP -->|"written into the Gated DeltaNet states"| M["olmo-tiny"]
|
| 80 |
+
GO["GO"] --> M
|
| 81 |
+
M -->|"greedy: the next trace token"| T["trace token"]
|
| 82 |
+
T -->|"fed back in"| M
|
| 83 |
+
T -->|"HALT"| END["done; putchar output is in the BYTE tokens"]
|
| 84 |
+
```
|
| 85 |
|
| 86 |
## Usage
|
| 87 |
|
| 88 |
```bash
|
| 89 |
+
pip install "chipformer @ https://github.com/JGalego/olmputer/archive/ff1f044bbd7aab21aa2e0da0dfe0a90fcf170be4.tar.gz#subdirectory=experimental/riscv" torch==2.14.1 transformers==5.19.0
|
| 90 |
```
|
| 91 |
|
| 92 |
```python
|
assets/checks.png
ADDED
|
assets/layers.png
ADDED
|
assets/logo.svg
ADDED
|
|