Text Generation
Transformers
Safetensors
olmo_hybrid
risc-v
compiled-weights
olmo-hybrid
gated-deltanet
tool-use
conversational
Instructions to use jgalego/olmputer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use jgalego/olmputer with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="jgalego/olmputer") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("jgalego/olmputer") model = AutoModelForCausalLM.from_pretrained("jgalego/olmputer", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use jgalego/olmputer with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "jgalego/olmputer" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jgalego/olmputer", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/jgalego/olmputer
- SGLang
How to use jgalego/olmputer with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "jgalego/olmputer" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jgalego/olmputer", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "jgalego/olmputer" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jgalego/olmputer", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use jgalego/olmputer with Docker Model Runner:
docker model run hf.co/jgalego/olmputer
Upload folder using huggingface_hub
Browse files- README.md +2 -26
- assets/diagram-call.png +0 -0
- assets/diagram-layers.png +0 -0
README.md
CHANGED
|
@@ -33,20 +33,7 @@ It ports [Qwenputer](https://github.com/divagr18/qwenputer) (the same machine in
|
|
| 33 |
|
| 34 |
### A calc call
|
| 35 |
|
| 36 |
-
|
| 37 |
-
sequenceDiagram
|
| 38 |
-
participant U as User
|
| 39 |
-
participant O as OLMo's 32 layers
|
| 40 |
-
participant C as The chip layers (the CPU)
|
| 41 |
-
U->>O: What is 123456789 * 987654321?
|
| 42 |
-
O->>C: function_calls opens: calc(expression='123456789*987654321')
|
| 43 |
-
Note over C: every token of the call is copied into the text buffer the firmware reads
|
| 44 |
-
O->>C: function_calls closes
|
| 45 |
-
Note over C: the CPU runs the call: 2,717 trace tokens through the chip layers only,<br/>which OLMo's layers, caches and positions never see
|
| 46 |
-
C->>O: im_end, im_start, environment, 121 932 631 112 635 269, im_end, im_start, assistant
|
| 47 |
-
Note over O: each echoed token is text: it runs through every layer
|
| 48 |
-
O->>U: The result of 123456789 multiplied by 987654321 is 121932631112635269.
|
| 49 |
-
```
|
| 50 |
|
| 51 |
|
| 52 |
|
|
@@ -54,18 +41,7 @@ sequenceDiagram
|
|
| 54 |
|
| 55 |
Each trace token runs once through the 11 chip layers, and each layer does one part of executing the instruction:
|
| 56 |
|
| 57 |
-
|
| 58 |
-
flowchart TB
|
| 59 |
-
IN["a trace token in"] --> L1["L1: the PC anchor and counters (record position, commits since the last jump)"]
|
| 60 |
-
L1 --> L2["L2: the PC word and the link address"]
|
| 61 |
-
L2 --> L3["L3: fetch: the code ROM, 23 heads of 2,944 words each"]
|
| 62 |
-
L3 --> L4["L4: decode; read 3 register ports"]
|
| 63 |
-
L4 --> L5["L5: the bignum digit file; adder and multiplier stages"]
|
| 64 |
-
L5 --> L6["L6 to L8: the RAM, 86 heads of 96 words; loads, stores, compares"]
|
| 65 |
-
L6 --> L9["L9: the data ROM; divider digits"]
|
| 66 |
-
L9 --> L10["L10 to L11: pick the result and write the next token's code"]
|
| 67 |
-
L10 --> OUT["the next trace token"]
|
| 68 |
-
```
|
| 69 |
|
| 70 |

|
| 71 |
|
|
|
|
| 33 |
|
| 34 |
### A calc call
|
| 35 |
|
| 36 |
+

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
|
| 39 |
|
|
|
|
| 41 |
|
| 42 |
Each trace token runs once through the 11 chip layers, and each layer does one part of executing the instruction:
|
| 43 |
|
| 44 |
+

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
|
| 46 |

|
| 47 |
|
assets/diagram-call.png
ADDED
|
assets/diagram-layers.png
ADDED
|