Instructions to use syaffers/OtakMini with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use syaffers/OtakMini with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="syaffers/OtakMini")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("syaffers/OtakMini") model = AutoModelForCausalLM.from_pretrained("syaffers/OtakMini", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use syaffers/OtakMini with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "syaffers/OtakMini" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "syaffers/OtakMini", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/syaffers/OtakMini
- SGLang
How to use syaffers/OtakMini with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "syaffers/OtakMini" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "syaffers/OtakMini", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "syaffers/OtakMini" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "syaffers/OtakMini", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use syaffers/OtakMini with Docker Model Runner:
docker model run hf.co/syaffers/OtakMini
|
Download README.md from syaffers/OtakMini: direct link, hf CLI and curl.
- Browser
- Download file 4.95 kB
-
https://huggingface.co/syaffers/OtakMini/resolve/main/README.md
- Command line
-
hf download hf://syaffers/OtakMini/README.md
-
curl -L -o README.md https://huggingface.co/syaffers/OtakMini/resolve/main/README.md
4.95 kB
| license: cc-by-nc-4.0 | |
| language: | |
| - en | |
| library_name: transformers | |
| pipeline_tag: text-generation | |
| datasets: | |
| - karpathy/climbmix-400b-shuffle | |
| - openbmb/Ultra-FineWeb | |
| - openbmb/Ultra-FineWeb-L3 | |
| - HuggingFaceFW/fineweb-edu | |
| - HuggingFaceFW/finephrase | |
| - HuggingFaceTB/smollm-corpus | |
| - nvidia/Nemotron-CC-Math-v1 | |
| - bigcode/starcoderdata | |
| # OtakMini | |
| OtakMini is a 135M-parameter English language model pretrained **from scratch** on a single consumer GPU (RTX 5050, 8 GB) in about 12 days, on 18B tokens. This is a **base model**: no instruction tuning, RLHF or safety tuning. It uses the Qwen3 architecture (grouped-query attention, QK-norm, SwiGLU, tied embeddings) so it loads with stock `transformers`. | |
| ## Model details | |
| | | | | |
| |---|---| | |
| | Parameters | 135.03M (18.9M of them in the tied embedding) | | |
| | Layers / width | 30 layers, hidden size 576, FFN 1728 | | |
| | Attention | 9 heads, 3 KV heads, head dim 64, RoPE (theta 1e5) | | |
| | Context length | 2048 tokens | | |
| | Vocabulary | 32,768 (custom byte-level BPE, digits split) | | |
| | Special tokens | `<\|endoftext\|>` (id 0: BOS/EOS/PAD), `<\|im_start\|>` (1), `<\|im_end\|>` (2, reserved for chat tuning, untrained here) | | |
| | Precision | bf16 weights (`model.safetensors`, 270 MB) | | |
| ## Quick start | |
| ```python | |
| import torch | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| repo = "syaffers/OtakMini" | |
| tok = AutoTokenizer.from_pretrained(repo) | |
| model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16) | |
| ids = tok("The capital of France is", return_tensors="pt") | |
| out = model.generate(**ids) # uses the sampling defaults in generation_config.json | |
| print(tok.decode(out[0])) | |
| ``` | |
| `generation_config.json` ships with `temperature=0.7`, `top_p=0.9`, `repetition_penalty=1.1`. | |
| ## Evaluation | |
| Zero-shot, full test sets, bf16, lm-evaluation-harness 0.4.13. `acc_norm` where available (HellaSwag, ARC, PIQA), otherwise `acc`. | |
| | Task | Score | ±stderr | Chance | | |
| |---|---|---|---| | |
| | HellaSwag (acc_norm) | 39.9 | 0.5 | 25.0 | | |
| | ARC-Easy (acc_norm) | 52.2 | 1.0 | 25.0 | | |
| | ARC-Challenge (acc_norm) | 29.4 | 1.3 | 25.0 | | |
| | PIQA (acc_norm) | 68.0 | 1.1 | 50.0 | | |
| | Winogrande (acc) | 51.8 | 1.4 | 50.0 | | |
| | LAMBADA (acc) | 33.9 | 0.7 | | | |
| | LAMBADA (perplexity) | 38.2 | 1.5 | | | |
| Validation loss on held-out web text is 2.606 (perplexity about 13.5). | |
| ## Training | |
| - **Data:** about 19B tokens of English text, 11 sources, one pass over the pool (about 0.95 epochs). Mix for most of training, with the mix for the last 10% in brackets: | |
| | Source | Share | | |
| |---|---| | |
| | ClimbMix | 30% (20%) | | |
| | Ultra-FineWeb | 20% (15%) | | |
| | Nemotron-CC-Math-v1 (4plus) | 10% (15%) | | |
| | Cosmopedia v2 | 10% (12%) | | |
| | FineWeb-Edu | 7% (10%) | | |
| | FinePhrase (FAQ / tutorial / math / table rewrites) | 15% (20%) | | |
| | Ultra-FineWeb-L3 (multi-style) | 5% (5%) | | |
| | StarCoderData (Python) | 3% (3%) | | |
| - **Decontamination:** documents with a 13-gram (word-level) overlap with HellaSwag, ARC, PIQA, Winogrande or LAMBADA were removed: 10,937 of 28.35M documents (0.04%). | |
| - **Optimiser:** Muon (lr 0.01, weight decay 0.01) on 2D weight matrices, AdamW (lr 1.5e-3) on embeddings and norms; batch 524,288 tokens; 34,332 steps. | |
| - **Schedule:** 2,000 warmup steps, constant lr, then a 1-sqrt cooldown over the last 10% of steps with the anneal data mix. | |
| - **Hardware:** one RTX 5050 (8 GB) at about 17k tokens/s, 2026-10-16 to 2026-10-28. | |
| ### Loss curves | |
|  | |
| Train loss (3-hour means) and validation loss over the run. The drop at the right is the cooldown with the anneal data mix. The hours axis is approximate. | |
| ## Intended use and limitations | |
| Meant for research, experiments with small models, and as a starting point for fine-tuning. It is not meant for anything where correctness matters. | |
| - It writes fluent text but **gets facts wrong** (for example, it will say Paris is the largest city in the world, or invent populations and dates), and it cannot do arithmetic reliably. | |
| - Without a repetition penalty it **loops**. Text drifts off-topic after a sentence or two. | |
| - English only, 2048-token context, no instruction following and no refusal behaviour. It can reproduce biases and unsafe content from web data. | |
| - The ChatML tokens exist in the tokenizer but the model has never seen them in use. A chat template is not included. | |
| ## License and data | |
| Released under **CC BY-NC 4.0 (non-commercial)**, because the training data includes non-commercial and agreement-restricted sources (ClimbMix is CC BY-NC 4.0; NVIDIA's Nemotron-CC-Math data agreement and the StarCoderData terms of use also apply). Check each dataset's own license before reusing the weights beyond research. | |
| ## Citation | |
| ```bibtex | |
| @misc{otakmini2026, | |
| title = {OtakMini: a 135M-parameter language model pretrained from scratch on a single consumer GPU}, | |
| author = {syaffers}, | |
| year = {2026}, | |
| url = {https://huggingface.co/syaffers/OtakMini} | |
| } | |
| ``` | |