Genrius AI
Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.
Try it: Will it fit your GPU? (calculator) | One-click build, Windows and Linux, NVIDIA RTX 30/40/50 |
Results across 5 models
| Model | Size | Perplexity: Pingala-8 vs IQ2_XXS | IQ2_XXS / Pingala-8 |
|---|---|---|---|
| Qwen3.5-0.8B | 0.35 GB | 25.97 vs 85.09 | 3.28x lower |
| Qwen3.5-2B | 0.78 GB | 15.62 vs 29.08 | 1.86x lower |
| Qwen3.5-4B | 1.51 GB | 10.97 vs 14.75 | 1.34x lower |
| Qwen3-4B | 1.24 GB | 16.64 vs 28.61 | 1.72x lower |
| Llama-3.1-8B | 2.39 GB | 9.14 vs 12.35 | 1.35x lower |
Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the v2.5 one-click build (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.
Pingala-8: 2-bit LLMs for llama.cpp, no retraining
- 85% smaller: Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.
- At the file size of llama.cpp's IQ2_XXS, our Qwen3-4B reaches WikiText-2 perplexity 16.64. IQ2_XXS reaches 28.61 and the best IQ2_KT file we built reaches 18.15 (lower is better).
- Where we lose: Qwen3-14B (IQ2_KT 9.40, ours 9.57), prompt processing (0.74x of IQ2_XXS), and retrained models such as Ternary Bonsai 4B.
Links
- VRAM calculator: normal vs Pingala-8
- One-click build (Windows and Linux, all five models): v2.5
- Code
- Paper
- Qwen3-4B-Pingala8-2bit, Llama-3.1-8B-Pingala8-2bit and Qwen3.5-4B-Pingala8-2bit (text only); Qwen3.5-2B-Pingala8-2bit and Qwen3.5-0.8B-Pingala8-2bit (text only; use the v2.5 one-click build)
- Benchmark tables