Spaces:
Running
Running
metadata
title: README
colorFrom: yellow
colorTo: gray
sdk: static
pinned: false
Genrius AI
Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.
Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus.
Try it: Will it fit your GPU? (calculator) | One-click build, Windows and Linux, NVIDIA RTX 30/40/50 |
Results across 5 models
| Model | Size | Perplexity: Pingala-8 vs IQ2_XXS | IQ2_XXS / Pingala-8 |
|---|---|---|---|
| Qwen3.5-0.8B | 0.35 GB | 25.97 vs 85.09 | 3.28x lower |
| Qwen3.5-2B | 0.78 GB | 15.62 vs 29.08 | 1.86x lower |
| Qwen3.5-4B | 1.51 GB | 10.97 vs 14.75 | 1.34x lower |
| Qwen3-4B | 1.24 GB | 16.64 vs 28.61 | 1.72x lower |
| Llama-3.1-8B | 2.39 GB | 9.14 vs 12.35 | 1.35x lower |
Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the v2.5 one-click build (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.
Pingala-8: 2-bit LLMs for llama.cpp, no retraining
- 85% smaller: Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.
- Better at the same size: at the file size of llama.cpp's IQ2_XXS, our Qwen3-4B reaches WikiText-2 perplexity 16.64. IQ2_XXS reaches 28.61 and the best IQ2_KT file we built reaches 18.15 (lower is better).
- Zero-shot (Qwen3-4B, 5-task average): we score 50.83, against 41.86 for IQ2_XXS. The 8-bit original scores 56.3.
- Where we lose: Qwen3-14B (IQ2_KT 9.40, ours 9.57), prompt processing (0.74x of IQ2_XXS), and retrained models such as Ternary Bonsai 4B.
- Models: Qwen3-4B-Pingala8-2bit and Llama-3.1-8B-Pingala8-2bit
- Benchmarks: pingala8-benchmarks
- One-click build (Windows and Linux, all five models): v2.5
- Code: Genrius-AI/pingala8-llm
- Paper: doi.org/10.5281/zenodo.23481757