README.md

---
title: README
colorFrom: yellow
colorTo: gray
sdk: static
pinned: false
---

# Genrius AI

**Generating Real Intelligence.** We build AI models and make them small and fast enough to run on hardware people already own.

Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus.

**Try it:** [Will it fit your GPU? (calculator)](https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator) | [One-click build, Windows and Linux, NVIDIA RTX 30/40/50](https://github.com/Genrius-AI/pingala8-llm/releases/latest) | <a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer"><img src="https://img.shields.io/github/stars/Genrius-AI/pingala8-llm?style=social" alt="Star Genrius-AI/pingala8-llm on GitHub" style="height:20px;width:auto;margin:0;vertical-align:middle;border-radius:0"></a>

**Results across 5 models**

![How much worse than 8-bit, at 2-bit size: Pingala-8 adds 23-66% perplexity over the 8-bit model, IQ2_XXS adds 65-443%, at the same file size](https://huggingface.co/spaces/Genrius-AI/README/resolve/main/results_summary.png)

| Model | Size | Perplexity: Pingala-8 vs IQ2_XXS | IQ2_XXS / Pingala-8 |
|---|---|---|---|
| [Qwen3.5-0.8B](https://huggingface.co/Genrius-AI/Qwen3.5-0.8B-Pingala8-2bit) | 0.35 GB | **25.97** vs 85.09 | 3.28x lower |
| [Qwen3.5-2B](https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit) | 0.78 GB | **15.62** vs 29.08 | 1.86x lower |
| [Qwen3.5-4B](https://huggingface.co/Genrius-AI/Qwen3.5-4B-Pingala8-2bit) | 1.51 GB | **10.97** vs 14.75 | 1.34x lower |
| [Qwen3-4B](https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit) | 1.24 GB | **16.64** vs 28.61 | 1.72x lower |
| [Llama-3.1-8B](https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit) | 2.39 GB | **9.14** vs 12.35 | 1.35x lower |

Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the [v2.5 one-click build](https://github.com/Genrius-AI/pingala8-llm/releases/latest) (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.

## Pingala-8: 2-bit LLMs for llama.cpp, no retraining

- **85% smaller:** Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.
- **Better at the same size:** at the file size of llama.cpp's IQ2_XXS, our Qwen3-4B reaches WikiText-2 perplexity 16.64. IQ2_XXS reaches 28.61 and the best IQ2_KT file we built reaches 18.15 (lower is better).
- **Zero-shot (Qwen3-4B, 5-task average):** we score 50.83, against 41.86 for IQ2_XXS. The 8-bit original scores 56.3.
- **Where we lose:** Qwen3-14B (IQ2_KT 9.40, ours 9.57), prompt processing (0.74x of IQ2_XXS), and retrained models such as Ternary Bonsai 4B.

![Pingala-8 Qwen3-4B (1.24 GB) answering in a local chat window on an RTX 5070 Ti Laptop GPU](https://huggingface.co/spaces/Genrius-AI/README/resolve/main/pingala8-poster.jpg)

- **Models:** [Qwen3-4B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit) and [Llama-3.1-8B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit)
- **Benchmarks:** [pingala8-benchmarks](https://huggingface.co/datasets/Genrius-AI/pingala8-benchmarks)
- **One-click build (Windows and Linux, all five models):** [v2.5](https://github.com/Genrius-AI/pingala8-llm/releases/latest)
- **Code:** [Genrius-AI/pingala8-llm](https://github.com/Genrius-AI/pingala8-llm)
- **Paper:** [doi.org/10.5281/zenodo.23481757](https://doi.org/10.5281/zenodo.23481757)