Spaces:

Genrius-AI
/

README

Running

App Files Files Community

README / README.md

harshit-genrius

Org card: results chart, 5-model table, GitHub star button

ddd51bb verified about 9 hours ago

preview code

Raw History Blame Contribute Delete

3.51 kB

---
title: README
colorFrom: yellow
colorTo: gray
sdk: static
pinned: false
---
# Genrius AI

**Generating Real Intelligence.** We build AI models and make them small and fast enough to run on hardware people already own.

Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus.

**Try it:** [Will it fit your GPU? (calculator)](https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator) | [One-click build, Windows and Linux, NVIDIA RTX 30/40/50](https://github.com/Genrius-AI/pingala8-llm/releases/latest) | <a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer"><img src="https://img.shields.io/github/stars/Genrius-AI/pingala8-llm?style=social" alt="Star Genrius-AI/pingala8-llm on GitHub" style="height:20px;width:auto;margin:0;vertical-align:middle;border-radius:0"></a>

**Results across 5 models**

![How much worse than 8-bit, at 2-bit size: Pingala-8 adds 23-66% perplexity over the 8-bit model, IQ2_XXS adds 65-443%, at the same file size](https://huggingface.co/spaces/Genrius-AI/README/resolve/main/results_summary.png)

| Model | Size | Perplexity: Pingala-8 vs IQ2_XXS | IQ2_XXS / Pingala-8 |
|---|---|---|---|
| [Qwen3.5-0.8B](https://huggingface.co/Genrius-AI/Qwen3.5-0.8B-Pingala8-2bit) | 0.35 GB | **25.97** vs 85.09 | 3.28x lower |
| [Qwen3.5-2B](https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit) | 0.78 GB | **15.62** vs 29.08 | 1.86x lower |
| [Qwen3.5-4B](https://huggingface.co/Genrius-AI/Qwen3.5-4B-Pingala8-2bit) | 1.51 GB | **10.97** vs 14.75 | 1.34x lower |
| [Qwen3-4B](https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit) | 1.24 GB | **16.64** vs 28.61 | 1.72x lower |
| [Llama-3.1-8B](https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit) | 2.39 GB | **9.14** vs 12.35 | 1.35x lower |

Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the [v2.5 one-click build](https://github.com/Genrius-AI/pingala8-llm/releases/latest) (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.

## Pingala-8: 2-bit LLMs for llama.cpp, no retraining

- **85% smaller:** Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.
- **Better at the same size:** at the file size of llama.cpp's IQ2_XXS, our Qwen3-4B reaches WikiText-2 perplexity 16.64. IQ2_XXS reaches 28.61 and the best IQ2_KT file we built reaches 18.15 (lower is better).
- **Zero-shot (Qwen3-4B, 5-task average):** we score 50.83, against 41.86 for IQ2_XXS. The 8-bit original scores 56.3.
- **Where we lose:** Qwen3-14B (IQ2_KT 9.40, ours 9.57), prompt processing (0.74x of IQ2_XXS), and retrained models such as Ternary Bonsai 4B.

![Pingala-8 Qwen3-4B (1.24 GB) answering in a local chat window on an RTX 5070 Ti Laptop GPU](https://huggingface.co/spaces/Genrius-AI/README/resolve/main/pingala8-poster.jpg)

- **Models:** [Qwen3-4B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit) and [Llama-3.1-8B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit)
- **Benchmarks:** [pingala8-benchmarks](https://huggingface.co/datasets/Genrius-AI/pingala8-benchmarks)
- **One-click build (Windows and Linux, all five models):** [v2.5](https://github.com/Genrius-AI/pingala8-llm/releases/latest)
- **Code:** [Genrius-AI/pingala8-llm](https://github.com/Genrius-AI/pingala8-llm)
- **Paper:** [doi.org/10.5281/zenodo.23481757](https://doi.org/10.5281/zenodo.23481757)

	---
	title: README
	colorFrom: yellow
	colorTo: gray
	sdk: static
	pinned: false
	---

	# Genrius AI

	Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.

	Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus.

	Try it: [Will it fit your GPU? (calculator)](https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator) \| [One-click build, Windows and Linux, NVIDIA RTX 30/40/50](https://github.com/Genrius-AI/pingala8-llm/releases/latest) \| <a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer"><img src="https://img.shields.io/github/stars/Genrius-AI/pingala8-llm?style=social" alt="Star Genrius-AI/pingala8-llm on GitHub" style="height:20px;width:auto;margin:0;vertical-align:middle;border-radius:0"></a>

	Results across 5 models

	![How much worse than 8-bit, at 2-bit size: Pingala-8 adds 23-66% perplexity over the 8-bit model, IQ2_XXS adds 65-443%, at the same file size](https://huggingface.co/spaces/Genrius-AI/README/resolve/main/results_summary.png)

	\| Model \| Size \| Perplexity: Pingala-8 vs IQ2_XXS \| IQ2_XXS / Pingala-8 \|
	\|---\|---\|---\|---\|
	\| [Qwen3.5-0.8B](https://huggingface.co/Genrius-AI/Qwen3.5-0.8B-Pingala8-2bit) \| 0.35 GB \| 25.97 vs 85.09 \| 3.28x lower \|
	\| [Qwen3.5-2B](https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit) \| 0.78 GB \| 15.62 vs 29.08 \| 1.86x lower \|
	\| [Qwen3.5-4B](https://huggingface.co/Genrius-AI/Qwen3.5-4B-Pingala8-2bit) \| 1.51 GB \| 10.97 vs 14.75 \| 1.34x lower \|
	\| [Qwen3-4B](https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit) \| 1.24 GB \| 16.64 vs 28.61 \| 1.72x lower \|
	\| [Llama-3.1-8B](https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit) \| 2.39 GB \| 9.14 vs 12.35 \| 1.35x lower \|

	Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the [v2.5 one-click build](https://github.com/Genrius-AI/pingala8-llm/releases/latest) (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.

	## Pingala-8: 2-bit LLMs for llama.cpp, no retraining

	- 85% smaller: Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.
	- Better at the same size: at the file size of llama.cpp's IQ2_XXS, our Qwen3-4B reaches WikiText-2 perplexity 16.64. IQ2_XXS reaches 28.61 and the best IQ2_KT file we built reaches 18.15 (lower is better).
	- Zero-shot (Qwen3-4B, 5-task average): we score 50.83, against 41.86 for IQ2_XXS. The 8-bit original scores 56.3.
	- Where we lose: Qwen3-14B (IQ2_KT 9.40, ours 9.57), prompt processing (0.74x of IQ2_XXS), and retrained models such as Ternary Bonsai 4B.

	![Pingala-8 Qwen3-4B (1.24 GB) answering in a local chat window on an RTX 5070 Ti Laptop GPU](https://huggingface.co/spaces/Genrius-AI/README/resolve/main/pingala8-poster.jpg)

	- Models: [Qwen3-4B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit) and [Llama-3.1-8B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit)
	- Benchmarks: [pingala8-benchmarks](https://huggingface.co/datasets/Genrius-AI/pingala8-benchmarks)
	- One-click build (Windows and Linux, all five models): [v2.5](https://github.com/Genrius-AI/pingala8-llm/releases/latest)
	- Code: [Genrius-AI/pingala8-llm](https://github.com/Genrius-AI/pingala8-llm)
	- Paper: [doi.org/10.5281/zenodo.23481757](https://doi.org/10.5281/zenodo.23481757)