Hugging Face's logo

Spaces:
Genrius-AI
/
README
Running

App Files Community
README / index.html
harshit-genrius's picture
harshit-genrius
Org card: results chart, 5-model table, GitHub star button
ddd51bb verified
Raw History Blame Contribute Delete
6.25 kB
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Genrius AI</title>
<style>
body { margin: 0; background: #0b0b0c; color: #e8e6e1; font-family: system-ui, -apple-system, Segoe UI, Roboto, sans-serif; line-height: 1.55; }
main { max-width: 760px; margin: 0 auto; padding: 40px 16px; }
h1 { color: #D4AF37; margin: 0 0 4px; font-size: 2rem; }
.lead { color: #b9b4a8; margin: 0 0 28px; }
h2 { color: #D4AF37; font-size: 1.15rem; margin-top: 28px; }
a { color: #D4AF37; }
img { width: 100%; height: auto; border-radius: 8px; margin: 16px 0; }
li { margin: 6px 0; }
table { border-collapse: collapse; width: 100%; margin: 8px 0; font-size: 0.95rem; }
th, td { border: 1px solid #333; padding: 6px 8px; text-align: left; }
</style>
</head>
<body>
<main>
<h1>Genrius AI</h1>
<p class="lead">Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.</p>
<p><strong>Try it:</strong> <a href="https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator" target="_blank" rel="noopener noreferrer">Will it fit your GPU? (calculator)</a> | <a href="https://github.com/Genrius-AI/pingala8-llm/releases/latest" target="_blank" rel="noopener noreferrer">One-click build, Windows and Linux, NVIDIA RTX 30/40/50</a> | <a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer"><img src="https://img.shields.io/github/stars/Genrius-AI/pingala8-llm?style=social" alt="Star Genrius-AI/pingala8-llm on GitHub" style="height:20px;width:auto;margin:0;vertical-align:middle;border-radius:0"></a></p>
<h2 style="margin-top:8px">Results across 5 models</h2>
<a href="https://huggingface.co/spaces/Genrius-AI/README/resolve/main/results_summary_dark.png" target="_blank" rel="noopener noreferrer"><img src="https://huggingface.co/spaces/Genrius-AI/README/resolve/main/results_summary_dark.png" width="1200" height="630" alt="How much worse than 8-bit, at 2-bit size: Pingala-8 adds 23-66% perplexity over the 8-bit model, IQ2_XXS adds 65-443%, at the same file size"></a>
<table>
<tr><th>Model</th><th>Size</th><th>Perplexity: Pingala-8 vs IQ2_XXS</th><th>IQ2_XXS / Pingala-8</th></tr>
<tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-0.8B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-0.8B</a></td><td>0.35 GB</td><td><strong>25.97</strong> vs 85.09</td><td>3.28x lower</td></tr>
<tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-2B</a></td><td>0.78 GB</td><td><strong>15.62</strong> vs 29.08</td><td>1.86x lower</td></tr>
<tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-4B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-4B</a></td><td>1.51 GB</td><td><strong>10.97</strong> vs 14.75</td><td>1.34x lower</td></tr>
<tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3-4B</a></td><td>1.24 GB</td><td><strong>16.64</strong> vs 28.61</td><td>1.72x lower</td></tr>
<tr><td><a href="https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Llama-3.1-8B</a></td><td>2.39 GB</td><td><strong>9.14</strong> vs 12.35</td><td>1.35x lower</td></tr>
</table>
<p class="lead">Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the <a href="https://github.com/Genrius-AI/pingala8-llm/releases/latest" target="_blank" rel="noopener noreferrer">v2.5 one-click build</a> (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.</p>
<h2>Pingala-8: 2-bit LLMs for llama.cpp, no retraining</h2>
<ul>
<li>85% smaller: Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.</li>
<li>At the file size of llama.cpp's IQ2_XXS, our Qwen3-4B reaches WikiText-2 perplexity 16.64. IQ2_XXS reaches 28.61 and the best IQ2_KT file we built reaches 18.15 (lower is better).</li>
<li>Where we lose: Qwen3-14B (IQ2_KT 9.40, ours 9.57), prompt processing (0.74x of IQ2_XXS), and retrained models such as Ternary Bonsai 4B.</li>
</ul>
<img src="https://huggingface.co/spaces/Genrius-AI/README/resolve/main/pingala8-poster.jpg" alt="Pingala-8 Qwen3-4B (1.24 GB) answering in a local chat window on an RTX 5070 Ti Laptop GPU">
<h2>Links</h2>
<ul>
<li><a href="https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator" target="_blank" rel="noopener noreferrer">VRAM calculator: normal vs Pingala-8</a></li>
<li><a href="https://github.com/Genrius-AI/pingala8-llm/releases/latest" target="_blank" rel="noopener noreferrer">One-click build (Windows and Linux, all five models): v2.5</a></li>
<li><a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer">Code</a></li>
<li><a href="https://doi.org/10.5281/zenodo.23481757" target="_blank" rel="noopener noreferrer">Paper</a></li>
<li><a href="https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3-4B-Pingala8-2bit</a>, <a href="https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Llama-3.1-8B-Pingala8-2bit</a> and <a href="https://huggingface.co/Genrius-AI/Qwen3.5-4B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-4B-Pingala8-2bit</a> (text only); <a href="https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-2B-Pingala8-2bit</a> and <a href="https://huggingface.co/Genrius-AI/Qwen3.5-0.8B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-0.8B-Pingala8-2bit</a> (text only; use the v2.5 one-click build)</li>
<li><a href="https://huggingface.co/datasets/Genrius-AI/pingala8-benchmarks" target="_blank" rel="noopener noreferrer">Benchmark tables</a></li>
</ul>
</main>
<script>document.addEventListener("click",function(e){var a=e.target.closest&&e.target.closest('a[href^="http"]');if(a){a.target="_blank";a.rel="noopener noreferrer";}},true);</script>
</body>
</html>