Spaces:
Running
Running
Org card: results chart, 5-model table, GitHub star button
Browse files- .gitattributes +2 -0
- README.md +11 -7
- index.html +10 -8
- results_summary.png +3 -0
- results_summary.svg +0 -0
- results_summary_dark.png +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
results_summary.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
results_summary_dark.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -12,15 +12,19 @@ pinned: false
|
|
| 12 |
|
| 13 |
Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus.
|
| 14 |
|
| 15 |
-
**Try it:** [Will it fit your GPU? (calculator)](https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator) | [One-click build, Windows and Linux, NVIDIA RTX 30/40/50](https://github.com/Genrius-AI/pingala8-llm/releases/latest)
|
| 16 |
|
| 17 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|---|---|---|---|
|
| 19 |
-
| [Qwen3-
|
| 20 |
-
| [
|
| 21 |
-
| [Qwen3.5-4B
|
| 22 |
-
| [Qwen3
|
| 23 |
-
| [
|
| 24 |
|
| 25 |
Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the [v2.5 one-click build](https://github.com/Genrius-AI/pingala8-llm/releases/latest) (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.
|
| 26 |
|
|
|
|
| 12 |
|
| 13 |
Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus.
|
| 14 |
|
| 15 |
+
**Try it:** [Will it fit your GPU? (calculator)](https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator) | [One-click build, Windows and Linux, NVIDIA RTX 30/40/50](https://github.com/Genrius-AI/pingala8-llm/releases/latest) | <a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer"><img src="https://img.shields.io/github/stars/Genrius-AI/pingala8-llm?style=social" alt="Star Genrius-AI/pingala8-llm on GitHub" style="height:20px;width:auto;margin:0;vertical-align:middle;border-radius:0"></a>
|
| 16 |
|
| 17 |
+
**Results across 5 models**
|
| 18 |
+
|
| 19 |
+

|
| 20 |
+
|
| 21 |
+
| Model | Size | Perplexity: Pingala-8 vs IQ2_XXS | IQ2_XXS / Pingala-8 |
|
| 22 |
|---|---|---|---|
|
| 23 |
+
| [Qwen3.5-0.8B](https://huggingface.co/Genrius-AI/Qwen3.5-0.8B-Pingala8-2bit) | 0.35 GB | **25.97** vs 85.09 | 3.28x lower |
|
| 24 |
+
| [Qwen3.5-2B](https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit) | 0.78 GB | **15.62** vs 29.08 | 1.86x lower |
|
| 25 |
+
| [Qwen3.5-4B](https://huggingface.co/Genrius-AI/Qwen3.5-4B-Pingala8-2bit) | 1.51 GB | **10.97** vs 14.75 | 1.34x lower |
|
| 26 |
+
| [Qwen3-4B](https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit) | 1.24 GB | **16.64** vs 28.61 | 1.72x lower |
|
| 27 |
+
| [Llama-3.1-8B](https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit) | 2.39 GB | **9.14** vs 12.35 | 1.35x lower |
|
| 28 |
|
| 29 |
Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the [v2.5 one-click build](https://github.com/Genrius-AI/pingala8-llm/releases/latest) (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.
|
| 30 |
|
index.html
CHANGED
|
@@ -21,15 +21,17 @@
|
|
| 21 |
<main>
|
| 22 |
<h1>Genrius AI</h1>
|
| 23 |
<p class="lead">Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.</p>
|
| 24 |
-
<p><strong>Try it:</strong> <a href="https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator" target="_blank" rel="noopener noreferrer">Will it fit your GPU? (calculator)</a> | <a href="https://github.com/Genrius-AI/pingala8-llm/releases/latest" target="_blank" rel="noopener noreferrer">One-click build, Windows and Linux, NVIDIA RTX 30/40/50</a></p>
|
|
|
|
|
|
|
| 25 |
<table>
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
<p class="lead">Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the <a href="https://github.com/Genrius-AI/pingala8-llm/releases/latest" target="_blank" rel="noopener noreferrer">v2.5 one-click build</a> (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.</p>
|
| 34 |
<h2>Pingala-8: 2-bit LLMs for llama.cpp, no retraining</h2>
|
| 35 |
<ul>
|
|
|
|
| 21 |
<main>
|
| 22 |
<h1>Genrius AI</h1>
|
| 23 |
<p class="lead">Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.</p>
|
| 24 |
+
<p><strong>Try it:</strong> <a href="https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator" target="_blank" rel="noopener noreferrer">Will it fit your GPU? (calculator)</a> | <a href="https://github.com/Genrius-AI/pingala8-llm/releases/latest" target="_blank" rel="noopener noreferrer">One-click build, Windows and Linux, NVIDIA RTX 30/40/50</a> | <a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer"><img src="https://img.shields.io/github/stars/Genrius-AI/pingala8-llm?style=social" alt="Star Genrius-AI/pingala8-llm on GitHub" style="height:20px;width:auto;margin:0;vertical-align:middle;border-radius:0"></a></p>
|
| 25 |
+
<h2 style="margin-top:8px">Results across 5 models</h2>
|
| 26 |
+
<a href="https://huggingface.co/spaces/Genrius-AI/README/resolve/main/results_summary_dark.png" target="_blank" rel="noopener noreferrer"><img src="https://huggingface.co/spaces/Genrius-AI/README/resolve/main/results_summary_dark.png" width="1200" height="630" alt="How much worse than 8-bit, at 2-bit size: Pingala-8 adds 23-66% perplexity over the 8-bit model, IQ2_XXS adds 65-443%, at the same file size"></a>
|
| 27 |
<table>
|
| 28 |
+
<tr><th>Model</th><th>Size</th><th>Perplexity: Pingala-8 vs IQ2_XXS</th><th>IQ2_XXS / Pingala-8</th></tr>
|
| 29 |
+
<tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-0.8B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-0.8B</a></td><td>0.35 GB</td><td><strong>25.97</strong> vs 85.09</td><td>3.28x lower</td></tr>
|
| 30 |
+
<tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-2B</a></td><td>0.78 GB</td><td><strong>15.62</strong> vs 29.08</td><td>1.86x lower</td></tr>
|
| 31 |
+
<tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-4B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-4B</a></td><td>1.51 GB</td><td><strong>10.97</strong> vs 14.75</td><td>1.34x lower</td></tr>
|
| 32 |
+
<tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3-4B</a></td><td>1.24 GB</td><td><strong>16.64</strong> vs 28.61</td><td>1.72x lower</td></tr>
|
| 33 |
+
<tr><td><a href="https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Llama-3.1-8B</a></td><td>2.39 GB</td><td><strong>9.14</strong> vs 12.35</td><td>1.35x lower</td></tr>
|
| 34 |
+
</table>
|
| 35 |
<p class="lead">Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the <a href="https://github.com/Genrius-AI/pingala8-llm/releases/latest" target="_blank" rel="noopener noreferrer">v2.5 one-click build</a> (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.</p>
|
| 36 |
<h2>Pingala-8: 2-bit LLMs for llama.cpp, no retraining</h2>
|
| 37 |
<ul>
|
results_summary.png
ADDED
|
Git LFS Details
|
results_summary.svg
ADDED
|
|
results_summary_dark.png
ADDED
|
Git LFS Details
|