Spaces:
Running
Running
Org card: Try-it line with calculator and one-click build, model table
Browse files- README.md +9 -0
- index.html +10 -0
README.md
CHANGED
|
@@ -12,6 +12,15 @@ pinned: false
|
|
| 12 |
|
| 13 |
Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus.
|
| 14 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
## Pingala-8: 2-bit LLMs for llama.cpp, no retraining
|
| 16 |
|
| 17 |
- **85% smaller:** Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.
|
|
|
|
| 12 |
|
| 13 |
Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus.
|
| 14 |
|
| 15 |
+
**Try it:** [Will it fit your GPU? (calculator)](https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator) | [One-click build, Windows and Linux, NVIDIA RTX 30/40/50](https://github.com/Genrius-AI/pingala8-llm/releases/latest)
|
| 16 |
+
|
| 17 |
+
| Model | Size | WikiText-2 PPL, ours | IQ2_XXS, same size |
|
| 18 |
+
|---|---|---|---|
|
| 19 |
+
| [Qwen3-4B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit) | 1.24 GB | 16.64 | 28.61 |
|
| 20 |
+
| [Llama-3.1-8B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit) | 2.39 GB | 9.14 | 12.35 |
|
| 21 |
+
|
| 22 |
+
Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them.
|
| 23 |
+
|
| 24 |
## Pingala-8: 2-bit LLMs for llama.cpp, no retraining
|
| 25 |
|
| 26 |
- **85% smaller:** Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.
|
index.html
CHANGED
|
@@ -13,12 +13,21 @@
|
|
| 13 |
a { color: #D4AF37; }
|
| 14 |
img { width: 100%; height: auto; border-radius: 8px; margin: 16px 0; }
|
| 15 |
li { margin: 6px 0; }
|
|
|
|
|
|
|
| 16 |
</style>
|
| 17 |
</head>
|
| 18 |
<body>
|
| 19 |
<main>
|
| 20 |
<h1>Genrius AI</h1>
|
| 21 |
<p class="tag">Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.</p>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
<h2>Pingala-8: 2-bit LLMs for llama.cpp, no retraining</h2>
|
| 23 |
<ul>
|
| 24 |
<li>85% smaller: Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.</li>
|
|
@@ -28,6 +37,7 @@
|
|
| 28 |
<img src="pingala8-poster.jpg" alt="Pingala-8 Qwen3-4B (1.24 GB) answering in a local chat window on an RTX 5070 Ti Laptop GPU">
|
| 29 |
<h2>Links</h2>
|
| 30 |
<ul>
|
|
|
|
| 31 |
<li><a href="https://github.com/Genrius-AI/pingala8-llm/releases/tag/v2.4-oneclick" target="_blank" rel="noopener noreferrer">One-click Windows build</a></li>
|
| 32 |
<li><a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer">Code</a></li>
|
| 33 |
<li><a href="https://doi.org/10.5281/zenodo.23481757" target="_blank" rel="noopener noreferrer">Paper</a></li>
|
|
|
|
| 13 |
a { color: #D4AF37; }
|
| 14 |
img { width: 100%; height: auto; border-radius: 8px; margin: 16px 0; }
|
| 15 |
li { margin: 6px 0; }
|
| 16 |
+
table { border-collapse: collapse; width: 100%; margin: 8px 0; font-size: 0.95rem; }
|
| 17 |
+
th, td { border: 1px solid #333; padding: 6px 8px; text-align: left; }
|
| 18 |
</style>
|
| 19 |
</head>
|
| 20 |
<body>
|
| 21 |
<main>
|
| 22 |
<h1>Genrius AI</h1>
|
| 23 |
<p class="tag">Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.</p>
|
| 24 |
+
<p><strong>Try it:</strong> <a href="https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator" target="_blank" rel="noopener noreferrer">Will it fit your GPU? (calculator)</a> | <a href="https://github.com/Genrius-AI/pingala8-llm/releases/latest" target="_blank" rel="noopener noreferrer">One-click build, Windows and Linux, NVIDIA RTX 30/40/50</a></p>
|
| 25 |
+
<table>
|
| 26 |
+
<tr><th>Model</th><th>Size</th><th>WikiText-2 PPL, ours</th><th>IQ2_XXS, same size</th></tr>
|
| 27 |
+
<tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3-4B-Pingala8-2bit</a></td><td>1.24 GB</td><td>16.64</td><td>28.61</td></tr>
|
| 28 |
+
<tr><td><a href="https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Llama-3.1-8B-Pingala8-2bit</a></td><td>2.39 GB</td><td>9.14</td><td>12.35</td></tr>
|
| 29 |
+
</table>
|
| 30 |
+
<p class="tag">Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them.</p>
|
| 31 |
<h2>Pingala-8: 2-bit LLMs for llama.cpp, no retraining</h2>
|
| 32 |
<ul>
|
| 33 |
<li>85% smaller: Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.</li>
|
|
|
|
| 37 |
<img src="pingala8-poster.jpg" alt="Pingala-8 Qwen3-4B (1.24 GB) answering in a local chat window on an RTX 5070 Ti Laptop GPU">
|
| 38 |
<h2>Links</h2>
|
| 39 |
<ul>
|
| 40 |
+
<li><a href="https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator" target="_blank" rel="noopener noreferrer">VRAM calculator: normal vs Pingala-8</a></li>
|
| 41 |
<li><a href="https://github.com/Genrius-AI/pingala8-llm/releases/tag/v2.4-oneclick" target="_blank" rel="noopener noreferrer">One-click Windows build</a></li>
|
| 42 |
<li><a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer">Code</a></li>
|
| 43 |
<li><a href="https://doi.org/10.5281/zenodo.23481757" target="_blank" rel="noopener noreferrer">Paper</a></li>
|