harshit-genrius commited on
Commit
ddd51bb
·
verified ·
1 Parent(s): d483205

Org card: results chart, 5-model table, GitHub star button

Browse files
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ results_summary.png filter=lfs diff=lfs merge=lfs -text
37
+ results_summary_dark.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -12,15 +12,19 @@ pinned: false
12
 
13
  Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus.
14
 
15
- **Try it:** [Will it fit your GPU? (calculator)](https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator) | [One-click build, Windows and Linux, NVIDIA RTX 30/40/50](https://github.com/Genrius-AI/pingala8-llm/releases/latest)
16
 
17
- | Model | Size | WikiText-2 PPL, ours | IQ2_XXS, same size |
 
 
 
 
18
  |---|---|---|---|
19
- | [Qwen3-4B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit) | 1.24 GB | 16.64 | 28.61 |
20
- | [Llama-3.1-8B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit) | 2.39 GB | 9.14 | 12.35 |
21
- | [Qwen3.5-4B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Qwen3.5-4B-Pingala8-2bit) | 1.51 GB | 10.97 | 14.75 |
22
- | [Qwen3.5-2B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit) | 0.78 GB | 15.62 | 29.08 |
23
- | [Qwen3.5-0.8B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Qwen3.5-0.8B-Pingala8-2bit) | 0.35 GB | 25.97 | 85.09 |
24
 
25
  Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the [v2.5 one-click build](https://github.com/Genrius-AI/pingala8-llm/releases/latest) (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.
26
 
 
12
 
13
  Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus.
14
 
15
+ **Try it:** [Will it fit your GPU? (calculator)](https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator) | [One-click build, Windows and Linux, NVIDIA RTX 30/40/50](https://github.com/Genrius-AI/pingala8-llm/releases/latest) | <a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer"><img src="https://img.shields.io/github/stars/Genrius-AI/pingala8-llm?style=social" alt="Star Genrius-AI/pingala8-llm on GitHub" style="height:20px;width:auto;margin:0;vertical-align:middle;border-radius:0"></a>
16
 
17
+ **Results across 5 models**
18
+
19
+ ![How much worse than 8-bit, at 2-bit size: Pingala-8 adds 23-66% perplexity over the 8-bit model, IQ2_XXS adds 65-443%, at the same file size](https://huggingface.co/spaces/Genrius-AI/README/resolve/main/results_summary.png)
20
+
21
+ | Model | Size | Perplexity: Pingala-8 vs IQ2_XXS | IQ2_XXS / Pingala-8 |
22
  |---|---|---|---|
23
+ | [Qwen3.5-0.8B](https://huggingface.co/Genrius-AI/Qwen3.5-0.8B-Pingala8-2bit) | 0.35 GB | **25.97** vs 85.09 | 3.28x lower |
24
+ | [Qwen3.5-2B](https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit) | 0.78 GB | **15.62** vs 29.08 | 1.86x lower |
25
+ | [Qwen3.5-4B](https://huggingface.co/Genrius-AI/Qwen3.5-4B-Pingala8-2bit) | 1.51 GB | **10.97** vs 14.75 | 1.34x lower |
26
+ | [Qwen3-4B](https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit) | 1.24 GB | **16.64** vs 28.61 | 1.72x lower |
27
+ | [Llama-3.1-8B](https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit) | 2.39 GB | **9.14** vs 12.35 | 1.35x lower |
28
 
29
  Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the [v2.5 one-click build](https://github.com/Genrius-AI/pingala8-llm/releases/latest) (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.
30
 
index.html CHANGED
@@ -21,15 +21,17 @@
21
  <main>
22
  <h1>Genrius AI</h1>
23
  <p class="lead">Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.</p>
24
- <p><strong>Try it:</strong> <a href="https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator" target="_blank" rel="noopener noreferrer">Will it fit your GPU? (calculator)</a> | <a href="https://github.com/Genrius-AI/pingala8-llm/releases/latest" target="_blank" rel="noopener noreferrer">One-click build, Windows and Linux, NVIDIA RTX 30/40/50</a></p>
 
 
25
  <table>
26
- <tr><th>Model</th><th>Size</th><th>WikiText-2 PPL, ours</th><th>IQ2_XXS, same size</th></tr>
27
- <tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3-4B-Pingala8-2bit</a></td><td>1.24 GB</td><td>16.64</td><td>28.61</td></tr>
28
- <tr><td><a href="https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Llama-3.1-8B-Pingala8-2bit</a></td><td>2.39 GB</td><td>9.14</td><td>12.35</td></tr>
29
- <tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-4B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-4B-Pingala8-2bit</a></td><td>1.51 GB</td><td>10.97</td><td>14.75</td></tr>
30
- <tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-2B-Pingala8-2bit</a></td><td>0.78 GB</td><td>15.62</td><td>29.08</td></tr>
31
- <tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-0.8B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-0.8B-Pingala8-2bit</a></td><td>0.35 GB</td><td>25.97</td><td>85.09</td></tr>
32
- </table>
33
  <p class="lead">Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the <a href="https://github.com/Genrius-AI/pingala8-llm/releases/latest" target="_blank" rel="noopener noreferrer">v2.5 one-click build</a> (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.</p>
34
  <h2>Pingala-8: 2-bit LLMs for llama.cpp, no retraining</h2>
35
  <ul>
 
21
  <main>
22
  <h1>Genrius AI</h1>
23
  <p class="lead">Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.</p>
24
+ <p><strong>Try it:</strong> <a href="https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator" target="_blank" rel="noopener noreferrer">Will it fit your GPU? (calculator)</a> | <a href="https://github.com/Genrius-AI/pingala8-llm/releases/latest" target="_blank" rel="noopener noreferrer">One-click build, Windows and Linux, NVIDIA RTX 30/40/50</a> | <a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer"><img src="https://img.shields.io/github/stars/Genrius-AI/pingala8-llm?style=social" alt="Star Genrius-AI/pingala8-llm on GitHub" style="height:20px;width:auto;margin:0;vertical-align:middle;border-radius:0"></a></p>
25
+ <h2 style="margin-top:8px">Results across 5 models</h2>
26
+ <a href="https://huggingface.co/spaces/Genrius-AI/README/resolve/main/results_summary_dark.png" target="_blank" rel="noopener noreferrer"><img src="https://huggingface.co/spaces/Genrius-AI/README/resolve/main/results_summary_dark.png" width="1200" height="630" alt="How much worse than 8-bit, at 2-bit size: Pingala-8 adds 23-66% perplexity over the 8-bit model, IQ2_XXS adds 65-443%, at the same file size"></a>
27
  <table>
28
+ <tr><th>Model</th><th>Size</th><th>Perplexity: Pingala-8 vs IQ2_XXS</th><th>IQ2_XXS / Pingala-8</th></tr>
29
+ <tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-0.8B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-0.8B</a></td><td>0.35 GB</td><td><strong>25.97</strong> vs 85.09</td><td>3.28x lower</td></tr>
30
+ <tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-2B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-2B</a></td><td>0.78 GB</td><td><strong>15.62</strong> vs 29.08</td><td>1.86x lower</td></tr>
31
+ <tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3.5-4B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3.5-4B</a></td><td>1.51 GB</td><td><strong>10.97</strong> vs 14.75</td><td>1.34x lower</td></tr>
32
+ <tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3-4B</a></td><td>1.24 GB</td><td><strong>16.64</strong> vs 28.61</td><td>1.72x lower</td></tr>
33
+ <tr><td><a href="https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Llama-3.1-8B</a></td><td>2.39 GB</td><td><strong>9.14</strong> vs 12.35</td><td>1.35x lower</td></tr>
34
+ </table>
35
  <p class="lead">Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them. All five run in the <a href="https://github.com/Genrius-AI/pingala8-llm/releases/latest" target="_blank" rel="noopener noreferrer">v2.5 one-click build</a> (Windows and Linux); Qwen3.5-2B and Qwen3.5-0.8B need v2.5 or later.</p>
36
  <h2>Pingala-8: 2-bit LLMs for llama.cpp, no retraining</h2>
37
  <ul>
results_summary.png ADDED

Git LFS Details

  • SHA256: 2adf0facae53b94cde3838e4d61c44ce77667784eb196a080ef88d79db2a2fd0
  • Pointer size: 131 Bytes
  • Size of remote file: 122 kB
results_summary.svg ADDED
results_summary_dark.png ADDED

Git LFS Details

  • SHA256: 6bcd197d75b87fc42c798759803f87aacf97b92a0e21a842972e95ec071230bc
  • Pointer size: 131 Bytes
  • Size of remote file: 121 kB