harshit-genrius commited on
Commit
c029215
·
verified ·
1 Parent(s): b9f9541

Org card: Try-it line with calculator and one-click build, model table

Browse files
Files changed (2) hide show
  1. README.md +9 -0
  2. index.html +10 -0
README.md CHANGED
@@ -12,6 +12,15 @@ pinned: false
12
 
13
  Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus.
14
 
 
 
 
 
 
 
 
 
 
15
  ## Pingala-8: 2-bit LLMs for llama.cpp, no retraining
16
 
17
  - **85% smaller:** Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.
 
12
 
13
  Founded by Harshit Goyal, former Google software engineer and IIIT Delhi alumnus.
14
 
15
+ **Try it:** [Will it fit your GPU? (calculator)](https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator) | [One-click build, Windows and Linux, NVIDIA RTX 30/40/50](https://github.com/Genrius-AI/pingala8-llm/releases/latest)
16
+
17
+ | Model | Size | WikiText-2 PPL, ours | IQ2_XXS, same size |
18
+ |---|---|---|---|
19
+ | [Qwen3-4B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit) | 1.24 GB | 16.64 | 28.61 |
20
+ | [Llama-3.1-8B-Pingala8-2bit](https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit) | 2.39 GB | 9.14 | 12.35 |
21
+
22
+ Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them.
23
+
24
  ## Pingala-8: 2-bit LLMs for llama.cpp, no retraining
25
 
26
  - **85% smaller:** Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.
index.html CHANGED
@@ -13,12 +13,21 @@
13
  a { color: #D4AF37; }
14
  img { width: 100%; height: auto; border-radius: 8px; margin: 16px 0; }
15
  li { margin: 6px 0; }
 
 
16
  </style>
17
  </head>
18
  <body>
19
  <main>
20
  <h1>Genrius AI</h1>
21
  <p class="tag">Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.</p>
 
 
 
 
 
 
 
22
  <h2>Pingala-8: 2-bit LLMs for llama.cpp, no retraining</h2>
23
  <ul>
24
  <li>85% smaller: Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.</li>
@@ -28,6 +37,7 @@
28
  <img src="pingala8-poster.jpg" alt="Pingala-8 Qwen3-4B (1.24 GB) answering in a local chat window on an RTX 5070 Ti Laptop GPU">
29
  <h2>Links</h2>
30
  <ul>
 
31
  <li><a href="https://github.com/Genrius-AI/pingala8-llm/releases/tag/v2.4-oneclick" target="_blank" rel="noopener noreferrer">One-click Windows build</a></li>
32
  <li><a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer">Code</a></li>
33
  <li><a href="https://doi.org/10.5281/zenodo.23481757" target="_blank" rel="noopener noreferrer">Paper</a></li>
 
13
  a { color: #D4AF37; }
14
  img { width: 100%; height: auto; border-radius: 8px; margin: 16px 0; }
15
  li { margin: 6px 0; }
16
+ table { border-collapse: collapse; width: 100%; margin: 8px 0; font-size: 0.95rem; }
17
+ th, td { border: 1px solid #333; padding: 6px 8px; text-align: left; }
18
  </style>
19
  </head>
20
  <body>
21
  <main>
22
  <h1>Genrius AI</h1>
23
  <p class="tag">Generating Real Intelligence. We build AI models and make them small and fast enough to run on hardware people already own.</p>
24
+ <p><strong>Try it:</strong> <a href="https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator" target="_blank" rel="noopener noreferrer">Will it fit your GPU? (calculator)</a> | <a href="https://github.com/Genrius-AI/pingala8-llm/releases/latest" target="_blank" rel="noopener noreferrer">One-click build, Windows and Linux, NVIDIA RTX 30/40/50</a></p>
25
+ <table>
26
+ <tr><th>Model</th><th>Size</th><th>WikiText-2 PPL, ours</th><th>IQ2_XXS, same size</th></tr>
27
+ <tr><td><a href="https://huggingface.co/Genrius-AI/Qwen3-4B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Qwen3-4B-Pingala8-2bit</a></td><td>1.24 GB</td><td>16.64</td><td>28.61</td></tr>
28
+ <tr><td><a href="https://huggingface.co/Genrius-AI/Llama-3.1-8B-Pingala8-2bit" target="_blank" rel="noopener noreferrer">Llama-3.1-8B-Pingala8-2bit</a></td><td>2.39 GB</td><td>9.14</td><td>12.35</td></tr>
29
+ </table>
30
+ <p class="tag">Lower is better. The files need our patched llama.cpp build (NVIDIA CUDA); stock llama.cpp, Ollama and LM Studio cannot load them.</p>
31
  <h2>Pingala-8: 2-bit LLMs for llama.cpp, no retraining</h2>
32
  <ul>
33
  <li>85% smaller: Qwen3-4B goes from 8.05 GB to 1.24 GB, and Llama 3.1 8B from 16.08 GB to 2.39 GB.</li>
 
37
  <img src="pingala8-poster.jpg" alt="Pingala-8 Qwen3-4B (1.24 GB) answering in a local chat window on an RTX 5070 Ti Laptop GPU">
38
  <h2>Links</h2>
39
  <ul>
40
+ <li><a href="https://huggingface.co/spaces/Genrius-AI/llm-vram-calculator" target="_blank" rel="noopener noreferrer">VRAM calculator: normal vs Pingala-8</a></li>
41
  <li><a href="https://github.com/Genrius-AI/pingala8-llm/releases/tag/v2.4-oneclick" target="_blank" rel="noopener noreferrer">One-click Windows build</a></li>
42
  <li><a href="https://github.com/Genrius-AI/pingala8-llm" target="_blank" rel="noopener noreferrer">Code</a></li>
43
  <li><a href="https://doi.org/10.5281/zenodo.23481757" target="_blank" rel="noopener noreferrer">Paper</a></li>