Spaces:
Running
Running
Add the page, the speed chart and the two demos
Browse files- .gitattributes +3 -0
- claude-code-demo.gif +3 -0
- index.html +61 -18
- llamacpp-comparison.gif +3 -0
- speed.png +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
claude-code-demo.gif filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
llamacpp-comparison.gif filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
speed.png filter=lfs diff=lfs merge=lfs -text
|
claude-code-demo.gif
ADDED
|
Git LFS Details
|
index.html
CHANGED
|
@@ -1,19 +1,62 @@
|
|
| 1 |
<!doctype html>
|
| 2 |
-
<html>
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
</
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
<!doctype html>
|
| 2 |
+
<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
|
| 3 |
+
<title>PolyStrata</title>
|
| 4 |
+
<style>
|
| 5 |
+
body{margin:0;background:#0f1116;color:#e6e6e6;font:16px/1.55 system-ui,-apple-system,Segoe UI,Roboto,sans-serif}
|
| 6 |
+
main{max-width:960px;margin:0 auto;padding:32px 20px 60px}
|
| 7 |
+
h1{font-size:2rem;margin:0 0 6px} h2{font-size:1.3rem;margin:36px 0 10px;border-bottom:1px solid #2a2e3a;padding-bottom:6px}
|
| 8 |
+
h3{font-size:1.05rem;margin:22px 0 8px;color:#c9d1e0}
|
| 9 |
+
p{margin:10px 0} a{color:#8ab4ff} img{max-width:100%;border-radius:8px;display:block}
|
| 10 |
+
table{border-collapse:collapse;width:100%;font-size:15px} th,td{text-align:left;padding:6px 8px;border-bottom:1px solid #2a2e3a} th{color:#aab}
|
| 11 |
+
.sub{color:#9aa3b2} code{background:#1b1f2a;padding:1px 5px;border-radius:4px}
|
| 12 |
+
.btn{display:inline-block;background:#2f6fed;color:#fff;padding:10px 16px;border-radius:8px;text-decoration:none;margin:8px 8px 8px 0}
|
| 13 |
+
</style></head><body><main>
|
| 14 |
+
<h1>PolyStrata</h1>
|
| 15 |
+
<p class="sub">Large mixture-of-experts models on the graphics card you already have. Open source, MIT, built on <a href="https://github.com/Niko1221/Strata">Strata</a>.</p>
|
| 16 |
+
<p><a class="btn" href="https://github.com/VecSzn/PolyStrata">GitHub</a> <a class="btn" href="https://github.com/VecSzn/PolyStrata/releases/latest">Latest release</a></p>
|
| 17 |
+
<p>A mixture-of-experts model only uses a few of its experts for each token. PolyStrata keeps the experts a model uses most on the GPU and computes the rest on the CPU from RAM, so a 35B model runs next to an 8 GB card and 32 GB of RAM. It reads the same GGUF files as llama.cpp, and its output is checked against llama.cpp token by token.</p>
|
| 18 |
+
<h2>Same laptop, same model file, same request</h2>
|
| 19 |
+
<p>PolyStrata on the left, llama.cpp b11540 on the right, on a laptop with an RTX 4070 (8 GB) and 32 GB of RAM, Qwen3.6-35B-A3B UD-Q3_K_M. On this laptop PolyStrata reads prompts 2.7 to 3.5 times as fast as llama.cpp and writes 1.5 to 1.8 times as fast; the comparison script is in the repository.</p>
|
| 20 |
+
<img src="llamacpp-comparison.gif" alt="PolyStrata and llama.cpp answering the same request on the same laptop">
|
| 21 |
+
<h2>Tokens per second</h2>
|
| 22 |
+
<img src="speed.png" alt="Tokens per second while writing an answer, laptop and workstation">
|
| 23 |
+
<h3>Laptop: RTX 4070 (8 GB), Core i7-14700HX, 32 GB RAM</h3>
|
| 24 |
+
<table><tr><th>Model</th><th>Size</th><th>Writes, tokens/s</th><th>Reads a 10K prompt, tokens/s</th></tr>
|
| 25 |
+
<tr><td>Qwen3.6-35B-A3B</td><td>UD-Q3_K_M</td><td>94-100</td><td>1,690</td></tr>
|
| 26 |
+
<tr><td>Qwen3.6-35B-A3B</td><td>UD-Q2_K_XL</td><td>123-124</td><td>1,590</td></tr>
|
| 27 |
+
<tr><td>Qwen3.6-35B-A3B</td><td>UD-Q4_K_M</td><td>82-86</td><td>1,560-1,570</td></tr>
|
| 28 |
+
<tr><td>Ornith 1.5</td><td>APEX-I-COMPACT</td><td>76-82</td><td>1,620</td></tr>
|
| 29 |
+
<tr><td>Ornith 1.5</td><td>APEX-I-MINI</td><td>80-86</td><td>1,550-1,560</td></tr>
|
| 30 |
+
<tr><td>Ornith 1.5</td><td>APEX-I-QUALITY</td><td>59-63</td><td>1,570-1,580</td></tr>
|
| 31 |
+
<tr><td>Gemma 4 26B-A4B</td><td>UD-Q3_K_M</td><td>92-93</td><td>2,240</td></tr>
|
| 32 |
+
<tr><td>Gemma 4 26B-A4B</td><td>UD-Q2_K_XL</td><td>101-104</td><td>2,150-2,160</td></tr>
|
| 33 |
+
<tr><td>Gemma 4 26B-A4B</td><td>UD-Q4_K_M</td><td>85</td><td>2,150</td></tr>
|
| 34 |
+
<tr><td>Tiel Coder 35B-A3B</td><td>UD-Q3_K_XL</td><td>83</td><td>1,700</td></tr>
|
| 35 |
+
<tr><td>Tiel Coder 35B-A3B</td><td>UD-IQ3_XXS</td><td>96-97</td><td>1,610</td></tr>
|
| 36 |
+
<tr><td>Tiel Coder 35B-A3B</td><td>UD-Q4_K_XL</td><td>74-79</td><td>1,660-1,670</td></tr>
|
| 37 |
+
</table>
|
| 38 |
+
<h3>Workstation: RTX 5090 (32 GB), Xeon Platinum 8470Q, 120 GB RAM</h3>
|
| 39 |
+
<table><tr><th>Model</th><th>Size</th><th>Writes, tokens/s</th><th>Reads a 10K prompt, tokens/s</th></tr>
|
| 40 |
+
<tr><td>GLM 5.3 Flash</td><td>3.0BIT</td><td>59-66</td><td>820</td></tr>
|
| 41 |
+
<tr><td>GLM 5.3 Flash</td><td>UD-IQ2_XXS</td><td>50-52</td><td>1,060-1,080</td></tr>
|
| 42 |
+
<tr><td>GLM 5.3 Flash</td><td>REAP50-Q3_K_M</td><td>59-66</td><td>1,300-1,310</td></tr>
|
| 43 |
+
<tr><td>Qwen3.6-35B-A3B</td><td>UD-Q3_K_M</td><td>394-395</td><td>6,350-6,550</td></tr>
|
| 44 |
+
<tr><td>Qwen3.6-35B-A3B</td><td>UD-Q2_K_XL</td><td>450</td><td>6,400-6,590</td></tr>
|
| 45 |
+
<tr><td>Qwen3.6-35B-A3B</td><td>UD-Q4_K_M</td><td>363-401</td><td>6,030-6,690</td></tr>
|
| 46 |
+
<tr><td>Ornith 1.5</td><td>APEX-I-COMPACT</td><td>316-319</td><td>6,830-6,860</td></tr>
|
| 47 |
+
<tr><td>Ornith 1.5</td><td>APEX-I-MINI</td><td>289-300</td><td>6,300-6,380</td></tr>
|
| 48 |
+
<tr><td>Ornith 1.5</td><td>APEX-I-QUALITY</td><td>289-298</td><td>6,040-6,390</td></tr>
|
| 49 |
+
<tr><td>Gemma 4 26B-A4B</td><td>UD-Q3_K_M</td><td>353-356</td><td>7,880-8,230</td></tr>
|
| 50 |
+
<tr><td>Gemma 4 26B-A4B</td><td>UD-Q2_K_XL</td><td>327-340</td><td>7,380-8,000</td></tr>
|
| 51 |
+
<tr><td>Gemma 4 26B-A4B</td><td>UD-Q4_K_M</td><td>335-346</td><td>7,040-8,050</td></tr>
|
| 52 |
+
<tr><td>Tiel Coder 35B-A3B</td><td>UD-Q3_K_XL</td><td>337-345</td><td>6,570-6,830</td></tr>
|
| 53 |
+
<tr><td>Tiel Coder 35B-A3B</td><td>UD-IQ3_XXS</td><td>339-351</td><td>6,090-6,520</td></tr>
|
| 54 |
+
<tr><td>Tiel Coder 35B-A3B</td><td>UD-Q4_K_XL</td><td>316-322</td><td>6,750-6,870</td></tr>
|
| 55 |
+
</table>
|
| 56 |
+
<p class="sub">GLM 5.3 Flash is a 79 to 114 GB file and needs about 100 GB of RAM beside a 32 GB card. Qwen3.8-Flash-Next runs on Strata's own engine; its numbers are in the README.</p>
|
| 57 |
+
<h2>Works with Claude Code</h2>
|
| 58 |
+
<p>The server speaks the OpenAI and Anthropic APIs on localhost, so Claude Code, Codex and other apps run against the local model. This is Claude Code on the laptop model, in real time:</p>
|
| 59 |
+
<img src="claude-code-demo.gif" alt="Claude Code working against the local model">
|
| 60 |
+
<h2>Install</h2>
|
| 61 |
+
<p>Windows: download the source from the <a href="https://github.com/VecSzn/PolyStrata/releases/latest">latest release</a> and run <code>START-HERE.bat</code>; it installs a ready-made engine and the model. Linux: <code>./setup.sh</code> builds the engine. AMD cards work on Linux as a beta. Details, every option and the community results are in the <a href="https://github.com/VecSzn/PolyStrata#readme">README</a>.</p>
|
| 62 |
+
</main></body></html>
|
llamacpp-comparison.gif
ADDED
|
Git LFS Details
|
speed.png
ADDED
|
Git LFS Details
|