VecSzn commited on
Commit
ab39a28
·
verified ·
1 Parent(s): e08c918

Add the page, the speed chart and the two demos

Browse files
Files changed (5) hide show
  1. .gitattributes +3 -0
  2. claude-code-demo.gif +3 -0
  3. index.html +61 -18
  4. llamacpp-comparison.gif +3 -0
  5. speed.png +3 -0
.gitattributes CHANGED
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ claude-code-demo.gif filter=lfs diff=lfs merge=lfs -text
37
+ llamacpp-comparison.gif filter=lfs diff=lfs merge=lfs -text
38
+ speed.png filter=lfs diff=lfs merge=lfs -text
claude-code-demo.gif ADDED

Git LFS Details

  • SHA256: b26e8f060eed03bc8bc95e877d3ab6c13b1500239b37776e2586d6988ee77dd3
  • Pointer size: 132 Bytes
  • Size of remote file: 1.79 MB
index.html CHANGED
@@ -1,19 +1,62 @@
1
  <!doctype html>
2
- <html>
3
- <head>
4
- <meta charset="utf-8" />
5
- <meta name="viewport" content="width=device-width" />
6
- <title>My static Space</title>
7
- <link rel="stylesheet" href="style.css" />
8
- </head>
9
- <body>
10
- <div class="card">
11
- <h1>Welcome to your static Space!</h1>
12
- <p>You can modify this app directly by editing <i>index.html</i> in the Files and versions tab.</p>
13
- <p>
14
- Also don't forget to check the
15
- <a href="https://huggingface.co/docs/hub/spaces" target="_blank">Spaces documentation</a>.
16
- </p>
17
- </div>
18
- </body>
19
- </html>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  <!doctype html>
2
+ <html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
3
+ <title>PolyStrata</title>
4
+ <style>
5
+ body{margin:0;background:#0f1116;color:#e6e6e6;font:16px/1.55 system-ui,-apple-system,Segoe UI,Roboto,sans-serif}
6
+ main{max-width:960px;margin:0 auto;padding:32px 20px 60px}
7
+ h1{font-size:2rem;margin:0 0 6px} h2{font-size:1.3rem;margin:36px 0 10px;border-bottom:1px solid #2a2e3a;padding-bottom:6px}
8
+ h3{font-size:1.05rem;margin:22px 0 8px;color:#c9d1e0}
9
+ p{margin:10px 0} a{color:#8ab4ff} img{max-width:100%;border-radius:8px;display:block}
10
+ table{border-collapse:collapse;width:100%;font-size:15px} th,td{text-align:left;padding:6px 8px;border-bottom:1px solid #2a2e3a} th{color:#aab}
11
+ .sub{color:#9aa3b2} code{background:#1b1f2a;padding:1px 5px;border-radius:4px}
12
+ .btn{display:inline-block;background:#2f6fed;color:#fff;padding:10px 16px;border-radius:8px;text-decoration:none;margin:8px 8px 8px 0}
13
+ </style></head><body><main>
14
+ <h1>PolyStrata</h1>
15
+ <p class="sub">Large mixture-of-experts models on the graphics card you already have. Open source, MIT, built on <a href="https://github.com/Niko1221/Strata">Strata</a>.</p>
16
+ <p><a class="btn" href="https://github.com/VecSzn/PolyStrata">GitHub</a> <a class="btn" href="https://github.com/VecSzn/PolyStrata/releases/latest">Latest release</a></p>
17
+ <p>A mixture-of-experts model only uses a few of its experts for each token. PolyStrata keeps the experts a model uses most on the GPU and computes the rest on the CPU from RAM, so a 35B model runs next to an 8 GB card and 32 GB of RAM. It reads the same GGUF files as llama.cpp, and its output is checked against llama.cpp token by token.</p>
18
+ <h2>Same laptop, same model file, same request</h2>
19
+ <p>PolyStrata on the left, llama.cpp b11540 on the right, on a laptop with an RTX 4070 (8 GB) and 32 GB of RAM, Qwen3.6-35B-A3B UD-Q3_K_M. On this laptop PolyStrata reads prompts 2.7 to 3.5 times as fast as llama.cpp and writes 1.5 to 1.8 times as fast; the comparison script is in the repository.</p>
20
+ <img src="llamacpp-comparison.gif" alt="PolyStrata and llama.cpp answering the same request on the same laptop">
21
+ <h2>Tokens per second</h2>
22
+ <img src="speed.png" alt="Tokens per second while writing an answer, laptop and workstation">
23
+ <h3>Laptop: RTX 4070 (8 GB), Core i7-14700HX, 32 GB RAM</h3>
24
+ <table><tr><th>Model</th><th>Size</th><th>Writes, tokens/s</th><th>Reads a 10K prompt, tokens/s</th></tr>
25
+ <tr><td>Qwen3.6-35B-A3B</td><td>UD-Q3_K_M</td><td>94-100</td><td>1,690</td></tr>
26
+ <tr><td>Qwen3.6-35B-A3B</td><td>UD-Q2_K_XL</td><td>123-124</td><td>1,590</td></tr>
27
+ <tr><td>Qwen3.6-35B-A3B</td><td>UD-Q4_K_M</td><td>82-86</td><td>1,560-1,570</td></tr>
28
+ <tr><td>Ornith 1.5</td><td>APEX-I-COMPACT</td><td>76-82</td><td>1,620</td></tr>
29
+ <tr><td>Ornith 1.5</td><td>APEX-I-MINI</td><td>80-86</td><td>1,550-1,560</td></tr>
30
+ <tr><td>Ornith 1.5</td><td>APEX-I-QUALITY</td><td>59-63</td><td>1,570-1,580</td></tr>
31
+ <tr><td>Gemma 4 26B-A4B</td><td>UD-Q3_K_M</td><td>92-93</td><td>2,240</td></tr>
32
+ <tr><td>Gemma 4 26B-A4B</td><td>UD-Q2_K_XL</td><td>101-104</td><td>2,150-2,160</td></tr>
33
+ <tr><td>Gemma 4 26B-A4B</td><td>UD-Q4_K_M</td><td>85</td><td>2,150</td></tr>
34
+ <tr><td>Tiel Coder 35B-A3B</td><td>UD-Q3_K_XL</td><td>83</td><td>1,700</td></tr>
35
+ <tr><td>Tiel Coder 35B-A3B</td><td>UD-IQ3_XXS</td><td>96-97</td><td>1,610</td></tr>
36
+ <tr><td>Tiel Coder 35B-A3B</td><td>UD-Q4_K_XL</td><td>74-79</td><td>1,660-1,670</td></tr>
37
+ </table>
38
+ <h3>Workstation: RTX 5090 (32 GB), Xeon Platinum 8470Q, 120 GB RAM</h3>
39
+ <table><tr><th>Model</th><th>Size</th><th>Writes, tokens/s</th><th>Reads a 10K prompt, tokens/s</th></tr>
40
+ <tr><td>GLM 5.3 Flash</td><td>3.0BIT</td><td>59-66</td><td>820</td></tr>
41
+ <tr><td>GLM 5.3 Flash</td><td>UD-IQ2_XXS</td><td>50-52</td><td>1,060-1,080</td></tr>
42
+ <tr><td>GLM 5.3 Flash</td><td>REAP50-Q3_K_M</td><td>59-66</td><td>1,300-1,310</td></tr>
43
+ <tr><td>Qwen3.6-35B-A3B</td><td>UD-Q3_K_M</td><td>394-395</td><td>6,350-6,550</td></tr>
44
+ <tr><td>Qwen3.6-35B-A3B</td><td>UD-Q2_K_XL</td><td>450</td><td>6,400-6,590</td></tr>
45
+ <tr><td>Qwen3.6-35B-A3B</td><td>UD-Q4_K_M</td><td>363-401</td><td>6,030-6,690</td></tr>
46
+ <tr><td>Ornith 1.5</td><td>APEX-I-COMPACT</td><td>316-319</td><td>6,830-6,860</td></tr>
47
+ <tr><td>Ornith 1.5</td><td>APEX-I-MINI</td><td>289-300</td><td>6,300-6,380</td></tr>
48
+ <tr><td>Ornith 1.5</td><td>APEX-I-QUALITY</td><td>289-298</td><td>6,040-6,390</td></tr>
49
+ <tr><td>Gemma 4 26B-A4B</td><td>UD-Q3_K_M</td><td>353-356</td><td>7,880-8,230</td></tr>
50
+ <tr><td>Gemma 4 26B-A4B</td><td>UD-Q2_K_XL</td><td>327-340</td><td>7,380-8,000</td></tr>
51
+ <tr><td>Gemma 4 26B-A4B</td><td>UD-Q4_K_M</td><td>335-346</td><td>7,040-8,050</td></tr>
52
+ <tr><td>Tiel Coder 35B-A3B</td><td>UD-Q3_K_XL</td><td>337-345</td><td>6,570-6,830</td></tr>
53
+ <tr><td>Tiel Coder 35B-A3B</td><td>UD-IQ3_XXS</td><td>339-351</td><td>6,090-6,520</td></tr>
54
+ <tr><td>Tiel Coder 35B-A3B</td><td>UD-Q4_K_XL</td><td>316-322</td><td>6,750-6,870</td></tr>
55
+ </table>
56
+ <p class="sub">GLM 5.3 Flash is a 79 to 114 GB file and needs about 100 GB of RAM beside a 32 GB card. Qwen3.8-Flash-Next runs on Strata's own engine; its numbers are in the README.</p>
57
+ <h2>Works with Claude Code</h2>
58
+ <p>The server speaks the OpenAI and Anthropic APIs on localhost, so Claude Code, Codex and other apps run against the local model. This is Claude Code on the laptop model, in real time:</p>
59
+ <img src="claude-code-demo.gif" alt="Claude Code working against the local model">
60
+ <h2>Install</h2>
61
+ <p>Windows: download the source from the <a href="https://github.com/VecSzn/PolyStrata/releases/latest">latest release</a> and run <code>START-HERE.bat</code>; it installs a ready-made engine and the model. Linux: <code>./setup.sh</code> builds the engine. AMD cards work on Linux as a beta. Details, every option and the community results are in the <a href="https://github.com/VecSzn/PolyStrata#readme">README</a>.</p>
62
+ </main></body></html>
llamacpp-comparison.gif ADDED

Git LFS Details

  • SHA256: 3393acfe9df935b760dfe3ccd480568b4ea0d58e99fd13d0ddf41174ae6883de
  • Pointer size: 132 Bytes
  • Size of remote file: 2.75 MB
speed.png ADDED

Git LFS Details

  • SHA256: ac975b57e82f3913e28f70ee6c82202e6d0c4dc3f7fa5c189394e1477cd9e60a
  • Pointer size: 131 Bytes
  • Size of remote file: 101 kB