Hugging Face's logo

Spaces:
mohdmahmodi
/
Spark-X2.6-WebGPU
Running

App Files Community
Spark-X2.6-WebGPU / benchmarks.html
mohdmahmodi's picture
mohdmahmodi
Small-GPU prefill fix, Transformers.js on the benchmarks page
9d57f66 verified
Raw History Blame Contribute Delete
14.1 kB
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="color-scheme" content="dark" />
<meta name="theme-color" content="#000000" />
<title>Spark-X2.6-4B in the browser</title>
<meta name="description" content="Spark-X2.6-4B running on WebGPU in Chrome and Edge, using the official 4-bit weights. Benchmarks, speed and how to use it." />
<link rel="canonical" href="https://mohdmahmodi-spark-x2-6-webgpu.static.hf.space/benchmarks.html" />
<meta property="og:type" content="article" />
<meta property="og:title" content="Spark-X2.6-4B in the browser" />
<meta property="og:description" content="Spark-X2.6-4B running on WebGPU with the official 4-bit weights, at about 90 tokens per second." />
<meta property="og:image" content="https://huggingface.co/mohdmahmodi/Spark-X2.6-4B-ONNX-WebGPU/resolve/main/assets/hero.png" />
<meta property="og:url" content="https://mohdmahmodi-spark-x2-6-webgpu.static.hf.space/benchmarks.html" />
<meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:creator" content="@mohdmahmodi" />
<meta name="author" content="Mohd Mahmodi" />
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Barlow:wght@500;600;700&family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap" />
<link rel="stylesheet" href="assets/base.css" />
<link rel="stylesheet" href="assets/site.css" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.29.0/prism.min.js" data-manual></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.29.0/plugins/autoloader/prism-autoloader.min.js"></script>
<script type="module" src="app/site.js"></script>
</head>
<body>
<div class="page">
<section id="social" aria-hidden="true">
<div class="eyebrow">Open weights on WebGPU</div>
<h1>Spark-X2.6-4B</h1>
<div class="tagline">Running in your browser</div>
<div class="key-numbers" data-figures></div>
</section>
<nav class="nav">
<div class="nav-inner">
<a class="name" href="./">Spark-X2.6-4B</a>
<a class="hide-sm" href="#compare">Benchmarks</a>
<a class="hide-sm" href="#speed">Speed</a>
<a class="hide-sm" href="#start">Get started</a>
<a class="hide-sm" href="https://github.com/mohdmahmodi/Spark-X2.6-4B-onnx-webgpu">GitHub</a>
<a class="btn" href="./">Try it</a>
</div>
</nav>
<header class="hero">
<div class="eyebrow">October 29, 2026</div>
<div class="hero-copy">
<h1>Spark-X2.6-4B</h1>
<div class="tagline">Running in your browser</div>
<p class="lead">
XHToken's 4B model, using its official 4-bit weights, running on the GPU in Chrome and Edge. In testing it loses no measurable
accuracy against the official INT4 release and generates about 90 tokens per second on a desktop GPU.
</p>
</div>
<div class="actions">
<a class="btn btn-solid" href="./">Start chatting<svg class="icon"><use href="assets/icons.svg#arrow-right" /></svg></a>
<a class="btn" href="https://huggingface.co/mohdmahmodi/Spark-X2.6-4B-ONNX-WebGPU"><svg class="icon"><use href="assets/icons.svg#huggingface" /></svg>Model files</a>
<a class="btn" href="https://github.com/mohdmahmodi/Spark-X2.6-4B-onnx-webgpu"><svg class="icon"><use href="assets/icons.svg#github" /></svg>Source</a>
</div>
<div class="byline">By <a href="https://x.com/mohdmahmodi">Mohd Mahmodi</a></div>
</header>
<article class="article">
<figure class="figure-block wide">
<img src="assets/chat-preview.png" width="2880" height="1800" alt="The chat app answering a coding question with thinking on" />
<figcaption>The chat app answering a coding question with thinking on. Everything runs on the local GPU.</figcaption>
</figure>
<div class="key-numbers wide" data-figures></div>
<p>
Spark-X2.6-4B is a 4-billion-parameter model from XHToken, aimed at on-device agents: tool use, coding and long context, with a
thinking budget that can be set to high, low or off. This build packages the official INT4 checkpoint as ONNX for
onnxruntime-web. The model runs inside a browser tab, and what you type stays on your machine.
</p>
<h2 id="compare">How Spark-X2.6-4B compares</h2>
<p>
XHToken reports these results for Spark-X2.6-4B against open models of similar size, all with thinking on. Across 30
benchmarks it averages <strong>63.3</strong>, ahead of the larger Qwen3.5-9B (49.1) and Gemma4-12B (47.0). The lead is widest
on agent work: web browsing, MCP tool use and software engineering.
</p>
<figure class="figure-block wide" id="quality">
<h2 class="shot-only shot-title">How Spark-X2.6-4B compares</h2>
<div class="multiples" id="quality-grid"></div>
<div class="legend" id="quality-legend"></div>
<figcaption>Reported by XHToken for the original Spark-X2.6-4B, thinking on at high effort. Source: the Spark-X2.6-4B model card on Hugging Face.</figcaption>
</figure>
<h3>All 30 benchmarks</h3>
<figure class="figure-block wide" id="results">
<h2 class="shot-only shot-title">Spark-X2.6-4B benchmark results</h2>
<div class="table-wrap"><table class="results" id="quality-table"></table></div>
<figcaption>* Taken by XHToken from that model's published card or paper. A dash means not reported. Gaokao 2026 is scored out of 150.</figcaption>
</figure>
<h2 id="accuracy">No measurable accuracy loss</h2>
<p>
A different runtime and 4-bit weights can change a model's answers, so the browser build was tested against the original model in
transformers: same prompts, same chat template, greedy decoding, thinking off. Every browser number comes from the actual web page,
run in headless Chrome.
</p>
<figure class="figure-block wide" id="parity">
<h2 class="shot-only shot-title">The browser build against the original model</h2>
<div class="chart" id="acc-chart"></div>
<div class="legend" id="acc-legend"></div>
<figcaption>Spark-X2.6-4B, q4f16 build in Chrome. GSM8K: all 1,319 test problems. BFCL v3: the first 200 simple function-calling cases. Needle in a haystack: 12 prompts from 1k to 16k tokens.</figcaption>
</figure>
<p>
The browser build is within measurement noise of the official INT4 checkpoint on all three. On GSM8K it solves 1,194
problems to the checkpoint's 1,203 and the original model's 1,198 (22 and 13 disagreements with the checkpoint, McNemar
p = 0.18); function calling and needle in a haystack score the same. At the level of individual tokens, the build adds a
mean KL divergence of 0.00055 on top of the INT4 quantization, and 30 minutes of back-to-back generation produced no
errors and no slowdown.
</p>
<figure class="figure-block" id="summary">
<h2 class="shot-only shot-title">Validation against the original model</h2>
<div class="table-wrap"><table class="checks" id="summary-table"></table></div>
<figcaption>A check passes when the browser build is not measurably worse than the official INT4 weights in transformers.</figcaption>
</figure>
<h2 id="speed">Speed</h2>
<p>
On a desktop GPU the model generates about <strong>90 tokens per second</strong> and starts answering a short prompt in under
100 ms. Generation slows gently as the conversation grows, to 69 tokens per second at 16,000 tokens of context. Long prompts
take longer to read before the first token: 1.7 seconds for 2,500 tokens.
</p>
<figure class="figure-block wide" id="history">
<h2 class="shot-only shot-title">From 21 to 88 tokens per second</h2>
<div class="chart" id="history-chart"></div>
<figcaption>Decode speed at 2,500 tokens of context after each change. The two largest steps came from keeping the KV cache in one GPU buffer that is updated in place, and from running the 4-bit matrix multiplies on float inputs.</figcaption>
</figure>
<figure class="figure-block wide" id="speed-lines">
<div class="multiples pair">
<div><h3>Decode speed</h3><p class="task">tokens per second</p><div class="chart" id="tps-chart"></div></div>
<div><h3>Time to first token</h3><p class="task">seconds</p><div class="chart" id="ttft-chart"></div></div>
</div>
<figcaption>Measured in Chrome on Windows with an NVIDIA RTX 3090 Ti, one request at a time. Speed depends on the GPU.</figcaption>
</figure>
<h2 id="start">Get started</h2>
<p>The model is a folder of static files and the loader is a few JavaScript modules, with no build step and no server code.</p>
<ol class="steps">
<li>
<h3>Chat in the browser</h3>
<p>Open the <a href="./">chat</a> in desktop Chrome or Edge. The model downloads once (2.77 GB) and loads from the browser cache afterwards.</p>
</li>
<li>
<h3>Run it locally</h3>
<p>Serve the app from a clone. It loads the model from Hugging Face unless a local build exists.</p>
<pre><code class="language-bash">git clone https://github.com/mohdmahmodi/Spark-X2.6-4B-onnx-webgpu
cd Spark-X2.6-4B-onnx-webgpu
npm install
npm run serve</code></pre>
</li>
<li>
<h3>Use it in your own app</h3>
<p>
Copy <code>web/lib/</code> and the import map from <code>web/index.html</code>. <code>generate</code> streams text and accepts
<code>temperature</code>, <code>maxNewTokens</code>, <code>effort</code>, <code>tools</code> and an abort <code>signal</code>.
</p>
<pre><code class="language-javascript">import { loadSpark } from "./lib/spark.js";
const spark = await loadSpark({
base: "https://huggingface.co/mohdmahmodi/Spark-X2.6-4B-ONNX-WebGPU/resolve/main",
});
const messages = [{ role: "user", content: "Hello!" }];
for await (const text of spark.generate(messages, { thinking: false })) {
output.textContent += text;
}</code></pre>
</li>
<li>
<h3>Or use Transformers.js</h3>
<p>
The same files load in Transformers.js 4.3 with <code>dtype: "q4f16"</code>, with identical output. The loader above is faster at long
context, since it keeps the KV cache in one preallocated buffer.
</p>
<pre><code class="language-javascript">import { pipeline } from "@huggingface/transformers";
const generator = await pipeline("text-generation", "mohdmahmodi/Spark-X2.6-4B-ONNX-WebGPU", { device: "webgpu", dtype: "q4f16" });
const out = await generator([{ role: "user", content: "Hello!" }], {
max_new_tokens: 256, tokenizer_encode_kwargs: { enable_thinking: false },
});
console.log(out[0].generated_text.at(-1).content);</code></pre>
</li>
</ol>
<h2 id="built">How it is built</h2>
<p>
The export writes the ONNX graph operator by operator from the checkpoint, using the fused kernels onnxruntime-web ships for
WebGPU: grouped-query attention with a 512-token sliding window on 27 of the 36 layers, fused RMSNorm, and 4-bit matrix
multiplies. The weights are XHToken's own INT4 codes, repacked without change, so the browser build runs the same quantized model
the official checkpoint does. Round-to-nearest 4-bit was tried first and was far less accurate on this model.
</p>
<p>
The KV cache stays on the GPU in one buffer per layer, and a follow-up message reuses the cached conversation instead of reading
it again. The <a href="https://github.com/mohdmahmodi/Spark-X2.6-4B-onnx-webgpu">source</a> includes the export, the loader, the
chat app and the full validation suite.
</p>
<h2 id="limits">Limitations</h2>
<ul>
<li>It needs WebGPU on a desktop browser, about 3.5 GB of free GPU memory at 4k tokens of context, and GPU buffers of at least 336 MB. A lower storage-binding limit works with the included loader, down to the 128 MB WebGPU default.</li>
<li>The browser's WebAssembly (CPU) backend cannot run this graph: it has no kernel for the quantized embedding.</li>
<li>Testing so far covers Windows with an NVIDIA RTX 3090 Ti. macOS, Android and other GPUs are untested.</li>
<li>The comparison with other models is XHToken's, measured on the original model with thinking on. It was not re-run for this build. The <a href="https://github.com/mohdmahmodi/Spark-X2.6-4B-onnx-webgpu/blob/main/docs/VALIDATION.md">validation report</a> covers what was.</li>
</ul>
</article>
<footer class="footer">
<span class="author">Mohd Mahmodi</span>
<a href="https://github.com/mohdmahmodi"><svg class="icon"><use href="assets/icons.svg#github" /></svg>GitHub</a>
<a href="https://x.com/mohdmahmodi"><svg class="icon"><use href="assets/icons.svg#brand-x" /></svg>@mohdmahmodi</a>
<a href="https://huggingface.co/mohdmahmodi"><svg class="icon"><use href="assets/icons.svg#huggingface" /></svg>Hugging Face</a>
<span>Spark-X2.6-4B is developed by XHToken and released under Apache-2.0.</span>
</footer>
</div>
<div class="tooltip" id="tooltip" role="tooltip"></div>
</body>
</html>