abhimittal commited on
Commit
f2b37cf
·
verified ·
1 Parent(s): ef68078

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +25 -4
README.md CHANGED
@@ -1,10 +1,31 @@
1
  ---
2
  title: Speculative Decoding Trace
3
- emoji: 👀
4
- colorFrom: green
5
- colorTo: gray
6
  sdk: static
 
7
  pinned: false
 
 
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  title: Speculative Decoding Trace
3
+ emoji: 🎯
4
+ colorFrom: blue
5
+ colorTo: green
6
  sdk: static
7
+ app_file: index.html
8
  pinned: false
9
+ license: mit
10
+ short_description: Watch rejection sampling accept/reject every draft token
11
  ---
12
 
13
+ # Speculative decoding, token by token
14
+
15
+ Two layers, both client-side (static Space — free tier, no ZeroGPU quota,
16
+ no cold starts):
17
+
18
+ 1. **Interactive explainer** (`index.html`) — drag toy draft/target
19
+ distributions and watch the acceptance probability min(1, p/q), the
20
+ residual norm(max(p − q, 0)), the losslessness histogram, and the
21
+ `E[tokens] = (1 − α^{γ+1})/(1 − α)` speedup curve respond live.
22
+ 2. **Real inference in your browser** (`live.html`) — quantized ONNX
23
+ `distilgpt2` drafts, `gpt2` verifies, via transformers.js
24
+ (WebGPU/WASM). The actual rejection sampling on actual logits, with the
25
+ per-position math rendered for every token.
26
+
27
+ The repo also ships a Gradio app with KV-cached speculative decoding,
28
+ draft-vs-target **attention-trace diffing**, and an optional 70B cloud
29
+ cross-check — run it locally with `python app.py`.
30
+
31
+ Source: https://github.com/aabhimittal/attention-trace-differ-for-speculative-decoding