anokimchen/BW_illustrations_t2i_512x512
Viewer ā¢ Updated ā¢ 504k ā¢ 465
Imchen
anokimchen
AI & ML interests
None yet
Recent Activity
updated a dataset about 19 hours ago
anokimchen/BW_illustrations_t2i_512x512 liked a dataset about 20 hours ago
limingcv/LAION_Aesthetics_1024 liked a model 8 days ago
anokimchen/sd-turbo-openvinoOrganizations
None yet
updated a
dataset about 19 hours ago
liked a
dataset about 20 hours ago
liked a
model 8 days ago
reacted to SeaWolf-AI's post with š 13 days ago
Post
5398
šµ VKUE ā No GPU? Runs anyway.
"Frontier models need a datacenter GPU" rests on a hidden assumption: that the model reads ALL its parameters every token. Decode is memory-bandwidth bound ā sweep 34B params/token and an 8 GB card dies at 1ā2 tok/s.
So we ran ONE 34.7B reasoning model ā Ourbox-35B-JGOS, a sparse Mixture-of-Experts ā as the identical weights across the whole hardware spectrum. All measured:
ā¢ B200: 18,057 tok/s (aggregate)
ā¢ 1Ć A10G: 126 tok/s
ā¢ 8 GB laptop (RTX 5060): 20 tok/s
ā¢ GPU-less CPU: 17 tok/s
Why it works: Ourbox holds 34.7B params but only ~3B are active per token (256 experts, top-8). Since decode is bandwidth-bound, a dense 34B moves ~16.7 GB/token while Ourbox moves ~1.45 GB ā ~11Ć less traffic. Put the experts in system RAM, keep attention/router/shared on the GPU, and a 34.7B reasoner runs on an 8 GB laptop ā or no GPU at all.
Sparsity alone, proven (same laptop, same quant, ~same footprint): Ourbox-35B (A3B) 20.01 tok/s vs Qwen2.5-32B (dense) 5.36 ā 3.7Ć from sparsity alone, ~2Ć the best dense-32B on any 8 GB machine. Not a toy: GPQA Diamond 86.4% (maj@8).
Try it live (same prompt, GPU vs GPU-less CPU, live tok/s). Honest scope: one machine's measurements; the CPU path proves it RUNS without a GPU, not that it beats one.
š Article: https://huggingface.co/blog/FINAL-Bench/vkue
šµ GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space/
šµ CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
š VKUE leaderboard: FINAL-Bench/VKUE
š¤ Model: FINAL-Bench/Ourbox-35B-JGOS-GGUF
ā” VKAE (speed): VIDraft/vkae
VKUE is the "runs anywhere" side of our serving line; VKAE the "fast on datacenter GPUs" side. VKAE is fast; VKUE is everywhere.
"Frontier models need a datacenter GPU" rests on a hidden assumption: that the model reads ALL its parameters every token. Decode is memory-bandwidth bound ā sweep 34B params/token and an 8 GB card dies at 1ā2 tok/s.
So we ran ONE 34.7B reasoning model ā Ourbox-35B-JGOS, a sparse Mixture-of-Experts ā as the identical weights across the whole hardware spectrum. All measured:
ā¢ B200: 18,057 tok/s (aggregate)
ā¢ 1Ć A10G: 126 tok/s
ā¢ 8 GB laptop (RTX 5060): 20 tok/s
ā¢ GPU-less CPU: 17 tok/s
Why it works: Ourbox holds 34.7B params but only ~3B are active per token (256 experts, top-8). Since decode is bandwidth-bound, a dense 34B moves ~16.7 GB/token while Ourbox moves ~1.45 GB ā ~11Ć less traffic. Put the experts in system RAM, keep attention/router/shared on the GPU, and a 34.7B reasoner runs on an 8 GB laptop ā or no GPU at all.
Sparsity alone, proven (same laptop, same quant, ~same footprint): Ourbox-35B (A3B) 20.01 tok/s vs Qwen2.5-32B (dense) 5.36 ā 3.7Ć from sparsity alone, ~2Ć the best dense-32B on any 8 GB machine. Not a toy: GPQA Diamond 86.4% (maj@8).
Try it live (same prompt, GPU vs GPU-less CPU, live tok/s). Honest scope: one machine's measurements; the CPU path proves it RUNS without a GPU, not that it beats one.
š Article: https://huggingface.co/blog/FINAL-Bench/vkue
šµ GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space/
šµ CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
š VKUE leaderboard: FINAL-Bench/VKUE
š¤ Model: FINAL-Bench/Ourbox-35B-JGOS-GGUF
ā” VKAE (speed): VIDraft/vkae
VKUE is the "runs anywhere" side of our serving line; VKAE the "fast on datacenter GPUs" side. VKAE is fast; VKUE is everywhere.
- 7 replies
liked a
dataset 13 days ago
published a
dataset 13 days ago
liked 2
datasets 14 days ago
liked a
dataset 15 days ago
updated a
dataset 23 days ago
published a
dataset 28 days ago
liked 3
models 6 months ago
liked a
model 7 months ago
liked a
dataset 7 months ago
liked a
Space 7 months ago
liked a
model 8 months ago
updated a
Space 8 months ago
published a
Space 8 months ago