Spaces:
Running on CPU Upgrade
Running on CPU Upgrade
Commit History
22 days ago
-
Recommend methods by source: on the fly or from a checkpoint
The receipt now lists a curated set of methods per precision, in recommendation order, chosen by a new "quantized on the fly / from a checkpoint" slot in the sentence. Adds NVFP4, GGUF, AutoRound and Quark; drops HQQ, optimum-quanto, HIGGS and AQLM, which no list includes. Also drops the ?/* qualifiers from the receipt and the focus outline on the precision hit targets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>3a22edb
-
Build the quantization picker
An interactive replacement for the quantization method table in the Transformers docs. The table lists bits; the deciding arithmetic is bytes, and that is where people stall. This does the arithmetic. - lib/data.ts holds the methods and the memory model: weights from params × bits, KV cache from the attention shape, activations as a flat estimate. - lib/hardware.ts is scraped from huggingface.co/hardware, one entry per memory configuration (316 of them) rather than per device, since an RTX 3060 is sold as both 8GB and 12GB. - The model is typed as a Hub repo id and resolved through /api/model, so any model works, not a curated five. - MemoryBar draws the selected precision solid with fp16 behind it as a ghost, so quantizing shows its own effect. - TradeoffMap places the matching methods by when they quantize and whether they serialize — the axes that actually separate them, since every method at a given precision lands at the same footprint. Deploys to a Space via the D…5ae37d4