Spaces:
Running on Zero
Running on Zero
| title: FocalCodec Quantization Lab | |
| emoji: ⚡ | |
| colorFrom: blue | |
| colorTo: purple | |
| sdk: gradio | |
| sdk_version: 6.30.0 | |
| app_file: app.py | |
| short_description: FocalCodec weights at int4–int32 vs fp4–fp32 | |
| python_version: "3.12" | |
| startup_duration_timeout: 1h | |
| # ⚡ FocalCodec 12.5 Hz — weight quantization lab | |
| Interactive demo of what happens to [`lucadellalib/focalcodec_12_5hz`](https://huggingface.co/lucadellalib/focalcodec_12_5hz) | |
| when its **145 M weights** are stored at different numeric precisions: | |
| `int4` · `int8` · `int16` · `int32` · `fp4` · `fp8` · `fp16` · `fp32` | |
| The token stream is untouched, so the codec still runs at **0.1625 kbps** | |
| (12.5 tokens/s × 13 bits) at every level — only the size of the network changes | |
| (72.6 MB at 4-bit down to 581.1 MB at fp32). | |
| ## Three views | |
| | Tab | What it shows | | |
| | --- | --- | | |
| | **Compare** | One reconstructed clip + input/reconstruction spectrograms, side by side for all 8 FocalCodec levels | | |
| | **Playground** | Encode → decode a single clip at one chosen codec + weight format, with its metrics row | | |
| | **Benchmark** | SI-SDR, STOI, LSD, stored model size, bitrate and real-time factor for all 18 levels — 8 FocalCodec + 8 LSCodec + 2 native-int8 rows — plus a summary chart | | |
| ## Method | |
| Post-training **fake quantization** of every floating-point tensor: per-output-channel | |
| absmax scale, round-to-nearest in the target format, dequantize back to fp32, then run | |
| the untouched codec forward pass. `fp4` is IEEE-style E2M1, `fp8` is E4M3, `fp16` | |
| and `fp32` are exact casts. Nothing is fine-tuned or calibrated on data. | |
| The `int8-native` and `lscodec-int8-native` rows skip the dequantize step entirely: | |
| the same int8 weights run through a real **W8A8 dynamic** runtime (`int8nn.py`, | |
| shipped with the packed checkpoints) — weights stay int8 in memory, each GEMM | |
| quantizes its activations per row and computes `int8 × int8 → int32` via | |
| `torch._int_mm`. Their STOI/LSD track the plain `int8` rows. | |
| The same sweep runs on a second codec, [LSCodec-25 Hz](https://github.com/X-LANCE/LSCodec-Inference), | |
| whose rows are prefixed `lscodec-`: its encoder, vocoder and codebook are quantized | |
| identically, while the WavLM-Large features it self-prompts with are not. It is a | |
| different model at a different bitrate (0.25 kbps vs 0.1625 kbps), so read it as a | |
| floor, not a control. | |
| ⚠️ **On SI-SDR:** it is deeply negative at every level, including `fp32`. That is | |
| expected — at 0.16 kbps FocalCodec does not preserve waveform phase, and the | |
| [FocalCodec paper](https://arxiv.org/abs/2502.04465) reports SI-SNR ≈ −37 dB for the | |
| untouched model. Use **STOI** (intelligibility, higher is better) and **LSD** | |
| (log-spectral distance, lower is better) to compare levels. | |
| ## Credits | |
| Model, method and example audio by | |
| [Luca Della Libera](https://huggingface.co/lucadellalib) et al. — Apache-2.0. | |
| [FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networks](https://arxiv.org/abs/2502.04465), NeurIPS 2025. | |
| The packed `int8` / `fp8` checkpoints produced by this lab live in | |
| [`ALWOAL/focalcodec_12_5hz_quantized`](https://huggingface.co/ALWOAL/focalcodec_12_5hz_quantized), | |
| and the packed LSCodec int8 files in [`ALWOAL/lscodec_25hz_int8`](https://huggingface.co/ALWOAL/lscodec_25hz_int8) — | |
| both ship `int8nn.py` for native W8A8 inference. | |
| The baseline codec is [LSCodec-Inference](https://github.com/X-LANCE/LSCodec-Inference) by | |
| Yiwei Guo et al. — MIT. | |