| --- |
| license: apache-2.0 |
| base_model: lucadellalib/focalcodec_12_5hz |
| tags: |
| - focalcodec |
| - audio-codec |
| - quantization |
| - post-training-quantization |
| library_name: other |
| --- |
| |
| # FocalCodec 12.5 Hz — packed int8 / fp8 weights |
|
|
| Post-training quantized copies of [`lucadellalib/focalcodec_12_5hz`](https://huggingface.co/lucadellalib/focalcodec_12_5hz) |
| (145.3 M weights) for running the codec from a half-size checkpoint. Two formats: |
|
|
| | Folder | Stored dtype | Bytes/weight | Total size | Demo level | |
| | --- | --- | --- | --- | --- | |
| | `int8/` | `int8` + fp32 per-channel scale | 1 | 145.9 MB | `int8` | |
| | `fp8/` | `float8_e4m3fn` + fp32 per-channel scale | 1 | 145.9 MB | `fp8` | |
|
|
| Scales are per-output-channel absmax (per-tensor for 1-D params), so each stored value is |
| `round(w / scale)`. `fp8` uses the IEEE-ish E4M3-FN grid (finite max 448), which every stored |
| value sits on exactly — dequantizing returns bit-identical results to the fake-quantized model |
| used in the demo Space. |
|
|
| This is post-training quantization only: no fine-tuning, no calibration set. Decoding quality |
| at 0.16 kbps (12.5 tokens/s × 13 bits) is unchanged at int8/fp8 within measurement noise; |
| see the [quantization lab Space](https://huggingface.co/spaces/ALWOAL/focalcodec-quantization) |
| for the full int4 … int32 / fp4 … fp32 sweep. |
|
|
| ## Load |
|
|
| ```python |
| import torch.hub |
| from huggingface_hub import hf_hub_download |
| from safetensors.torch import load_file |
| |
| def dequantize(path: str) -> dict: |
| raw = load_file(path) |
| return { |
| k: (v.float() if (s := raw.get("_scale/" + k)) is None else v.float() * s) |
| for k, v in raw.items() if not k.startswith("_scale/") |
| } |
| |
| codec = torch.hub.load("lucadellalib/focalcodec", "focalcodec", config="lucadellalib/focalcodec_12_5hz", trust_repo=True) |
| state = dequantize(hf_hub_download("ALWOAL/focalcodec_12_5hz_quantized", "int8/model.safetensors", repo_type="model")) |
| codec.load_state_dict(state) |
| codec.eval() |
| ``` |
|
|
| Quantization reference: `core.int_levels` / `core.fp_levels` in the Space's `core.py`. |
|
|
| ## Native int8 — no dequantization |
|
|
| `int8nn.py` (shipped in this repo) swaps every Linear/Conv1d/Embedding for an int8 twin: |
| weights stay int8 in memory, each GEMM quantizes its activations per row on the fly and |
| computes `int8 x int8 -> int32` via `torch._int_mm` (W8A8 dynamic). The dequantize path |
| above is only for kernels without int8 support — `fp8/` has no int8 GEMM path at all. |
|
|
| ```python |
| import torch.hub |
| from huggingface_hub import hf_hub_download |
| from safetensors.torch import load_file |
| import int8nn # from this repo |
| |
| codec = torch.hub.load("lucadellalib/focalcodec", "focalcodec", config="lucadellalib/focalcodec_12_5hz", trust_repo=True).eval() |
| packed = load_file(hf_hub_download("ALWOAL/focalcodec_12_5hz_quantized", "int8/model.safetensors", repo_type="model")) |
| int8nn.apply_int8(codec, packed) |
| ``` |
|
|
| Measured on a 3 s LibriSpeech clip: STOI 0.722 running natively versus 0.701 through the |
| dequantized fake-quant path (fp32: 0.745) — the Space's `int8-native` row. |
|
|
