README.md

metadata

license: apache-2.0
base_model: lucadellalib/focalcodec_12_5hz
tags:
  - focalcodec
  - audio-codec
  - quantization
  - post-training-quantization
library_name: other

FocalCodec 12.5 Hz — packed int8 / fp8 weights

Post-training quantized copies of lucadellalib/focalcodec_12_5hz (145.3 M weights) for running the codec from a half-size checkpoint. Two formats:

Folder	Stored dtype	Bytes/weight	Total size	Demo level
int8/	int8 + fp32 per-channel scale	1	145.9 MB	int8
fp8/	float8_e4m3fn + fp32 per-channel scale	1	145.9 MB	fp8

Scales are per-output-channel absmax (per-tensor for 1-D params), so each stored value is round(w / scale). fp8 uses the IEEE-ish E4M3-FN grid (finite max 448), which every stored value sits on exactly — dequantizing returns bit-identical results to the fake-quantized model used in the demo Space.

This is post-training quantization only: no fine-tuning, no calibration set. Decoding quality at 0.16 kbps (12.5 tokens/s × 13 bits) is unchanged at int8/fp8 within measurement noise; see the quantization lab Space for the full int4 … int32 / fp4 … fp32 sweep.

Load

import torch.hub
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

def dequantize(path: str) -> dict:
    raw = load_file(path)
    return {
        k: (v.float() if (s := raw.get("_scale/" + k)) is None else v.float() * s)
        for k, v in raw.items() if not k.startswith("_scale/")
    }

codec = torch.hub.load("lucadellalib/focalcodec", "focalcodec", config="lucadellalib/focalcodec_12_5hz", trust_repo=True)
state = dequantize(hf_hub_download("ALWOAL/focalcodec_12_5hz_quantized", "int8/model.safetensors", repo_type="model"))
codec.load_state_dict(state)
codec.eval()

Quantization reference: core.int_levels / core.fp_levels in the Space's core.py.

Native int8 — no dequantization

int8nn.py (shipped in this repo) swaps every Linear/Conv1d/Embedding for an int8 twin: weights stay int8 in memory, each GEMM quantizes its activations per row on the fly and computes int8 x int8 -> int32 via torch._int_mm (W8A8 dynamic). The dequantize path above is only for kernels without int8 support — fp8/ has no int8 GEMM path at all.

import torch.hub
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
import int8nn  # from this repo

codec = torch.hub.load("lucadellalib/focalcodec", "focalcodec", config="lucadellalib/focalcodec_12_5hz", trust_repo=True).eval()
packed = load_file(hf_hub_download("ALWOAL/focalcodec_12_5hz_quantized", "int8/model.safetensors", repo_type="model"))
int8nn.apply_int8(codec, packed)

Measured on a 3 s LibriSpeech clip: STOI 0.722 running natively versus 0.701 through the dequantized fake-quant path (fp32: 0.745) — the Space's int8-native row.