---
license: apache-2.0
library_name: voxcpm
pipeline_tag: text-to-speech
base_model: openbmb/VoxCPM2
base_model_relation: quantized
quantized_by: ShinePixelOrg
language:
  - zh
  - en
  - ar
  - my
  - da
  - nl
  - fi
  - fr
  - de
  - el
  - he
  - hi
  - id
  - ja
  - km
  - ko
  - lo
  - ms
  - no
  - pl
  - pt
  - ru
  - es
  - sw
  - sv
  - tl
  - th
  - tr
  - vi
tags:
  - text-to-speech
  - tts
  - multilingual
  - voice-cloning
  - nvfp4
  - gptq
  - quantized
---

# VoxCPM2 NVFP4 (GPTQ)

Weight-only NVFP4 quantization of [openbmb/VoxCPM2](https://huggingface.co/openbmb/VoxCPM2) revision `32279effe8c19989596f05d353d1447f51d9e915`, produced with [llm-compressor](https://github.com/vllm-project/llm-compressor) GPTQ and saved in [compressed-tensors](https://github.com/vllm-project/compressed-tensors) `nvfp4-pack-quantized` format.

These weights are a derivative of the official OpenBMB checkpoint. **Use is governed by the same Apache-2.0 license as the original weights.** Upstream model card: https://huggingface.co/openbmb/VoxCPM2. Project: https://github.com/OpenBMB/VoxCPM.

Only the two language-model backbones are quantized, and only their weights. Activations stay BF16. Attention `v_proj` and FFN `down_proj` stay BF16 in both language models, together with LocDiT, LocEnc, FSQ, the projection heads, the stop head and AudioVAE. This is the second revision of this repository: the first revision quantized every language-model matrix, including `v_proj` and `down_proj`, with plain min/max rounding.

## Size

The comparison is the weight payload: `model.safetensors` plus `audiovae.pth`. The AudioVAE file is byte-identical in both trees.

| | BF16 source | This checkpoint | Delta |
| --- | ---: | ---: | ---: |
| `model.safetensors` | 4.266 GiB | 2.623 GiB | −1.643 GiB, −38.5% |
| `audiovae.pth` | 0.351 GiB (359.5 MiB) | 0.351 GiB (359.5 MiB) | 0 |
| Weight total | 4.617 GiB | 2.974 GiB | −1.643 GiB, 64.4% of the source |

vLLM-Omni 0.30.0 reported **3.21 GiB** of weight memory for this checkpoint and **4.86 GiB** for the BF16 checkpoint. The engine log line to look for is `ATS VoxCPM2 quant: W4A4=0 A16=180 int4=0` (see Loading).

## What was quantized

| Precision | Modules |
| --- | --- |
| NVFP4, weights only | `q_proj`, `k_proj`, `o_proj`, `gate_proj`, `up_proj` in BaseLM (28 layers) and ResidualLM (8 layers). 180 modules |
| BF16 | `v_proj` and `down_proj` in both language models, token embeddings, norms, LocEnc, LocDiT, FSQ, projection heads, stop head |
| FP32 | AudioVAE (`audiovae.pth`, unchanged) |

Attention V and the FFN down projection are the two tensor families that community VoxCPM2 quantizations keep at higher precision: they carry the content-bearing part of the signal, and 4-bit rounding of them is what breaks long sentences. Keeping them dense is the main change from the first revision of this repository.

NVFP4 settings: 4-bit float weights, symmetric, tensor-group quantization, group size 16, per-group scales in `float8_e4m3fn`, plus one `weight_global_scale` per matrix. There is no `input_global_scale`, so activations are not quantized.

### Method

GPTQ, block size 128, dampening 0.01, static activation ordering, Hessians offloaded to CPU.

Calibration used 48 real VoxCPM2 synthesizations (216.8 seconds of 48 kHz audio) produced by the BF16 model at temperature 0, each paired with its own transcript, covering Chinese and English statements, questions, digit strings and long passages. For every sample the waveform was encoded by the model's own AudioVAE into latent patches, embedded by LocEnc, and passed through both language models with the same text-mask / audio-mask interleaving the server uses at inference time. No random-noise calibration was used.

One honest note on the numbers: the mean relative weight error against BF16 is 14.1% for this checkpoint and 9.5% for plain min/max rounding. That is expected. GPTQ minimizes output error `WX - ŴX` under real activations, which accepts larger individual weight changes. Audio below is what it optimizes, and audio is better.

## Audio comparison

Measured 2026-11-03. Both engines used temperature 0 and seed 42, zero-shot, no reference audio, same GPU, one request at a time. Every file is 48 kHz, 16-bit, mono, and neither side clipped. The engine is deterministic: the same BF16 model run twice scores 0.0% duration error, 1.00 contour correlation and 0.00 dB MCD, so every number below is quantization, not run-to-run noise.

MCD is mel-cepstral distortion in dB after aligning the candidate cepstral track onto the reference frame grid. Pitch-contour correlation is the Pearson correlation of 16 bins of voiced F0. Spectral similarity is the cosine of the mean log-mel spectrum.

| Sentence | Duration, BF16 → NVFP4 | Mean F0 | F0-contour correlation | Spectral similarity | MCD |
| --- | ---: | ---: | ---: | ---: | ---: |
| Short Chinese | 3.36s → 3.36s (0%) | 148 → 149 Hz | 0.96 | 0.999 | 12.4 dB |
| Chinese question | 4.32s → 4.00s (−7%) | 141 → 134 Hz | 0.85 | 0.997 | 20.2 dB |
| Numbers and time | 5.60s → 5.76s (+3%) | 145 → 132 Hz | 0.71 | 0.999 | 16.7 dB |
| Long Chinese | 17.44s → 17.92s (+3%) | 144 → 166 Hz | −0.17 | 0.999 | 21.7 dB |
| Chinese story | 34.24s → 35.04s (+2%) | 211 → 238 Hz | −0.38 | 0.990 | 20.2 dB |
| English paragraph | 17.12s → 17.12s (0%) | 119 → 121 Hz | 0.70 | 0.999 | 19.7 dB |
| Chinese news | 15.84s → 16.96s (+7%) | 144 → 131 Hz | −0.00 | 0.996 | 23.6 dB |
| Dialogue | 17.76s → 16.96s (−5%) | 268 → 273 Hz | 0.84 | 1.000 | 16.1 dB |
| **Mean** | **3.4% absolute drift** | — | **0.44** | **0.997** | **18.8 dB** |

Against the previous revision of this repository, which quantized all 252 language-model matrices with min/max rounding, on the same eight sentences:

| | Previous revision (252 matrices, RTN) | This checkpoint (180 matrices, GPTQ) |
| --- | ---: | ---: |
| Mean absolute duration drift | 7.7% | **3.4%** |
| Mean F0-contour correlation | 0.29 | **0.44** |
| Mean spectral similarity | 0.9952 | **0.9973** |
| Mean MCD | 17.96 dB | 18.84 dB |
| Language-model weight memory | 2.58 GiB | 3.21 GiB |

Short sentences are close to the source. Intonation on long passages is where quantization still shows: the two longest utterances keep their length and their timbre (0.990 and 0.999) but do not follow the BF16 pitch contour.

### Stop behaviour and repetition

Repetition and failure to stop come from the language model, so they were checked directly. On the long Chinese sentence this checkpoint produces **109 audio tokens, the same count as BF16**. Over 128 requests at concurrency 64 the mean is 59.8 tokens against 57.6 for BF16, with a maximum of 74 against 65. Across those 128 outputs the longest fixed-period repetition measured 1.08 seconds, against 1.12 seconds measured on the BF16 outputs from the same protocol. Per-utterance comparisons against BF16 score up to 1.04 seconds on the reference side, so these values are the measurement floor and not a looping defect.

## Output speed

Measured 2026-11-03 on one RTX PRO 6000 Blackwell (SM 120), vLLM-Omni 0.30.0, `max_num_seqs` 64, `max_model_len` 1024, temperature 0, seed 42. Each engine ran alone on the card. Both engines used their own deploy config: 6 GiB KV cache for BF16 and 2.25 GiB for this checkpoint, 174,752 and 65,536 tokens of capacity. Peak concurrent resident demand in these runs stayed under 4,000 tokens, so neither engine was scheduling against its cache ceiling.

Single stream, long Chinese sentence, one warmup call discarded, median of three. One audio token is 1/6.25 s of 48 kHz audio.

| | Wall | Audio | Tokens | Realtime | ms/token |
| --- | ---: | ---: | ---: | ---: | ---: |
| BF16 | 1.219 s | 17.44 s | 109 | 14.31× | 11.18 |
| This checkpoint | 1.168 s | 17.44 s | 109 | 14.93× | 10.71 |

Concurrency 64, 128 requests over three fixed paragraphs. The first pass captures the batch-64 decode graph and is not shown; the table is the warm second pass.

| | Elapsed | Audio | Realtime | Tokens | ms/token | p50 | p95 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| BF16 | 11.811 s | 1180.2 s | 99.9× | 7376 (mean 57.6) | 1.60 | 5.41 s | 6.13 s |
| This checkpoint | 11.402 s | 1225.6 s | 107.5× | 7660 (mean 59.8) | 1.49 | 5.52 s | 6.51 s |

Read this as a memory optimization with a small speed side effect, not as a speed switch. VoxCPM2's language-model GEMMs are narrow, the DiT accounts for roughly three times the per-token FLOPs of BaseLM and stays BF16, and SM 120 has no native FP4 path for weight-only Marlin, so the gain is single-digit percent. On this card the measured gain was 4% single stream and 7% per token at concurrency 64. Take the halved weight memory and the smaller KV budget; do not plan around a large latency win.

## Loading

Packed `weight_packed` tensors are not a drop-in for upstream `VoxCPM.from_pretrained`, and vLLM-Omni 0.30.0 does not install them on this architecture by itself. Serving needs a small runtime adapter that maps `base_lm.*` and `residual_lm.*` packed tensors onto the paged LM linears through `MarlinNvFp4LinearKernel`, leaves every other module on its dense weight, and reports what it installed.

Two log lines confirm a correct load:

```
ATS VoxCPM2 quant: W4A4=0 A16=180 int4=0
Model loading took 3.21 GiB memory
```

If the second line says `4.86 GiB` and the first line is missing, the adapter did not run inside the spawned engine process. That failure is silent and sounds like short loud clicks: the language model is left as an unloaded dense shell, so generation stops after one token (0.16 s of audio per request). The usual cause is an adapter that skips patching whenever `sys.argv[0] == "-c"`, because a multiprocessing spawn child starts as `['-c', '--multiprocessing-fork']`. Skip the healthcheck on its command text (`import urllib.request`), which does not match the spawn child.

With the adapter in place, `GET /health` returns 200 and `/v1/audio/speech` returns 48 kHz WAV that stops inside the context limit.

## Recipe

`recipe.yaml` in this repository records the modifier, target and ignore patterns. Quantization ran in a host environment with llmcompressor 0.10.0.1, compressed-tensors 0.14.0.1 and torch 2.11.0+cu130; serving was tested against the vLLM-Omni v0.30.0 image.