GSV.CPP — GPT-SoVITS V5 weights and server for gsv.cpp

Model files and prebuilt servers for gsv.cpp, a C++/ggml inference engine for GPT-SoVITS V5 only — v5turbo (4-step DMD) and v5dev (32-step). v2 / v3 / v4 checkpoints are not supported.

The engine runs the whole pipeline in C++ (AR → cond → DiT/CFM → vocoder) on a ggml Vulkan or CPU backend. The vocoder is a ggml HiFi-GAN with sub-pixel upsampling; ONNX Runtime is not used anywhere. A small HTTP service exposes the same API as upstream api_v2.py, so the GAG GUI can use it directly.

Models

file	role	notes
gsv-ar-f16.gguf	AR (s1v3)	fp16
gsv-ar-q8.gguf	AR (s1v3)	Q8_0, ~2x smaller, default
gsv-cond-f16.gguf	cond (enc_p / ref_enc / wns1 / rvq)	v5turbo
gsv-dit-f16all.gguf	DiT / CFM	v5turbo
gsv-refcode.gguf	reference encoder projection	v5turbo
gsv-v5dev-cond-f16.gguf	cond	v5dev
gsv-v5dev-dit-f16all.gguf	DiT / CFM	v5dev
gsv-v5dev-refcode.gguf	reference encoder projection	v5dev
gsv-bert-f16.gguf	BERT (Chinese RoBERTa)	shared
gsv-hubert-f16.gguf	CN-HuBERT	shared
gsv-vocoder-f32.gguf	HiFi-GAN vocoder, 48 kHz, sub-pixel (ggml, F32)	shared
resampy_kaiser_best.bin	resampling filter table	shared

gsv-vocoder-f32.gguf stores phase-major voc.upsub.* tensors. Older voc.ups.* layouts are not supported.

Server binaries

Prebuilt engines live in server/:

file	contents
server/gsv.cpp-windows-x64-vulkan.zip	gsv_server.exe + ggml DLLs (Vulkan, AVX2 baseline)
server/gsv.cpp-linux-x64-vulkan.tar.gz	gsv_server + libggml*.so* (Vulkan, AVX2 baseline, glibc 2.35+)

They contain the engine only: add the model files, the Python text sidecar, and a reference WAV. --device cpu works without a Vulkan driver.

The Windows build is the 0.1.2 engine (second-render mask fix, reference/text caching, absolute voice paths, working skip-first-step, CFG skip removed). The Linux tarball is still the previous build and does not include those fixes yet — build from source if you need them on Linux.

Windows one-click package

windows/GSV.CPP-win64-0.1.0.zip (3.16 GB, sha256 852eaceb994d7e7422c62cb8153e80f19b2d759382e744d98ba186b3f16fccab) is a complete portable folder: extract it anywhere and double-click Start GAG.bat.

Patch 0.1.2 - patch/GSV.CPP-win64-patch-0.1.2.zip (14 MB, sha256 a6733730f64b079d91bdd893957ae45c17d8b9dbfe83d569196e51a12c0834e0): the current patch, cumulative (it contains everything from 0.1.1). Extract it over the 0.1.0 folder - patch order does not matter and there is no need to download the full package again.

What it fixes:

• the second and later renders of a session came out garbled and much too long (an 18 s sentence became 49 s): the HuBERT attention mask was written once at graph-build time but lived in the reusable galloc arena, so the first inference overwrote it and every later one ran with a garbage mask. It now owns a persistent buffer; tests/test_prep_repeat.cpp guards it.
• the reference audio was re-preprocessed on every render (the GUI uploads it to a fresh temp path each time, which always missed a path-keyed cache). The cache is keyed on the file content now; reference-side decode_encp and the text frontend (G2P + BERT) are memoised too. Repeats drop by ~0.6-1.0 s per render (prep 700->0 ms, cond 650->450 ms, g2p 200->0 ms).
• switching to the v5dev / v5turbo voice failed with engine load failed for voice: v5dev when the server was started with a relative --models (which the GUI does). Model paths are resolved to absolute paths, and a missing file is now named in the error.
• the GUI voice dropdown only ever listed default: the voice-list thread's result signal raced with deleteLater and was dropped. Fixed, with retries, and voice <-> model-version are now synchronised in both directions.
• "skip first step" was silently ignored on /tts (only the batch paths forwarded it).
• the CFG-skip switch was removed: it changed the timbre noticeably, so it is gone from the engine, the API, the log and the GUI. Requests that still send dit_cfg_off_t are ignored.
• the [tts] log line now reports the effective cfg / steps / voice version and the cache reuse counters, which is what made several of the above visible.

Patch 0.1.1 is superseded by this one; patch/GSV.CPP-win64-patch-0.1.1.zip stays online for reference (mixed-language modes, auto_yue, non-ASCII reference paths, clear language-mismatch errors, mp3/flac/ogg/m4a references in the GUI).

The package contains a fork of GAG (GPT-SoVITS-Api-GUI by 领航员未鸟, MIT - see the credits below) adapted to drive gsv.cpp, runtime/gsv (engine + DLLs + VC runtime), an embedded Python with the GUI and frontend dependencies, runtime/frontend (text sidecar code), and runtime/models (turbo-q8 and v5dev GGUF, frontend/g2pw, frontend/bert_tokenizer, nltk_data). No reference audio is bundled: choose a 3-10 s clip in the GUI.

Frontend data (language G2P)

The frontend code ships with the engine (frontend/); this repository carries its data:

path	point the sidecar at it with
frontend/g2pw/ (g2pW.onnx + dicts, ~608 MB)	GSV_G2PW_MODEL
frontend/bert_tokenizer/ (vocab.txt + configs)	GSV_BERT_DIR
frontend/nltk_data.tar.gz (extract first)	NLTK_DATA
frontend/requirements-zh.txt, frontend/requirements-multi.txt	pip install -r

Recommended layout (the sidecar finds these without environment variables):

models/
  frontend/
    g2pw/
    bert_tokenizer/
  nltk_data/

See frontend/README.md for details and licenses.

Layout

v5turbo setup:

models/
  gsv-ar-q8.gguf
  gsv-cond-f16.gguf
  gsv-dit-f16all.gguf
  gsv-refcode.gguf
  gsv-bert-f16.gguf
  gsv-hubert-f16.gguf
  gsv-vocoder-f32.gguf
  resampy_kaiser_best.bin
  frontend/g2pw/...
  frontend/bert_tokenizer/...

Additional voices (v5dev or your own fine-tunes) are described by a manifest, e.g. models/voices/v5dev/manifest.json:

{
  "name": "v5dev",
  "version": "v5dev",
  "ar": "gsv-ar-q8.gguf",
  "cond": "gsv-v5dev-cond-f16.gguf",
  "dit": "gsv-v5dev-dit-f16all.gguf",
  "refcode": "gsv-v5dev-refcode.gguf",
  "bert": "gsv-bert-f16.gguf",
  "hubert": "gsv-hubert-f16.gguf",
  "vocoder": "gsv-vocoder-f32.gguf"
}

Usage

# text frontend (zh; add requirements-multi.txt for en/ja/ko/yue)
python -m venv .venv-sidecar
.venv-sidecar/bin/pip install -r frontend/requirements-zh.txt
export GSV_G2PW_MODEL=./models/frontend/g2pw
export GSV_BERT_DIR=./models/frontend/bert_tokenizer
export NLTK_DATA=./models/nltk_data
.venv-sidecar/bin/python tools/text_sidecar.py --port 9882

# engine
./gsv_server --device vulkan --port 9880   # or --device cpu

Documentation in the engine repository:

• docs/build.md — build gsv.cpp from zero (Linux/Windows)
• docs/build.zh.md — 从零构建 gsv.cpp（中文）
• docs/usage.md — artifacts, models, conversion, running the service
• docs/api.md — HTTP API, streaming, references, error codes
• docs/benchmark.md — full-pipeline timings

Credits

This project builds on other people's work; the pieces redistributed in this repository are:

• GAG - GPT-SoVITS-Api-GUI by 领航员未鸟 (AliceNavigator), MIT. The Windows one-click package bundles a modified fork (KakaruHayate/GPT-SoVITS-Api-GUI) whose backend tab launches gsv_server instead of api_v2.py, with V5-specific controls (voice / sample steps / cache-dit / skip-first-step). Its license text ships as licenses/GAG-LICENSE.txt and as LICENSE inside the package.
• GPT-SoVITS (MIT): model checkpoints, and the vendored text frontend under frontend/GPT_SoVITS/text.
• G2PW (Apache-2.0): polyphone disambiguation model and code.
• llama.cpp / ggml (MIT): the inference backend (built from a patched tree, see the build guide).

If you redistribute the Windows package, keep the bundled LICENSE (GAG, MIT) and the notices in THIRD_PARTY_NOTICES.md.

License

The engine code is MIT (see LICENSE). The weights are derived from GPT-SoVITS checkpoints (MIT), the vendored text frontend keeps its upstream licenses (GPT-SoVITS MIT, G2PW Apache-2.0); see THIRD_PARTY_NOTICES.md and licenses/.