OpenMOSS-Team

/

MOSS-TTS-GGUF

Like 29

Follow

OpenMOSS 1.38k

Context

Got MOSS-TTS-GGUF (Delay architecture, 8B, Q5_K_M) running end-to-end on
Windows, CPU-only (no CUDA), via the moss_tts_delay.llama_cpp + ONNX
torch-free pipeline. None of this seems to be covered in the current docs,
which appear to assume Linux + MSVC/gcc are interchangeable — they aren't,
on Windows. Sharing what we hit and how we fixed it, in case it saves
someone else the same afternoon.

Setup: Windows 11, no GPU used for inference, MinGW-w64 GCC (via
winget install BrechtSanders.WinLibs.POSIX.UCRT), CMake, VS Build Tools
(only needed transitively by CMake's toolchain detection).

1. Build llama.cpp with MinGW, not the default MSVC generator

cmake -B build on Windows defaults to the MSVC generator. The resulting
.lib/.dll import libraries are in a format GNU ld (used by
build_bridge.sh) cannot link against — you'll get cannot find -lllama
even though the file exists.

Fix: force the MinGW generator explicitly:

cmake -B build -G "MinGW Makefiles"
cmake --build build --config Release --target llama -j

Note --target llama specifically — building the default all target
fails on vendor/cpp-httplib's use of CreateFile2 under MinGW
('::CreateFile2' has not been declared). That's the bundled HTTP server,
unused by this pipeline, so skipping it is safe.

With this generator, the static libs land in build/src/*.a and
build/ggml/src/*.a rather than build/bin/, and the ggml*.a files need
a lib prefix added (copy or symlink) for GNU -l flag resolution.

2. build_bridge.sh needs several Windows/MinGW-specific edits

The stock script assumes a gcc -shared one-liner works. On Windows it
needs:

• gcc → g++ as the linker driver — the static libs contain C++
  symbols, and plain gcc won't auto-link libstdc++
  (undefined reference to '__gxx_personality_seh0').
• -Wl,--start-group -lllama -lggml -lggml-base -lggml-cpu -lllama-common-base -lvendor-hash -Wl,--end-group instead of a bare
  -lllama — these libs have circular dependencies that GNU ld can't
  resolve in a single left-to-right pass otherwise
  (undefined reference to 'ggml_free' etc.).
• -fopenmp — ggml-cpu is built with OpenMP; without this flag
  you'll get undefined reference to 'GOMP_*' / omp_get_thread_num.

3. The real gotcha: g++ compiles .c files as C++ by default

This one cost the most time. Even after fixing the link line, the built
DLL loaded fine via ctypes.CDLL(...) but every symbol lookup failed:
AttributeError: function 'bridge_create' not found.

Root cause: g++ always compiles a .c file in C++ mode unless told
otherwise, so the plain C functions in backbone_bridge.c
(bridge_create, bridge_get_logits, etc.) get C++ name-mangled in the
object file (e.g. bridge_create → _Z13bridge_createPKciiiiiii).
-Wl,--export-all-symbols doesn't help — it happily exports the mangled
names, just not the plain ones ctypes is looking for.

Fix: force C mode explicitly for that one file, then reset:

g++ ... -x c backbone_bridge.c -x none ...

Combined with an explicit .def file (safer/more deterministic than
--export-all-symbols on this mingw-w64 version) listing the real
exports:

EXPORTS
bridge_create
bridge_decode_embd
bridge_decode_embd_batch
bridge_get_embeddings
bridge_get_logits
bridge_n_embd
bridge_n_vocab
bridge_clear_kv
bridge_free

passed to the linker (bridge.def alongside the .c file in the link
command).

4. ctypes on Windows won't find the bridge's dependent DLLs

Python 3.8+ no longer searches PATH for a loaded DLL's own dependencies
on Windows. libbackbone_bridge.so (a PE DLL despite the extension) needs
the MinGW runtime DLLs (libgcc_s_seh-1.dll, libstdc++-6.dll,
libwinpthread-1.dll, libgomp-1.dll, etc., from the MinGW bin/
folder) copied into the same directory it lives in, or ctypes.CDLL
raises FileNotFoundError: Could not find module ... (or one of its dependencies).

5. moss_audio_tokenizer is a git submodule, not a standalone pip package

This is the one that isn't obvious at all from the docs. pipeline.py
does from moss_audio_tokenizer.onnx import OnnxAudioTokenizer, which
looks like it wants the standalone OpenMOSS/MOSS-Audio-Tokenizer repo
pip-installed separately. We tried exactly that
(git clone + pip install -e .) — it "succeeds" with no error, but
still raises ModuleNotFoundError: No module named 'moss_audio_tokenizer.onnx' at runtime, because that standalone repo's
pyproject.toml has py-modules = [] and no package-discovery config —
it isn't set up to be pip-installed on its own.

The actual answer is in MOSS-TTS/.gitmodules: moss_audio_tokenizer is
declared as a git submodule of the MOSS-TTS repo itself, pinned to a
specific commit. A plain git clone of MOSS-TTS leaves that directory
empty. Fix:

git submodule update --init --recursive

run from the MOSS-TTS repo root. That populates
MOSS-TTS/moss_audio_tokenizer/onnx/... directly, and since python -m moss_tts_delay.llama_cpp is run from that root (which Python adds to
sys.path), the import resolves correctly with zero extra pip installs.
Might be worth a one-line mention in the main README or the
llama_cpp/README.md.

Result

With all of the above, python -m moss_tts_delay.llama_cpp --config configs/llama_cpp/default.yaml --text "..." --output output.wav runs
fully CPU-only on Windows (Q5_K_M, 32GB RAM, no GPU), ~5.9 tok/s text
generation on a mid-range mobile CPU. Quality is good; the only rough
edge we found is loanwords in non-English text (e.g. English tech terms
inside Turkish sentences) sometimes get an odd pronunciation — respelling
them phonetically in the source text (rather than relying on the
--language tag) fixed it reliably in our testing.

A note on how this was put together

I went through this whole build step by step together with Claude
(Anthropic's AI assistant) — running each command myself, debugging as we
went, rather than following a pre-written script. I'm not a professional
C/C++ developer, so it's entirely possible something above is imprecise,
incomplete, or just wrong in a way I didn't catch. If you spot a mistake,
please point it out — happy to be corrected, and apologies in advance for
anything off. Just hoping the parts that did work save someone else the
time we spent.

Happy to answer questions if anyone hits the same walls — hope this saves
someone a few hours.