scripts/README.md

Conversation app on a self-hosted LLM

Point the Reachy Mini conversation app at your own LLM instead of OpenAI. Optional — nothing here is part of the ReachyPi image.

The conversation app does not speak Chat Completions — it speaks the OpenAI Realtime API over a WebSocket (speech in, speech out). So a plain vLLM endpoint can't be plugged in directly. huggingface/speech-to-speech bridges the gap: it exposes a Realtime endpoint and internally chains VAD → STT → your LLM → TTS. Its README notes it "runs in production as the conversation backend for thousands of Reachy Mini robots".

Reachy Mini ──ws──> speech-to-speech ──http──> vLLM (Qwen3.5-9B-AWQ)
                    VAD + STT + TTS            :50000
                    (CPU)                      (GPU)

Run it

On the GPU box, with vLLM already up — two variants, each in its own venv, both strictly CPU:

./qwen-realtime.sh     # Qwen3-TTS — the default voices the Reachy app GUI offers
./kokoro-realtime.sh   # Kokoro-82M — lighter, but its own voice set (af_heart)

Either serves ws://<server>:8765/v1/realtime.

Point the robot at it

On the Pi, in the conversation app's .env:

HF_REALTIME_WS_URL=ws://<server-ip>:8765/v1/realtime

The path must end in /realtime — the app validates it. Use wss:// only if you terminate TLS in front (and then no port, or the real one — not :443 on a ws:// URL).

Gotchas

No Nvidia card needed. Qwen3-TTS runs on GGML; CUDA is only an optional accelerator. What assumes CUDA 12.8 is the qwentts-cpp-python wheel published on PyPI, so the script pulls the +cpu build from the upstream wheelhouse instead.

speech-to-speech is installed from git main. The GGML backend isn't in a PyPI release yet (latest is 0.2.10, git-tagged the same), so qwen-realtime.sh reinstalls from main on every launch. Swap for a plain pip install speech-to-speech once PyPI ships > 0.2.10.

VAD tuned for the robot's mic. The XVF3800 stream is quiet after beamforming: with the stock thresholds (--thresh 0.6, 384 ms of speech required) the VAD discarded whole utterances and the robot never answered. But the speaker sits centimeters from the mic, so the residual echo of the robot's own voice can be louder than a person across the room — no threshold separates the two, and with barge-in active the robot kept cutting itself mid-sentence. So qwen-realtime.sh disables voice barge-in in the launcher (the app's stop button still interrupts) and runs a sensitive --thresh 0.3 --min_speech_ms 256. The trade-off: you can't interrupt the robot by talking over it — wait for the end of its sentence.

pip install speech-to-speech can drag the CUDA torch back in, even after installing the CPU trio — that CUDA torch crashes the VAD on a non-Nvidia box. qwen-realtime.sh therefore force-reinstalls the CPU torch stack last, uninstalls any nvidia-* leftovers, and asserts torch.__version__ ends in +cpu before launching.

Everything vocal runs on CPU (--device cpu). Parakeet TDT (0.6B) and Qwen3-TTS are small; the GPU stays dedicated to the LLM, which is where --gpu-memory-utilization 0.84 needs it.

TTS sized for CPU real-time. The default Qwen3-TTS (1.7B in BF16, 4.2 GB of GGUF weights) can't keep up on a CPU — the audio stutters. Quality lives in two places with opposite needs: the talker (prosody) tolerates hard quantization, while the codec (the actual waveform) is tiny (0.36 GB) and sounds metallic when quantized. So qwen-realtime.sh runs the 1.7B talker in Q4_K_M with the codec kept in BF16 (~1.5 GB total). Upstream has no CLI flag for any of this, so the script launches through a small wrapper that downloads the two GGUFs and passes their paths. Knobs at the top of the script: TTS_MODEL (drop to 0.6B-CustomVoice if it still stutters — same named voices), TTS_TALKER_QUANT, TTS_CODEC_QUANT.

Tool calls may not fire. vLLM runs --tool-call-parser qwen3_coder, which is tuned for the Coder models. On a Qwen3.5 instruct model the Reachy tools (dance, move_head, emotions) may be parsed wrong or ignored. If the robot talks but never moves, try --tool-call-parser hermes on the vLLM side.

The camera tool is disabled by vLLM's --limit-mm-per-prompt '{"image": 0, "video": 0}'. The robot won't be able to look at anything.