| --- |
| license: cc-by-nc-4.0 |
| base_model: google/gemma-4-12B-it |
| tags: |
| - full-duplex |
| - speech |
| - gemma-4 |
| - csm |
| - lora |
| --- |
| |
| # Parley |
|
|
| Parley is a full-duplex speech model. It listens while it talks and decides every 80 ms |
| whether to speak, keep speaking, or listen. |
|
|
| It's the model from the blog post |
| [Inside Frankenstein's lab: a full-duplex model at home](https://walln.dev/blog/full-duplex-at-home/), |
| which covers how it works, how it was trained, and where it falls short. |
|
|
| ## What it is |
|
|
| - **Thinker:** Gemma 4 12B (audio-native, no speech encoder), adapted with LoRA. It hears |
| 16 kHz audio in 40 ms slices and writes text or control tokens on an 80 ms clock: `SIL` |
| while listening, `EPS` while still speaking, or words when it has them. |
| - **Talker:** CSM-1B (Sesame) speaks the words Gemma writes, in chunks, through a playout |
| queue that handles interruption and yielding. |
| - **Codec:** Mimi (Kyutai) provides the audio codes CSM renders. |
|
|
| This repo holds the thinker's LoRA adapter and processor. CSM-1B and Mimi are used as |
| released. |
|
|
| ## Training |
|
|
| - **Supervised:** synthetic spoken conversations, with replies rewritten to sound like |
| speech and training examples where the user interrupts mid-reply. User audio is synthetic |
| voices only. |
| - **Reinforcement learning:** GRPO on the decision to speak only, on mid-sentence pause clips |
| from Meta's Seamless Interaction, heard over room tone. This is what teaches the model to |
| wait through pauses. The released adapter is the step-20 save of that run. |
|
|
| ## Intended use |
|
|
| Research, experimentation, and demos of full-duplex voice interaction. Not a production |
| voice agent. The code is at https://github.com/walln/parley, including a live demo you can |
| run yourself: `parley serve` starts a full-duplex server with a browser page to talk to it. |
|
|
| ## Limitations |
|
|
| - Trained on synthetic voices only; recorded human speech is weaker. |
| - It waits through pauses well, but answers later than it should at the end of a turn. |
| - One voice (the CSM reference clip). |
| - Gemma gets one text token per frame, so long text and tool calls don't fit. |
| - It stops for any interruption, including backchannels like "okay". |
| - Research-only weights. |
|
|
| ## Components and licenses |
|
|
| - Gemma 4 12B (`google/gemma-4-12B-it`): Apache 2.0. |
| - CSM-1B (`sesame/csm-1b`): Apache 2.0, built on Llama 3.2 (Llama Community License). |
| - Mimi (`kyutai/mimi`): CC BY 4.0. |
|
|
| ## Training data and attributions |
|
|
| - [Seamless Interaction](https://ai.meta.com/research/seamless-interaction/) (Meta): CC BY-NC 4.0. |
| Pause clips for reinforcement learning. |
| - [databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) (Databricks): CC BY-SA 3.0. |
| - [alpaca-cleaned](https://huggingface.co/datasets/yahma/alpaca-cleaned): CC BY 4.0. Answers originally written by an OpenAI model. |
| - [no_robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots) (Hugging Face H4): CC BY-NC 4.0. |
| - [hh-rlhf](https://huggingface.co/datasets/Anthropic/hh-rlhf) (Anthropic), helpful split: MIT. Source of the multi-turn conversations. |
| - Training replies were partly rewritten by a teacher model. |
|
|
| ## Weights license |
|
|
| Research-only, under CC BY-NC 4.0. Commercial use requires a separate agreement. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{wall2026fullduplex, |
| author = {Nick Wall}, |
| title = {Inside Frankenstein's lab: a full-duplex model at home}, |
| year = {2026}, |
| month = {oct}, |
| howpublished = {\url{https://walln.dev/blog/full-duplex-at-home/}}, |
| note = {Blog post} |
| } |
| ``` |
|
|
