Blame for README.md

---
license: cc-by-nc-4.0
base_model: google/gemma-4-12B-it
tags:
- full-duplex
- speech
- gemma-4
- csm
- lora
---

# Parley

Parley is a full-duplex speech model. It listens while it talks and decides every 80 ms
whether to speak, keep speaking, or listen.

It's the model from the blog post
[Inside Frankenstein's lab: a full-duplex model at home](https://walln.dev/blog/full-duplex-at-home/),
which covers how it works, how it was trained, and where it falls short.

## What it is

- **Thinker:** Gemma 4 12B (audio-native, no speech encoder), adapted with LoRA. It hears
  16 kHz audio in 40 ms slices and writes text or control tokens on an 80 ms clock: `SIL`
  while listening, `EPS` while still speaking, or words when it has them.
- **Talker:** CSM-1B (Sesame) speaks the words Gemma writes, in chunks, through a playout
  queue that handles interruption and yielding.
- **Codec:** Mimi (Kyutai) provides the audio codes CSM renders.

This repo holds the thinker's LoRA adapter and processor. CSM-1B and Mimi are used as
released.

## Training

- **Supervised:** synthetic spoken conversations, with replies rewritten to sound like
  speech and training examples where the user interrupts mid-reply. User audio is synthetic
  voices only.
- **Reinforcement learning:** GRPO on the decision to speak only, on mid-sentence pause clips
  from Meta's Seamless Interaction, heard over room tone. This is what teaches the model to
  wait through pauses. The released adapter is the step-20 save of that run.

## Intended use

Research, experimentation, and demos of full-duplex voice interaction. Not a production
voice agent. The code is at https://github.com/walln/parley, including a live demo you can
run yourself: `parley serve` starts a full-duplex server with a browser page to talk to it.

## Limitations

- Trained on synthetic voices only; recorded human speech is weaker.
- It waits through pauses well, but answers later than it should at the end of a turn.
- One voice (the CSM reference clip).
- Gemma gets one text token per frame, so long text and tool calls don't fit.
- It stops for any interruption, including backchannels like "okay".
- Research-only weights.

## Components and licenses

- Gemma 4 12B (`google/gemma-4-12B-it`): Apache 2.0.
- CSM-1B (`sesame/csm-1b`): Apache 2.0, built on Llama 3.2 (Llama Community License).
- Mimi (`kyutai/mimi`): CC BY 4.0.

## Training data and attributions

- [Seamless Interaction](https://ai.meta.com/research/seamless-interaction/) (Meta): CC BY-NC 4.0.
  Pause clips for reinforcement learning.
- [databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) (Databricks): CC BY-SA 3.0.
- [alpaca-cleaned](https://huggingface.co/datasets/yahma/alpaca-cleaned): CC BY 4.0. Answers originally written by an OpenAI model.
- [no_robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots) (Hugging Face H4): CC BY-NC 4.0.
- [hh-rlhf](https://huggingface.co/datasets/Anthropic/hh-rlhf) (Anthropic), helpful split: MIT. Source of the multi-turn conversations.
- Training replies were partly rewritten by a teacher model.

## Weights license

Research-only, under CC BY-NC 4.0. Commercial use requires a separate agreement.

## Citation

```bibtex
@misc{wall2026fullduplex,
  author = {Nick Wall},
  title = {Inside Frankenstein's lab: a full-duplex model at home},
  year = {2026},
  month = {oct},
  howpublished = {\url{https://walln.dev/blog/full-duplex-at-home/}},
  note = {Blog post}
}
```