Marimba

A System One model for speech. Ask typed questions about an eight-second audio window and get typed answers: noul (yes/no), score (a probability) or choice (a distribution over your options). One pass of a frozen Qwen3-ASR-0.6B encoder serves every question.

Quickstart

pip install "marimba-audio[torch]"

from marimba import Marimba, Question

model = Marimba.from_pretrained(backend="torch", device="cpu", dtype="float32")
questions = [
    Question.score("Has the speaker finished?"),
    Question.noul("Is someone speaking?"),
    Question.choice("What language?", ["en", "pt", "es"]),
    Question.noul('Did someone say "hello"?'),
    Question.choice("How many speakers?", ["0", "1", "2", "3+"]),
]
for decision in model("speech.wav", questions):
    print(decision.to_dict())

What it answers

Question	Output	Status	Result
Turn end	score	validated	AUROC 0.930 on the Smart Turn v3.2 test (Smart Turn v3.2: 0.982, in-distribution for it); 0.716 vs. 0.712 on ICSI meetings, a tie
Voice activity	noul	validated	AUROC 0.995 on AMI, 0.9996 on VoxConverse
Language	choice	validated	30 languages; closed-set accuracy 100% on FLEURS, 93% on Common Voice (300 clips each)
Term	noul	validated	AUROC 0.92 on Common Voice, 0.97 on FLEURS; scores are uncalibrated (raw fraction of greedy + four beam hypotheses)
Same speaker	noul / score	validated	AUROC 0.998 on AMI, 0.996 on VoxConverse; on par with ECAPA (0.985, 0.995)
Speaker change	noul / score	validated	AUROC 0.966 on AMI, 0.980 on VoxConverse; ECAPA 0.924, 0.962
Speaker count (0/1/2/3+)	choice	validated	Accuracy 0.778 on AMI, 0.859 on VoxConverse; pyannote 3.1 0.738, 0.836; undercounts 3+
Personal information	noul	experimental	Needs your text PII detector (none included); F1 0.904 on 200 synthetic pt-BR clips with an in-house detector

Speaker questions are per-window decisions: same speaker compares the two four-second halves, speaker change compares the last two seconds with the previous six. They are not diarization and produce no identities or timelines.

On one NVIDIA L4 (bf16, batch size 1), p95 latency per question is 13 ms for turn end, 16 ms for voice activity, 161 ms for language and 2.35 s for term detection.

How it works

Term questions score the frozen decoder's greedy and four-beam hypotheses by exact match, so any word can be asked without training; the score is the fraction of the five hypotheses that contain it. Language uses Qwen3-ASR's native language scores. Voice activity uses a small question-conditioned head; turn end and the speaker questions use linear or small MLP readouts on frozen encoder layers. Personal information passes the greedy transcript to a detector you supply.

Limitations

• Windows are eight seconds; decoder context above eight seconds is unvalidated.
• Term detection inherits ASR misses on rare words.
• Voice-activity prompts were evaluated in Portuguese and in held-out Arabic, Hindi, Korean and Russian (AUROC at least 0.98); recognized questions use the canonical prompt.
• Personal information was evaluated only on synthetic Brazilian Portuguese speech.
• Sound and scene classification is not supported; the head is at chance on CochlScene.
• Calibration shifts across domains: on ICSI, 200 labeled windows per fold reduced turn-end ECE from 0.200 to 0.011.
• Not for diagnosis, identity verification, surveillance or safety-critical automation.

License and credits

Head and readout weights are Apache-2.0.

• Qwen/Qwen3-ASR-0.6B (Apache-2.0), revision 5eb144179a02acc5e5ba31e748d22b0cf3e303b0: frozen encoder and decoder.
• pipecat-ai/smart-turn-data-v3.2-train (CC BY 4.0), revision f8c95d284595defb6b88a715c356cbc5c31b7ad1: turn-end readout training data. No dataset audio is distributed here.

Solano Amadori, charcot.ai