crele/nimbus-v1

Crele Nimbus v1

A real-time audio decision model. Ask yes/no questions about live audio in plain English, and get an answer every 40 ms, plus events when an answer starts and stops. It also knows who is speaking: the user, or someone else.

Stream audio in at any chunk size; each answer arrives about a tenth of a second after the moment it describes. Give it a few seconds of the user's voice, and two built-in signals follow the user and everyone else, live (see The user's voice).

Built for voice agents: barge-in, end of turn, background sounds, another voice in the room.

Website: crele.co/models/nimbus · Code: github.com/Crele-AI/nimbus

The user's voice

Most voice activity detectors answer one question: is anyone speaking? Nimbus answers two, because a voice agent needs to know whose voice it is:

Signal	Answers
user	Is the user speaking? The voice you enrolled
other	Is someone else speaking? Any other voice: another person in the room, or a voice from a TV or a loudspeaker

Both are always on, answered every 40 ms beside your own questions, with events when they start and stop.

What it makes possible:

• Barge-in from the user only. Stop talking when the user interrupts, not when someone in the background says something.
• The user's end of turn, even while other people are still talking.
• Talking over each other. Both signals can be yes at once, so you can see when someone speaks over the user, or the user over them.
• Someone joins. Notice another person or a TV voice, and adapt.

Enrollment takes a few seconds of the user speaking, recorded live or from a file (the playground records five). A speaker-embedding model compares the voices in the stream with that one.

session.enroll(user_voice_16k)     # float32, mono, 16 kHz, a few seconds of speech

for update in session.push(pcm_16k):
    for event in update.events:
        if event.question == "user" and event.kind == "started":
            ...                    # the user started talking: stop the agent's speech

Without an enrollment, the model has no reference: user answers cant_tell, and other cannot tell the user apart from anyone else. Enroll before relying on either signal.

Shown live on a microphone: the user alone, another voice near and far, turn taking, and overlapping speech.

The enrolled voice becomes a voiceprint, which is personal data in many places: get the user's consent, and store it as you would any personal data.

What you can ask

Any yes/no question about what can be heard right now, in your own words:

• "Is music playing?"
• "Is someone typing on a keyboard?"
• "Is someone whistling?"
• "Is a siren wailing?"

Every answer is yes, no or cant_tell, with its probability. The two voice signals, user and other, are always on beside them.

Quick start

pip install git+https://github.com/Crele-AI/nimbus

from crele_nimbus import Nimbus

model = Nimbus.from_pretrained("crele/nimbus-v1")
session = model.session()
session.ask("music", "Is music playing?")
session.ask("whistle", "Is someone whistling?", threshold=0.5)
session.enroll(voice_16k)          # optional: a few seconds of the user's voice (see above)

for update in session.push(pcm_16k):   # float32, mono, 16 kHz, any chunk length
    for event in update.events:
        print(event.question, event.kind, event.start_s)

To try it in a browser with your microphone or an audio file:

python -m crele_nimbus.playground

The first run downloads about 1.5 GB: this model and the text encoder it uses.

Tuning

Each question has four settings. The defaults suit most questions; change them per question.

Setting	Default	What it does
threshold	0.70	An event starts when the probability rises above this
start_secs	0.12	... and stays up this long
stop_threshold	0.35	The event stops when the probability falls below this
stop_secs	0.30	... and stays down this long

• A sound is missed: lower threshold for that question, for example to 0.5.
• A question fires too often: raise threshold.
• An event does not stop: the probability is resting above stop_threshold; raise it.
• Speech turns split at short pauses: raise stop_secs, for example to 1 s.

Speed

• About 0.7 seconds of compute per second of audio (real-time factor 0.7) on an Apple M4 Pro laptop CPU, one stream, with 8 or with 32 questions.
• Asking more questions costs almost nothing: going from 8 to 32 questions adds about 1 ms per 80 ms step. The audio is processed once per stream; every question reads from it.
• An answer arrives 80 to 120 ms of audio after the moment it describes, plus compute.

How well it works

Measured on 500 real AudioSet clips with the default settings:

• Says yes about 0.2 s after a sound starts: a median of 0.24 s over the sounds it caught. For speech, 0.21 s, with 97% of 181 speech onsets caught.
• Ask in your own words: questions worded in ways it never trained on scored 0.892 against 0.911 for its own wording (mean AUROC over 40 sounds, the same clips).
• Quiet in a silent room: the highest resting answer across twelve sound questions was 0.03.
• Tells the user's voice from other voices: shown live on a microphone with the user alone, another voice near and far, turn taking, and overlapping speech (see The user's voice).

Limits

• English questions only.
• One user voice per session. Without a voice clip, the two voice signals have no reference.
• Audio is 16 kHz mono. Each answer is about the last few seconds of audio.
• Some sounds are recognised better than others; tune threshold for the ones that matter to you, and test on your own audio.

How it is built

• A streaming audio encoder distilled from Dasheng-Base.
• A speaker-embedding model, CAM++ from 3D-Speaker, for the two voice signals.
• Qwen3-Embedding-0.6B, unchanged, which reads each question once.
• A small decision head that answers every question from the shared audio.

About 97M parameters run on the audio. The text encoder (0.6B) runs once per question.

Trained with AudioSet and AudioSet-Strong, the AudioSet ontology, LibriSpeech, MUSAN and FSD50K (CC0 and CC BY clips). The models and datasets, with their licences, are listed in NOTICE.

Files

File	What
listener.safetensors	the streaming audio encoder
head.safetensors	the decision head
speaker.safetensors	the speaker-embedding model
config.json	the settings the model runs with
LICENSE, NOTICE	the licence and the attributions

Licence

Apache-2.0. See LICENSE and NOTICE.

Citation

@misc{crele2026nimbus,
  title  = {Crele Nimbus v1: a real-time audio decision model},
  author = {Crele},
  year   = {2026},
  url    = {https://huggingface.co/crele/nimbus-v1}
}