Ming-3.0-audio-tiny is a compact native audio-language model from inclusionAI. It accepts audio together with text instructions and supports speech recognition, spoken question answering, audio understanding, audio reasoning, and optional synthesized-speech responses.
## Model Summary
| Property | Description |
| --- | --- |
| Developer | inclusionAI |
| Model type | Compact native audio-language Mixture-of-Experts model |
| Parameters | 9.3B total parameters, 2.7B activated per inference step |
| Inputs | Audio and text instructions |
| Outputs | Text, or text with optional synthesized speech |
| Primary languages | Chinese and English, including varied accents and dialects |
| License | MIT |
## Highlights
- **Efficient audio intelligence:** Provides speech and audio understanding in a compact Mixture-of-Experts model.
- **Complex speech recognition:** Handles Chinese and English speech, dialects, accents, noisy environments, and multi-speaker recordings.
- **Long-audio understanding:** Comprehends and analyzes long-form recordings.
- **Multidimensional perception:** Understands speech, non-speech sounds, acoustic scenes, and music.
- **Audio reasoning:** Answers questions and performs reasoning over audio content.
- **Audio interaction:** Produces text responses and can optionally synthesize speech with built-in voices or a reference voice.
## Architecture
Ming-3.0-audio-tiny combines an audio encoder and projector with a compact Mixture-of-Experts language backbone. An optional Talker generates speech responses from the model's output.
## Quick Start with Transformers
The model uses custom Transformers code. Review the repository code before enabling `trust_remote_code=True`.
```bash
git clone https://github.com/inclusionAI/Ming-3.0-audio-flash.git
cd Ming-3.0-audio-flash
python -m venv .venv-transformers
source .venv-transformers/bin/activate
pip install -r requirements-transformers.txt
```
### Audio Understanding
```bash
python examples/transformers_audio.py examples/audio/introduce_ant_group.wav \
--model inclusionAI/Ming-3.0-audio-tiny \
--prompt "Please answer the question in the audio."
```
### Speech Recognition
```bash
python examples/transformers_audio.py examples/audio/introduce_ant_group.wav \
--model inclusionAI/Ming-3.0-audio-tiny \
--task asr \
--max-new-tokens 256 \
--no-repeat-ngram-size 10
```
The validated ASR prompt asks the model to identify the spoken language and place a tab between the language label and transcription.
### Direct Python Usage
```python
import torch
from transformers import AutoModelForCausalLM, AutoProcessor
model_id = "inclusionAI/Ming-3.0-audio-tiny"
audio_path = "examples/audio/introduce_ant_group.wav"
processor = AutoProcessor.from_pretrained(
model_id,
trust_remote_code=True,
use_fast=False,
)
model = AutoModelForCausalLM.from_pretrained(
model_id,
dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
).eval()
messages = [
{
"role": "user",
"content": [
{"type": "audio", "audio": audio_path},
{"type": "text", "text": "Please answer the question in the audio."},
],
}
]
inputs = processor(messages).to(model.device)
inputs["audio_feats"] = inputs["audio_feats"].to(model.dtype)
with torch.inference_mode():
output_ids = model.generate(
**inputs,
max_new_tokens=512,
do_sample=False,
)
generated_ids = output_ids[0, inputs["input_ids"].shape[1] :]
response = processor.decode(generated_ids, skip_special_tokens=True).strip()
print(response)
```
## Speech Output
The Talker example supports built-in voices and reference-audio voice prompting:
```bash
python examples/transformers_thinker_talker.py \
--model inclusionAI/Ming-3.0-audio-tiny \
--audio examples/audio/introduce_ant_group.wav \
--voice US_Stella \
--output output.wav
```
See the [voice guide](https://github.com/inclusionAI/Ming-3.0-audio-flash/blob/main/docs/voices.md) for available voice names, invocation rules, and reference-audio usage.
## Serving
### vLLM-Omni
Text output or ASR uses the Tiny Thinker on one GPU:
```bash
export MODEL_PATH=inclusionAI/Ming-3.0-audio-tiny
MING_MODEL_VARIANT=tiny MING_OUTPUT_MODE=text \
./scripts/serve_vllm_omni.sh "$MODEL_PATH"
```
For synthesized-speech output, select the full-audio profile:
```bash
export MODEL_PATH=inclusionAI/Ming-3.0-audio-tiny
MING_MODEL_VARIANT=tiny MING_OUTPUT_MODE=audio \
./scripts/serve_vllm_omni.sh "$MODEL_PATH"
```
The reference full-audio profile places the Thinker on GPU 0 and the Talker on GPU 1. Select the deployment YAML that matches text-only, ASR, or Talker output. See the [deployment guide](https://github.com/inclusionAI/Ming-3.0-audio-flash/blob/main/README.md#vllm-omni) for details.
Select an IP or registered voice with the vLLM-Omni client's `--voice` option:
```bash
python examples/vllm_omni_audio_chat.py examples/audio/introduce_ant_group.wav \
--audio-response \
--voice US_Stella \
--output-audio outputs/answer.wav
```
The client sends this selection as Chat Completions `audio.voice: {"id": "..."}`. vLLM-Omni uses the built-in IP voice `灵小柔` when no voice is specified. The model directory also provides `灵楷` and `US_James` as preset reference voices under `voice_presets/voice_audio/`, with their matching transcripts in `voice_presets/voice_prompts.json`. Both names are also available as built-in IP voices.
### SGLang
```bash
export MODEL_PATH=inclusionAI/Ming-3.0-audio-tiny
MING_MODEL_VARIANT=tiny CUDA_VISIBLE_DEVICES=0 \
./scripts/serve_sglang.sh "$MODEL_PATH"
```
The validated Tiny SGLang profile uses one GPU and provides text output. See the [SGLang example](https://github.com/inclusionAI/Ming-3.0-audio-flash/blob/main/examples/sglang_audio_chat.py) for client usage.
### SGLang-Omni
[SGLang-Omni](https://sgl-project.github.io/sglang-omni/) serves transcription, audio chat, and speech synthesis from one server. The Tiny checkpoint fits on a single GPU; `--talker.gpu 0` colocates speech synthesis with the other stages:
```bash
sgl-omni serve --model-path inclusionAI/Ming-3.0-audio-tiny --talker.gpu 0
```
The server exposes OpenAI-compatible `/v1/audio/transcriptions`, `/v1/audio/speech`, and `/v1/chat/completions` endpoints. The [SGLang-Omni cookbook](https://sgl-project.github.io/sglang-omni/cookbook/ming3_audio.html) shows client examples; the same requests work with the Tiny model name.
## Evaluation
The shared benchmark figures below include results for both Ming-3.0-audio-tiny and Ming-3.0-audio-flash.
### Speech Recognition
### Audio Intelligence
### Text Benchmarks
All benchmark values are percentages. Higher is better unless the metric is marked with a downward arrow. Results marked with an asterisk are self-evaluated. Refer to the [GitHub README](https://github.com/inclusionAI/Ming-3.0-audio-flash#benchmarks) for the latest notes and evaluation details.
## License
This model is released under the [MIT License](https://github.com/inclusionAI/Ming-3.0-audio-flash/blob/main/LICENSE). See [LEGAL.md](https://github.com/inclusionAI/Ming-3.0-audio-flash/blob/main/LEGAL.md) for the legal disclaimer.
GitHub
