zylin12 commited on
Commit
1694c49
·
verified ·
1 Parent(s): 0b0295a

Update README model capabilities

Browse files
Files changed (1) hide show
  1. README.md +2 -62
README.md CHANGED
@@ -28,7 +28,7 @@ pipeline_tag: audio-text-to-text
28
 
29
  MOSS-Transcribe-Diarize 0.9B is an end-to-end audio understanding model for long-form multi-speaker transcription, diarization, timestamps, and acoustic event awareness.
30
 
31
- It supports transcription and diarization across 50+ languages.
32
 
33
  Given an audio or video file, the model generates a compact speaker-aware transcript in one pass, including timestamps and anonymous speaker labels such as `[S01]`, `[S02]`, and beyond.
34
 
@@ -44,7 +44,6 @@ Given an audio or video file, the model generates a compact speaker-aware transc
44
  - [Quickstart](#quickstart)
45
  - [Environment Setup](#environment-setup)
46
  - [Python Usage](#python-usage)
47
- - [Custom Prompt and Hotwords](#custom-prompt-and-hotwords)
48
  - [Serve with vLLM and SGLang](#serve-with-vllm-and-sglang)
49
  - [Subtitle Web App](#subtitle-web-app)
50
  - [Output Format](#output-format)
@@ -70,14 +69,6 @@ Core capabilities:
70
  <img src="Model_Architecture.png" alt="MOSS-Transcribe-Diarize 0.9B model architecture" width="900">
71
  </p>
72
 
73
- | Component | Specification |
74
- |---|---|
75
- | Text backbone | Qwen3-0.6B style causal decoder |
76
- | Audio encoder | Whisper-Medium encoder configuration |
77
- | Audio frontend | `WhisperFeatureExtractor`, 16 kHz, 80 mel bins, 30 s chunks |
78
- | Audio-text bridge | 4x temporal merge + MLP adaptor |
79
- | Fusion | Audio features replace <code>&lt;&#124;audio_pad&#124;&gt;</code> embeddings via `masked_scatter` |
80
- | Output format | Compact `[start][Sxx]text[end]` transcript with speaker tags such as `[S01]` |
81
 
82
  This Hugging Face repository includes the custom Transformers remote code required to load the model with `trust_remote_code=True`.
83
 
@@ -308,27 +299,6 @@ curl -X POST http://localhost:8000/v1/audio/transcriptions \
308
  -F response_format=verbose_json
309
  ```
310
 
311
- ```python
312
- import requests
313
-
314
- with open("audio.wav", "rb") as f:
315
- resp = requests.post(
316
- "http://localhost:8000/v1/audio/transcriptions",
317
- data={
318
- "model": "OpenMOSS-Team/MOSS-Transcribe-Diarize",
319
- "response_format": "verbose_json",
320
- },
321
- files={"file": ("audio.wav", f, "audio/wav")},
322
- timeout=300,
323
- )
324
-
325
- resp.raise_for_status()
326
- payload = resp.json()
327
- print(payload["text"])
328
- for segment in payload.get("segments", []):
329
- print(f"[{segment['start']:.2f}-{segment['end']:.2f}] {segment['text']}")
330
- ```
331
-
332
  For longer multi-speaker audio, raise `max_new_tokens` so the decoder can finish the full diarized transcript:
333
 
334
  ```bash
@@ -339,37 +309,7 @@ curl -X POST http://localhost:8000/v1/audio/transcriptions \
339
  -F max_new_tokens=65536
340
  ```
341
 
342
- | Parameter | Type | Default | Description |
343
- |---|---|---|---|
344
- | `file` | file | required | Audio file uploaded as multipart form data |
345
- | `model` | string | server default | Model identifier |
346
- | `language` | string | unset | Optional language hint |
347
- | `response_format` | string | `json` | `json`, `verbose_json`, or `text` |
348
- | `temperature` | float | model default (`0.0`) | Sampling temperature |
349
- | `max_new_tokens` | int | `5120` | Max generated tokens; raise for long audio, for example `65536` |
350
- | `prompt` | string | unset | Optional instruction override; omit to use the built-in transcribe+diarize prompt |
351
-
352
- For benchmarking, performance numbers, and implementation details, see the [SGLang Omni cookbook](https://github.com/sgl-project/sglang-omni/blob/main/docs/cookbook/moss_transcribe_diarize.md). The following single-H100 results are reported for short- and long-sequence multi-speaker ASR tasks.
353
-
354
- `movies` short-sequence ASR:
355
-
356
- | Concurrency | Throughput (req/s) | Mean latency (s) | RTF mean | audio_s/s |
357
- |---:|---:|---:|---:|---:|
358
- | 1 | 2.57 | 0.388 | 0.0612 | 29.76 |
359
- | 2 | 4.89 | 0.409 | 0.0659 | 56.55 |
360
- | 4 | 6.62 | 0.513 | 0.0790 | 76.64 |
361
- | 8 | 6.80 | 0.533 | 0.0810 | 78.70 |
362
- | 16 | 7.08 | 0.659 | 0.0922 | 81.98 |
363
-
364
- `aishell4_long` long-sequence ASR:
365
-
366
- | Concurrency | Throughput (req/s) | Mean latency (s) | RTF mean | audio_s/s |
367
- |---:|---:|---:|---:|---:|
368
- | 1 | 0.022 | 45.2 | 0.0197 | 50.64 |
369
- | 2 | 0.032 | 60.7 | 0.0265 | 74.25 |
370
- | 4 | 0.036 | 105.6 | 0.0461 | 81.64 |
371
- | 8 | 0.040 | 172.6 | 0.0754 | 90.62 |
372
- | 16 | 0.043 | 282.8 | 0.1237 | 98.83 |
373
 
374
  ### Subtitle Web App
375
 
 
28
 
29
  MOSS-Transcribe-Diarize 0.9B is an end-to-end audio understanding model for long-form multi-speaker transcription, diarization, timestamps, and acoustic event awareness.
30
 
31
+ It supports transcription and diarization across 50+ languages, single-pass inference on audio recordings up to 90 minutes long, and custom hotword prompting for domain-specific terms.
32
 
33
  Given an audio or video file, the model generates a compact speaker-aware transcript in one pass, including timestamps and anonymous speaker labels such as `[S01]`, `[S02]`, and beyond.
34
 
 
44
  - [Quickstart](#quickstart)
45
  - [Environment Setup](#environment-setup)
46
  - [Python Usage](#python-usage)
 
47
  - [Serve with vLLM and SGLang](#serve-with-vllm-and-sglang)
48
  - [Subtitle Web App](#subtitle-web-app)
49
  - [Output Format](#output-format)
 
69
  <img src="Model_Architecture.png" alt="MOSS-Transcribe-Diarize 0.9B model architecture" width="900">
70
  </p>
71
 
 
 
 
 
 
 
 
 
72
 
73
  This Hugging Face repository includes the custom Transformers remote code required to load the model with `trust_remote_code=True`.
74
 
 
299
  -F response_format=verbose_json
300
  ```
301
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
302
  For longer multi-speaker audio, raise `max_new_tokens` so the decoder can finish the full diarized transcript:
303
 
304
  ```bash
 
309
  -F max_new_tokens=65536
310
  ```
311
 
312
+
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
313
 
314
  ### Subtitle Web App
315