Document Whisper word timestamps and automatic speaker diarization

README.md

@@ -950,6 +950,7 @@ All protected examples below assume `MEDIAFORGE_API_KEY` is set and include `Aut
 | Image | `/v1/image/resize`, `crop`, `convert`, `slideshow`, `sequence`, `video`, `watermark`, `overlay` |
 | yt-dlp | `/v1/ytdlp/download` |
 | Whisper | `/v1/whisper/transcribe`, `subtitles`, `detect-language` |
+| Diarization | `/v1/diarization/diarize` — automatic speaker diarization using pyannote |
 | Templates | `GET /v1/templates`, `GET /v1/templates/categories`, `GET /v1/templates/{reference}`, `POST /v1/templates/run` |
 
 Common video parameters include `format`, `width`, `height`, `fit`, `crf`, `preset`, `start`, `end`, `duration`, `fps`, `bitrate`, `factor`, `position`, and `opacity`. Swagger lists every route; the defaults and bounds are enforced by the operation that consumes each field.
@@ -986,6 +987,69 @@ curl -X POST http://localhost:7860/v1/ytdlp/download \
 
 Models: `tiny`, `base`, `small`, `medium`, and `large-v3`. The default is `small`. Models load only on their first request and always use `device="cpu"` and `compute_type="int8"`. Tasks are `transcribe` and `translate`; omit `language` for automatic language detection. Formats are `txt`, `srt`, `vtt`, `json`, and `tsv`.
 
+#### Word-level timestamps and speaker diarization
+
+Whisper transcription supports canonical word-level timestamps and optional speaker attribution. The JSON transcript schema is versioned and includes segment-level and word-level timing; when diarization is applied, segments and words can carry a stable speaker label such as `SPEAKER_00`.
+
+Diarization can be supplied in three ways:
+
+1. **Automatic remote diarization** — set `diarization=true` and configure MediaRouter's `DIARIZATION_URL` and `DIARIZATION_API_KEY`. The backend can call this Space's `POST /v1/diarization/diarize` endpoint.
+2. **Inline spans** — provide a `diarization` array containing objects with `speaker`, `start`, and `end`.
+3. **Diarization JSON input** — provide a second input containing a JSON document with `spans`, `segments`, or `speakers`.
+
+The dedicated diarization API in this Space uses `pyannote/speaker-diarization-3.1`, converts the source media to mono 16 kHz PCM for analysis, and returns normalized speaker spans:
+
+```json
+{
+  "spans": [
+    {"speaker": "SPEAKER_00", "start": 0.0, "end": 3.42},
+    {"speaker": "SPEAKER_01", "start": 3.42, "end": 7.18}
+  ],
+  "speakers": ["SPEAKER_00", "SPEAKER_01"],
+  "filename": "interview.mp4"
+}
+```
+
+The diarization endpoint is protected by the normal API-key authentication/scope layer and additionally requires `DIARIZATION_API_KEY` to be configured on the Space. The configured key must have `operations:execute`. Access to the gated pyannote model also requires `HF_TOKEN` with the model's access granted.
+
+**Diarization endpoint:**
+
+```http
+POST /v1/diarization/diarize
+Authorization: Bearer <api-key>
+Content-Type: application/octet-stream
+X-Filename: interview.mp4
+X-Min-Speakers: 2
+X-Max-Speakers: 4
+```
+
+The request body is the raw media bytes. `X-Min-Speakers` and `X-Max-Speakers` are optional bounds from 1–32. The service enforces bounded input/response sizes and returns normalized spans ordered by start time. It does not expose model or subprocess internals in API errors.
+
+Example:
+
+```bash
+curl -X POST "https://<owner>-<space>.hf.space/v1/diarization/diarize" \
+  -H "Authorization: Bearer $MEDIAFORGE_API_KEY" \
+  -H "Content-Type: application/octet-stream" \
+  -H "X-Filename: interview.mp4" \
+  --data-binary "@interview.mp4"
+```
+
+For a complete transcription with speaker labels through MediaRouter, enable diarization in the transcription request:
+
+```bash
+curl -X POST "https://<media-router-space>.hf.space/v1/whisper/transcribe" \
+  -H "Authorization: Bearer $MEDIAFORGE_API_KEY" \
+  -F "file=@interview.mp4" \
+  -F "model=small" \
+  -F "output_format=json" \
+  -F "word_timestamps=true" \
+  -F "diarization=true"
+```
+
+The resulting canonical JSON contains `schema_version: "1.1"`, `speakers`, and speaker fields on matching segments/words. SRT/VTT continue to use the canonical timing data; use JSON when downstream automation needs machine-readable speaker attribution.
+
+
 ```bash
 curl -X POST http://localhost:7860/v1/whisper/transcribe \
   -H "Authorization: Bearer $MEDIAFORGE_API_KEY" \
@@ -1093,6 +1157,11 @@ For large files, prefer n8n's multipart binary option or raw binary body; JSON B
 | `TEMPLATE_DIR` | built-in `app/templates/categories` | Recursively scanned YAML workflow catalog |
 | `MAX_UPLOAD_SIZE` | `1073741824` | Maximum bytes for each upload/download |
 | `WHISPER_MODEL` | `small` | Default faster-whisper model |
+| `DIARIZATION_API_KEY` | empty | Dedicated bearer secret for the diarization endpoint; must also correspond to an authenticated API key with `operations:execute` |
+| `DIARIZATION_MODEL_ID` | `pyannote/speaker-diarization-3.1` | pyannote diarization pipeline |
+| `DIARIZATION_MAX_INPUT_BYTES` | `1073741824` | Maximum diarization request body |
+| `DIARIZATION_MAX_RESPONSE_SPANS` | `10000` | Maximum normalized diarization spans returned |
+| `HF_TOKEN` | empty | Hugging Face token with access to the gated diarization model |
 | `CLEANUP_MINUTES` | `60` | TTL for request workspaces and outputs |
 | `CLEANUP_INTERVAL_SECONDS` | `60` | Cleanup scan interval |
 | `MAX_WORKERS` | `2` | Shared per-service CPU process/task concurrency |
@@ -1150,6 +1219,8 @@ This contract keeps new operations independent of multipart, URLs, Base64, n8n,
 
 - **Space is out of memory:** use `tiny`, `base`, or `small`; set `MAX_WORKERS=1`; do not launch multiple Uvicorn workers. `medium` and `large-v3` require substantially more RAM even with int8.
 - **First transcription is slow:** the model is downloaded and initialized lazily. Use persistent Space storage or pre-warm with a short request after deployment.
+- **Speaker diarization returns 503:** configure `DIARIZATION_API_KEY` on the Space and ensure the bearer key is a valid authenticated API key with `operations:execute`; configure `HF_TOKEN` and accept access to `pyannote/speaker-diarization-3.1`. The model is loaded lazily on the first diarization request.
+- **Speaker diarization is unavailable through MediaRouter:** set MediaRouter `DIARIZATION_URL` to `https://<owner>-<space>.hf.space/v1/diarization/diarize` and configure the same `DIARIZATION_API_KEY`. Keep the URL HTTPS in production.
 - **FFmpeg reports incompatible streams during concat:** leave `stream_copy` false (the default), which normalizes dimensions and re-encodes. Enable it only for files with identical stream layout, codec, time base, and parameters.
 - **Remote URL is rejected:** private and non-global addresses are blocked intentionally. Set `ALLOW_PRIVATE_URLS=true` only in a trusted network and never on a public Space.
 - **Download URL returns 404:** the request TTL expired, the Space restarted without persistent storage, or the filename/request ID was changed. Download results promptly.