Ping Long commited on
Commit
ee33001
Β·
unverified Β·
2 Parent(s): 9667cc6cd2c99f

Merge pull request #1 from SiliconLanguage/copilot/scaffold-open-asr-model-explorer

Browse files
.gitignore CHANGED
@@ -1,3 +1,9 @@
 
 
 
 
 
 
1
  # Byte-compiled / optimized / DLL files
2
  __pycache__/
3
  *.py[codz]
 
1
+ # Node / frontend
2
+ node_modules/
3
+ frontend/node_modules/
4
+ frontend/dist/
5
+ frontend/.vite/
6
+
7
  # Byte-compiled / optimized / DLL files
8
  __pycache__/
9
  *.py[codz]
README.md CHANGED
@@ -1,2 +1,228 @@
1
- # model-explorer-open-asr
2
- A hybrid testbed for evaluating top open-source ASR models (Cohere, Qwen3-ASR, IBM Granite) featuring vLLM server-side routing and transformers.js WebGPU client-side inference.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: Open-ASR Model Explorer
3
+ emoji: πŸŽ™
4
+ colorFrom: indigo
5
+ colorTo: purple
6
+ sdk: docker
7
+ app_port: 8000
8
+ pinned: false
9
+ ---
10
+
11
+ # Open-ASR Model Explorer
12
+
13
+ [![Deploy to Hugging Face Spaces](https://img.shields.io/badge/πŸ€—%20Deploy-Hugging%20Face%20Spaces-orange?style=for-the-badge)](https://huggingface.co/spaces)
14
+ [![Python](https://img.shields.io/badge/Python-3.11-blue?style=for-the-badge&logo=python)](https://python.org)
15
+ [![FastAPI](https://img.shields.io/badge/FastAPI-0.120-green?style=for-the-badge&logo=fastapi)](https://fastapi.tiangolo.com)
16
+ [![React](https://img.shields.io/badge/React-18-61DAFB?style=for-the-badge&logo=react)](https://react.dev)
17
+ [![vLLM](https://img.shields.io/badge/vLLM-0.18-purple?style=for-the-badge)](https://vllm.ai)
18
+
19
+ A hybrid inference testbed for evaluating top open-source ASR models (Cohere, Qwen3-ASR, IBM Granite) featuring vLLM server-side routing and transformers.js WebGPU client-side inference.
20
+
21
+ ---
22
+
23
+ ## Architecture
24
+
25
+ ```
26
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
27
+ β”‚ Browser (React/Vite) β”‚
28
+ β”‚ β”‚
29
+ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” Strategy Router β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
30
+ β”‚ β”‚ Model Select │──────────────────► β”‚ InferRouter β”‚ β”‚
31
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
32
+ β”‚ β”‚ β”‚
33
+ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β” β”‚
34
+ β”‚ β”‚ β”‚ β”‚
35
+ β”‚ "WebGPU" in name? else β”‚
36
+ β”‚ β”‚ β”‚ β”‚
37
+ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β” β”‚
38
+ β”‚ β”‚ WebWorker β”‚ β”‚ fetch() β”‚ β”‚
39
+ β”‚ β”‚ (transforms β”‚ β”‚ SSE β”‚ β”‚
40
+ β”‚ β”‚ .js ONNX) β”‚ β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ β”‚
41
+ β”‚ β”‚ LocalAgrmt2 β”‚ β”‚ β”‚
42
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚
43
+ β”‚ β”‚ β”‚ β”‚
44
+ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β” β”‚
45
+ β”‚ β”‚ Metrics Dashboard β”‚ β”‚
46
+ β”‚ β”‚ TTFT Β· ITL Β· RTFx β”‚ β”‚
47
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
48
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
49
+ β”‚ HTTP POST (multipart/SSE)
50
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
51
+ β”‚ FastAPI Backend β”‚
52
+ β”‚ β”‚
53
+ β”‚ Audio Normaliser β”‚
54
+ β”‚ (pad/truncate 30s) β”‚
55
+ β”‚ β”‚ β”‚
56
+ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β” β”‚
57
+ β”‚ β”‚ vLLM Engine β”‚ β”‚
58
+ β”‚ β”‚ chunked β”‚ β”‚
59
+ β”‚ β”‚ prefill β”‚ β”‚
60
+ β”‚ β”‚ max_tokens β”‚ β”‚
61
+ β”‚ β”‚ =2048 β”‚ β”‚
62
+ β”‚ β”‚ gpu_util β”‚ β”‚
63
+ β”‚ β”‚ =0.88 β”‚ β”‚
64
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
65
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
66
+ ```
67
+
68
+ ### Strategy-Based Routing
69
+
70
+ | Model | Execution Mode | Description |
71
+ |---|---|---|
72
+ | `Cohere-transcribe-03-2026` | **Server (vLLM)** | Routes to FastAPI backend |
73
+ | `Qwen3-ASR-1.7B` | **Server (vLLM)** | Routes to FastAPI backend |
74
+ | `ibm-granite/granite-4.0-1b-speech` | **Server (vLLM)** | Routes to FastAPI backend |
75
+ | `Cohere-Transcribe-WebGPU` | **Client (WebGPU)** | Runs in-browser via transformers.js |
76
+ | `Whisper-Large-v3-Turbo-WebGPU` | **Client (WebGPU)** | Whisper Large v3 Turbo in-browser via transformers.js |
77
+
78
+ ### WebGPU Client-Side Features
79
+ - **Zero-server transcription** – audio never leaves the browser
80
+ - **Persistent ONNX cache** – model weights cached via the browser Cache API
81
+ - **LocalAgreement-2 streaming** – unconfirmed partial hypotheses are held back until two consecutive passes agree on a prefix
82
+
83
+ ### Backend vLLM Configuration
84
+ - **Chunked Prefill** – `enable_chunked_prefill=True` prevents long audio prefills from starving decode steps
85
+ - **Pre-batching normalisation** – all audio padded/truncated to 30 s before entering the scheduler
86
+ - **Resource constraints** – `max_num_batched_tokens=2048`, `gpu_memory_utilization=0.88`
87
+
88
+ ---
89
+
90
+ ## Project Structure
91
+
92
+ ```
93
+ model-explorer-open-asr/
94
+ β”œβ”€β”€ docker-compose.yml # Full-stack orchestration (backend + frontend)
95
+ β”œβ”€β”€ backend/
96
+ β”‚ β”œβ”€β”€ app.py # FastAPI application with vLLM integration
97
+ β”‚ β”œβ”€β”€ requirements.txt # Python dependencies
98
+ β”‚ └── Dockerfile # CUDA 12.1 container image
99
+ └── frontend/
100
+ β”œβ”€β”€ Dockerfile # Multi-stage Vite build β†’ Nginx serve
101
+ β”œβ”€β”€ nginx.conf # Nginx: API proxy + SSE streaming + SPA fallback
102
+ β”œβ”€β”€ index.html
103
+ β”œβ”€β”€ package.json
104
+ β”œβ”€β”€ vite.config.js
105
+ └── src/
106
+ β”œβ”€β”€ main.jsx
107
+ β”œβ”€β”€ App.jsx
108
+ β”œβ”€β”€ index.css
109
+ β”œβ”€β”€ components/
110
+ β”‚ β”œβ”€β”€ ModelSelector.jsx
111
+ β”‚ β”œβ”€β”€ AudioRecorder.jsx
112
+ β”‚ β”œβ”€β”€ MetricsDashboard.jsx
113
+ β”‚ └── TranscriptDisplay.jsx
114
+ β”œβ”€β”€ services/
115
+ β”‚ └── inferenceRouter.js # Strategy-based routing logic
116
+ └── workers/
117
+ └── webgpu.worker.js # transformers.js + LocalAgreement-2
118
+ ```
119
+
120
+ ---
121
+
122
+ ## Metrics
123
+
124
+ | Metric | Definition |
125
+ |---|---|
126
+ | **TTFT** (Time-to-First-Token) | Time from request submission to the first generated token |
127
+ | **ITL** (Inter-Token Latency) | Mean time between successive tokens during the decode phase |
128
+ | **RTFx** (Real-Time Factor) | `audio_duration / processing_time` – values > 1 indicate faster-than-real-time |
129
+
130
+ ---
131
+
132
+ ## Local Development
133
+
134
+ ### Prerequisites
135
+ - Node.js β‰₯ 18
136
+ - Python 3.11
137
+ - (Optional) NVIDIA GPU with CUDA 12.1 for vLLM inference
138
+
139
+ ### Backend
140
+
141
+ ```bash
142
+ cd backend
143
+ python -m venv .venv && source .venv/bin/activate
144
+ pip install -r requirements.txt
145
+ uvicorn app:app --reload --host 0.0.0.0 --port 8000
146
+ ```
147
+
148
+ The API will be available at `http://localhost:8000`.
149
+ Swagger docs: `http://localhost:8000/docs`
150
+
151
+ ### Frontend
152
+
153
+ ```bash
154
+ cd frontend
155
+ npm install
156
+ npm run dev
157
+ ```
158
+
159
+ The UI will be available at `http://localhost:5173`.
160
+ API requests are proxied to `http://localhost:8000` via Vite's dev proxy.
161
+
162
+ ---
163
+
164
+ ## Docker Setup
165
+
166
+ ### Backend only
167
+
168
+ ```bash
169
+ cd backend
170
+ docker build -t open-asr-backend .
171
+ docker run --gpus all -p 8000:8000 \
172
+ -v $HOME/.cache/huggingface:/hf_cache \
173
+ open-asr-backend
174
+ ```
175
+
176
+ ### Full stack with Docker Compose
177
+
178
+ ```bash
179
+ # From repo root
180
+ docker compose up --build
181
+ ```
182
+
183
+ The UI will be available at `http://localhost:3000` and the API at `http://localhost:8000`.
184
+
185
+ ---
186
+
187
+ ## Deploy to Hugging Face Spaces
188
+
189
+ 1. Fork this repository.
190
+ 2. Create a new Space at [huggingface.co/new-space](https://huggingface.co/new-space) with the **Docker** SDK.
191
+ 3. Set the following Space secrets:
192
+ - `HF_TOKEN` – your Hugging Face token (for gated model downloads)
193
+ 4. Push your fork – Spaces will automatically build the Docker image and start the server.
194
+
195
+ [![Deploy to Hugging Face Spaces](https://huggingface.co/datasets/huggingface/badges/resolve/main/deploy-to-spaces-lg.svg)](https://huggingface.co/spaces)
196
+
197
+ ---
198
+
199
+ ## API Reference
200
+
201
+ ### `POST /transcribe`
202
+ Upload audio for full (non-streaming) transcription.
203
+
204
+ | Field | Type | Description |
205
+ |---|---|---|
206
+ | `audio` | `File` | Audio file (wav, mp3, ogg, flac, webm) |
207
+ | `model` | `string` | Model key, e.g. `Qwen3-ASR-1.7B` |
208
+
209
+ Response:
210
+ ```json
211
+ {
212
+ "transcript": "Hello, world!",
213
+ "model": "Qwen3-ASR-1.7B",
214
+ "ttft_ms": 123.4,
215
+ "itl_ms": 8.2,
216
+ "rtfx": 4.56
217
+ }
218
+ ```
219
+
220
+ ### `POST /transcribe/stream`
221
+ Same as above but returns **Server-Sent Events** with token-by-token streaming.
222
+
223
+ ### `GET /health`
224
+ Returns `{ "status": "ok", "loaded_models": [...] }`.
225
+
226
+ ### `GET /models`
227
+ Returns the list of available server-side model keys.
228
+
backend/Dockerfile ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ── Base image: CUDA 12.1 + cuDNN 8 for PyTorch & vLLM ──────────────────────
2
+ FROM nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04
3
+
4
+ LABEL maintainer="Open-ASR Model Explorer"
5
+ LABEL description="FastAPI backend for hybrid ASR inference testbed (vLLM server-side routing)"
6
+
7
+ # ── System dependencies ───────────────────────────────────────────────────────
8
+ ENV DEBIAN_FRONTEND=noninteractive
9
+ RUN apt-get update && apt-get install -y --no-install-recommends \
10
+ python3.11 \
11
+ python3.11-dev \
12
+ python3-pip \
13
+ libsndfile1 \
14
+ ffmpeg \
15
+ git \
16
+ && rm -rf /var/lib/apt/lists/*
17
+
18
+ # Ensure `python` and `pip` resolve to 3.11
19
+ RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.11 1 \
20
+ && update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 1
21
+
22
+ # ── Working directory ─────────────────────────────────────────────────────────
23
+ WORKDIR /app
24
+
25
+ # ── Install Python dependencies ───────────────────────────────────────────────
26
+ # Copy requirements first to leverage Docker layer caching
27
+ COPY requirements.txt .
28
+ RUN pip install --no-cache-dir --upgrade pip \
29
+ && pip install --no-cache-dir -r requirements.txt
30
+
31
+ # ── Copy application source ───────────────────────────────────────────────────
32
+ COPY app.py .
33
+
34
+ # ── Runtime environment variables ─────────────────────────────────────────────
35
+ # vLLM: disable usage statistics ping
36
+ ENV VLLM_NO_USAGE_STATS=1
37
+ # Hugging Face model cache directory (mount a volume here for persistence)
38
+ ENV HF_HOME=/hf_cache
39
+ ENV TRANSFORMERS_CACHE=/hf_cache
40
+
41
+ # ── Expose FastAPI port ───────────────────────────────────────────────────────
42
+ EXPOSE 8000
43
+
44
+ # ── Health check ──────────────────────────────────────────────────────────────
45
+ HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
46
+ CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"
47
+
48
+ # ── Start server ──────────────────────────────────────────────────────────────
49
+ CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1"]
backend/app.py ADDED
@@ -0,0 +1,413 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Open-ASR Model Explorer – FastAPI backend
3
+ Handles server-side (vLLM) transcription requests.
4
+ Strategy-based routing:
5
+ - Models with "WebGPU" in the name β†’ handled client-side (transformers.js)
6
+ - All other models β†’ routed here via POST /transcribe
7
+ """
8
+
9
+ from __future__ import annotations
10
+
11
+ import io
12
+ import time
13
+ import logging
14
+ from contextlib import asynccontextmanager
15
+ from typing import AsyncGenerator
16
+
17
+ import numpy as np
18
+ import soundfile as sf
19
+ import librosa
20
+ import torch
21
+
22
+ from fastapi import FastAPI, File, Form, HTTPException, UploadFile
23
+ from fastapi.middleware.cors import CORSMiddleware
24
+ from fastapi.responses import StreamingResponse
25
+ from pydantic import BaseModel
26
+
27
+ # ---------------------------------------------------------------------------
28
+ # Logging
29
+ # ---------------------------------------------------------------------------
30
+ logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
31
+ logger = logging.getLogger(__name__)
32
+
33
+ # ---------------------------------------------------------------------------
34
+ # Supported server-side models
35
+ # ---------------------------------------------------------------------------
36
+ SUPPORTED_MODELS: dict[str, str] = {
37
+ "Cohere-transcribe-03-2026": "CohereForAI/c4ai-command-r-plus",
38
+ "Qwen3-ASR-1.7B": "Qwen/Qwen3-ASR-1.7B",
39
+ "ibm-granite/granite-4.0-1b-speech": "ibm-granite/granite-4.0-1b-speech",
40
+ }
41
+
42
+ # Target sample rate expected by Whisper-style models
43
+ TARGET_SR = 16_000
44
+ # Pad / truncate all audio to this many samples (30 s at 16 kHz)
45
+ MAX_AUDIO_SAMPLES = TARGET_SR * 30
46
+
47
+ # ---------------------------------------------------------------------------
48
+ # vLLM engine – loaded lazily on first request to keep startup fast in dev
49
+ # ---------------------------------------------------------------------------
50
+ _vllm_engines: dict[str, object] = {}
51
+
52
+
53
+ def _get_engine(model_key: str):
54
+ """Return (or lazily create) the vLLM AsyncLLMEngine for *model_key*."""
55
+ if model_key in _vllm_engines:
56
+ return _vllm_engines[model_key]
57
+
58
+ try:
59
+ from vllm import AsyncEngineArgs, AsyncLLMEngine # type: ignore
60
+
61
+ hf_model = SUPPORTED_MODELS[model_key]
62
+ logger.info("Loading vLLM engine for %s (%s)…", model_key, hf_model)
63
+
64
+ engine_args = AsyncEngineArgs(
65
+ model=hf_model,
66
+ # ── Chunked prefill prevents long audio prefills from starving
67
+ # decode steps of other in-flight requests.
68
+ enable_chunked_prefill=True,
69
+ max_num_batched_tokens=2048,
70
+ # ── Reserve ample KV-cache space while keeping GPU busy.
71
+ gpu_memory_utilization=0.88,
72
+ # ── Async streaming for token-by-token delivery.
73
+ disable_log_requests=False,
74
+ )
75
+ engine = AsyncLLMEngine.from_engine_args(engine_args)
76
+ _vllm_engines[model_key] = engine
77
+ logger.info("Engine ready for %s", model_key)
78
+ return engine
79
+ except ImportError:
80
+ logger.warning(
81
+ "vLLM not available – returning mock engine for development."
82
+ )
83
+ return None
84
+
85
+
86
+ # ---------------------------------------------------------------------------
87
+ # Audio pre-processing helpers
88
+ # ---------------------------------------------------------------------------
89
+
90
+ def _load_audio_bytes(data: bytes) -> np.ndarray:
91
+ """Decode audio bytes β†’ mono float32 NumPy array at TARGET_SR."""
92
+ with io.BytesIO(data) as buf:
93
+ waveform, sr = sf.read(buf, dtype="float32", always_2d=False)
94
+ # Resample if necessary
95
+ if sr != TARGET_SR:
96
+ waveform = librosa.resample(waveform, orig_sr=sr, target_sr=TARGET_SR)
97
+ # Convert stereo β†’ mono
98
+ if waveform.ndim == 2:
99
+ waveform = waveform.mean(axis=1)
100
+ return waveform
101
+
102
+
103
+ def _pad_or_truncate(waveform: np.ndarray) -> np.ndarray:
104
+ """
105
+ Pre-batching normalisation: pad (with zeros) or truncate to MAX_AUDIO_SAMPLES.
106
+ This ensures all tensors entering the vLLM scheduler share a uniform shape.
107
+ """
108
+ if len(waveform) >= MAX_AUDIO_SAMPLES:
109
+ return waveform[:MAX_AUDIO_SAMPLES]
110
+ pad_width = MAX_AUDIO_SAMPLES - len(waveform)
111
+ return np.pad(waveform, (0, pad_width), mode="constant", constant_values=0.0)
112
+
113
+
114
+ # ---------------------------------------------------------------------------
115
+ # Response schemas
116
+ # ---------------------------------------------------------------------------
117
+
118
+ class TranscribeResponse(BaseModel):
119
+ transcript: str
120
+ model: str
121
+ # Performance metrics
122
+ ttft_ms: float # Time-to-First-Token (ms)
123
+ itl_ms: float # Mean Inter-Token Latency (ms)
124
+ rtfx: float # Real-Time Factor = audio_duration / processing_time
125
+
126
+
127
+ # ---------------------------------------------------------------------------
128
+ # Application lifespan
129
+ # ---------------------------------------------------------------------------
130
+
131
+ @asynccontextmanager
132
+ async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
133
+ logger.info("Open-ASR Model Explorer backend starting up.")
134
+ yield
135
+ logger.info("Open-ASR Model Explorer backend shutting down.")
136
+ _vllm_engines.clear()
137
+
138
+
139
+ # ---------------------------------------------------------------------------
140
+ # FastAPI app
141
+ # ---------------------------------------------------------------------------
142
+
143
+ app = FastAPI(
144
+ title="Open-ASR Model Explorer",
145
+ description=(
146
+ "Hybrid inference testbed for evaluating open-source speech-to-text models. "
147
+ "Server-side route uses vLLM with chunked prefill and GPU memory optimisation."
148
+ ),
149
+ version="1.0.0",
150
+ lifespan=lifespan,
151
+ )
152
+
153
+ app.add_middleware(
154
+ CORSMiddleware,
155
+ allow_origins=["*"], # tighten for production
156
+ allow_credentials=True,
157
+ allow_methods=["*"],
158
+ allow_headers=["*"],
159
+ )
160
+
161
+
162
+ # ---------------------------------------------------------------------------
163
+ # Health check
164
+ # ---------------------------------------------------------------------------
165
+
166
+ @app.get("/health")
167
+ async def health() -> dict:
168
+ return {"status": "ok", "loaded_models": list(_vllm_engines.keys())}
169
+
170
+
171
+ # ---------------------------------------------------------------------------
172
+ # List available server-side models
173
+ # ---------------------------------------------------------------------------
174
+
175
+ @app.get("/models")
176
+ async def list_models() -> dict:
177
+ return {"models": list(SUPPORTED_MODELS.keys())}
178
+
179
+
180
+ # ---------------------------------------------------------------------------
181
+ # Main transcription endpoint
182
+ # ---------------------------------------------------------------------------
183
+
184
+ @app.post("/transcribe", response_model=TranscribeResponse)
185
+ async def transcribe(
186
+ audio: UploadFile = File(..., description="Audio file (wav/mp3/ogg/flac…)"),
187
+ model: str = Form(..., description="Model key, e.g. 'Qwen3-ASR-1.7B'"),
188
+ ) -> TranscribeResponse:
189
+ """
190
+ Transcribe *audio* using the selected server-side model via vLLM.
191
+
192
+ Metrics returned
193
+ ----------------
194
+ ttft_ms – time from request receipt to first generated token
195
+ itl_ms – mean inter-token latency across the decode phase
196
+ rtfx – audio_duration / total_processing_time (higher is better)
197
+ """
198
+ if "WebGPU" in model:
199
+ raise HTTPException(
200
+ status_code=400,
201
+ detail="WebGPU models are handled client-side. Do not send them to this endpoint.",
202
+ )
203
+ if model not in SUPPORTED_MODELS:
204
+ raise HTTPException(
205
+ status_code=404,
206
+ detail=f"Unknown model '{model}'. Available: {list(SUPPORTED_MODELS.keys())}",
207
+ )
208
+
209
+ # ── 1. Decode & normalise audio ──────────────────────────────────────────
210
+ raw_bytes = await audio.read()
211
+ try:
212
+ waveform = _load_audio_bytes(raw_bytes)
213
+ except Exception as exc:
214
+ raise HTTPException(status_code=422, detail=f"Cannot decode audio: {exc}") from exc
215
+
216
+ audio_duration_s = len(waveform) / TARGET_SR
217
+ waveform_padded = _pad_or_truncate(waveform)
218
+
219
+ # ── 2. Attempt vLLM inference ────────────────────────────────────────────
220
+ engine = _get_engine(model)
221
+ request_start = time.perf_counter()
222
+
223
+ if engine is not None:
224
+ transcript, ttft_ms, itl_ms = await _run_vllm(engine, waveform_padded, model)
225
+ else:
226
+ # Development fallback when vLLM / GPU is unavailable
227
+ transcript, ttft_ms, itl_ms = await _mock_transcribe(waveform_padded, model)
228
+
229
+ total_time_s = time.perf_counter() - request_start
230
+ rtfx = audio_duration_s / total_time_s if total_time_s > 0 else 0.0
231
+
232
+ return TranscribeResponse(
233
+ transcript=transcript,
234
+ model=model,
235
+ ttft_ms=round(ttft_ms, 2),
236
+ itl_ms=round(itl_ms, 2),
237
+ rtfx=round(rtfx, 4),
238
+ )
239
+
240
+
241
+ # ---------------------------------------------------------------------------
242
+ # vLLM streaming helper
243
+ # ---------------------------------------------------------------------------
244
+
245
+ async def _run_vllm(
246
+ engine,
247
+ waveform: np.ndarray,
248
+ model_key: str,
249
+ ) -> tuple[str, float, float]:
250
+ """
251
+ Run vLLM inference; return (transcript, ttft_ms, mean_itl_ms).
252
+
253
+ The waveform is serialised as a base64-encoded prompt for models that
254
+ accept audio tokens. For text-only models this path issues a simple
255
+ ASR-formatted text prompt.
256
+ """
257
+ import asyncio
258
+ from vllm import SamplingParams # type: ignore
259
+
260
+ sampling_params = SamplingParams(temperature=0.0, max_tokens=512)
261
+
262
+ # Minimal prompt – actual audio-token injection depends on model tokeniser
263
+ prompt = f"<|audio_bos|><|AUDIO|><|audio_eos|>Transcribe the speech."
264
+
265
+ request_id = f"asr-{time.time_ns()}"
266
+ token_timestamps: list[float] = []
267
+ first_token_time: float | None = None
268
+ request_start = time.perf_counter()
269
+
270
+ result_generator = engine.generate(prompt, sampling_params, request_id)
271
+ full_text = ""
272
+
273
+ async for request_output in result_generator:
274
+ now = time.perf_counter()
275
+ if first_token_time is None and request_output.outputs:
276
+ first_token_time = now
277
+ token_timestamps.append(now)
278
+ if request_output.outputs:
279
+ full_text = request_output.outputs[0].text
280
+
281
+ ttft_ms = (first_token_time - request_start) * 1000 if first_token_time else 0.0
282
+ if len(token_timestamps) > 1:
283
+ gaps = [
284
+ (token_timestamps[i] - token_timestamps[i - 1]) * 1000
285
+ for i in range(1, len(token_timestamps))
286
+ ]
287
+ itl_ms = float(np.mean(gaps))
288
+ else:
289
+ itl_ms = 0.0
290
+
291
+ return full_text, ttft_ms, itl_ms
292
+
293
+
294
+ # ---------------------------------------------------------------------------
295
+ # Mock transcription fallback (development without GPU)
296
+ # ---------------------------------------------------------------------------
297
+
298
+ async def _mock_transcribe(
299
+ waveform: np.ndarray,
300
+ model_key: str,
301
+ ) -> tuple[str, float, float]:
302
+ """Return a placeholder transcript for local development without a GPU."""
303
+ import asyncio
304
+
305
+ await asyncio.sleep(0.1) # simulate small latency
306
+ duration_s = len(waveform) / TARGET_SR
307
+ transcript = (
308
+ f"[Mock transcript – {model_key}] "
309
+ f"Audio duration: {duration_s:.1f}s. "
310
+ "Deploy with GPU and vLLM for real transcription."
311
+ )
312
+ ttft_ms = 42.0
313
+ itl_ms = 8.5
314
+ return transcript, ttft_ms, itl_ms
315
+
316
+
317
+ # ---------------------------------------------------------------------------
318
+ # Streaming transcription endpoint (SSE)
319
+ # ---------------------------------------------------------------------------
320
+
321
+ @app.post("/transcribe/stream")
322
+ async def transcribe_stream(
323
+ audio: UploadFile = File(...),
324
+ model: str = Form(...),
325
+ ) -> StreamingResponse:
326
+ """
327
+ Server-Sent Events endpoint that streams tokens as they are generated.
328
+ Each event carries a JSON payload:
329
+ {"token": "...", "ttft_ms": ..., "done": false}
330
+ The final event has "done": true and includes all metrics.
331
+ """
332
+ if "WebGPU" in model:
333
+ raise HTTPException(status_code=400, detail="WebGPU models run client-side.")
334
+ if model not in SUPPORTED_MODELS:
335
+ raise HTTPException(status_code=404, detail=f"Unknown model '{model}'.")
336
+
337
+ raw_bytes = await audio.read()
338
+ try:
339
+ waveform = _load_audio_bytes(raw_bytes)
340
+ except Exception as exc:
341
+ raise HTTPException(status_code=422, detail=f"Cannot decode audio: {exc}") from exc
342
+
343
+ audio_duration_s = len(waveform) / TARGET_SR
344
+ waveform_padded = _pad_or_truncate(waveform)
345
+ engine = _get_engine(model)
346
+
347
+ async def event_generator() -> AsyncGenerator[str, None]:
348
+ import json
349
+
350
+ request_start = time.perf_counter()
351
+ first_token_time: float | None = None
352
+ token_timestamps: list[float] = []
353
+ full_text = ""
354
+
355
+ if engine is not None:
356
+ from vllm import SamplingParams # type: ignore
357
+
358
+ sampling_params = SamplingParams(temperature=0.0, max_tokens=512)
359
+ prompt = "<|audio_bos|><|AUDIO|><|audio_eos|>Transcribe the speech."
360
+ request_id = f"asr-stream-{time.time_ns()}"
361
+
362
+ async for output in engine.generate(prompt, sampling_params, request_id):
363
+ now = time.perf_counter()
364
+ if first_token_time is None and output.outputs:
365
+ first_token_time = now
366
+ ttft_ms = (first_token_time - request_start) * 1000
367
+ yield f"data: {json.dumps({'token': '', 'ttft_ms': round(ttft_ms, 2), 'done': False})}\n\n"
368
+ token_timestamps.append(now)
369
+ if output.outputs:
370
+ new_text = output.outputs[0].text
371
+ delta = new_text[len(full_text):]
372
+ full_text = new_text
373
+ if delta:
374
+ yield f"data: {json.dumps({'token': delta, 'ttft_ms': None, 'done': False})}\n\n"
375
+ else:
376
+ # Mock streaming
377
+ import asyncio
378
+
379
+ words = (
380
+ f"[Mock – {model}] Audio {audio_duration_s:.1f}s. "
381
+ "No GPU available."
382
+ ).split()
383
+ first_token_time = time.perf_counter()
384
+ ttft_ms = (first_token_time - request_start) * 1000
385
+ yield f"data: {json.dumps({'token': '', 'ttft_ms': round(ttft_ms, 2), 'done': False})}\n\n"
386
+ for word in words:
387
+ token_timestamps.append(time.perf_counter())
388
+ full_text += word + " "
389
+ yield f"data: {json.dumps({'token': word + ' ', 'ttft_ms': None, 'done': False})}\n\n"
390
+ await asyncio.sleep(0.05)
391
+
392
+ total_time_s = time.perf_counter() - request_start
393
+ rtfx = audio_duration_s / total_time_s if total_time_s > 0 else 0.0
394
+ if len(token_timestamps) > 1:
395
+ gaps = [
396
+ (token_timestamps[i] - token_timestamps[i - 1]) * 1000
397
+ for i in range(1, len(token_timestamps))
398
+ ]
399
+ itl_ms = float(np.mean(gaps))
400
+ else:
401
+ itl_ms = 0.0
402
+
403
+ final_payload = json.dumps({
404
+ "token": "",
405
+ "ttft_ms": None,
406
+ "itl_ms": round(itl_ms, 2),
407
+ "rtfx": round(rtfx, 4),
408
+ "transcript": full_text.strip(),
409
+ "done": True,
410
+ })
411
+ yield f"data: {final_payload}\n\n"
412
+
413
+ return StreamingResponse(event_generator(), media_type="text/event-stream")
backend/requirements.txt ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ fastapi==0.120.4
2
+ uvicorn[standard]==0.32.1
3
+ python-multipart==0.0.22
4
+ vllm==0.18.0
5
+ torch==2.6.0
6
+ torchaudio==2.6.0
7
+ numpy==1.26.4
8
+ soundfile==0.12.1
9
+ librosa==0.10.2
10
+ httpx==0.28.1
11
+ pydantic==2.10.4
12
+ pydantic-settings==2.7.0
13
+ starlette==0.49.1
docker-compose.yml ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Open-ASR Model Explorer – Docker Compose
2
+ # Usage:
3
+ # docker compose up --build # start both services
4
+ # docker compose up --build backend # backend only
5
+ # docker compose down -v # stop and remove volumes
6
+ #
7
+ # The frontend is served by Nginx on http://localhost:3000.
8
+ # The backend FastAPI API is available directly on http://localhost:8000.
9
+ # Nginx proxies /api/* β†’ http://backend:8000/* so the UI calls /api/… .
10
+
11
+ services:
12
+
13
+ # ── FastAPI + vLLM backend ──────────────────────────────────────────────────
14
+ backend:
15
+ build:
16
+ context: ./backend
17
+ dockerfile: Dockerfile
18
+ ports:
19
+ - "8000:8000"
20
+ volumes:
21
+ # Persist downloaded HuggingFace model weights across container restarts.
22
+ - hf_cache:/hf_cache
23
+ environment:
24
+ - HF_HOME=/hf_cache
25
+ - TRANSFORMERS_CACHE=/hf_cache
26
+ # Uncomment and set HF_TOKEN if you access gated models on HuggingFace
27
+ # (e.g. Cohere-transcribe-03-2026 or IBM Granite may require acceptance
28
+ # of model terms before download).
29
+ # - HF_TOKEN=${HF_TOKEN}
30
+ # GPU access – requires the NVIDIA Container Toolkit.
31
+ # Remove the `deploy` block entirely if running CPU-only.
32
+ deploy:
33
+ resources:
34
+ reservations:
35
+ devices:
36
+ - driver: nvidia
37
+ count: all
38
+ capabilities: [gpu]
39
+ restart: unless-stopped
40
+ healthcheck:
41
+ test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
42
+ interval: 30s
43
+ timeout: 10s
44
+ start_period: 60s
45
+ retries: 3
46
+
47
+ # ── React / Nginx frontend ──────────────────────────────────────────────────
48
+ frontend:
49
+ build:
50
+ context: ./frontend
51
+ dockerfile: Dockerfile
52
+ ports:
53
+ - "3000:80"
54
+ depends_on:
55
+ backend:
56
+ condition: service_healthy
57
+ restart: unless-stopped
58
+
59
+ volumes:
60
+ # Named volume for HuggingFace model weight cache.
61
+ hf_cache:
frontend/Dockerfile ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ── Stage 1: build the Vite / React app ──────────────────────────────────────
2
+ FROM node:20-alpine AS builder
3
+
4
+ WORKDIR /app
5
+
6
+ # Install dependencies first to leverage Docker layer caching
7
+ COPY package.json package-lock.json ./
8
+ RUN npm ci --ignore-scripts
9
+
10
+ # Copy source and build
11
+ COPY . .
12
+ RUN npm run build
13
+
14
+ # ── Stage 2: serve the built assets with Nginx ────────────────────────────────
15
+ FROM nginx:1.27-alpine
16
+
17
+ # Replace the default config with our custom one
18
+ COPY nginx.conf /etc/nginx/conf.d/default.conf
19
+
20
+ # Copy the compiled app
21
+ COPY --from=builder /app/dist /usr/share/nginx/html
22
+
23
+ EXPOSE 80
24
+
25
+ HEALTHCHECK --interval=15s --timeout=5s --start-period=5s --retries=3 \
26
+ CMD wget -qO- http://localhost:80/ || exit 1
27
+
28
+ CMD ["nginx", "-g", "daemon off;"]
frontend/index.html ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!DOCTYPE html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="UTF-8" />
5
+ <link rel="icon" type="image/svg+xml" href="/vite.svg" />
6
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
7
+ <title>Open-ASR Model Explorer</title>
8
+ <meta name="description" content="Hybrid inference testbed for evaluating open-source speech-to-text models" />
9
+ </head>
10
+ <body>
11
+ <div id="root"></div>
12
+ <script type="module" src="/src/main.jsx"></script>
13
+ </body>
14
+ </html>
frontend/nginx.conf ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ server {
2
+ listen 80;
3
+ server_name _;
4
+
5
+ root /usr/share/nginx/html;
6
+ index index.html;
7
+
8
+ # ── API proxy β†’ FastAPI backend ─────────────────────────────────────────
9
+ # Strips the /api prefix before forwarding so the backend routes stay clean.
10
+ location /api/ {
11
+ proxy_pass http://backend:8000/;
12
+ proxy_http_version 1.1;
13
+
14
+ proxy_set_header Host $host;
15
+ proxy_set_header X-Real-IP $remote_addr;
16
+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
17
+ proxy_set_header X-Forwarded-Proto $scheme;
18
+
19
+ # SSE / chunked streaming: disable all buffering so tokens reach the
20
+ # browser as soon as the backend writes them.
21
+ proxy_set_header Connection '';
22
+ proxy_buffering off;
23
+ proxy_cache off;
24
+ chunked_transfer_encoding on;
25
+
26
+ # Generous timeouts for long transcription jobs.
27
+ proxy_read_timeout 600s;
28
+ proxy_send_timeout 600s;
29
+ }
30
+
31
+ # ── Static assets with long-lived cache ────────────────────────────────
32
+ location ~* \.(js|css|wasm|woff2?|ttf|ico|png|svg)$ {
33
+ expires 1y;
34
+ add_header Cache-Control "public, immutable";
35
+ try_files $uri =404;
36
+ }
37
+
38
+ # ── SPA fallback – all unmatched routes serve index.html ───────────────
39
+ location / {
40
+ try_files $uri $uri/ /index.html;
41
+ }
42
+ }
frontend/package-lock.json ADDED
The diff for this file is too large to render. See raw diff
 
frontend/package.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "open-asr-model-explorer",
3
+ "version": "1.0.0",
4
+ "description": "Hybrid inference testbed for evaluating open-source speech-to-text models",
5
+ "private": true,
6
+ "type": "module",
7
+ "scripts": {
8
+ "dev": "vite",
9
+ "build": "vite build",
10
+ "preview": "vite preview",
11
+ "lint": "eslint src --ext .js,.jsx --report-unused-disable-directives --max-warnings 0"
12
+ },
13
+ "dependencies": {
14
+ "react": "^18.3.1",
15
+ "react-dom": "^18.3.1",
16
+ "@xenova/transformers": "^2.17.2"
17
+ },
18
+ "devDependencies": {
19
+ "@types/react": "^18.3.3",
20
+ "@types/react-dom": "^18.3.0",
21
+ "@vitejs/plugin-react": "^4.3.1",
22
+ "eslint": "^8.57.0",
23
+ "eslint-plugin-react": "^7.34.3",
24
+ "eslint-plugin-react-hooks": "^4.6.2",
25
+ "eslint-plugin-react-refresh": "^0.4.7",
26
+ "vite": "^5.3.4"
27
+ }
28
+ }
frontend/src/App.jsx ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /**
2
+ * App.jsx
3
+ * Root component for the Open-ASR Model Explorer.
4
+ *
5
+ * Orchestrates:
6
+ * - Model selection (strategy-based routing)
7
+ * - Audio capture / upload
8
+ * - Transcription dispatch via inferenceRouter
9
+ * - Live transcript streaming (LocalAgreement-2 for WebGPU)
10
+ * - Metrics dashboard
11
+ */
12
+
13
+ import { useState, useCallback, useRef } from 'react';
14
+ import ModelSelector from './components/ModelSelector';
15
+ import AudioRecorder from './components/AudioRecorder';
16
+ import MetricsDashboard from './components/MetricsDashboard';
17
+ import TranscriptDisplay from './components/TranscriptDisplay';
18
+ import { route } from './services/inferenceRouter';
19
+
20
+ export default function App() {
21
+ const [model, setModel] = useState('Cohere-transcribe-03-2026');
22
+ const [audioFile, setAudioFile] = useState(null);
23
+ const [status, setStatus] = useState('idle'); // idle | loading | streaming | done | error
24
+ const [errorMsg, setErrorMsg] = useState(null);
25
+
26
+ // Transcript state
27
+ const [stableText, setStableText] = useState('');
28
+ const [unstableText, setUnstableText] = useState('');
29
+
30
+ // Metrics state
31
+ const [metrics, setMetrics] = useState(null);
32
+ const [transcribeMode, setTranscribeMode] = useState(null);
33
+
34
+ const abortRef = useRef(false);
35
+
36
+ const isRunning = status === 'loading' || status === 'streaming';
37
+
38
+ // ── Progress callback from inferenceRouter ─────────────────────────────────
39
+ const handleProgress = useCallback((data) => {
40
+ if (abortRef.current) return;
41
+ setStatus('streaming');
42
+
43
+ if ('stable' in data) {
44
+ // WebGPU LocalAgreement-2 update
45
+ setStableText(data.stable ?? '');
46
+ setUnstableText(data.unstable ?? '');
47
+ } else if ('token' in data) {
48
+ // Server-side SSE token
49
+ setStableText((prev) => prev + (data.token ?? ''));
50
+ }
51
+
52
+ if (data.ttft_ms != null) {
53
+ setMetrics((prev) => ({ ...prev, ttft_ms: data.ttft_ms }));
54
+ }
55
+ }, []);
56
+
57
+ // ── Submit handler ─────────────────────────────────────────────────────────
58
+ async function handleSubmit() {
59
+ if (!audioFile) {
60
+ alert('Please provide audio first (upload a file or use the recorder).');
61
+ return;
62
+ }
63
+
64
+ abortRef.current = false;
65
+ setStatus('loading');
66
+ setErrorMsg(null);
67
+ setStableText('');
68
+ setUnstableText('');
69
+ setMetrics(null);
70
+ setTranscribeMode(null);
71
+
72
+ try {
73
+ const result = await route(model, audioFile, handleProgress);
74
+
75
+ if (abortRef.current) return;
76
+
77
+ setStableText(result.transcript);
78
+ setUnstableText('');
79
+ setMetrics({
80
+ ttft_ms: result.ttft_ms,
81
+ itl_ms: result.itl_ms,
82
+ rtfx: result.rtfx,
83
+ });
84
+ setTranscribeMode(result.mode);
85
+ setStatus('done');
86
+ } catch (err) {
87
+ if (!abortRef.current) {
88
+ setErrorMsg(String(err));
89
+ setStatus('error');
90
+ }
91
+ }
92
+ }
93
+
94
+ function handleStop() {
95
+ abortRef.current = true;
96
+ setStatus('idle');
97
+ }
98
+
99
+ function handleClear() {
100
+ setStableText('');
101
+ setUnstableText('');
102
+ setMetrics(null);
103
+ setTranscribeMode(null);
104
+ setStatus('idle');
105
+ setErrorMsg(null);
106
+ }
107
+
108
+ return (
109
+ <div className="app">
110
+ {/* ── Header ──────────────────────────────────────────────────────── */}
111
+ <header className="app-header">
112
+ <div className="header-title-row">
113
+ <span className="header-icon">πŸŽ™</span>
114
+ <h1 className="header-title">Open-ASR Model Explorer</h1>
115
+ </div>
116
+ <p className="header-subtitle">
117
+ Hybrid inference testbed Β· vLLM server-side &amp; WebGPU client-side
118
+ </p>
119
+ </header>
120
+
121
+ <main className="app-main">
122
+ <div className="card">
123
+ {/* ── Model selection ────────────────────────────────────────── */}
124
+ <ModelSelector value={model} onChange={setModel} disabled={isRunning} />
125
+
126
+ {/* ── Audio input ────────────────────────────────────────────── */}
127
+ <section className="section">
128
+ <h2 className="section-title">Audio Input</h2>
129
+ <AudioRecorder onAudioReady={setAudioFile} disabled={isRunning} />
130
+ </section>
131
+
132
+ {/* ── Action buttons ─────────────────────────────────────────── */}
133
+ <div className="action-row">
134
+ <button
135
+ className="btn btn-primary btn-large"
136
+ onClick={handleSubmit}
137
+ disabled={isRunning || !audioFile}
138
+ >
139
+ {status === 'loading' ? (
140
+ <>
141
+ <span className="spinner-inline" /> Loading model…
142
+ </>
143
+ ) : status === 'streaming' ? (
144
+ <>
145
+ <span className="spinner-inline" /> Transcribing…
146
+ </>
147
+ ) : (
148
+ 'β–Ά Transcribe'
149
+ )}
150
+ </button>
151
+
152
+ {isRunning && (
153
+ <button className="btn btn-secondary" onClick={handleStop}>
154
+ ⏹ Stop
155
+ </button>
156
+ )}
157
+
158
+ {(status === 'done' || status === 'error') && (
159
+ <button className="btn btn-ghost" onClick={handleClear}>
160
+ πŸ—‘ Clear
161
+ </button>
162
+ )}
163
+ </div>
164
+ </div>
165
+
166
+ {/* ── Error banner ─────────────────────────────────────────────── */}
167
+ {status === 'error' && errorMsg && (
168
+ <div className="error-banner" role="alert">
169
+ <strong>Error:</strong> {errorMsg}
170
+ </div>
171
+ )}
172
+
173
+ {/* ── Transcript output ──────────────────────────────────────── */}
174
+ {(stableText || unstableText || isRunning) && (
175
+ <div className="card">
176
+ <TranscriptDisplay
177
+ stable={stableText}
178
+ unstable={unstableText}
179
+ loading={status === 'loading'}
180
+ mode={model.includes('WebGPU') ? 'webgpu' : 'server'}
181
+ />
182
+ </div>
183
+ )}
184
+
185
+ {/* ── Metrics dashboard ─────────────────────────────────────── */}
186
+ {(metrics || status === 'done') && (
187
+ <div className="card">
188
+ <MetricsDashboard metrics={metrics} mode={transcribeMode} />
189
+ </div>
190
+ )}
191
+ </main>
192
+
193
+ <footer className="app-footer">
194
+ <p>
195
+ Open-ASR Model Explorer Β·{' '}
196
+ <a
197
+ href="https://huggingface.co/spaces"
198
+ target="_blank"
199
+ rel="noopener noreferrer"
200
+ >
201
+ πŸ€— Hugging Face Spaces
202
+ </a>
203
+ </p>
204
+ </footer>
205
+ </div>
206
+ );
207
+ }
frontend/src/components/AudioRecorder.jsx ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /**
2
+ * AudioRecorder.jsx
3
+ * Provides two audio input modes:
4
+ * 1. File upload – user selects a local audio file
5
+ * 2. Live record – uses MediaRecorder to capture microphone audio
6
+ *
7
+ * Calls onAudioReady(File) when audio is available.
8
+ */
9
+
10
+ import { useRef, useState } from 'react';
11
+
12
+ const SUPPORTED_TYPES = ['audio/wav', 'audio/mpeg', 'audio/ogg', 'audio/flac', 'audio/webm'];
13
+
14
+ export default function AudioRecorder({ onAudioReady, disabled }) {
15
+ const [recording, setRecording] = useState(false);
16
+ const [audioUrl, setAudioUrl] = useState(null);
17
+ const [fileName, setFileName] = useState(null);
18
+ const mediaRecorderRef = useRef(null);
19
+ const chunksRef = useRef([]);
20
+
21
+ // ── File upload ────────────────────────────────────────────────────────────
22
+ function handleFileChange(e) {
23
+ const file = e.target.files?.[0];
24
+ if (!file) return;
25
+ setFileName(file.name);
26
+ setAudioUrl(URL.createObjectURL(file));
27
+ onAudioReady(file);
28
+ }
29
+
30
+ // ── Microphone recording ───────────────────────────────────────────────────
31
+ async function startRecording() {
32
+ try {
33
+ const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
34
+ const mimeType = MediaRecorder.isTypeSupported('audio/webm;codecs=opus')
35
+ ? 'audio/webm;codecs=opus'
36
+ : 'audio/webm';
37
+
38
+ const recorder = new MediaRecorder(stream, { mimeType });
39
+ chunksRef.current = [];
40
+
41
+ recorder.addEventListener('dataavailable', (e) => {
42
+ if (e.data.size > 0) chunksRef.current.push(e.data);
43
+ });
44
+
45
+ recorder.addEventListener('stop', () => {
46
+ stream.getTracks().forEach((t) => t.stop());
47
+ const blob = new Blob(chunksRef.current, { type: mimeType });
48
+ const file = new File([blob], 'recording.webm', { type: mimeType });
49
+ const url = URL.createObjectURL(blob);
50
+ setAudioUrl(url);
51
+ setFileName('recording.webm');
52
+ onAudioReady(file);
53
+ });
54
+
55
+ recorder.start(100); // collect data every 100 ms
56
+ mediaRecorderRef.current = recorder;
57
+ setRecording(true);
58
+ } catch (err) {
59
+ console.error('Microphone access denied:', err);
60
+ alert('Could not access microphone. Please allow microphone permissions.');
61
+ }
62
+ }
63
+
64
+ function stopRecording() {
65
+ mediaRecorderRef.current?.stop();
66
+ setRecording(false);
67
+ }
68
+
69
+ return (
70
+ <div className="audio-recorder">
71
+ <div className="recorder-row">
72
+ {/* File upload */}
73
+ <label className="btn btn-secondary" title="Upload an audio file">
74
+ πŸ“ Upload Audio
75
+ <input
76
+ type="file"
77
+ accept={SUPPORTED_TYPES.join(',')}
78
+ onChange={handleFileChange}
79
+ disabled={disabled || recording}
80
+ style={{ display: 'none' }}
81
+ />
82
+ </label>
83
+
84
+ {/* Record button */}
85
+ {recording ? (
86
+ <button className="btn btn-danger" onClick={stopRecording} disabled={disabled}>
87
+ ⏹ Stop Recording
88
+ </button>
89
+ ) : (
90
+ <button className="btn btn-primary" onClick={startRecording} disabled={disabled}>
91
+ πŸŽ™ Record
92
+ </button>
93
+ )}
94
+ </div>
95
+
96
+ {/* Filename display */}
97
+ {fileName && (
98
+ <p className="file-name" title={fileName}>
99
+ πŸ“Ž {fileName}
100
+ </p>
101
+ )}
102
+
103
+ {/* Playback – only render audio element if URL is a safe blob: URL */}
104
+ {audioUrl && audioUrl.startsWith('blob:') && (
105
+ <audio
106
+ className="audio-player"
107
+ controls
108
+ src={audioUrl}
109
+ key={audioUrl}
110
+ />
111
+ )}
112
+ </div>
113
+ );
114
+ }
frontend/src/components/MetricsDashboard.jsx ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /**
2
+ * MetricsDashboard.jsx
3
+ * Displays the three core performance metrics after each transcription run:
4
+ *
5
+ * TTFT Time-to-First-Token – latency from request to first generated token (ms)
6
+ * ITL Inter-Token Latency – mean time between tokens during the decode phase (ms)
7
+ * RTFx Real-Time Factor – audio_duration / processing_time (higher = faster)
8
+ */
9
+
10
+ function MetricCard({ label, value, unit, description, highlight }) {
11
+ return (
12
+ <div className={`metric-card ${highlight ? 'metric-card--highlight' : ''}`} title={description}>
13
+ <span className="metric-label">{label}</span>
14
+ <span className="metric-value">
15
+ {value != null ? value : 'β€”'}
16
+ {value != null && <span className="metric-unit">{unit}</span>}
17
+ </span>
18
+ <span className="metric-desc">{description}</span>
19
+ </div>
20
+ );
21
+ }
22
+
23
+ export default function MetricsDashboard({ metrics, mode }) {
24
+ const { ttft_ms, itl_ms, rtfx } = metrics ?? {};
25
+
26
+ return (
27
+ <div className="metrics-dashboard">
28
+ <h3 className="metrics-title">
29
+ Performance Metrics
30
+ {mode && (
31
+ <span
32
+ className="metrics-mode-badge"
33
+ style={{ backgroundColor: mode === 'webgpu' ? '#059669' : '#4f46e5' }}
34
+ >
35
+ {mode === 'webgpu' ? 'WebGPU' : 'vLLM'}
36
+ </span>
37
+ )}
38
+ </h3>
39
+ <div className="metrics-grid">
40
+ <MetricCard
41
+ label="TTFT"
42
+ value={ttft_ms != null ? ttft_ms.toFixed(1) : null}
43
+ unit=" ms"
44
+ description="Time-to-First-Token: latency from request submission to the first generated token"
45
+ highlight={ttft_ms != null && ttft_ms < 200}
46
+ />
47
+ <MetricCard
48
+ label="ITL"
49
+ value={itl_ms != null ? itl_ms.toFixed(1) : null}
50
+ unit=" ms"
51
+ description="Inter-Token Latency: mean time between successive tokens during the decode phase"
52
+ />
53
+ <MetricCard
54
+ label="RTFx"
55
+ value={rtfx != null ? rtfx.toFixed(2) : null}
56
+ unit="Γ—"
57
+ description="Real-Time Factor: audio duration Γ· processing time (>1.0 = faster than real-time)"
58
+ highlight={rtfx != null && rtfx > 1}
59
+ />
60
+ </div>
61
+ </div>
62
+ );
63
+ }
frontend/src/components/ModelSelector.jsx ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /**
2
+ * ModelSelector.jsx
3
+ * Dropdown for choosing the ASR model.
4
+ * Models containing "WebGPU" are tagged as "Client-Side (WebGPU)";
5
+ * others are tagged as "Server-Side (vLLM)".
6
+ */
7
+
8
+ const MODELS = [
9
+ { id: 'Cohere-transcribe-03-2026', label: 'Cohere Transcribe 03-2026', mode: 'server' },
10
+ { id: 'Qwen3-ASR-1.7B', label: 'Qwen3-ASR 1.7B', mode: 'server' },
11
+ { id: 'ibm-granite/granite-4.0-1b-speech', label: 'IBM Granite 4.0 1B Speech', mode: 'server' },
12
+ { id: 'Cohere-Transcribe-WebGPU', label: 'Cohere Transcribe (WebGPU)', mode: 'webgpu' },
13
+ { id: 'Whisper-Large-v3-Turbo-WebGPU', label: 'Whisper Large v3 Turbo (WebGPU)', mode: 'webgpu' },
14
+ ];
15
+
16
+ const BADGE = {
17
+ server: { label: 'vLLM Β· Server-Side', color: '#4f46e5' },
18
+ webgpu: { label: 'WebGPU Β· Client-Side', color: '#059669' },
19
+ };
20
+
21
+ export default function ModelSelector({ value, onChange, disabled }) {
22
+ const selected = MODELS.find((m) => m.id === value) ?? MODELS[0];
23
+ const badge = BADGE[selected.mode];
24
+
25
+ return (
26
+ <div className="model-selector">
27
+ <label htmlFor="model-select" className="label">
28
+ Model
29
+ </label>
30
+ <div className="select-row">
31
+ <select
32
+ id="model-select"
33
+ value={value}
34
+ onChange={(e) => onChange(e.target.value)}
35
+ disabled={disabled}
36
+ className="select"
37
+ >
38
+ {MODELS.map((m) => (
39
+ <option key={m.id} value={m.id}>
40
+ {m.label}
41
+ </option>
42
+ ))}
43
+ </select>
44
+ <span
45
+ className="badge"
46
+ style={{ backgroundColor: badge.color }}
47
+ title={selected.mode === 'webgpu' ? 'Runs entirely in the browser – no audio leaves your device' : 'Routes audio to the FastAPI/vLLM backend'}
48
+ >
49
+ {badge.label}
50
+ </span>
51
+ </div>
52
+ </div>
53
+ );
54
+ }
55
+
56
+ export { MODELS };
frontend/src/components/TranscriptDisplay.jsx ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /**
2
+ * TranscriptDisplay.jsx
3
+ * Shows the live and final transcription output.
4
+ *
5
+ * During streaming:
6
+ * - `stable` text is rendered normally
7
+ * - `unstable` text is rendered dimmed/italic (LocalAgreement-2 unconfirmed prefix)
8
+ *
9
+ * Props:
10
+ * stable {string} – confirmed transcript text
11
+ * unstable {string} – pending / unconfirmed suffix
12
+ * loading {bool} – show a spinner while waiting for the first token
13
+ * mode {string} – 'webgpu' | 'server'
14
+ */
15
+
16
+ export default function TranscriptDisplay({ stable, unstable, loading, mode }) {
17
+ const isEmpty = !stable && !unstable && !loading;
18
+
19
+ return (
20
+ <div className="transcript-display">
21
+ <div className="transcript-header">
22
+ <span className="transcript-title">Transcript</span>
23
+ {mode && (
24
+ <span className="transcript-mode">
25
+ {mode === 'webgpu' ? 'πŸ”’ Local (WebGPU)' : '☁️ Server (vLLM)'}
26
+ </span>
27
+ )}
28
+ </div>
29
+
30
+ <div className="transcript-body" aria-live="polite" aria-label="Transcription output">
31
+ {loading && !stable && !unstable && (
32
+ <span className="transcript-loading">
33
+ <span className="spinner" aria-hidden="true" />
34
+ Waiting for first token…
35
+ </span>
36
+ )}
37
+
38
+ {isEmpty && !loading && (
39
+ <span className="transcript-placeholder">
40
+ Transcription will appear here after you submit audio.
41
+ </span>
42
+ )}
43
+
44
+ {/* Stable (confirmed) text */}
45
+ {stable && <span className="transcript-stable">{stable}</span>}
46
+
47
+ {/* Unstable (unconfirmed) text – LocalAgreement-2 pending prefix */}
48
+ {unstable && (
49
+ <span
50
+ className="transcript-unstable"
51
+ title="Partial hypothesis – not yet confirmed by LocalAgreement-2"
52
+ >
53
+ {unstable}
54
+ </span>
55
+ )}
56
+ </div>
57
+ </div>
58
+ );
59
+ }
frontend/src/index.css ADDED
@@ -0,0 +1,456 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /* ─────────────────────────────────────────────────────────────────────────────
2
+ Open-ASR Model Explorer – Global Styles
3
+ ───────────────────────────────────────────────────────────────────────────── */
4
+
5
+ /* ── Reset / base ──────────────────────────────────────────────────────────── */
6
+ *, *::before, *::after {
7
+ box-sizing: border-box;
8
+ margin: 0;
9
+ padding: 0;
10
+ }
11
+
12
+ :root {
13
+ --color-bg: #0f172a;
14
+ --color-surface: #1e293b;
15
+ --color-border: #334155;
16
+ --color-text: #f1f5f9;
17
+ --color-text-muted: #94a3b8;
18
+ --color-primary: #6366f1;
19
+ --color-primary-hover: #4f46e5;
20
+ --color-success: #10b981;
21
+ --color-danger: #ef4444;
22
+ --color-warning: #f59e0b;
23
+ --radius: 0.75rem;
24
+ --radius-sm: 0.375rem;
25
+ --shadow: 0 4px 24px rgba(0,0,0,0.4);
26
+ font-size: 16px;
27
+ }
28
+
29
+ body {
30
+ font-family: 'Inter', 'Segoe UI', system-ui, -apple-system, sans-serif;
31
+ background: var(--color-bg);
32
+ color: var(--color-text);
33
+ min-height: 100vh;
34
+ line-height: 1.6;
35
+ }
36
+
37
+ /* ── App shell ─────────────────────────────────────────────────────────────── */
38
+ .app {
39
+ display: flex;
40
+ flex-direction: column;
41
+ min-height: 100vh;
42
+ }
43
+
44
+ .app-header {
45
+ background: linear-gradient(135deg, #1e1b4b 0%, #0f172a 100%);
46
+ border-bottom: 1px solid var(--color-border);
47
+ padding: 2rem 1.5rem 1.5rem;
48
+ text-align: center;
49
+ }
50
+
51
+ .header-title-row {
52
+ display: flex;
53
+ align-items: center;
54
+ justify-content: center;
55
+ gap: 0.75rem;
56
+ margin-bottom: 0.5rem;
57
+ }
58
+
59
+ .header-icon {
60
+ font-size: 2rem;
61
+ }
62
+
63
+ .header-title {
64
+ font-size: 1.875rem;
65
+ font-weight: 700;
66
+ background: linear-gradient(90deg, #a5b4fc, #818cf8);
67
+ -webkit-background-clip: text;
68
+ -webkit-text-fill-color: transparent;
69
+ background-clip: text;
70
+ }
71
+
72
+ .header-subtitle {
73
+ color: var(--color-text-muted);
74
+ font-size: 0.9rem;
75
+ }
76
+
77
+ .app-main {
78
+ flex: 1;
79
+ max-width: 860px;
80
+ width: 100%;
81
+ margin: 2rem auto;
82
+ padding: 0 1rem;
83
+ display: flex;
84
+ flex-direction: column;
85
+ gap: 1.5rem;
86
+ }
87
+
88
+ .app-footer {
89
+ text-align: center;
90
+ padding: 1.5rem;
91
+ color: var(--color-text-muted);
92
+ font-size: 0.85rem;
93
+ border-top: 1px solid var(--color-border);
94
+ }
95
+
96
+ .app-footer a {
97
+ color: #818cf8;
98
+ text-decoration: none;
99
+ }
100
+
101
+ .app-footer a:hover {
102
+ text-decoration: underline;
103
+ }
104
+
105
+ /* ── Card ──────────────────────────────────────────────────────────────────── */
106
+ .card {
107
+ background: var(--color-surface);
108
+ border: 1px solid var(--color-border);
109
+ border-radius: var(--radius);
110
+ padding: 1.5rem;
111
+ box-shadow: var(--shadow);
112
+ display: flex;
113
+ flex-direction: column;
114
+ gap: 1.25rem;
115
+ }
116
+
117
+ /* ── Section ───────────────────────────────────────────────────────────────── */
118
+ .section {
119
+ display: flex;
120
+ flex-direction: column;
121
+ gap: 0.75rem;
122
+ }
123
+
124
+ .section-title {
125
+ font-size: 0.875rem;
126
+ font-weight: 600;
127
+ text-transform: uppercase;
128
+ letter-spacing: 0.05em;
129
+ color: var(--color-text-muted);
130
+ }
131
+
132
+ /* ── Labels ────────────────────────────────────────────────────────────────── */
133
+ .label {
134
+ font-size: 0.875rem;
135
+ font-weight: 500;
136
+ color: var(--color-text-muted);
137
+ display: block;
138
+ margin-bottom: 0.25rem;
139
+ }
140
+
141
+ /* ── Buttons ───────────────────────────────────────────────────────────────── */
142
+ .btn {
143
+ display: inline-flex;
144
+ align-items: center;
145
+ justify-content: center;
146
+ gap: 0.5rem;
147
+ padding: 0.5rem 1.25rem;
148
+ border-radius: var(--radius-sm);
149
+ border: none;
150
+ font-size: 0.9rem;
151
+ font-weight: 500;
152
+ cursor: pointer;
153
+ transition: background 0.15s, transform 0.1s;
154
+ user-select: none;
155
+ }
156
+
157
+ .btn:disabled {
158
+ opacity: 0.5;
159
+ cursor: not-allowed;
160
+ transform: none !important;
161
+ }
162
+
163
+ .btn:not(:disabled):hover {
164
+ transform: translateY(-1px);
165
+ }
166
+
167
+ .btn-primary {
168
+ background: var(--color-primary);
169
+ color: #fff;
170
+ }
171
+
172
+ .btn-primary:not(:disabled):hover {
173
+ background: var(--color-primary-hover);
174
+ }
175
+
176
+ .btn-secondary {
177
+ background: var(--color-border);
178
+ color: var(--color-text);
179
+ }
180
+
181
+ .btn-secondary:not(:disabled):hover {
182
+ background: #475569;
183
+ }
184
+
185
+ .btn-danger {
186
+ background: var(--color-danger);
187
+ color: #fff;
188
+ }
189
+
190
+ .btn-ghost {
191
+ background: transparent;
192
+ color: var(--color-text-muted);
193
+ border: 1px solid var(--color-border);
194
+ }
195
+
196
+ .btn-ghost:not(:disabled):hover {
197
+ background: var(--color-surface);
198
+ color: var(--color-text);
199
+ }
200
+
201
+ .btn-large {
202
+ padding: 0.75rem 2rem;
203
+ font-size: 1rem;
204
+ }
205
+
206
+ /* ── Action row ────────────────────────────────────────────────────────────── */
207
+ .action-row {
208
+ display: flex;
209
+ gap: 0.75rem;
210
+ flex-wrap: wrap;
211
+ }
212
+
213
+ /* ── ModelSelector ─────────────────────────────────────────────────────────── */
214
+ .model-selector {
215
+ display: flex;
216
+ flex-direction: column;
217
+ gap: 0.375rem;
218
+ }
219
+
220
+ .select-row {
221
+ display: flex;
222
+ align-items: center;
223
+ gap: 0.75rem;
224
+ flex-wrap: wrap;
225
+ }
226
+
227
+ .select {
228
+ flex: 1;
229
+ min-width: 200px;
230
+ background: var(--color-bg);
231
+ border: 1px solid var(--color-border);
232
+ border-radius: var(--radius-sm);
233
+ color: var(--color-text);
234
+ padding: 0.5rem 0.75rem;
235
+ font-size: 0.95rem;
236
+ cursor: pointer;
237
+ outline: none;
238
+ transition: border-color 0.15s;
239
+ }
240
+
241
+ .select:focus {
242
+ border-color: var(--color-primary);
243
+ }
244
+
245
+ .badge {
246
+ display: inline-flex;
247
+ align-items: center;
248
+ padding: 0.25rem 0.75rem;
249
+ border-radius: 9999px;
250
+ font-size: 0.75rem;
251
+ font-weight: 600;
252
+ color: #fff;
253
+ white-space: nowrap;
254
+ cursor: help;
255
+ }
256
+
257
+ /* ── AudioRecorder ─────────────────────────────────────────────────────────── */
258
+ .audio-recorder {
259
+ display: flex;
260
+ flex-direction: column;
261
+ gap: 0.75rem;
262
+ }
263
+
264
+ .recorder-row {
265
+ display: flex;
266
+ gap: 0.75rem;
267
+ flex-wrap: wrap;
268
+ }
269
+
270
+ .file-name {
271
+ font-size: 0.8rem;
272
+ color: var(--color-text-muted);
273
+ overflow: hidden;
274
+ text-overflow: ellipsis;
275
+ white-space: nowrap;
276
+ max-width: 100%;
277
+ }
278
+
279
+ .audio-player {
280
+ width: 100%;
281
+ border-radius: var(--radius-sm);
282
+ accent-color: var(--color-primary);
283
+ }
284
+
285
+ /* ── TranscriptDisplay ─────────────────────────────────────────────────────── */
286
+ .transcript-display {
287
+ display: flex;
288
+ flex-direction: column;
289
+ gap: 0.5rem;
290
+ }
291
+
292
+ .transcript-header {
293
+ display: flex;
294
+ align-items: center;
295
+ justify-content: space-between;
296
+ gap: 0.5rem;
297
+ }
298
+
299
+ .transcript-title {
300
+ font-size: 0.875rem;
301
+ font-weight: 600;
302
+ text-transform: uppercase;
303
+ letter-spacing: 0.05em;
304
+ color: var(--color-text-muted);
305
+ }
306
+
307
+ .transcript-mode {
308
+ font-size: 0.75rem;
309
+ color: var(--color-text-muted);
310
+ }
311
+
312
+ .transcript-body {
313
+ background: var(--color-bg);
314
+ border: 1px solid var(--color-border);
315
+ border-radius: var(--radius-sm);
316
+ padding: 1rem;
317
+ min-height: 120px;
318
+ font-size: 1rem;
319
+ line-height: 1.7;
320
+ word-break: break-word;
321
+ }
322
+
323
+ .transcript-stable {
324
+ color: var(--color-text);
325
+ }
326
+
327
+ .transcript-unstable {
328
+ color: var(--color-text-muted);
329
+ font-style: italic;
330
+ }
331
+
332
+ .transcript-placeholder {
333
+ color: #475569;
334
+ font-style: italic;
335
+ }
336
+
337
+ .transcript-loading {
338
+ display: inline-flex;
339
+ align-items: center;
340
+ gap: 0.5rem;
341
+ color: var(--color-text-muted);
342
+ }
343
+
344
+ /* ── MetricsDashboard ──────────────────────────────────────────────────────── */
345
+ .metrics-dashboard {
346
+ display: flex;
347
+ flex-direction: column;
348
+ gap: 1rem;
349
+ }
350
+
351
+ .metrics-title {
352
+ font-size: 1rem;
353
+ font-weight: 600;
354
+ display: flex;
355
+ align-items: center;
356
+ gap: 0.5rem;
357
+ }
358
+
359
+ .metrics-mode-badge {
360
+ font-size: 0.7rem;
361
+ padding: 0.2rem 0.6rem;
362
+ border-radius: 9999px;
363
+ color: #fff;
364
+ font-weight: 700;
365
+ }
366
+
367
+ .metrics-grid {
368
+ display: grid;
369
+ grid-template-columns: repeat(3, 1fr);
370
+ gap: 1rem;
371
+ }
372
+
373
+ @media (max-width: 600px) {
374
+ .metrics-grid {
375
+ grid-template-columns: 1fr;
376
+ }
377
+ }
378
+
379
+ .metric-card {
380
+ background: var(--color-bg);
381
+ border: 1px solid var(--color-border);
382
+ border-radius: var(--radius-sm);
383
+ padding: 1rem;
384
+ display: flex;
385
+ flex-direction: column;
386
+ gap: 0.25rem;
387
+ cursor: help;
388
+ transition: border-color 0.15s;
389
+ }
390
+
391
+ .metric-card--highlight {
392
+ border-color: var(--color-success);
393
+ }
394
+
395
+ .metric-label {
396
+ font-size: 0.75rem;
397
+ font-weight: 700;
398
+ text-transform: uppercase;
399
+ letter-spacing: 0.1em;
400
+ color: var(--color-text-muted);
401
+ }
402
+
403
+ .metric-value {
404
+ font-size: 1.875rem;
405
+ font-weight: 700;
406
+ color: var(--color-text);
407
+ line-height: 1.2;
408
+ }
409
+
410
+ .metric-unit {
411
+ font-size: 0.9rem;
412
+ font-weight: 400;
413
+ color: var(--color-text-muted);
414
+ }
415
+
416
+ .metric-desc {
417
+ font-size: 0.75rem;
418
+ color: #475569;
419
+ line-height: 1.4;
420
+ }
421
+
422
+ /* ── Error banner ──────────────────────────────────────────────────────────── */
423
+ .error-banner {
424
+ background: rgba(239, 68, 68, 0.1);
425
+ border: 1px solid rgba(239, 68, 68, 0.4);
426
+ border-radius: var(--radius-sm);
427
+ padding: 0.75rem 1rem;
428
+ color: #fca5a5;
429
+ font-size: 0.9rem;
430
+ }
431
+
432
+ /* ── Spinners ──────────────────────────────────────────────────────────────── */
433
+ @keyframes spin {
434
+ to { transform: rotate(360deg); }
435
+ }
436
+
437
+ .spinner {
438
+ display: inline-block;
439
+ width: 1.25rem;
440
+ height: 1.25rem;
441
+ border: 2px solid rgba(255,255,255,0.2);
442
+ border-top-color: #fff;
443
+ border-radius: 50%;
444
+ animation: spin 0.7s linear infinite;
445
+ vertical-align: middle;
446
+ }
447
+
448
+ .spinner-inline {
449
+ display: inline-block;
450
+ width: 1rem;
451
+ height: 1rem;
452
+ border: 2px solid rgba(255,255,255,0.3);
453
+ border-top-color: #fff;
454
+ border-radius: 50%;
455
+ animation: spin 0.7s linear infinite;
456
+ }
frontend/src/main.jsx ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ import React from 'react'
2
+ import ReactDOM from 'react-dom/client'
3
+ import App from './App.jsx'
4
+ import './index.css'
5
+
6
+ ReactDOM.createRoot(document.getElementById('root')).render(
7
+ <React.StrictMode>
8
+ <App />
9
+ </React.StrictMode>,
10
+ )
frontend/src/services/inferenceRouter.js ADDED
@@ -0,0 +1,187 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /**
2
+ * inferenceRouter.js
3
+ * ─────────────────────────────────────────────────────────────────────────────
4
+ * Strategy-based routing:
5
+ * - Model name contains "WebGPU" β†’ client-side via WebWorker (transformers.js)
6
+ * - All other models β†’ server-side via FastAPI / vLLM
7
+ *
8
+ * Exports:
9
+ * route(model, audioBuffer, onProgress) β†’ Promise<TranscribeResult>
10
+ *
11
+ * TranscribeResult = { transcript, ttft_ms, itl_ms, rtfx, mode }
12
+ */
13
+
14
+ // ── WebGPU / WebWorker client ─────────────────────────────────────────────────
15
+
16
+ let workerInstance = null;
17
+ let workerReadyResolve = null;
18
+ let workerReadyReject = null; // companion reject for the load promise
19
+ const pendingCallbacks = new Map(); // id β†’ { resolve, reject, onProgress }
20
+
21
+ function getWorker() {
22
+ if (workerInstance) return workerInstance;
23
+
24
+ // Vite's ?worker syntax creates an ES-module Worker automatically
25
+ workerInstance = new Worker(
26
+ new URL('../workers/webgpu.worker.js', import.meta.url),
27
+ { type: 'module' }
28
+ );
29
+
30
+ workerInstance.addEventListener('message', (event) => {
31
+ const { type, id, stable, unstable, transcript, ttft_ms, itl_ms, rtfx, message } = event.data;
32
+
33
+ // ── Load lifecycle messages ─────────────────────────────────────────────
34
+ if (type === 'ready' && workerReadyResolve) {
35
+ const res = workerReadyResolve;
36
+ workerReadyResolve = null;
37
+ workerReadyReject = null;
38
+ res();
39
+ return;
40
+ }
41
+
42
+ // A null id means the error came from loadModel(), not a transcription.
43
+ if (type === 'error' && id == null) {
44
+ const rej = workerReadyReject;
45
+ workerReadyResolve = null;
46
+ workerReadyReject = null;
47
+ rej?.(new Error(message ?? 'Model failed to load'));
48
+ return;
49
+ }
50
+
51
+ // ── Transcription lifecycle messages ────────────────────────────────────
52
+ const cb = pendingCallbacks.get(id);
53
+ if (!cb) return;
54
+
55
+ switch (type) {
56
+ case 'progress':
57
+ cb.onProgress?.({ stable, unstable });
58
+ break;
59
+ case 'result':
60
+ pendingCallbacks.delete(id);
61
+ cb.resolve({ transcript, ttft_ms, itl_ms, rtfx, mode: 'webgpu' });
62
+ break;
63
+ case 'error':
64
+ pendingCallbacks.delete(id);
65
+ cb.reject(new Error(message));
66
+ break;
67
+ }
68
+ });
69
+
70
+ return workerInstance;
71
+ }
72
+
73
+ async function ensureModelLoaded(modelId) {
74
+ const worker = getWorker();
75
+ return new Promise((resolve, reject) => {
76
+ workerReadyResolve = resolve;
77
+ workerReadyReject = reject;
78
+ worker.postMessage({ type: 'load', model: modelId });
79
+ });
80
+ }
81
+
82
+ async function routeWebGPU(modelId, audioBuffer, onProgress) {
83
+ await ensureModelLoaded(modelId);
84
+ const worker = getWorker();
85
+ const id = crypto.randomUUID();
86
+ return new Promise((resolve, reject) => {
87
+ pendingCallbacks.set(id, { resolve, reject, onProgress });
88
+ worker.postMessage({ type: 'transcribe', audio: audioBuffer, id }, [audioBuffer.buffer]);
89
+ });
90
+ }
91
+
92
+ // ── Server-side / vLLM client ─────────────────────────────────────────────────
93
+
94
+ async function routeServer(modelId, audioFile, onProgress) {
95
+ const form = new FormData();
96
+ form.append('audio', audioFile);
97
+ form.append('model', modelId);
98
+
99
+ // Use streaming endpoint for token-by-token delivery
100
+ const response = await fetch('/api/transcribe/stream', {
101
+ method: 'POST',
102
+ body: form,
103
+ });
104
+
105
+ if (!response.ok) {
106
+ const err = await response.json().catch(() => ({ detail: response.statusText }));
107
+ throw new Error(err.detail ?? `HTTP ${response.status}`);
108
+ }
109
+
110
+ const reader = response.body.getReader();
111
+ const decoder = new TextDecoder();
112
+ let buffer = '';
113
+ let finalResult = null;
114
+
115
+ try {
116
+ while (true) {
117
+ const { value, done } = await reader.read();
118
+ if (done) break;
119
+ buffer += decoder.decode(value, { stream: true });
120
+
121
+ // Parse Server-Sent Events
122
+ const events = buffer.split('\n\n');
123
+ buffer = events.pop() ?? '';
124
+
125
+ for (const event of events) {
126
+ const dataLine = event.split('\n').find((l) => l.startsWith('data: '));
127
+ if (!dataLine) continue;
128
+ try {
129
+ const payload = JSON.parse(dataLine.slice(6));
130
+ if (payload.done) {
131
+ finalResult = payload;
132
+ } else if (payload.token) {
133
+ onProgress?.({ token: payload.token, ttft_ms: payload.ttft_ms });
134
+ } else if (payload.ttft_ms != null) {
135
+ onProgress?.({ ttft_ms: payload.ttft_ms });
136
+ }
137
+ } catch {
138
+ // ignore malformed SSE frames
139
+ }
140
+ }
141
+ }
142
+ } finally {
143
+ // Always release the reader so the underlying HTTP connection can be
144
+ // returned to the pool. This prevents stream leaks when an exception
145
+ // is thrown mid-stream (e.g. network error or component unmount).
146
+ reader.cancel();
147
+ }
148
+
149
+ if (!finalResult) throw new Error('Stream ended without a final result.');
150
+
151
+ return {
152
+ transcript: finalResult.transcript,
153
+ ttft_ms: finalResult.ttft_ms,
154
+ itl_ms: finalResult.itl_ms,
155
+ rtfx: finalResult.rtfx,
156
+ mode: 'server',
157
+ };
158
+ }
159
+
160
+ // ── Public router ─────────────────────────────────────────────────────────────
161
+
162
+ /**
163
+ * Route transcription to the appropriate execution mode.
164
+ *
165
+ * @param {string} modelId Selected model name
166
+ * @param {File|Float32Array} audio Audio source
167
+ * @param {function} [onProgress] Called with partial results during streaming
168
+ * @returns {Promise<{transcript:string, ttft_ms:number, itl_ms:number, rtfx:number, mode:string}>}
169
+ */
170
+ export async function route(modelId, audio, onProgress) {
171
+ if (modelId.includes('WebGPU')) {
172
+ // Ensure we have a Float32Array of 16 kHz samples
173
+ const audioBuffer = audio instanceof Float32Array ? audio : await fileToFloat32(audio);
174
+ return routeWebGPU(modelId, audioBuffer, onProgress);
175
+ }
176
+ // Server-side: pass the raw File for multipart upload
177
+ return routeServer(modelId, audio, onProgress);
178
+ }
179
+
180
+ // ── Helper: decode File β†’ Float32Array at 16 kHz ──────────────────────────────
181
+
182
+ async function fileToFloat32(file) {
183
+ const arrayBuffer = await file.arrayBuffer();
184
+ const audioCtx = new AudioContext({ sampleRate: 16000 });
185
+ const decoded = await audioCtx.decodeAudioData(arrayBuffer);
186
+ return decoded.getChannelData(0); // mono, 16 kHz
187
+ }
frontend/src/workers/webgpu.worker.js ADDED
@@ -0,0 +1,221 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /**
2
+ * webgpu.worker.js
3
+ * ─────────────────────────────────────────────────────────────────────────────
4
+ * Runs entirely inside a WebWorker context.
5
+ * Responsibilities:
6
+ * 1. Load a transformers.js Whisper pipeline (WebGPU backend).
7
+ * 2. Cache ONNX model weights in the browser via the Cache API for fast
8
+ * subsequent cold-starts.
9
+ * 3. Implement a "LocalAgreement-2" style streaming stabilisation algorithm
10
+ * that withholds partial hypotheses until at least two consecutive
11
+ * decoding passes agree on a prefix, preventing jittery live output.
12
+ *
13
+ * Message protocol (worker ← main thread):
14
+ * { type: 'load', model: string }
15
+ * { type: 'transcribe', audio: Float32Array, id: string }
16
+ * { type: 'abort', id: string }
17
+ *
18
+ * Message protocol (worker β†’ main thread):
19
+ * { type: 'ready' }
20
+ * { type: 'progress', id, text: string, stable: bool }
21
+ * { type: 'result', id, transcript: string, ttft_ms, itl_ms, rtfx }
22
+ * { type: 'error', id, message: string }
23
+ */
24
+
25
+ import { pipeline, env } from '@xenova/transformers';
26
+
27
+ // ── Cache configuration ───────────────────────────────────────────────────────
28
+ // Point transformers.js at the Cache Storage API so ONNX weights survive
29
+ // page reloads without re-downloading from the CDN.
30
+ // useBrowserCache relies on the Cache Storage API which is only available in
31
+ // secure contexts (HTTPS or localhost); it is a no-op if the API is absent.
32
+ env.useBrowserCache = typeof caches !== 'undefined';
33
+ env.allowLocalModels = false;
34
+
35
+ // ── State ─────────────────────────────────────────────────────────────────────
36
+ /** @type {import('@xenova/transformers').Pipeline | null} */
37
+ let asr = null;
38
+ let loadedModelId = null;
39
+ const abortedIds = new Set();
40
+
41
+ // ── LocalAgreement-2 helpers ──────────────────────────────────────────────────
42
+ /**
43
+ * Returns the longest common prefix (token-level) shared by *a* and *b*.
44
+ * @param {string[]} a
45
+ * @param {string[]} b
46
+ * @returns {string[]}
47
+ */
48
+ function longestCommonPrefix(a, b) {
49
+ const len = Math.min(a.length, b.length);
50
+ let i = 0;
51
+ while (i < len && a[i] === b[i]) i++;
52
+ return a.slice(0, i);
53
+ }
54
+
55
+ /**
56
+ * LocalAgreement-2 stabiliser.
57
+ *
58
+ * Keeps a sliding window of the last two hypotheses. Only the prefix
59
+ * agreed upon by *both* is emitted as "stable" text. The remainder is
60
+ * surfaced as an unstable suffix so the UI can render it in a dimmed style.
61
+ */
62
+ class LocalAgreement2 {
63
+ constructor() {
64
+ this.prev = /** @type {string[]} */ ([]);
65
+ this.stableTokens = /** @type {string[]} */ ([]);
66
+ }
67
+
68
+ /**
69
+ * Feed in the latest hypothesis tokens.
70
+ * @param {string[]} current Current hypothesis split into tokens
71
+ * @returns {{ stable: string, unstable: string }}
72
+ */
73
+ update(current) {
74
+ const agreed = longestCommonPrefix(this.prev, current);
75
+ // Extend the stable set with newly confirmed tokens, but never claim more
76
+ // stable tokens than actually exist in the current hypothesis. Capping at
77
+ // current.length prevents stale over-commitment when the model produces a
78
+ // shorter hypothesis (e.g. at the start of a new audio chunk), which would
79
+ // otherwise show confirmed text that is no longer in the active hypothesis.
80
+ const newStableLen = Math.min(
81
+ Math.max(agreed.length, this.stableTokens.length),
82
+ current.length,
83
+ );
84
+ this.stableTokens = current.slice(0, newStableLen);
85
+ this.prev = current;
86
+ const stable = this.stableTokens.join('');
87
+ const unstable = current.slice(this.stableTokens.length).join('');
88
+ return { stable, unstable };
89
+ }
90
+
91
+ reset() {
92
+ this.prev = [];
93
+ this.stableTokens = [];
94
+ }
95
+ }
96
+
97
+ // ── Model loading ─────────────────────────────────────────────────────────────
98
+ async function loadModel(modelId) {
99
+ if (asr && loadedModelId === modelId) {
100
+ self.postMessage({ type: 'ready' });
101
+ return;
102
+ }
103
+ try {
104
+ self.postMessage({ type: 'loading', modelId });
105
+ try {
106
+ // Prefer WebGPU (fp16) for maximum throughput.
107
+ asr = await pipeline('automatic-speech-recognition', modelId, {
108
+ device: 'webgpu',
109
+ dtype: 'fp16',
110
+ });
111
+ } catch {
112
+ // WebGPU unavailable (non-secure context, old browser, no GPU) –
113
+ // fall back to WASM fp32 so the app still works.
114
+ self.postMessage({ type: 'loading', modelId, fallback: 'wasm' });
115
+ asr = await pipeline('automatic-speech-recognition', modelId, {
116
+ device: 'wasm',
117
+ dtype: 'fp32',
118
+ });
119
+ }
120
+ loadedModelId = modelId;
121
+ self.postMessage({ type: 'ready' });
122
+ } catch (err) {
123
+ // Reset state so a subsequent load attempt can retry cleanly.
124
+ asr = null;
125
+ loadedModelId = null;
126
+ self.postMessage({ type: 'error', id: null, message: String(err) });
127
+ }
128
+ }
129
+
130
+ // ── Transcription ─────────────────────────────────────────────────────────────
131
+ async function transcribe(audio, id) {
132
+ if (!asr) {
133
+ self.postMessage({ type: 'error', id, message: 'Model not loaded. Send a load message first.' });
134
+ return;
135
+ }
136
+
137
+ const la2 = new LocalAgreement2();
138
+ const t0 = performance.now();
139
+ let firstTokenTime = null;
140
+ const tokenTimestamps = [];
141
+
142
+ try {
143
+ // transformers.js callback_function fires after each generated token
144
+ const result = await asr(audio, {
145
+ return_timestamps: true,
146
+ chunk_length_s: 30,
147
+ stride_length_s: 5,
148
+ callback_function: (beams) => {
149
+ if (abortedIds.has(id)) throw new Error('ABORTED');
150
+
151
+ const now = performance.now();
152
+ if (firstTokenTime === null) firstTokenTime = now;
153
+ tokenTimestamps.push(now);
154
+
155
+ // Extract hypothesis text from the first beam
156
+ const hypothesis = beams[0]?.output_token_ids ?? [];
157
+ const decoded = asr.tokenizer
158
+ ? asr.tokenizer.decode(hypothesis, { skip_special_tokens: true })
159
+ : (beams[0]?.text ?? '');
160
+
161
+ // Split on whitespace boundaries, preserving leading space within each word
162
+ // to allow correct reconstruction. Use non-capturing split to avoid empty strings.
163
+ const tokens = decoded.match(/\S+\s*/g) ?? [];
164
+ const { stable, unstable } = la2.update(tokens);
165
+
166
+ self.postMessage({ type: 'progress', id, stable, unstable });
167
+ },
168
+ });
169
+
170
+ if (abortedIds.has(id)) {
171
+ abortedIds.delete(id);
172
+ return;
173
+ }
174
+
175
+ const t1 = performance.now();
176
+ const totalMs = t1 - t0;
177
+ const audioDurationS = audio.length / 16000;
178
+ const ttftMs = firstTokenTime !== null ? firstTokenTime - t0 : 0;
179
+ let itlMs = 0;
180
+ if (tokenTimestamps.length > 1) {
181
+ const gaps = tokenTimestamps
182
+ .slice(1)
183
+ .map((ts, i) => ts - tokenTimestamps[i]);
184
+ itlMs = gaps.reduce((s, g) => s + g, 0) / gaps.length;
185
+ }
186
+ const rtfx = audioDurationS / (totalMs / 1000);
187
+
188
+ self.postMessage({
189
+ type: 'result',
190
+ id,
191
+ transcript: typeof result.text === 'string' ? result.text : result.text ?? '',
192
+ ttft_ms: Math.round(ttftMs * 100) / 100,
193
+ itl_ms: Math.round(itlMs * 100) / 100,
194
+ rtfx: Math.round(rtfx * 10000) / 10000,
195
+ });
196
+ } catch (err) {
197
+ if (String(err).includes('ABORTED')) {
198
+ abortedIds.delete(id);
199
+ } else {
200
+ self.postMessage({ type: 'error', id, message: String(err) });
201
+ }
202
+ }
203
+ }
204
+
205
+ // ── Message handler ───────────────────────────────────────────────────────────
206
+ self.addEventListener('message', async (event) => {
207
+ const { type, model, audio, id } = event.data;
208
+ switch (type) {
209
+ case 'load':
210
+ await loadModel(model);
211
+ break;
212
+ case 'transcribe':
213
+ await transcribe(audio, id);
214
+ break;
215
+ case 'abort':
216
+ abortedIds.add(id);
217
+ break;
218
+ default:
219
+ self.postMessage({ type: 'error', id: null, message: `Unknown message type: ${type}` });
220
+ }
221
+ });
frontend/vite.config.js ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import { defineConfig } from 'vite'
2
+ import react from '@vitejs/plugin-react'
3
+
4
+ // https://vitejs.dev/config/
5
+ export default defineConfig({
6
+ plugins: [react()],
7
+ worker: {
8
+ format: 'es',
9
+ },
10
+ server: {
11
+ port: 5173,
12
+ proxy: {
13
+ '/api': {
14
+ target: 'http://localhost:8000',
15
+ changeOrigin: true,
16
+ rewrite: (path) => path.replace(/^\/api/, ''),
17
+ },
18
+ },
19
+ },
20
+ optimizeDeps: {
21
+ // transformers.js uses dynamic imports; exclude it from pre-bundling
22
+ exclude: ['@xenova/transformers'],
23
+ },
24
+ })