Mac: the server uses 1 CPU thread on macOS (more threads are slower on Apple silicon); card: measured M3 Max speeds, and the 8-thread CPU figure is a server, not a laptop

README.md

@@ -37,7 +37,7 @@ Streaming Thai speech-to-text with 0.64B parameters. It writes Thai speech down
 <tr><td><b>Training data</b></td><td>12,373 hours of Thai speech and 961 hours of English (LibriSpeech)</td></tr>
 <tr><td><b>Input</b></td><td>16 kHz mono audio (128 log-mel bands); any length when cut at pauses, as the included server does</td></tr>
 <tr><td><b>Files</b></td><td>Two <code>.nemo</code> files of 2.56 GB each, 32-bit weights (see <a href="#use">Use</a>). An ONNX export, full precision and int8, is at <a href="https://huggingface.co/iapp/OpenThai2.0-ASR-Streaming-ONNX">iapp/OpenThai2.0-ASR-Streaming-ONNX</a></td></tr>
-<tr><td><b>Speed</b></td><td>One NVIDIA H100: files about 257× faster than real time, about 20 live streams at once at 560 ms. A CPU with 8 threads: files about 4× real time, a live stream keeps up at 560 and 320 ms</td></tr>
+<tr><td><b>Speed</b></td><td>One NVIDIA H100: files about 257× faster than real time, about 20 live streams at once at 560 ms. A server CPU limited to 8 threads: files about 4× real time, a live stream keeps up at 560 and 320 ms. Apple M3 Max with 1 thread: files about 3× real time, a live stream keeps up at 560 and 320 ms</td></tr>
 <tr><td><b>Memory</b></td><td>About 2.6 GB for the weights; about 6.5 GB of RAM in all on a CPU</td></tr>
 <tr><td><b>Licence</b></td><td>OpenMDW-1.1</td></tr>
 </table>
@@ -220,7 +220,9 @@ print(transcribe_long(asr, load_audio("talk.wav"), policy=MODES["standard"])["di
 
 All fields, environment variables and error codes: [`serve/README.md`](serve/README.md). The service has no authentication; keep it on localhost or behind your own gateway.
 
-**Speed.** On one NVIDIA H100, files decode about 257 times faster than real time, and one copy of the model keeps up with about 20 live streams at 560 ms. On a laptop-class CPU with 8 threads, files decode about 4 times faster than real time and live streams keep up at 560 and 320 ms, with about 6.5 GB of memory.
+**Speed.** On one NVIDIA H100, files decode about 257 times faster than real time, and one copy of the model keeps up with about 20 live streams at 560 ms. On a server CPU limited to 8 threads, files decode about 4 times faster than real time and live streams keep up at 560 and 320 ms, with about 6.5 GB of memory.
+
+**On a Mac (Apple silicon), use one CPU thread.** There, every extra thread makes decoding slower: on an M3 Max a 7.6-second clip takes 2.3 s with 1 thread, 6.4 s with 4 and 20.7 s with 16. The reference server uses 1 thread on macOS unless `CHINDA_NUM_THREADS` says otherwise. For NeMo's streaming script, run it with `OMP_NUM_THREADS=1`; in your own Python code, call `torch.set_num_threads(1)`. With 1 thread on the M3 Max, live streams keep up at 560 and 320 ms; at 80 ms it decodes at only about real time, with no margin, and memory peaks at about 6 GB. The model runs on the CPU only on a Mac: the server does not use the Apple GPU (MPS).
 
 ## What the output looks like
 

serve/README.md

@@ -46,7 +46,7 @@ The blank bias δ controls how readily the model writes something: the more nega
 | `CHINDA_MODEL` | `../openthai2.0-asr-streaming_raw.nemo` | must be the raw file; `*_bias-*` files are refused (the bias would apply twice) |
 | `CHINDA_PRELOAD` | unset | `1` = load at start-up; otherwise the first request waits about 20 s |
 | `CHINDA_REPLICAS` | `1` | model copies, about 2.6 GB each; each copy runs one decode step at a time |
-| `CHINDA_NUM_THREADS` | min(16, CPUs) | CPU threads |
+| `CHINDA_NUM_THREADS` | min(16, CPUs); 1 on macOS | CPU threads. On Apple silicon each extra thread makes decoding slower, so macOS uses 1 |
 | `CHINDA_DIAR_MODEL` | download | path to a local `Nemotron-3-Diarization.nemo` |
 | `CHINDA_CUDA_GRAPHS` | `0` | leave at 0: `1` crashes on the request after speaker labels |
 | `CHINDA_SPLIT_FROM_S` | `30` | `split=auto` cuts files longer than this many seconds |

serve/app.py

@@ -5,7 +5,7 @@
 Environment:
   CHINDA_MODEL        raw (unbaked) .nemo (default ../openthai2.0-asr-streaming_raw.nemo)
   CHINDA_DEVICE       cpu (default) | cuda
-  CHINDA_NUM_THREADS  torch CPU threads (default min(16, cpu_count))
+  CHINDA_NUM_THREADS  torch CPU threads (default min(16, cpu_count); 1 on macOS, where more threads are slower)
   CHINDA_REPLICAS     model replicas (default 1; each ~2.6 GB)
   CHINDA_PRELOAD      1 = load the model at startup instead of on the first request
   CHINDA_DIAR_MODEL   Nemotron-3-Diarization .nemo for speaker labels (default: downloaded from Hugging Face)

serve/chinda_asr.py

@@ -53,6 +53,22 @@ import numpy as np
 log = logging.getLogger("chinda_asr")
 
 SAMPLE_RATE = 16000
+
+
+def default_threads() -> int:
+    """CPU threads when CHINDA_NUM_THREADS is not set.
+
+    macOS: 1. On Apple silicon, PyTorch's CPU path gets slower with every extra thread for this model
+    (M3 Max, a 7.6 s clip: 1 thread about 2 s, 16 threads about 21 s; measured 13 Nov 2026).
+    Elsewhere: min(16, CPUs).
+    """
+    import sys
+
+    if sys.platform == "darwin":
+        return 1
+    return min(16, os.cpu_count() or 1)
+
+
 FEATURE_TOL = 1e-3  # max |log-mel difference| accepted between streamed and whole-utterance features
                     # (GPU rounding noise); larger => the stream is re-decoded from the buffered audio
 HOP = 160  # samples per feature frame (preprocessor window_stride 0.01 s)
@@ -394,7 +410,7 @@ class ChindaASR:
             )
         self.model_path = model_path
         self.device = device or os.environ.get("CHINDA_DEVICE", "cpu")   # "cuda" / "cuda:0" on a GPU host
-        self.num_threads = num_threads or int(os.environ.get("CHINDA_NUM_THREADS", "0")) or min(16, os.cpu_count() or 1)
+        self.num_threads = num_threads or int(os.environ.get("CHINDA_NUM_THREADS", "0")) or default_threads()
         self.n_replicas = max(1, int(replicas))
         self.verify_stream_max_s = verify_stream_max_s
         self._load_lock = threading.Lock()