Update README.md

README.md

@@ -50,6 +50,45 @@ model-index:
 
 # Model card: whisper-large-v3 + LoRA "main" (machine-sound adapter) and voice agent
 
+## Summary
+
+This model is a LoRA adapter for OpenAI's Whisper large-v3. It teaches Whisper to **describe industrial
+machine sounds** as structured text instead of transcribing speech. Given a 10-second clip, it outputs a
+line such as `machine: fan; condition: high load; status: anomalous; fault: overheating; noise: low noise`.
+Engine clips get an RPM and torque estimate instead. It still transcribes English speech as well as the
+original model.
+
+A companion voice agent puts it behind a phone line (Azure Communication Services media stream), a browser
+microphone, or file upload. Speech gets a spoken read-back ("I heard: …"), and machine sounds get a spoken
+diagnosis ("Sound check. This is a fan under high load. It sounds abnormal. Possible fault: overheating.").
+Speech recognition and synthesis run locally, with no cloud AI services.
+
+**Training**
+- **Data:** about 24,500 clips (70 h). 20,000 come from AnomalyMachine-50K (6 machine types, 3 load
+  conditions, 5 fault types, 4 noise levels, balanced normal/anomalous) and 4,497 from Procedural Engine
+  Sounds. **Both datasets are synthetic**, and neither contains speech.
+- **Method:** LoRA (r 32, alpha 64) on all attention and feed-forward layers of the encoder and decoder,
+  with the base model frozen in 8-bit.
+  - 3 epochs, effective batch 32, learning rate 1e-4 with cosine decay.
+  - 9 hours on one RTX 5070 (12 GB).
+- **Results:** measured on a 600-clip validation subset ([§5](#5-evaluation)):
+
+  | Metric | Value |
+  |---|---|
+  | Template WER | 0.185 |
+  | Exact template match | 22% |
+  | Mean per-field accuracy | 60% |
+  | Normal/anomalous F1 | 1.00 |
+
+  The test split has not been evaluated yet.
+
+**Limits:** it is untested on real machines, real microphones or real operators. Engine RPM and torque
+estimates are unreliable, and spoken alarm codes can be misread ([§7](#7-limitations-and-risks)). It is a
+research prototype and **non-commercial only**, because both the engine training data and the voice model
+are CC-BY-NC-4.0 ([§9](#9-licences)).
+
+---
+
 | | |
 |---|---|
 | **Model** | `whisper_ft/out/whisper-lora/main/adapter`: LoRA adapter for `openai/whisper-large-v3` |
@@ -242,6 +281,52 @@ so there are no pauses between code characters.
 
 ## 6. How to run inference
 
+### 6.1 Quick start: audio file in → voice file out
+
+One command creates the environment on its first run (skipping steps already done), then answers the
+audio with a spoken reply: [run_voice_inference.sh](run_voice_inference.sh).
+
+```bash
+cd finetune_whisper
+bash run_voice_inference.sh recording/section_00_source_test_anomaly_0021.wav
+```
+
+Output (verified 2026-11-12):
+
+```
+[   0.0-  10.0s] heard: machine sound: machine: fan; condition: high load; status: anomalous; fault: overheating; noise: low noise
+                 reply: Sound check. This is a fan under high load. It sounds abnormal. Possible fault: overheating. The recording has low background noise.
+{"segments": 1, "spoken": 1, "skipped": 0, "reply_s": 11.02, "out_wav": ".../out/speak/section_00_source_test_anomaly_0021_reply.wav"}
+```
+
+The voice reply is in `out/speak/<audio name>_reply.wav`. For speech, the output is `heard: <transcript>`
+and `reply: I heard: <transcript>`.
+
+### 6.2 The same steps by hand
+
+```bash
+cd finetune_whisper
+
+# 1. Python environment (once)
+conda create -n mfg_whisper python=3.11 -y
+conda activate mfg_whisper
+pip install torch                      # PyPI wheel supports RTX 50-series (sm_120)
+pip install -r requirements.txt
+
+# 2. Models (once): base Whisper (~3 GB, pinned revision) and the TTS voice (~145 MB)
+python scripts/download_model.py --no-xet
+python scripts/download_tts_model.py --no-xet
+#    The adapter whisper_ft/out/whisper-lora/main/adapter comes from training (whisper_ft/README.md).
+
+# 3. Audio file → Whisper + adapter → text → TTS → voice file
+python speak_transcript.py recording/section_00_source_test_anomaly_0021.wav --show
+#    options: --out-wav reply.wav   --out-json reply.json (texts, mode 0600)
+#             --reply-prefix ""     (speak the transcript without "I heard:")
+#             --adapter ""          (stock Whisper: no machine-sound diagnosis)
+```
+
+### 6.3 Other entry points
+
 Conda env `mfg_whisper`; run from `finetune_whisper/`.
 
 ```bash