mlboydaisuke commited on
Commit
3835eff
·
verified ·
1 Parent(s): a37547b

README and manifest notes: plainer wording ("chunked by the runtime"; the 20 LibriSpeech clips named)

Browse files
Files changed (2) hide show
  1. README.md +2 -2
  2. litertlm_manifest.json +1 -1
README.md CHANGED
@@ -104,7 +104,7 @@ If you downloaded this file with the litert-lm CLI's `--from-huggingface-repo` b
104
  Notes:
105
 
106
  - Greedy decoding (top_k 1) is what the vendor runtime does; the model emits plain text and stops on `<|im_end|>`.
107
- - The encoder window is **30 s**. Longer clips are chunked by the engine (30 s windows, no overlap), which resets the encoder's context at each boundary; a 10 s window measured +1.6 pp WER on this set from those resets, which is why the shipped window is 30 s.
108
  - KV budget 2048 tokens: 30 s of audio is 225 tokens plus the ~40-token prompt, leaving room for the transcript.
109
 
110
  ## Performance
@@ -120,7 +120,7 @@ Everything in this section above its last part was measured on earlier files (th
120
 
121
  Init is the `Init time` the CLI prints; that value includes both `Init Total` and the phases inside it, so it counts engine creation twice ([LiteRT-LM #4057](https://github.com/google-ai-edge/LiteRT-LM/issues/4057)).
122
 
123
- End-to-end on the fixture set (engine already loaded, one conversation per clip): real-time factor **0.24 on CPU** and **0.19 with the LM on Metal**, i.e. a 10 s clip transcribes in about 2 s.
124
 
125
  Galaxy S26 (SM-S942Q, Snapdragon SM8850, Adreno), `litert_lm_advanced_main` built from the litert-lm v0.16.1 tag, `--benchmark` with a 264-token text prompt, one reading per configuration (LM only):
126
 
 
104
  Notes:
105
 
106
  - Greedy decoding (top_k 1) is what the vendor runtime does; the model emits plain text and stops on `<|im_end|>`.
107
+ - The encoder window is **30 s**. Longer clips are chunked by the runtime (30 s windows, no overlap), which resets the encoder's context at each boundary; a 10 s window measured +1.6 pp WER on this set from those resets, which is why the shipped window is 30 s.
108
  - KV budget 2048 tokens: 30 s of audio is 225 tokens plus the ~40-token prompt, leaving room for the transcript.
109
 
110
  ## Performance
 
120
 
121
  Init is the `Init time` the CLI prints; that value includes both `Init Total` and the phases inside it, so it counts engine creation twice ([LiteRT-LM #4057](https://github.com/google-ai-edge/LiteRT-LM/issues/4057)).
122
 
123
+ End-to-end on the 20 LibriSpeech clips above (engine already loaded, one conversation per clip): real-time factor **0.24 on CPU** and **0.19 with the LM on Metal**, i.e. a 10 s clip transcribes in about 2 s.
124
 
125
  Galaxy S26 (SM-S942Q, Snapdragon SM8850, Adreno), `litert_lm_advanced_main` built from the litert-lm v0.16.1 tag, `--benchmark` with a 264-token text prompt, one reading per configuration (LM only):
126
 
litertlm_manifest.json CHANGED
@@ -83,7 +83,7 @@
83
  "platform_notes": [
84
  "Audio only through the runtime's generic audio path: send the clip as an audio content item (wav/mp3/flac; any sample rate — the runtime resamples to 24 kHz) plus the text 'This is a X.XX seconds audio, please transcribe it.' (X.XX = clip length); with no text the file's template sends 'Please transcribe it.' (+0.9 pp WER on the 20 LibriSpeech clips)",
85
  "audio_backend must be cpu (--audio_backend=cpu / audio_backend=CPU()); the LM may run on gpu",
86
- "The encoder window is 30 s; longer clips are chunked by the engine with a context reset at each 30 s boundary",
87
  "Accuracy: 20 LibriSpeech dev-clean clips (448 words): WER 2.68 % on macOS CPU and 2.46 % LM-on-GPU (fp32 PyTorch reference 2.68 %), 3.12 % Galaxy S26 CPU, 2.46 % Galaxy S26 LM-on-GPU; vendor card Libri-clean 2.41 % (full test set, VibeASR.cpp)",
88
  "Peak private footprint on Galaxy S26: 3227 MB CPU/CPU, 2268 MB LM-on-GPU (--report_peak_memory_footprint, 5.9 s clip)",
89
  "2026-11-17 GPU graph (weights unchanged): the prefill/decode graph has the GPU composite form (28 odml.cache_update composites for the KV cache and 56 attention composites; the previous file had none); every weight and every section outside that graph has the same size and sha256 as in the previous file (Hub revision 0d3aca6f). This file was tested on litert-lm 0.19.0-0.19.2 (macOS) and litert_lm_advanced_main v0.19.0 (Galaxy S26, Pixel 11 Pro). On macOS with litert-lm 0.19.2 this file and the previous file gave the same transcripts of the 20 clips: 18/448 word errors (4.02 %) with both files on the CPU with 1 thread, 11/448 (2.46 %) with both files with the LM on the GPU. The figures in the notes above, peak_ram_mb, the macOS recommendation and the measured rows dated 2026-09-04 were measured on earlier files."
 
83
  "platform_notes": [
84
  "Audio only through the runtime's generic audio path: send the clip as an audio content item (wav/mp3/flac; any sample rate — the runtime resamples to 24 kHz) plus the text 'This is a X.XX seconds audio, please transcribe it.' (X.XX = clip length); with no text the file's template sends 'Please transcribe it.' (+0.9 pp WER on the 20 LibriSpeech clips)",
85
  "audio_backend must be cpu (--audio_backend=cpu / audio_backend=CPU()); the LM may run on gpu",
86
+ "The encoder window is 30 s; longer clips are chunked by the runtime with a context reset at each 30 s boundary",
87
  "Accuracy: 20 LibriSpeech dev-clean clips (448 words): WER 2.68 % on macOS CPU and 2.46 % LM-on-GPU (fp32 PyTorch reference 2.68 %), 3.12 % Galaxy S26 CPU, 2.46 % Galaxy S26 LM-on-GPU; vendor card Libri-clean 2.41 % (full test set, VibeASR.cpp)",
88
  "Peak private footprint on Galaxy S26: 3227 MB CPU/CPU, 2268 MB LM-on-GPU (--report_peak_memory_footprint, 5.9 s clip)",
89
  "2026-11-17 GPU graph (weights unchanged): the prefill/decode graph has the GPU composite form (28 odml.cache_update composites for the KV cache and 56 attention composites; the previous file had none); every weight and every section outside that graph has the same size and sha256 as in the previous file (Hub revision 0d3aca6f). This file was tested on litert-lm 0.19.0-0.19.2 (macOS) and litert_lm_advanced_main v0.19.0 (Galaxy S26, Pixel 11 Pro). On macOS with litert-lm 0.19.2 this file and the previous file gave the same transcripts of the 20 clips: 18/448 word errors (4.02 %) with both files on the CPU with 1 thread, 11/448 (2.46 %) with both files with the LM on the GPU. The figures in the notes above, peak_ram_mb, the macOS recommendation and the measured rows dated 2026-09-04 were measured on earlier files."