You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Zapformer-48 streaming ASR, Indonesian + English: epoch 48 (final)

A 167M-parameter streaming transducer for Indonesian, with English as a second language, built to run on device. This revision ships the same checkpoint in three formats:

  • ONNX for sherpa-onnx (patched; patch included).
  • LiteRT (.tflite) for Android/iOS LiteRT runtimes.
  • PyTorch weights for re-export.

The architecture is Zapformer, Dan Povey's successor to Zipformer (icefall PR #2082).

Status: this is the final checkpoint of the v2.0 run: epoch 48 of 48. The learning rate has decayed to about 2% of peak. main = epoch 48 + the lm/ folder.

Quick start

Access. This repo is private. Log in once with an account that has access: hf auth login, or set HF_TOKEN.

Pick a runtime:

You want Use Download Needs
Transcribe files or live audio on a desktop or server, with LM / hotwords / two-pass sherpa-onnx (patched, built from source) ~380 MB C++ toolchain + cmake; build β‰ˆ 5 min
The simplest check that the model works, or a reference loop for a mobile app LiteRT (.tflite, pure Python) ~170 MB pip only
Re-export, fine-tune or debug PyTorch (model_code/) ~640 MB pip only

Tested with Python 3.12 on macOS (Apple Silicon) and Linux x86-64. The whole repo is 3.6 GB; download only what you need:

pip install -U huggingface_hub          # provides the `hf` command
R=acul3/zapformer-id-en-streaming
# common files (tokens + example scripts), always needed
hf download $R --local-dir zapformer --include "tokens.txt" --include "bpe.*" --include "examples/*.py"
# sherpa-onnx: 640 ms streaming (~170 MB) + the sherpa patches
hf download $R --local-dir zapformer --include "streaming-640ms/*int8.onnx" --include "streaming-640ms/decoder-*" \
  --include "sherpa-onnx/*"
#   + offline model for --two-pass (~180 MB): --include "offline/*int8.onnx" --include "offline/decoder-*"
#   + language model for --lm (~30 MB):        --include "lm/*int8.onnx" --include "lm/*.fst"
#   + other latencies:                         --include "streaming-320ms/*" --include "streaming-1280ms/*"
# LiteRT (~170 MB)
hf download $R --local-dir zapformer --include "litert/*int8.tflite" --include "litert/init_states.npz" \
  --include "litert/meta.json"
# PyTorch (~640 MB)
hf download $R --local-dir zapformer --include "pretrained-epoch-48-avg-2.pt" --include "model_code/*"

Audio. The model takes 16 kHz mono. examples/decode_file.py resamples WAV/FLAC itself. decode_file_litert.py and model_code/infer.py need a 16 kHz file:

ffmpeg -i input.m4a -ac 1 -ar 16000 audio.wav

Run (from the zapformer/ folder):

# LiteRT: no build needed
pip install ai-edge-litert kaldi-native-fbank soundfile numpy
python examples/decode_file_litert.py audio.wav
# sherpa-onnx: build it first (see "How to run: sherpa-onnx" below)
python examples/decode_file.py audio.wav                       # streaming, one line per utterance
python examples/decode_file.py audio.wav --two-pass --lm       # best accuracy for a final transcript
# PyTorch
pip install torch numpy soundfile kaldi-native-fbank
python model_code/infer.py --weights pretrained-epoch-48-avg-2.pt --tokens tokens.txt --blank-penalty 0.5 audio.wav

Output is cased and punctuated text. decode_file.py prints [mm:ss.ss] text per utterance.

Before using it on code-switched audio, read Known issues.

Results

Real streaming decoding at 640 ms (chunk by chunk, with state carried between chunks), int8, blank penalty 0.5. WER in %; lower is better.

Benchmark (normalizer) This model, greedy This model + LM (beam 4, lm/) Qwen3-ASR-1.7B (offline, ~2B) Nemotron-0.6B P13 (our previous best, 1,040 ms)
conversational-v2: Indonesian conversational speech, 357 human-reviewed clips (lowercase, punctuation stripped) 18.66 18.06 33.45 29.80
GigaSpeechBench-IDN: 21 h, human labels (official IDN normalizer) 21.44 20.31 22.29 29.67
GigaSpeech 2 id test (num2words) 18.07 17.38 17.03 23.38
FLEURS id test (lowercase, punctuation stripped / numbers spelled out) 12.02 / 9.76 10.46 / 8.29 5.65 / 5.12 13.65
FLEURS en test 19.22 17.72 – 15.23
  • Columns:

    • Greedy is the default and cheapest setting.
    • + LM adds a 25 MB LSTM language model and a 4.9 MB LODR bigram with beam search (lm/, ~1.4Γ— compute, ~+100 MB memory). LM settings were tuned on dev sets only; see lm/README.md.
  • Decoding: both columns were decoded with the patched sherpa-onnx (Linux build), at batch 8.

  • Test sets: conversational-v2 is our internal set of 357 human-reviewed Indonesian conversational clips (not public). GSB-IDN, GigaSpeech 2 and FLEURS are public.

  • Comparisons:

    • Qwen3-ASR-1.7B was scored by us on the same clips with the same normalizer (GS2: the full 17,271-clip test). The exception is GSB, where we use the paper's figure.
    • Nemotron P13 is NVIDIA nemotron-3.5-asr-streaming-0.6b fine-tuned on our Indonesian data. It is 3.6Γ— larger and uses 1,040 ms of lookahead.
  • Blank penalty subtracts a constant from the blank logit at every frame (blank_penalty= in sherpa-onnx). It costs nothing at runtime. See "Decoding settings" below for why 0.5 is the default.

GigaSpeechBench-IDN in context (arXiv 2606.28884, Table 2):

System IDN WER
FunASR-Realtime 14.87
Qwen3.5-Omni-Plus 18.05
Chirp 3 19.98
Qwen3-ASR-Flash 20.45
This model + LM (640 ms streaming, 167M + 30 MB LM, int8, on device) 20.31
This model, greedy (640 ms streaming, 167M, int8, on device) 21.44
Qwen3-ASR-1.7B 22.29
ElevenLabs Scribe v2 22.91
Gemini 3.0 Flash 24.18
Azure 25.50
Whisper large-v3 27.40
FunASR-MLT-Nano 27.68
GPT-4o Transcribe / Meta OmniASR-3B 37.95 / 37.91

GSB-IDN has 122 videos. By our video-level bootstrap, a single system's score is uncertain by about Β±2.2 points (95%). The systems between 20 and 23 are therefore effectively tied. Paired differences on the same model, such as two checkpoints or two decoding settings, are good to about Β±0.2.

Learning curve (640 ms, blank penalty 0):

ep 5 ep 10 ep 15 ep 20 ep 25 ep 29 ep 35 ep 40 ep 48
conversational-v2 29.22 26.86 24.83 23.87 21.91 21.51 21.05 19.09 19.17
GSB-IDN 28.28 25.85 25.18 24.18 23.51 22.88 22.80 22.52 22.75
GS2 22.01 20.50 20.07 19.67 19.30 19.04 19.28 18.94 18.80
FLEURS-id 13.84 13.45 13.56 12.90 13.10 13.29 12.86 12.46 12.13
FLEURS-en 35.77 31.03 28.45 26.82 25.11 23.47 22.37 20.80 19.59

Epoch 48 was decoded with our Python streaming loop (same search as sherpa-onnx, batch 1); earlier epochs with sherpa-onnx. Small differences from the Results table (blank penalty 0.5, batch 8) come from that.

Decoding settings

Blank penalty 0.5 (recommended default). As the learning rate decays, the model's wrong-word errors keep falling, but it outputs nothing on more very short clips:

GSB empty outputs (of 22,978 clips) ep 25 ep 29 ep 35 ep 40 ep 48
no output at all 1,659 1,807 2,267 2,449 2,860

A small blank penalty corrects this. Sweep on epoch 35 (GSB quarter subset, which is harder than the full set; full conversational-v2):

blank penalty GSB subset (sub / del / ins) conversational-v2 (sub / del / ins)
0 25.89 (11.2 / 13.6 / 1.0) 21.03 (11.7 / 7.1 / 2.3)
0.5 24.68 (12.0 / 11.4 / 1.3) 20.49 (12.2 / 5.2 / 3.0)
1.0 24.15 (12.7 / 9.9 / 1.6) 20.80 (13.1 / 3.8 / 3.9)
1.5 24.02 (13.2 / 8.7 / 2.1) 21.85 (13.7 / 2.9 / 5.3)
  • Default: use 0.5 for mixed audio.
  • Short-clip audio: 1.0 helps a little more on short clips (full GSB 21.01) but costs about 0.3 on conversational-v2. The 1.0 result was picked by looking at GSB, so we don't headline it.

Punctuation boost. The model outputs cased, punctuated text. In streaming it emits a full stop only once the next sentence starts, so the last sentence of an utterance is often left open.

  • ONNX: the *.punct.* joiners have a small logit boost baked in: . +2, ? +2, , +1.
  • LiteRT: apply the same boost in the search loop; examples/decode_file_litert.py does this.
  • Either way: add . to any utterance that ends without a mark at an endpoint.

On conversational-v2 at epoch 15, the boost raised comma F1 from 24 to 49 and sentence-end F1 from 40 to 74, with WER unchanged. The offline export closes sentences by itself.

Latency modes. One checkpoint supports several modes. These numbers are from epoch 10; the relative gaps have held:

320 ms 640 ms 1,280 ms offline (full context)
conversational-v2 27.79 26.86 26.24 23.34
GSB-IDN 26.96 25.85 25.95 23.96
GS2 21.05 20.50 20.70 19.58
FLEURS-id 13.87 13.45 13.13 12.10
FLEURS-en 33.01 31.03 29.73 27.89
  • Live captions: 640 ms is the best choice.
  • Final transcript: use offline mode, re-decoding each finished utterance ("two-pass", see below).

Files

Path What it is
streaming-640ms/ Main ONNX model. encoder-…int8.onnx, decoder-….onnx, joiner-…int8.onnx, joiner-…punct.int8.onnx, plus fp32 encoder and joiners (for other quantization schemes)
streaming-320ms/, streaming-1280ms/ The same checkpoint exported for 320 ms and 1,280 ms chunks (int8; plain and punct joiners)
offline/ Full-context ONNX export for a second pass or file decoding (int8 + fp32)
litert/ LiteRT 640 ms streaming model: encoder/decoder/joiner.int8.tflite (recommended) and .fp32.tflite, init_states.npz, meta.json, plus the conversion scripts
pretrained-epoch-48-avg-2.pt PyTorch weights ({"model": state_dict}), the averaged model over epochs 46–48, for icefall export*.py
model_code/ Pinned PyTorch model code (icefall PR #2082 at 0ee2a6d5, the version this model was trained with) + zap_model.py loader + infer.py streaming transcription. Runs without icefall/k2/lhotse. Newer upstream Zapformer code has a different architecture and cannot load these weights
lm/ Language model for rare words: lstm-lm-2x1024.int8.onnx (25 MB) + lodr-2gram-p300k.fst (4.9 MB), fp32 LM, and its README (results, settings, how it was built)
tokens.txt, bpe.model, bpe.vocab SentencePiece unigram, 4,096 tokens, cased and punctuated, byte fallback, digits split. bpe.vocab is what sherpa needs to tokenize hotwords
sherpa-onnx/ The sherpa-onnx patches this model needs, plus the ONNX export scripts
examples/decode_file.py Transcribe a file with sherpa-onnx: streaming pass, optional --two-pass, --lm, --hotwords "A/B" or --hotwords-file, --latency 320/640/1280, --beam, --threads, --no-punct-boost. Resamples WAV/FLAC to 16 kHz
examples/decode_file_litert.py Transcribe a file with the LiteRT model in pure Python; this is the reference loop for a mobile runtime
examples/poc/ Internal web demo (live mic, latency switch, two-pass). Not runnable from this repo: it is hard-coded to the author's machine layout. Use decode_file.py instead

How to run: sherpa-onnx (ONNX)

1. Build sherpa-onnx with the Zapformer patch. Stock sherpa-onnx can't load this model: Zapformer has 9 streaming states per layer (326 in total), and its causal norms start from learned values.

Needs git, cmake (β‰₯ 3.13) and a C++17 compiler (Xcode command-line tools on macOS, build-essential on Linux). The build downloads a prebuilt ONNX Runtime and takes about 5 minutes. Clone sherpa-onnx next to the model folder, not inside it (the model repo already has a sherpa-onnx/ folder holding the patches):

ZAP=$PWD/zapformer                                        # the folder you downloaded the model into
git clone https://github.com/k2-fsa/sherpa-onnx sherpa-onnx-src && cd sherpa-onnx-src
git checkout 040afe360a38e25daaa325ce8889abf93ea02609
git apply $ZAP/sherpa-onnx/sherpa-onnx-zapformer.patch    # model type "zapformer"
git apply $ZAP/sherpa-onnx/sherpa-onnx-ort-1.30.patch     # ONNX Runtime 1.30: ~150 MB less memory
python -m venv .venv && . .venv/bin/activate              # any venv; Python 3.12 tested
pip install . soundfile numpy scipy                       # Python package, used by examples/decode_file.py
cd $ZAP

For Android/iOS, run ./build-android-arm64-v8a.sh or ./build-ios.sh in sherpa-onnx-src instead of pip install . (details in sherpa-onnx/README.md).

2. Streaming (live captions):

import sherpa_onnx
d, s = "streaming-640ms", "epoch-48-avg-2-chunk-32-left-128"
rec = sherpa_onnx.OnlineRecognizer.from_transducer(
    tokens="tokens.txt", encoder=f"{d}/encoder-{s}.int8.onnx", decoder=f"{d}/decoder-{s}.onnx",
    joiner=f"{d}/joiner-{s}.punct.int8.onnx", num_threads=1, decoding_method="greedy_search", blank_penalty=0.5,
    enable_endpoint_detection=True, rule2_min_trailing_silence=1.2, rule3_min_utterance_length=20)
stream = rec.create_stream()
# loop: stream.accept_waveform(16000, pcm_float32); while rec.is_ready(stream): rec.decode_stream(stream)
#       rec.get_result(stream) -> partial text
#       rec.is_endpoint(stream) -> utterance done: take the text, then start a NEW stream (not rec.reset(stream)),
#       feeding it the last ~1 s of audio first (see Known issues; examples/decode_file.py does this)
  • Audio: 16 kHz mono float32, samples in [βˆ’1, 1].
  • New stream per utterance: rec.reset(stream) clears the text but keeps the encoder state. Keeping that state across utterances makes the model drop English that follows Indonesian, so create a fresh stream at each endpoint.
  • Tail padding: append about 1 s of silence before input_finished() when decoding a file or push-to-talk. Otherwise the last chunk is never decoded.

3. Two-pass / offline. Use sherpa_onnx.OfflineRecognizer.from_transducer(...) with the offline/ files, which needs no patch. Two quirks of this export:

  • Length limit: inputs must be at most about 30 s, so split longer audio.
  • Allowed lengths: it only accepts fbank lengths with frames % 8 == 5. Pad up to 70 ms of silence, as examples/decode_file.py does. Without the padding ONNX Runtime fails with an error like Where node … broadcast 513 by 516.
  • Cutting utterances: cut each one using the streaming decoder's timing (start_time + last token timestamp + ~0.7 s), not the moment the endpoint fired.
python examples/decode_file.py audio.wav --latency 640 --two-pass

decode_file.py keeps the streaming text for a segment when the offline pass returns far fewer words (the offline model can drop leading English, see Known issues). Segment boundaries can repeat a word.

4. Rare words: LM and hotwords (both need decoding_method="modified_beam_search", max_active_paths=4).

  • LM (lm/): general language knowledge from a large text corpus. It fixes common words and text-known names, and improves every test set (table above). Use lm_scale=0.3 together with lodr_fst=lm/lodr-2gram-p300k.fst, lodr_scale=-0.25. Without LODR the decoder drops words.
  • Hotwords: boost phrases you know in advance, such as speaker names, client names or a product glossary. Pass them per stream with rec.create_stream(hotwords="Yohannis Lema/RKP/Bhinneka") or globally with hotwords_file=, plus modeling_unit="bpe", bpe_vocab="bpe.vocab", hotwords_score=2.0.
    • Case-sensitive: lowercase-only lists lose ~8 points of recall on names. Add case variants (as given + lowercase + Title + UPPER). In our test that beat even perfectly cased lists: conversational rare-word recall 60% vs 58% (vs 38% without hotwords).
    • Our test: each utterance got its rare words plus 100 distractors. Rare-word recall rose from 38–45% to 58–67% with no harm to other words.
    • With the LM: LM + hotwords (score 2.0) is the best combination. On FLEURS-id dev, WER drops 9.9 β†’ 8.0 (LM alone 8.8, hotwords alone 9.2). On conversational speech, hotwords carry the gain.
  • Score: 2.0 is the tuned default over many utterances. A single stubborn word may need more (in one test, "Bhinneka" needed 8 with hotwords alone but was fixed at 2.0 together with --lm). Higher scores add false insertions, so check on your own audio.
  • They complement each other: the LM can't know session-specific names, and hotwords can't fix general language. Both can run together.
python examples/decode_file.py audio.wav --lm --hotwords "Yohannis Lema/Bhinneka/CCUS"

How to run: LiteRT (.tflite)

litert/ holds the 640 ms streaming model, converted from PyTorch with litert-torch 0.9.4 and quantized with ai-edge-quantizer (dynamic int8: int8 weights, activations quantized on the fly). This is the same scheme as the ONNX int8 export. Accuracy matches ONNX int8 within eval noise (Β±0.25), and the encoder is about 25% faster per chunk on Mac CPU.

Epoch 48 (same Python loop, batch 1) LiteRT int8 ONNX int8
encoder size 155.7 MiB 155.8 MiB
conversational-v2, blank penalty 0 18.95 19.17
FLEURS-id (norm / n2w), blank penalty 0 11.55 / 9.61 12.13 / 9.96
encoder time per 640 ms chunk (Mac M5, 1 thread; measured on epoch 35, same model size) 16.4 ms 21.8 ms
peak memory of the benchmark process (Python) 424 MB 351 MB
pip install ai-edge-litert kaldi-native-fbank soundfile numpy
python examples/decode_file_litert.py audio.wav            # int8, blank penalty 0.5, punctuation boost

This decodes the whole file as one stream with no endpointing, so it shows the code-switching issue below on mixed audio. Use it for single utterances or as the reference loop; use decode_file.py for long recordings.

Interface (what an Android/iOS runtime must implement):

  • Encoder (serving_default): 327 inputs, 327 outputs.
    • Inputs: args_0 is fbank [1, 71, 80] float32; args_1_<i> is state i.
    • Outputs: output_0 is encoder_out [1, 16, 576]; output_<i+1> is the new state i.
    • States: 36 layers Γ— 9 (cached_key, cached_value, cached_conv, cached_norm_stats, cached_norm_len, cached_attn_wm_sum, cached_attn_wm_num_frames, cached_conv_wm_sum, cached_conv_wm_num_frames), then embed_cache [1, 128, 6, 19] and processed_lens [1]. 73 states are int64.
    • Shapes and dtypes are in meta.json.
    • Start every stream from init_states.npz, not zeros. The causal-norm states have learned initial values.
  • Features: kaldi fbank, 80 bins, dither 0, snip_edges=false, high_freq=-400, samples in [βˆ’1, 1].
    • Feed 71 frames and advance by 64.
    • At end of stream, pad ~1 s of silence.
  • Decoder: args_0 is the context [1, 2] int64. Start with [-1, 0]; βˆ’1 is masked inside the model. Output is [1, 576].
  • Joiner: args_0 is encoder frame [1, 576], args_1 is decoder out [1, 576]. Output is logits [1, 4096].
  • Search: greedy, at most one symbol per frame, blank = 0.
    • Subtract the blank penalty (0.5) from logit[0].
    • Optionally add the punctuation boost (token 260 . +2, 275 ? +2, 259 , +1).
  • Tokens: ▁ means a space; <0xNN> are byte-fallback pieces (decode as UTF-8).
  • Not provided yet:
    • A Kotlin/C++ runtime. sherpa-onnx can't load .tflite, so use the LiteRT CompiledModel API plus kaldi-native-fbank plus the loop above.
    • GPU/NPU delegates. These need int32 states, state packing and full static int8.
    • A LiteRT offline (two-pass) encoder.
  • Re-export: the conversion scripts are in litert/scripts/, with their own README.

How to run: PyTorch

model_code/ has the exact model code this checkpoint was trained with, frozen. Upstream Zapformer changed the architecture after 2026-08: the central stack has no conv and no second feed-forward, and it uses SiLU. Newer upstream code cannot load these weights.

pip install torch numpy soundfile kaldi-native-fbank
python model_code/infer.py --weights pretrained-epoch-48-avg-2.pt --tokens tokens.txt --blank-penalty 0.5 audio.wav

Pass --blank-penalty 0.5 explicitly (the script's default is 0). Like the LiteRT example, it decodes the whole file as one stream, with no endpointing and no punctuation boost.

infer.py runs real 640 ms streaming with greedy search, the same algorithm as sherpa-onnx. On FLEURS-id, 17/20 outputs were identical to the int8 ONNX model; the rest differed by a comma or one word (fp32 vs int8). For your own code, use from zap_model import load_model.

Speed and memory

Apple M5, single stream, int8, ONNX Runtime 1.30 (the model size is the same at every epoch):

value
RTF, 640 ms, 1 thread, greedy 0.037–0.039 (about 24 ms of compute per 640 ms chunk)
RTF, 640 ms, 1 thread, beam 4 + LM + LODR (300k) 0.052 (33 ms per chunk), peak memory +100 MB
RTF, 320 ms / 1,280 ms (2 threads) 0.055 / 0.027
Memory while streaming (Python process) ~370 MB
Time to first partial result 0.8 s at 640 ms

One thread is the best setting. Phone numbers are still to be measured; expect roughly 2–6Γ— the Mac's RTF, which is still well within real time.

Throughput vs battery. A single live stream is light: about 4% of one core. Batch-decoding hours of audio keeps every core busy, so plan for mains power.

Model

Architecture Zapformer (icefall PR #2082, danpovey/zapformer3127 @ 0ee2a6d5), causal, --base-dim 48
Parameters 166.9M total (encoder 156.5M; stateless decoder with a 2-token context; joiner)
Frontend 80-dim fbank (lhotse = kaldi-native-fbank; sherpa defaults)
Training losses pruned RNN-T + CTC auxiliary (scale 0.2)
Streaming training random chunk sizes {16, 32, 64, full} Γ— left context {64, 128, 256, full}, at 50 Hz after the embed layer
Optimizer / schedule BatchedRubik (Muon-style), base LR 0.02, InterpCosine over 48 virtual epochs
Hardware / precision 2 Γ— A100-80GB, bf16, 1,000 s of audio per GPU per batch
Augmentation synthetic room reverb (p = 0.2), Opus/MP3 recompression (p = 0.1), on-GPU alternating SpecAugment
Checkpoint epoch 48 (final), --avg 2 --use-averaged-model 1 (averaged model between epochs 46 and 48)

Training data

Indonesian and English speech, mostly Indonesian. The data sources are not disclosed. All test sets reported above were excluded from training.

Known issues

  • English after Indonesian in the same stream is dropped. Once the encoder has processed Indonesian speech, English that follows in the same stream comes out empty or nearly empty. Silence in between does not help (tested 0–5 s); English β†’ Indonesian and Indonesian β†’ Indonesian are fine. It happens in all three runtimes and comes from the encoder's streaming state (its causal norms keep running statistics over the whole stream).
    • Workaround (verified): start a new stream at every endpoint, re-feeding the last ~1 s of audio. examples/decode_file.py does this by default (--keep-state turns it off). English then decodes normally after Indonesian utterances.
    • Still affected: English inside the same utterance as preceding Indonesian (no pause long enough to trigger an endpoint), and the single-stream examples (decode_file_litert.py, model_code/infer.py).
    • Offline model: in full-context mode, English at the start of a segment that is mostly Indonesian can be dropped.
    • Found on short test clips (conversational Indonesian + read English, and synthetic voices); not yet measured on real code-switched recordings.

Limitations

  • Spurious "Hai". On read speech the model sometimes starts an utterance with "Hai" (18 times in the 687 FLEURS-id clips; never on conversational-v2). This is a label artifact in the training data. It will be cleaned for the next model.
  • Rare words and names. The prediction network only sees the last 2 tokens. Foreign names, technical terms and Indonesian look-alike words are the main real errors. This is why FLEURS-id is far behind LLM-based ASR (Qwen3-ASR: 5.7). The bundled LM recovers part of this (FLEURS-id 12.0 β†’ 10.5). Hotwords fix names you know in advance. Targeted rare-vocabulary training data is the next fix.
  • Very short clips. Clips under 1.5 s are often left empty, and this grew in the last epochs (GSB: 2,860 empty clips at epoch 48 with blank penalty 0). Use blank penalty 0.5. Most live speech is longer, so the effect on real use is smaller than on GSB.
  • English is secondary (FLEURS-en 19.6). More English data is planned for the next, larger model.
  • Long silences inside a sentence (over 1.2 s) end an utterance with the default endpoint rules.
  • Quantization is dynamic int8 only, in both ONNX and LiteRT. There's no static int8 for NPUs yet.

Other runtimes (not in this repo)

An ExecuTorch (.pte) export of this checkpoint for react-native-executorch (greedy, 640 ms, int8, fbank inside the model) exists but is not published here; ask the maintainer.

Intended use

Internal research and product evaluation (on-device Indonesian speech recognition). Private repository.

Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Paper for acul3/zapformer-id-en-streaming