scripts/README.md

Qwen2.5-Instruct for mobile (ONNX) and the transcript refiner

Converts Qwen/Qwen2.5-1.5B-Instruct to int4 ONNX files that ONNX Runtime runs on Android and iOS, ports the transcript refiner (punctuation, misheard words, fillers, numbers, dates and units, bullet lists) to Kotlin and Swift, checks everything against the Python reference, and publishes the public Hugging Face repo pdnhan/Qwen2.5-1.5B-Instruct-mobile (Apache 2.0, used by the app). The same steps convert arcee-ai/Arcee-VyLinh, a Vietnamese merge of VyLinh-3B with Qwen2.5-3B-Instruct (same architecture and tokenizer), to pdnhan/Arcee-VyLinh-mobile (--size VyLinh-3B). It is published under Apache 2.0, the license Arcee AI gives it.

uv venv .venv --python 3.12
uv pip install --python .venv/bin/python --extra-index-url https://download.pytorch.org/whl/cpu \
    "torch==2.6.*" transformers onnx onnx_ir onnxscript onnxruntime onnxruntime-genai numpy regex jiwer \
    protobuf safetensors tokenizers huggingface_hub

hf download Qwen/Qwen2.5-1.5B-Instruct --local-dir data/Qwen2.5-1.5B-Instruct
hf download arcee-ai/Arcee-VyLinh --revision 5f9751c095cdfa14c2e77489f8aef8a037da1b1c --local-dir data/Arcee-VyLinh
cp data/Qwen2.5-1.5B-Instruct/LICENSE data/Arcee-VyLinh/LICENSE   # the repo has no LICENSE file; this is the Apache 2.0 text
# FLEURS Vietnamese test clips and PhoWhisper's int8 exports: see ../mobile_export

.venv/bin/python -I export.py                          # out/<size>/repo (sizes: 1.5B, VyLinh-3B): onnx/, vocab.txt, merges.txt, LICENSE
../mobile_export/.venv/bin/python -I asr_hyps.py       # data/asr_hyps.tsv: PhoWhisper tiny and small transcripts
.venv/bin/python -I validate.py                        # tokenizer, prompt, WER and punctuation -> out/<size>/validation.json, fixtures
(cd kotlin && gradle -q run --args="../out/1.5B/repo ../out/1.5B/fixtures")   # QwenRefiner.kt on the JVM, real ONNX Runtime
sh swift/check/run.sh                                  # the Swift package in a swift:6.0 container, with a stand-in for ONNX Runtime
sh swift/check/real-ort/run.sh 1.5B                    # the same sources with real ONNX Runtime (C library) on the real model
.venv/bin/python -I publish.py                         # model card + code + upload (needs hf auth login)

export.py uses the safetensors weights and runs no code from the model repos (hf_remote=false).

qwen_refine.py is the reference pipeline the Kotlin (kotlin/src/main/kotlin/QwenRefiner.kt) and Swift (swift/Sources/QwenRefiner/) ports follow step by step. The app copies the Kotlin file to app/src/main/java/vn/thithom/stt/refine/QwenRefiner.kt; keep them in sync. A prompt change changes the expected outputs: rerun validate.py and both checks, and update the app's device test.

The Swift code targets ONNX Runtime's Objective-C API, which only exists on Apple platforms. swift/check/run.sh replays the Python reference's tokens through a stand-in; swift/check/real-ort/ implements the same Objective-C API on ONNX Runtime's C library, so the package's unchanged sources run the real model on Linux.