mlboydaisuke/Qwen3-0.6B-Code-Expert-LiteRT

Qwen3-0.6B-Code-Expert — LiteRT-LM (int8)

suayptalha/Qwen3-0.6B-Code-Expert, a full fine-tune of Qwen/Qwen3-0.6B on nvidia/OpenCodeReasoning by its author (suayptalha), converted to the .litertlm format for Google's LiteRT-LM runtime. It is the representative model of the recipe How do I run a fine-tuned Hugging Face model on iPhone?: a community fine-tune under 1B parameters, Apache-2.0, converted with one command and loaded in an iOS app through swift-litert-lm.

File	model.litertlm, 613,406,208 bytes
sha256	7a4ec53564ef677f64dcb2de88017fccd1c00aa610495b24c089493278720fe6
Quantization	int8 weights (the converter's default)
Context (KV cache)	4096 tokens
Template	the fine-tune's own Qwen3 chat template, embedded verbatim; thought channel <think> / </think>; stop tokens <|im_end|> (151645) and <|endoftext|> (151643)
Source revision	02c021d (2025-05-11)

2026-09-21: chat template updated to accept the 0.18 content-parts form (string form unchanged); weights, tokenizer and executor metadata byte-identical.

How it was made

git clone https://github.com/john-rocky/hf-to-litertlm && cd hf-to-litertlm
pip install litert-torch ai-edge-quantizer "transformers==5.14.*" huggingface_hub litert-lm
python scripts/convert.py suayptalha/Qwen3-0.6B-Code-Expert     # 217 s on a Mac Studio M4 Max

Stock litert-torch 0.9.3 export (ai-edge-quantizer 0.8.0, transformers 5.14.1, litert-lm-builder 0.15.0), 2026-09-05. The converter's exit gate asks 8 fixed questions through the runtime: this bundle answered 7 of 8 with no degenerate output (the miss: "Roses are red, violets are ___" answered "violet"); median decode during the gate 138 tok/s on the Mac GPU.

Run it

Desktop, with the LiteRT-LM CLI (macOS, Linux, Windows):

pip install litert-lm
litert-lm run --from-huggingface-repo mlboydaisuke/Qwen3-0.6B-Code-Expert-LiteRT model.litertlm \
  --prompt "What is 17 + 25? Answer briefly." --thinking false --temperature 0 --top-k 1
# 17 + 25 = 42

If run stops right after the download with no error message from the CLI (a bus error; exit status 138 on a Mac) or with an error log that contains too large for file size, the downloaded file may be incomplete. The litert-lm CLI (0.14.0 to 0.19.2, and litert-lm-nightly 0.20.0.dev20261112) can keep an incomplete download in ~/.litert-lm/cache/huggingface/mlboydaisuke/Qwen3-0.6B-Code-Expert-LiteRT/ and uses it again on later runs without checking its size. Delete that file and run the command again.

Keep --thinking false when you sample greedily. With thinking on and --temperature 0 --top-k 1 the same prompt looped on "17 + 25." inside the thought channel for 128 s and never answered; the Qwen3 model card says not to use greedy decoding in thinking mode. With the CLI's default sampler and thinking on (litert-lm 0.17.0), the model thought for one paragraph and answered 17 + 25 = 42.

iOS, in an existing app, through swift-litert-lm (the same runtime, Metal GPU):

import LiteRTFoundation
let chat = try await LiteRTChat(
  huggingFaceRepo: "mlboydaisuke/Qwen3-0.6B-Code-Expert-LiteRT", fileName: "model.litertlm",
  modalities: [])                       // text-only: no vision or audio tower
for try await delta in chat.stream("What is 17 + 25? Answer briefly.") { print(delta, terminator: "") }

Dependency, stop, release and the verify command are in the recipe.

Android: the file is the format Google AI Edge Gallery imports; not tested on Android here.

Measured (2026-09-05)

Mac Studio, Apple M4 Max, 128 GB, macOS 27.0, prompt "What is 17 + 25? Answer briefly.", reply 17 + 25 = 42 in every row:

path	backend	decode	prefill	first turn after load	note
swift-litert-lm LiteRTChat (LiteRT-LM v0.15.0 xcframework)	GPU	142.7 tok/s	469.2 tok/s	2.2 s	footprint 1,411 MB; prewarm: false
swift-litert-lm LiteRTChat (LiteRT-LM v0.15.0 xcframework)	CPU	33.3 tok/s	95.2 tok/s	11.4 s	footprint 1,107 MB
litert-lm CLI 0.17.0, --thinking false, greedy	default			1.1 s wall
litert-lm CLI 0.16.0, --thinking false, greedy	default			1.0 s wall

Known limits

• One message per conversation on LiteRT-LM (measured on the v0.15.0 Swift binaries, litert-lm 0.17.1 and 0.18.0). A second message on the same Conversation fails with INTERNAL: The new rendered template string does not start with the previous rendered template string. On the v0.15.0 Swift binaries this happened with thinking off and on, GPU and CPU; in my regression run on litert-lm 0.18.0 (Mac, CPU backend, Python API) it happened in 6 of 6 conversations tried (two probes, each run twice; the first message of every conversation returned, the second raised), and my 2026-09-22 run of this template saw the same on litert-lm 0.17.1. The cause is the chat template, not the weights: before each message the runtime re-renders the conversation so far and requires the new render to extend the previous one, and this template (the fine-tune's Qwen3 template) renders the assistant turn after the latest user message with its <think>…</think> block — an empty one when there was no reasoning — and drops the block once a newer user message follows, so the two renders differ even with thinking off (LiteRT-LM#3443 is my report of the same template on another bundle; #3166 describes the mechanism). Workaround: a new conversation per turn with the history in the prompt. A re-pack with LiteRT-LM's own models/qwen3 template, which renders history turns the same way wherever they sit, would be a metadata-only fix for litert-lm 0.18.0 and newer; it is not in this file yet.
• Greedy decoding with thinking on loops (see above). Use the sampler the Qwen3 card recommends for thinking mode (temperature 0.6, top-p 0.95, top-k 20) or turn thinking off.
• Not verified: any iPhone (the recipe's device row is pending), Android, and the converter's --int4 recipe.

Provenance

• Converted and verified by: john-rocky (GitHub) / mlboydaisuke (Hugging Face)
• Recipe: https://github.com/john-rocky/swift-litert-lm/blob/main/docs/recipe-hf-finetune-to-iphone.md
• Measurements: https://github.com/john-rocky/swift-litert-lm/blob/main/docs/recipe-hf-finetune-to-iphone.md#7-verified--unverified (Mac Studio M4 Max, 2026-09-05)
• Commit: john-rocky/hf-to-litertlm@5ffe9ee (converter); john-rocky/swift-litert-lm@09f04f1 (Swift runs)
• Maintained at: https://github.com/john-rocky/swift-litert-lm/issues

Known runtime issues

On LiteRT-LM releases before 0.17.0 (read in the source of 0.10.1 to 0.16.1), an answer whose last punctuation mark is a token of its own, such as a . or ? after a word, arrives without that mark: this file carries stop sequences of <|im_end|> and a newline with each of 13 characters in front (12 punctuation marks and a space), so the runtime holds the mark back to see whether <|im_end|> and a newline follow, the answer ends on <|im_end|> alone, and the held mark is never written. LiteRT-LM 0.17.0 and later write the held mark (StopTokenStreamFilter in runtime/core/tasks.cc, read in the source of 0.17.0 to 0.18.1). Not measured on this file: this follows from the runtime's source read with this file's stop list and tokenizer; the drop itself was seen with LiteRT-LM 0.16.1 on a Galaxy S26, on another bundle whose stop sequences have the same form.

From a conversation preface (the system message, initial messages, tools and extra context a conversation is created with) that has tools and no messages, LiteRT-LM renders this file's chat template with an empty message list, and the template reads messages[0] without checking that the list is non-empty, so the render fails with Failed to apply template: undefined value. This happens in two places: when the preface is prefilled at creation (prefillPrefaceOnInit = true in Kotlin, prefillPrefaceOnInit: true on the web, SetPrefillPrefaceOnInit(true) in C++; the default is false), where creating the conversation fails with that error, and in the preface render (renderPrefaceIntoString() in Kotlin and Swift, RenderPrefaceIntoString() in C++, litert_lm_conversation_render_preface_to_string in the C API). The same happens with extra context (such as enable_thinking) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2). Read in the source of LiteRT-LM 0.17.0, 0.18.1 and main; measured with LiteRT-LM 0.18.1 (the litert-lm package from PyPI) through its C API on this file's template text. The default path, where the first message is rendered together with the preface, is not affected: the Python API and the CLI of 0.18.1 expose neither the flag nor the preface render, and Google AI Edge Gallery sets neither (its source as read on 2026-10-25). A system message or any initial message in the conversation config avoids it.

License

Apache-2.0, inherited from the fine-tune (its card metadata declares Apache-2.0) and from Qwen3-0.6B. The weights are the fine-tune author's work quantized to int8; nothing else was changed. Qwen3 is by the Qwen team at Alibaba Cloud. This is an independent conversion, not affiliated with the model author, Alibaba, or Google.