Instructions to use mlboydaisuke/Tashkeel-350M-v2-LiteRT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use mlboydaisuke/Tashkeel-350M-v2-LiteRT with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- LiteRT-LM
How to use mlboydaisuke/Tashkeel-350M-v2-LiteRT with LiteRT-LM:
# LiteRT-LM runs on various platforms (Android, iOS, Windows, Linux, macOS, IoT, Web/WASM) # and supports many APIs (C++, Python, Kotlin, Swift, JavaScript, Flutter). # For platform-specific integration guides, please refer to the official developer website: # https://ai.google.dev/edge/litert-lm # To try LiteRT-LM, the easiest way is to use our CLI tool. # 1. Install the LiteRT-LM CLI tool: pip install -U litert-lm # 2. Download and run this model locally: # See: https://ai.google.dev/edge/litert-lm/cli # A single .litertlm file in the repo is picked automatically; otherwise the CLI asks which one to run # (or pass its name right after the repo id). litert-lm run \ --from-huggingface-repo=mlboydaisuke/Tashkeel-350M-v2-LiteRT \ --prompt="Write me a poem"
- Notebooks
- Google Colab
- Kaggle
Tashkeel-350M-v2 β LiteRT-LM (on-device Arabic diacritization)
On-device conversion of Etherll/Tashkeel-350M-v2 β
an Arabic diacritization (tashkeel) fine-tune of
ibm-granite/granite-4.0-h-350m trained on
Misraj/Sadeed_Tashkeela β to .litertlm
bundles for the LiteRT-LM runtime. All credit for
the model itself goes to its author; this repo only packages it for phones and desktops.
Requires litert-lm β₯ 0.15 to run (the Mamba2 hybrid state binds through the
ExecutorMetadata section). CPU is the backend to use on phones. The int8 file does run on
mobile GPUs β on a Galaxy S26 (Adreno) and a Pixel 8a (Mali) it delegates fully to LiteRT GPU
(litert-lm 0.16.0, 2026-09-05) β but on both phones the CPU decodes faster (S26: 67β69 vs 46 tok/s;
Pixel 8a: 47β49 vs 18) at a third to a half of the GPU's peak memory and without its 22.4β25.5 s engine
init. On a 10-word test sentence both GPUs also added one letter (alef, Ψ§) to the end of the last
word, which the CPU did not. An earlier version of this card said the selective-scan ops exceed the
GPU delegate's rank limit; that describes the base model's CPU-lineage exports and did not hold for
this bundle when it was checked. The fp16 file was measured on the CPU only.
| file | quant | size | task-check parity vs HF fp32 |
|---|---|---|---|
Tashkeel-350M-v2_fp16.litertlm |
fp16 weights (convs/SSM fp32) | 769 MB | 10/10 byte-identical |
Tashkeel-350M-v2_int8.litertlm |
dynamic int8 (linears + embedding) | 481 MB | 8/10 (see note) |
Which file: fp16 is the exact-parity build β greedy output byte-identical to the HF fp32 reference on all 10 diacritization probes. int8 is 38% of the size; its two misses are single-diacritic greedy flips (both probes are byte-exact on the float parent, i.e. quantization cost, not conversion error β the same 350M-scale int8 sensitivity the base model's card documents). On phones the CPU runtime unpacks fp16 to fp32 in RAM, so int8 is the practical mobile choice; use fp16 where RAM allows.
Conversion & verification
Converted with hf-to-litertlm
(family recipe granite_work/convert_granite4h.py, 2026-08-25):
- The checkpoint's chat template (byte-equal to the granite base's, 6418/6418) is embedded and applied at runtime.
- The metadata start token is dropped β granite's template has no leading BOS, and at 350M
scale a prepended
<|end_of_text|>flips correct diacritization into garbage (measured on this checkpoint). - Task check: the model card's worked example plus nine undiacritized MSA probes, bundle vs
HF fp32 greedy on identical rendered strings (
granite_work/gate_tashkeel.py). - One Engine per conversation. On litert-lm 0.18.0 the runtime starts a new Conversation on an Engine that already ran another conversation from the earlier conversation's recurrent state: it reuses the prompt tokens the two conversations share and does not reset the state (my open report). In a test that asks three questions (What is the capital of France? / What is 17 times 23? / What color is the sky on a clear day?) first in a pirate-persona conversation and then in a conversation told to answer in one word, on the same Engine against a fresh Engine in a new process (answers capped at 200 tokens; CPU backend; litert-lm 0.18.0 on a Mac), the
_int8file answered the capital question with an empty reply on the same Engine against "The answer is Paris." on a fresh one, in both of its runs (a test is run a second time when its first run shows a difference or when the fresh Engine misses one of the three expected answers: Paris, 391, blue). The_fp16file gave all three expected answers on both Engines in its one run. It answered the capital question with a sentence ("The capital of France is Paris.") on the same Engine against the one word "Paris" on a fresh one. The test's questions are outside this model's task (Arabic diacritization), so the test shows the Engine's behaviour, not the model's quality. Those three short questions say nothing about longer conversations. Create a new Engine for each conversation. The first conversation on an Engine is unaffected. A fix landed on LiteRT-LM main on 2026-11-03 (e1f86b05). No release carries it yet (v0.19.1 does not); this note will be updated when one does.
Performance (Apple M4 Max, litert-lm 0.16.0, -p 256 -d 256 --runs 3 --cache no, CPU)
| file | prefill tok/s | decode tok/s | TTFT |
|---|---|---|---|
| fp16 | 661.6 | 58.2 | 0.40 s |
| int8 | 761.3 | 97.2 | 0.35 s |
Performance (Android, litert-lm 0.16.0 litert_lm_advanced_main, 224-token prompt, --benchmark, 2 cold runs per row)
| phone | file | backend | prefill tok/s | decode tok/s | peak RSS |
|---|---|---|---|---|---|
| Galaxy S26 (SM8850, Adreno) | int8 | CPU | 192β212 | 67.2β69.0 | 1110 MiB |
| Galaxy S26 | int8 | GPU (LITERT_CL, full delegation) |
752β755 | 46.3 | 2254β2256 MiB |
| Galaxy S26 | fp16 | CPU | 66β77 | 23.2β23.7 | 5149β5151 MiB |
| Pixel 8a (Tensor G3, Mali) | int8 | CPU | 112 | 47.3β49.1 | 1068β1072 MiB |
| Pixel 8a | int8 | GPU (LITERT_CL, full delegation) |
251β263 | 18.3β18.4 | 3672β3689 MiB |
Each GPU/CPU pair is the same handset, binary and prompt, taken 2026-09-05 with the phone cooled
between runs. A separate run on each phone and backend diacritized a 10-word test sentence in the
trained prompt form (see Usage): both GPUs returned the CPU's output plus one extra letter (alef, Ψ§)
at the end of the last word. The same rows, with conditions, are in this repo's
litertlm_manifest.json.
Usage
The model diacritizes the text you send as a user message in its trained prompt form:
pip install litert-lm
litert-lm run Tashkeel-350M-v2_fp16.litertlm --backend cpu \
--prompt 'ΩΩ
Ψ¨ΨͺΨ΄ΩΩΩ ΩΨ°Ψ§ Ψ§ΩΩΨ΅ :
Ψ§ΩΨ³ΩΨ§Ω
ΨΉΩΩΩΩ
'
Output: Ψ§ΩΨ³ΩΩΩΩΨ§Ω
Ω ΨΉΩΩΩΩΩΩΩΩ
Ω
Known runtime issues
From a conversation preface (the system message, initial messages, tools and extra context a conversation is created with) that has tools and no messages, LiteRT-LM renders these files' chat template with an empty message list, and the template reads messages[0] without checking that the list is non-empty, so the render fails with Failed to apply template: undefined value. This happens in two places: when the preface is prefilled at creation (prefillPrefaceOnInit = true in Kotlin, prefillPrefaceOnInit: true on the web, SetPrefillPrefaceOnInit(true) in C++; the default is false), where creating the conversation fails with that error, and in the preface render (renderPrefaceIntoString() in Kotlin and Swift, RenderPrefaceIntoString() in C++, litert_lm_conversation_render_preface_to_string in the C API). The same happens with extra context (such as enable_thinking) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2). Read in the source of LiteRT-LM 0.17.0, 0.18.1 and main; measured with LiteRT-LM 0.18.1 (the litert-lm package from PyPI) through its C API on these files' template text. The default path, where the first message is rendered together with the preface, is not affected: the Python API and the CLI of 0.18.1 expose neither the flag nor the preface render, and Google AI Edge Gallery sets neither (its source as read on 2026-10-25). A system message or any initial message in the conversation config avoids it.
License
apache-2.0, inherited from the source model and its granite base.
- Downloads last month
- -
Model tree for mlboydaisuke/Tashkeel-350M-v2-LiteRT
Base model
ibm-granite/granite-4.0-h-350m-base