Hugging Face's logo

webAI-Official
/
TwIL-LM3-Pro-LiteRT-LM

Text Generation
LiteRT-LM
LiteRT
on-device
android
ios
granite
granite-4.2
reasoning
formal-logic
Model card Files
xet
 Community
TwIL-LM3-Pro-LiteRT-LM / CONVERSION.md
anurag051194's picture
anurag051194
Add LiteRT-LM INT4 and INT8 release
bb30df2 verified
preview code
|
Raw History Blame Contribute Delete
2.4 kB

Conversion recipe

Source model: webAI-Official/TwIL-LM3-Pro

Converter source: john-rocky/hf-to-litertlm@cb2ed3c53bd397acb82f132ee3d4b4313827ef30

Environment 

Python                 3.10
litert-torch           0.9.3
litert-converter       0.4.0
ai-edge-quantizer      0.9.0
litert-lm-builder      0.16.1
litert-lm              0.16.1
transformers           5.14.1

Export settings 

cache length           4096
prefill signatures     1024,256,64,16,4,1
externalize embedder   true
start token            omitted
tokenizer              embedded Hugging Face tokenizer.json
INT4 recipe            blockwise-32 + OCTAV; INT8 embedding
INT8 recipe            dynamic per-channel INT8

The start token is deliberately omitted because Granite 4.2 declares <s> as BOS but its tokenizer post-processor does not prepend it. Adding it at runtime changes the prompt seen by the model.

The exporter was supplied the local Hugging Face checkpoint and the templates/granite42_think.jinja structured-template extraction shim:

export CACHE=4096
export PREFILL=1024,256,64,16,4,1
export EXTERNALIZE_EMBEDDER=1
export NO_START_TOKEN=1

python scripts/export_simple_template.py \
  /path/to/TwIL-LM3-Pro \
  out/int4 \
  templates/granite42_think.jinja \
  BOCTAV4

python scripts/export_simple_template.py \
  /path/to/TwIL-LM3-Pro \
  out/int8 \
  templates/granite42_think.jinja \
  dynamic_wi8_afp32

After conversion, two metadata-only operations were applied:

  1. Declare the thought channel as <think> … </think>.
  2. Bundle the LiteRT-compatible canonical Granite 4.2 Jinja chat template.

Both utilities unpacked and repacked the bundle, then verified that every non-metadata section remained byte-identical.

Final artifacts 

f3b47325fb31189b93c9e55e6bb3a89d8f4aa4cbccd44d537100efcdb75b4641  TwIL-LM3-Pro_int4.litertlm
956d86ec174cafc864a65d18c17b0f3a3425b0cecbe7c9bfc866032eacf373ea  TwIL-LM3-Pro_int8.litertlm

Validation

Both bundles loaded and generated the expected Paris answer through LiteRT-LM 0.16.1 on Linux CPU with thinking disabled. The INT4 bundle was also run with a 64-token thinking budget on an entailment prompt; reasoning arrived through the declared thought channel and the final answer was yes.

No Track A, Track B, mobile-device, or GPU-delegation benchmark is claimed by this conversion release.