| # Conversion recipe |
|
|
| Source model: `webAI-Official/TwIL-LM3-Pro` |
|
|
| Converter source: |
| `john-rocky/hf-to-litertlm@cb2ed3c53bd397acb82f132ee3d4b4313827ef30` |
|
|
| ## Environment |
|
|
| ```text |
| Python 3.10 |
| litert-torch 0.9.3 |
| litert-converter 0.4.0 |
| ai-edge-quantizer 0.9.0 |
| litert-lm-builder 0.16.1 |
| litert-lm 0.16.1 |
| transformers 5.14.1 |
| ``` |
|
|
| ## Export settings |
|
|
| ```text |
| cache length 4096 |
| prefill signatures 1024,256,64,16,4,1 |
| externalize embedder true |
| start token omitted |
| tokenizer embedded Hugging Face tokenizer.json |
| INT4 recipe blockwise-32 + OCTAV; INT8 embedding |
| INT8 recipe dynamic per-channel INT8 |
| ``` |
|
|
| The start token is deliberately omitted because Granite 4.2 declares `<s>` as |
| BOS but its tokenizer post-processor does not prepend it. Adding it at runtime |
| changes the prompt seen by the model. |
|
|
| The exporter was supplied the local Hugging Face checkpoint and the |
| `templates/granite42_think.jinja` structured-template extraction shim: |
|
|
| ```bash |
| export CACHE=4096 |
| export PREFILL=1024,256,64,16,4,1 |
| export EXTERNALIZE_EMBEDDER=1 |
| export NO_START_TOKEN=1 |
| |
| python scripts/export_simple_template.py \ |
| /path/to/TwIL-LM3-Pro \ |
| out/int4 \ |
| templates/granite42_think.jinja \ |
| BOCTAV4 |
| |
| python scripts/export_simple_template.py \ |
| /path/to/TwIL-LM3-Pro \ |
| out/int8 \ |
| templates/granite42_think.jinja \ |
| dynamic_wi8_afp32 |
| ``` |
|
|
| After conversion, two metadata-only operations were applied: |
|
|
| 1. Declare the `thought` channel as `<think>` … `</think>`. |
| 2. Bundle the LiteRT-compatible canonical Granite 4.2 Jinja chat template. |
|
|
| Both utilities unpacked and repacked the bundle, then verified that every |
| non-metadata section remained byte-identical. |
|
|
| ## Final artifacts |
|
|
| ```text |
| f3b47325fb31189b93c9e55e6bb3a89d8f4aa4cbccd44d537100efcdb75b4641 TwIL-LM3-Pro_int4.litertlm |
| 956d86ec174cafc864a65d18c17b0f3a3425b0cecbe7c9bfc866032eacf373ea TwIL-LM3-Pro_int8.litertlm |
| ``` |
|
|
| ## Validation |
|
|
| Both bundles loaded and generated the expected `Paris` answer through |
| LiteRT-LM 0.16.1 on Linux CPU with thinking disabled. The INT4 bundle was also |
| run with a 64-token thinking budget on an entailment prompt; reasoning arrived |
| through the declared thought channel and the final answer was `yes`. |
|
|
| No Track A, Track B, mobile-device, or GPU-delegation benchmark is claimed by |
| this conversion release. |
|
|
