File size: 2,397 Bytes
bb30df2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 | # Conversion recipe
Source model: `webAI-Official/TwIL-LM3-Pro`
Converter source:
`john-rocky/hf-to-litertlm@cb2ed3c53bd397acb82f132ee3d4b4313827ef30`
## Environment
```text
Python 3.10
litert-torch 0.9.3
litert-converter 0.4.0
ai-edge-quantizer 0.9.0
litert-lm-builder 0.16.1
litert-lm 0.16.1
transformers 5.14.1
```
## Export settings
```text
cache length 4096
prefill signatures 1024,256,64,16,4,1
externalize embedder true
start token omitted
tokenizer embedded Hugging Face tokenizer.json
INT4 recipe blockwise-32 + OCTAV; INT8 embedding
INT8 recipe dynamic per-channel INT8
```
The start token is deliberately omitted because Granite 4.2 declares `<s>` as
BOS but its tokenizer post-processor does not prepend it. Adding it at runtime
changes the prompt seen by the model.
The exporter was supplied the local Hugging Face checkpoint and the
`templates/granite42_think.jinja` structured-template extraction shim:
```bash
export CACHE=4096
export PREFILL=1024,256,64,16,4,1
export EXTERNALIZE_EMBEDDER=1
export NO_START_TOKEN=1
python scripts/export_simple_template.py \
/path/to/TwIL-LM3-Pro \
out/int4 \
templates/granite42_think.jinja \
BOCTAV4
python scripts/export_simple_template.py \
/path/to/TwIL-LM3-Pro \
out/int8 \
templates/granite42_think.jinja \
dynamic_wi8_afp32
```
After conversion, two metadata-only operations were applied:
1. Declare the `thought` channel as `<think>` … `</think>`.
2. Bundle the LiteRT-compatible canonical Granite 4.2 Jinja chat template.
Both utilities unpacked and repacked the bundle, then verified that every
non-metadata section remained byte-identical.
## Final artifacts
```text
f3b47325fb31189b93c9e55e6bb3a89d8f4aa4cbccd44d537100efcdb75b4641 TwIL-LM3-Pro_int4.litertlm
956d86ec174cafc864a65d18c17b0f3a3425b0cecbe7c9bfc866032eacf373ea TwIL-LM3-Pro_int8.litertlm
```
## Validation
Both bundles loaded and generated the expected `Paris` answer through
LiteRT-LM 0.16.1 on Linux CPU with thinking disabled. The INT4 bundle was also
run with a 64-token thinking budget on an entailment prompt; reasoning arrived
through the declared thought channel and the final answer was `yes`.
No Track A, Track B, mobile-device, or GPU-delegation benchmark is claimed by
this conversion release.
|