Hugging Face's logo

webAI-Official
/
TwIL-LM3-Pro-LiteRT-LM

Text Generation
LiteRT-LM
LiteRT
on-device
android
ios
granite
granite-4.2
reasoning
formal-logic
Model card Files
xet
 Community
TwIL-LM3-Pro-LiteRT-LM
File size: 4,473 Bytes
bb30df2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
---
license: other
license_name: webai-non-commercial-license-ver.-1.0
license_link: https://huggingface.co/webAI-Official/TwIL-LM3-Pro/blob/main/LICENSE.md
base_model: webAI-Official/TwIL-LM3-Pro
pipeline_tag: text-generation
library_name: litert-lm
tags:
- litert
- litert-lm
- on-device
- android
- ios
- granite
- granite-4.2
- reasoning
- formal-logic
---

# TwIL-LM3-Pro — LiteRT-LM

[webAI-Official/TwIL-LM3-Pro](https://huggingface.co/webAI-Official/TwIL-LM3-Pro)
converted to Google's LiteRT-LM (`.litertlm`) format for on-device inference.
Requires LiteRT-LM 0.16 or newer.

TwIL-LM3-Pro is a 3.66B-parameter formal-logic reasoning model based on
IBM Granite 4.2-3B. These bundles preserve its ChatML prompt format, pre-open
the `<think>` block, and declare a `thought` channel so LiteRT-LM can expose or
budget reasoning separately from the final answer.

## Files

| File | Quantization | Size | Intended use |
| --- | --- | ---: | --- |
| `TwIL-LM3-Pro_int4.litertlm` | INT4 blockwise-32 + OCTAV linears; INT8 embedding | 2.19 GB | Mobile / smallest deployable build |
| `TwIL-LM3-Pro_int8.litertlm` | Dynamic per-channel INT8 linears and embedding | 3.76 GB | Desktop / quality-oriented build |

Both bundles use a 4,096-token KV cache and six prefill signatures:
1,024, 256, 64, 16, 4, and 1 tokens. The input embedding is externalized into
its own bundle section to keep the primary graph below mobile mmap limits.

## Usage

Install a recent LiteRT-LM runtime:

```bash
pip install "litert-lm>=0.16"
```

Run locally:

```bash
litert-lm run ./TwIL-LM3-Pro_int4.litertlm \
  --backend gpu \
  --thinking true \
  --prompt "Does 'All dogs are mammals. Rex is a dog.' entail 'Rex is a mammal'?"
```

Run directly from Hugging Face:

```bash
litert-lm run \
  --from-huggingface-repo webAI-Official/TwIL-LM3-Pro-LiteRT-LM \
  TwIL-LM3-Pro_int4.litertlm \
  --backend gpu \
  --thinking true \
  --prompt "What is the capital of France?"
```

For reasoning-heavy prompts, allow at least 2,048 output tokens. A generation
that reaches its limit inside `<think>` may never produce a final answer.
Use `--thinking false` for lower latency when reasoning is not needed.

## Validation performed

The final repacked artifacts were checked with LiteRT-LM 0.16.1 on Linux CPU:

- INT4, thinking disabled: `What is the capital of France?` → `Paris`
- INT8, thinking disabled: `What is the capital of France?` → `Paris`
- INT4, thinking budget 64: the runtime emitted a separate `[thought]` channel
  and answered the syllogism prompt with `yes`
- Both metadata repacks proved all non-metadata sections byte-identical to the
  converter output

This is a conversion smoke test, not a benchmark. The bf16 Track A and Track B
scores from the source model card must not be attributed to either quantized
bundle until those evaluations are rerun through LiteRT-LM. In particular,
reasoning models can be more sensitive to INT4 quantization than ordinary chat
models; prefer INT8 when memory permits.

## Conversion

Converted from the released `model.safetensors`, tokenizer, and configuration
in `webAI-Official/TwIL-LM3-Pro` using:

- `litert-torch` 0.9.3
- `litert-converter` 0.4.0
- `ai-edge-quantizer` 0.9.0
- `litert-lm-builder` 0.16.1
- `transformers` 5.14.1
- `john-rocky/hf-to-litertlm` revision
  `cb2ed3c53bd397acb82f132ee3d4b4313827ef30`

The release uses the LiteRT-compatible Granite 4.2 Jinja template from that
conversion project. It supports normal thinking on/off behavior and
multi-turn history. The source checkpoint's extra `reasoning_effort="low"`
template option is not exposed by this LiteRT bundle.

See `CONVERSION.md` and `litertlm_manifest.json` for the exact recipe and
artifact hashes.

## Limitations

- Context is capped at 4,096 tokens in these on-device bundles, not the
  source checkpoint's inherited 131,072-token maximum.
- Track A generations are often long. Mobile sessions need a sufficiently
  large output budget and enough memory for the selected backend.
- Device-specific CPU/GPU performance and full delegation have not yet been
  measured for these TwIL weights.
- The INT4 and INT8 quantized models have not yet been evaluated on the
  published formal-logic or held-out benchmark suites.

## License

Released under the
[webAI Non-Commercial License ver. 1.0](./LICENSE.md), matching the source
TwIL-LM3-Pro model. The Granite 4.2 base attribution and Apache 2.0 license
text are retained in `apache-2.0-LICENSE.txt`.