LFM2-2.6B-Exp — LiteRT-LM

LiquidAI/LFM2-2.6B-Exp converted to the LiteRT-LM (.litertlm) format for Google's LiteRT-LM runtime. Tested on litert-lm 0.18.0 on a Mac; the phone rows ran the prebuilt Android binary named below. On litert-lm 0.17.1 the int8 file answered "What is the capital of France?" as if no question had been asked (one CPU run; the int4 file was not tried on 0.17.1).

Liquid AI describes LFM2-2.6B-Exp as an experimental checkpoint built on LFM2-2.6B using pure reinforcement learning, trained on instruction following, knowledge and math, and suggests the LFM2 models for agentic tasks, data extraction, RAG, creative writing and multi-turn conversations rather than knowledge-intensive tasks or programming. Config, tokenizer and chat template match LFM2-2.6B's: 2,569,272,320 parameters in 30 layers (22 convolution + 8 attention), tied embeddings, a 65,536-token vocabulary, a 32,768-token context (4096 here); English, Arabic, Chinese, French, German, Japanese, Korean and Spanish. Liquid AI's card says nothing about reasoning, yet in the runs below the model sometimes wrote a trace between <think> and </think>.

File	Recipe	Size	sha256
LFM2-2.6B-Exp_int8.litertlm	int8 dynamic on linears + int8 embedding, convs float	2.72 GB	a44c3cfd0644e9fe9036c0efe3955df48a91990fe6e42855a9de0020c4e76258
LFM2-2.6B-Exp_int4.litertlm	int4 blockwise-32 + OCTAV on linears, int8 lm_head + embedding	1.53 GB	1d3401ca1b81dc40de154b88a13b654d8ac1838281fd47303aa161f0b825f877

On GSM8K (n=100) and IFEval-lite (n=120) the two files score within two items of the bf16 model (table in Accuracy). The int4 file is 43.8 % smaller. Time for a 1024-token prefill plus 256 decoded tokens at context 2048, from the rates in Performance:

Seconds	Mac M4 Max	Galaxy S26	Pixel 8a
int4 GPU	2.00	10.81	22.06
int8 GPU	2.49	13.42	25.14
int8 CPU	9.40	17.39	58.42
int4 CPU	11.11	25.91	65.57

So int4 is the faster file on every GPU and int8 on every CPU. On the Mac CPU the int4 file sometimes kept repeating after its answer (Usage); the int8 file did not. Not measured on iOS: the int8 file's one weights section is 2,722,843,088 bytes (2.54 GiB), and the LFM2.5-2.6B card reports an int8 file with a 2.87 GB weights section running on an iPhone 17 Pro only when the app carries the extended-virtual-addressing and increased-memory entitlements.

Correctness

8-question check on the Mac M4 Max (the questions with "Answer briefly." appended, --temperature 0, one litert-lm run process each): int8 7/8 on CPU and GPU, int4 7/8 on CPU and GPU, bf16 (transformers) 7/8; every row misses the same rhyme question.

Both files' 12 signatures were fully delegated on the three GPUs measured. On both phones, both files answered "What is the capital of France? Answer in one sentence." on GPU and CPU with "Paris" in every run.

Accuracy

GSM8K: test questions 0–99, prompted for step-by-step reasoning and a final #### <number> line. IFEval-lite: the first 120 google/IFEval rows with mechanically checkable instructions, prompt-level strict. Greedy, at most 2048 output tokens; litert-lm on the Mac GPU, a fresh process per item. Scores are comparable only within this table; Liquid AI publishes no benchmark table for this checkpoint.

Configuration	GSM8K (n=100)	IFEval-lite (n=120)
bf16 reference (transformers, MPS)	89	102
int8, litert-lm	87	102
int4, litert-lm	89	101

Scores exclude the reasoning trace: no GSM8K reply had one, and the one IFEval-lite prompt that traced per row passed on its reply.

Usage

litert-lm run ./LFM2-2.6B-Exp_int8.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17."
litert-lm run ./LFM2-2.6B-Exp_int8.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17." --backend gpu --cache no
litert-lm run ./LFM2-2.6B-Exp_int4.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17." --backend gpu --cache no
litert-lm benchmark ./LFM2-2.6B-Exp_int8.litertlm -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no --backend cpu

The three run commands listed the two dates with no chat marker such as <|im_end|> and no [thought] block. On the Mac CPU the int4 file is not reliable with this prompt: in 9 runs, 3 repeated lines until the 4096-token context was full (2 of 4 with the bundle's sampler, 1 of 5 with --top-k 1) and 1 kept correcting itself; on the GPU it ended normally in 4 of 4 runs, as did the int8 file on the CPU in 3 of 3. The cause was not established (its other CPU checks asked for short answers; GSM8K and IFEval-lite ran on the GPU). Without --backend, run uses the CPU; without --cache, it writes a compiled-weights cache next to the model (2.58 GB for int8, 1.51 GB for int4). For Android, see android-gpu.md.

The bundle's default sampler is temperature 0.1, top-k 50, top-p 1.0; Liquid AI recommends temperature 0.3, min_p 0.15 and repetition_penalty 1.05, and the bundle has no min_p or repetition penalty. run uses the bundle's sampler unless you pass --top-k, --top-p, --temperature or --seed; --top-k 1 gives greedy output.

The chat template is byte-identical to LiteRT-LM's models/lfm2/chat_template.jinja at commit 86a7e6d0, with its stops <|im_end|> and <|endoftext|> and start token <|startoftext|>. A trace between <think> and </think> goes to the thought channel, printed by the CLI between [thought] and [/thought]. One appeared for "What is the capital of France?" (int8, CPU and GPU; with a 16-token output limit the reply was empty, all 16 tokens being trace) and for 1 of 120 IFEval-lite prompts per row; none for GSM8K, the checks, the phone prompts or the commands above. --thinking-budget 0 and 32 did not suppress it.

Tool calling is declared (model type lfm2): both files returned a parsed get_weather call (Python API, CPU, greedy) and, with automatic tool calling, used its result. Without tools the template renders the same text as Liquid AI's (four conversations checked); with tools it omits Liquid AI's <|tool_list_start|> / <|tool_list_end|> and <|tool_response_start|> / <|tool_response_end|> markers, wraps the tool result in [ … ], and quotes call arguments with double quotes instead of single quotes.

Engine reuse (int8 file, Python API, CPU, greedy, 3 runs each on a shared and on a fresh Engine): after one conversation, a second conversation with a different system prompt answered its two factual questions ("Paris"; "391") exactly as a fresh Engine did. With the same system prompt in both conversations and user messages that begin with the same eight words, the condition of the open issue LiteRT-LM #3165, the shared Engine answered "The third item in the list is cherry." where a fresh Engine answered "cherry" (3 of 3 each, both correct). Open a new Engine for each conversation.

Performance

Mac Studio M4 Max, litert-lm 0.18.0: the benchmark command above with --backend cpu or gpu (1024-token prefill, 256-token decode, context 2048), one warmup, then the mean of 3 iterations; GPU is WebGPU on Metal. Init is the first iteration's engine init.

Mac M4 Max	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)	Init (s)
int8 GPU	2441.36	123.39	0.4275	2.8241
int4 GPU	2475.63	161.77	0.4198	3.8491
int8 CPU	377.26	38.27	2.7405	31.9277
int4 CPU	177.07	48.04	5.8085	8.6964

Phones, Android 16: Galaxy S26 (SM-S942Q, Qualcomm SM8850, Adreno 840, MemTotal 10.9 GiB) and Pixel 8a (Tensor G3, Mali-G715, MemTotal 7.4 GiB), the same workload through the prebuilt litert_lm_advanced_main (sha256 adac974b…, 40,382,152 bytes) published 2026-09-18 in LiteRT-LM's public binary bucket (binaries/latest/android_arm64/litert_lm/), with --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3 --report_peak_memory_footprint=true. Rates are the mean of iterations 2 and 3 (GPU: LITERT_CL). Init cold = the first process after the compiled-program caches were removed, warm = the benchmark process; peak = the largest of the benchmark process's Peak system ram usage lines.

Galaxy S26	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)	Init cold / warm (s)	Peak (MB)
int8 GPU	1140.5	20.44	0.950	8.66 / 4.76	779.4
int4 GPU	664.3	27.62	1.585	9.88 / 6.53	1029.2
int8 CPU	285.6	18.55	3.645	3.12 / 0.22	4163.8
int4 CPU	74.0	21.21	13.995	6.01 / 4.77	3111.8

The clocks lowered during these runs: the GPU clock ceiling fell from 1300 MHz to 826 MHz (int8) and 726 MHz (int4) inside the benchmark, and in the CPU runs the CPU clock was lowered in 23 of 24 (int8) and 36 of 37 (int4) samples.

Pixel 8a	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)	Init cold / warm (s)	Peak (MB)
int8 GPU	498.5	11.09	2.145	106.18 / 12.50	1667.2
int4 GPU	171.7	15.90	6.030	26.29 / 12.95	1504.6
int8 CPU	70.2	5.84	15.045	11.00 / 0.71	3659.5
int4 CPU	30.5	8.00	33.715	12.46 / 10.84	2597.2

In the CPU runs the CPU clock was lowered in 51 of 53 (int8) and 65 of 65 (int4) samples; the sampler reads only Adreno GPU clocks, so no Pixel 8a GPU clock was recorded. In the int8 GPU benchmark process the lowest MemAvailable sample was 501,160 kB. Not run on the NPU.

Conversion notes

• Source and stack: LiquidAI/LFM2-2.6B-Exp at revision 2251c865; litert-torch 0.9.4, litert-converter 0.4.0, ai-edge-quantizer 0.9.0, transformers 5.14.1, torch 2.13.0; packed by the litert-lm 0.18.0 CLI. lfm_work/convert_lfm2_26b_exp.sh in hf-to-litertlm rebuilt both files, all four sections byte-identical (only the packer's uuid and timestamp differ).
• Order: unquantized export (11 prefill signatures, 1024 down to 1, plus decode, over a 4096-position KV cache) → post-hoc int8 and int4 quantization → int4 zero-scale fix (746,432 zero block scales in 26 tensors, replaced by each tensor's smallest nonzero scale, block payload zeroed) → metadata repack with LiteRT-LM's lfm2 configuration.
• Template: Liquid AI's chat template (the LFM2-2.6B file) fails at render on litert-lm 0.18.0 (map has no method named get), so the bundle carries LiteRT-LM's lfm2 template, which reads message content as a list of parts.

License and changes

Distributed under the LFM Open License v1.0 inherited from the base model (LICENSE); its section 5 conditions commercial use on an annual revenue below 10 million US dollars. Changes from the original work: exported to the LiteRT-LM format and quantized to int8 and int4 as described above; chat template and stop tokens taken from LiteRT-LM's models/lfm2 configuration, with max_num_tokens changed from 32768 to 4096. This conversion was made and is published independently of Liquid AI.