LFM2.5-Encoder-350M-Policy-Linter — LiteRT

LiquidAI/LFM2.5-Encoder-350M-Policy-Linter converted to LiteRT (.tflite) for on-device inference. Zero-shot policy linting: write your rules as free text and the model scores every token against every rule in one CPU pass (demo Space).

Model description

File	Recipe	Size	Target
LFM2.5-Encoder-350M-Policy-Linter_wi8fc.tflite	int8 dynamic-range (linears + embedding, convs float)	365 MB	mobile + desktop
LFM2.5-Encoder-350M-Policy-Linter_fp16.tflite	fp16 weights, float compute	713 MB	desktop CPU — phone CPU memory limits (XNNPACK per-signature fp32 unpacking); this is the file the Snapdragon NPU runs, AOT-compiled (see Snapdragon NPU (Hexagon))

Two signatures, lint_128 and lint_512 (S = 128 / 512, batch 1, right-padded, up to 8 rule slots):

Tensor	Shape	Meaning
input_ids	int32 [1, S]	Policy:\n- <rule 1>\n- <rule 2>…\n\nText:\n<document>
attention_mask	int32 [1, S]	1 = real token, 0 = pad
rule_pool	float32 [1, 8, S]	row r = mean-pool weights over rule r's tokens (1/n each); unused rows all-zero
output	float32 [1, S, 8]	per-token, per-rule scores, zeroed at padded positions

sigmoid(score[t, r]) > 0.5 flags token t under rule r. Read the flags only for real rules and only over the document's token range — the prompt's own header and rule text sit in the same sequence.

How to use

1. Install dependencies

pip install ai-edge-litert numpy tokenizers huggingface_hub

2. Save the script below as lint_text.py:

#!/usr/bin/env python3
"""Flag policy violations with litert-community/LFM2.5-Encoder-350M-Policy-Linter."""
import argparse

import numpy as np
from ai_edge_litert.interpreter import Interpreter
from huggingface_hub import hf_hub_download
from tokenizers import Tokenizer

REPO = "litert-community/LFM2.5-Encoder-350M-Policy-Linter"
MAX_RULES = 8


def build_inputs(text, rules, tokenizer, seq_len):
    """Builds input_ids/attention_mask plus the per-rule mean-pool matrix."""
    body = "\n".join(f"- {rule}" for rule in rules)
    prefix = f"Policy:\n{body}\n\nText:\n"
    encoding = tokenizer.encode(prefix + text)
    ids, offsets = encoding.ids, encoding.offsets
    if len(ids) > seq_len:
        raise SystemExit(f"{len(ids)} tokens exceed --seq-len {seq_len}")

    input_ids = np.zeros((1, seq_len), np.int32)
    attention_mask = np.zeros((1, seq_len), np.int32)
    input_ids[0, : len(ids)] = ids
    attention_mask[0, : len(ids)] = 1

    rule_pool = np.zeros((1, MAX_RULES, seq_len), np.float32)
    pos = len("Policy:\n")
    for r, rule in enumerate(rules):
        start, end = pos + 2, pos + 2 + len(rule)
        pos = end + 1
        idx = [i for i, (a, b) in enumerate(offsets) if a < end and b > start and a != b]
        rule_pool[0, r, idx] = 1 / len(idx)

    doc_idx = [i for i, (a, b) in enumerate(offsets) if b > len(prefix) and a != b]
    return ({"input_ids": input_ids, "attention_mask": attention_mask,
             "rule_pool": rule_pool}, encoding, doc_idx)


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--text", required=True, help="The document to lint.")
    parser.add_argument("--rule", action="append", required=True,
                        help="A policy rule in plain language, repeatable (up to 8).")
    parser.add_argument("--seq-len", type=int, default=512, choices=[128, 512])
    args = parser.parse_args()
    if len(args.rule) > MAX_RULES:
        raise SystemExit(f"at most {MAX_RULES} rules")

    model_path = hf_hub_download(REPO, "LFM2.5-Encoder-350M-Policy-Linter_wi8fc.tflite")
    tokenizer = Tokenizer.from_file(hf_hub_download(REPO, "tokenizer.json"))

    feed, encoding, doc_idx = build_inputs(args.text, args.rule, tokenizer, args.seq_len)
    interpreter = Interpreter(model_path=model_path)
    runner = interpreter.get_signature_runner(f"lint_{args.seq_len}")
    scores = list(runner(**feed).values())[0][0]

    for r, rule in enumerate(args.rule):
        flagged = [encoding.tokens[i] for i in doc_idx
                   if 1 / (1 + np.exp(-scores[i, r])) > 0.5]
        print(f"rule: {rule}")
        print(f"  flagged: {''.join(flagged).replace(chr(0x120), ' ').strip() or '(none)'}")


if __name__ == "__main__":
    main()

3. Run it

python lint_text.py \
  --text "Hi Tom, my personal email is tom@example.com. We will definitely deliver by March 3." \
  --rule "Do not share personal contact details" \
  --rule "Do not promise specific delivery dates"

rule: Do not share personal contact details
  flagged: email tom@example.com
rule: Do not promise specific delivery dates
  flagged: will definitely deliver March 3

On Android/iOS use the LiteRT runtime's SignatureRunner APIs with the same signature names; the tokenizer is the standard Hugging Face tokenizer.json. The rule_pool construction mirrors the router sibling's (LFM2.5-Encoder-350M-Prompt-Router) with the Policy: header in place of Categories:.

Performance

One pass over a padded sequence with the int8 (wi8fc) file, CPU only.

Device	Threads	lint_128	lint_512
Apple M4 Max (macOS)	8	35.1 ms	114.6 ms
iPhone 17 Pro	6	not measured	143 ms

Mac figures are the median of 20 warm runs (ai-edge-litert 2.1.6, XNNPACK, otherwise idle machine). The iPhone figure comes from the on-device gate (TFLite C API + SignatureRunner + XNNPACK) and is a single run, not a median.

Budget for one slow first call. The first inference after loading pays a one-time graph preparation: on the Mac it took 379 ms against a 35.1 ms steady state. Later signatures on the same loaded model do not pay it again — lint_512 measured 114 ms cold against 115 ms warm. Model load itself was 0.38 s on the iPhone, with a peak footprint of 649 MiB.

One pass scores every token against all eight rule slots at once, so the cost does not grow with the number of rules. The signatures are fixed-shape, so input language or content does not change the time.

Accuracy note

Task-level parity against the PyTorch reference on the demo document — an email-address share and a delivery-date promise checked against two rules: fp32, fp16 and int8 all flag the identical 10-token spans for both rules. That is a single-document spot check, not a benchmark over a labelled corpus.

On the iPhone 17 Pro the int8 file reproduces the desktop outputs bit-exactly — cosine 1.000000, max absolute difference 0.0.

Android (Pixel 8a)

Android figures use the standard TFLite benchmark_model on a Pixel 8a (Tensor G3, Android 16) — 5 warm-up runs then 20 timed runs, the signature selected explicitly with --signature_to_run_for, CPU at 4 threads.

Signature	GPU (OpenCL, previous export)	CPU (XNNPACK, 4 threads)
lint_128	354 ms	119 ms
lint_512	1509 ms	591 ms

GPU works as of the 2026-08-13 re-export. The re-export respells the one idiom mobile GPU delegates refuse — transformers' rank-5 repeat_kv expand — into an equivalent rank-4 matmul (outputs bitwise-identical on CPU); the OpenCL delegate now takes the whole graph. Measured with the LiteRT CompiledModel API (fp32 GPU precision, real inputs incl. the rule-pool matrix, best of 3 warm runs): lint_512 14.1 ms on the Pixel 8a, cosine 0.9992 vs the fp32 desktop reference; iPhone 17 Pro Metal lint_512 171 ms, cosine 0.9999. Set the GPU precision to fp32 — at fp16 GPU precision this family's norm reductions overflow and every output is NaN. These CompiledModel timings are not comparable to the classic-delegate benchmark_model timings above (different GPU runtime).

Snapdragon NPU (Hexagon)

• LFM2.5-Encoder-350M-Policy-Linter_fp16.tflite — the NPU runs it at 175.3 ms. The GPU does not — LiteRtException: Failed to compile model.
• LFM2.5-Encoder-350M-Policy-Linter_wi8fc.tflite — the GPU runs it at 80.57 ms. The NPU does not — LiteRtException: Failed to compile model.

file	backend	compiled	inference (median / min)	load
LFM2.5-Encoder-350M-Policy-Linter_fp16.tflite	NPU (Hexagon v81)	AOT (SM8850)	175.3 ms / 171.4 ms	438 ms
LFM2.5-Encoder-350M-Policy-Linter_wi8fc.tflite	GPU (Adreno)	—	80.57 ms / 79.75 ms	7362 ms

Measured on a Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status NONE throughout. Headroom 0.74–0.79, where 1.0 is the throttling threshold.

The NPU row marked AOT ran an artifact compiled ahead of time for SM8850 (ai-edge-litert 2.2.0 + QAIRT 2.47.0), not the published file. That artifact is not distributed here; the compile is one command in the NPU guide.

GPU wiring: GPU guide.

License

LFM Open License v1.0 (see LICENSE, unchanged from the base model). Note the license's commercial-use threshold (Section 5). This repository redistributes converted Derivative Works of LiquidAI/LFM2.5-Encoder-350M-Policy-Linter with modification notices per Section 4; all credit for the model to Liquid AI.