GLiClass-Edge v3.0 for LiteRT: S128 and S256 graphs (fp32 and fp16 weights), float16 token table, tokenizer, Python host, Android sample, conversion scripts, fixtures and card

88c9095 verified about 1 month ago

preview code

Raw History Blame Contribute Delete

11.3 kB

GLiClass Edge — Android zero-shot text classification

Type English text and the labels to choose from, one per line. Pick Single label (softmax, the top label) or Multi-label (sigmoid, every label at or above a threshold), GPU or CPU, and tap Classify for every label's score with the chosen labels highlighted, the graph window and timings. An optional prompt (a task description) goes right before the text. For the prefilled request (an invented support message with five labels) the app answers connectivity problem (0.856), the same decision as the official gliclass pipeline.

Model and requirements

Model: litert-community/GLiClass-Edge-v3.0-LiteRT.
Upstream: knowledgator/gliclass-edge-v3.0 revision df03993a2ed98e5e4a0d2dd7efbbd105abe874cf, Apache-2.0; encoder jhu-clsp/ettin-encoder-32m (ModernBERT, MIT).
Semantics: gliclass 0.1.20 ZeroShotClassificationPipeline (uni-encoder): single-label and multi-label classification with an optional prompt.
Android: arm64-v8a, Android 8.0 / API 26 or newer; compile/target SDK 35.
Runtime: LiteRT 2.2.0 CompiledModel; Material 1 Compose with MVVM.

Explicit GPU FP32 precision is mandatory. On a Galaxy S26 the default GPU precision ran the 128-token graph but changed decisions near the decision boundary: 475 of 482 single-label and 455 of 482 multi-label decisions matched the official pipeline. The app requests CompiledModel.GpuOptions(precision = CompiledModel.GpuOptions.Precision.FP32). CPU (four threads) is a second backend you choose in the app; the app never switches to it on its own.

Download, build and install

Use JDK 17, Android SDK platform 35 / build-tools 35.0.0, Android platform-tools (adb) and the Hugging Face CLI (hf). The app needs four files of the model repository:

hf download litert-community/GLiClass-Edge-v3.0-LiteRT \
  gliclass_edge_v3_s128_fp32.tflite gliclass_edge_v3_s256_fp32.tflite \
  host_assets/tok_embeddings_fp16.bin host_assets/tokenizer.json \
  --local-dir "$HOME/Downloads/GLiClass-Edge-v3.0-LiteRT"
./gradlew :app:assembleDebug
# Optional when multiple devices are connected:
# export ANDROID_SERIAL=your-device-serial
adb install app/build/outputs/apk/debug/app-debug.apk
./scripts/install_to_device.sh "$HOME/Downloads/GLiClass-Edge-v3.0-LiteRT"
adb shell am start -n com.gliclass/.MainActivity

The install script's argument defaults to the download directory above. It validates all four files before contacting the device, pushes through /data/local/tmp/gliclass/, copies with run-as com.gliclass into private files/ and removes the temporary files. Install the debug APK first; run-as requires a debuggable package. Model files are external and are not needed to build the APK. A launch before the install script ran shows "Model unavailable" with "Missing . Run scripts/install_to_device.sh, then retry."

External files

The app loads the graphs by path and memory-maps the embedding table. Four files, 149,936,711 bytes.

File	Bytes	Purpose
`gliclass_edge_v3_s128_fp32.tflite`	53,703,480	Up to 128 tokens (labels + prompt + text)
`gliclass_edge_v3_s256_fp32.tflite`	53,965,476	Up to 256 tokens
`host_assets/tok_embeddings_fp16.bin`	38,684,160	Float16 `[50370,384]` token-embedding table, upcast to float32 on lookup
`host_assets/tokenizer.json`	3,583,595	Byte-level BPE vocabulary, merges and added tokens (the upstream file, unchanged)

Each graph takes inputs_embeds [1,N,384], attention_mask [1,N] and label_routing [1,25,N] (float32) and returns logits [1,1,1,25]; the first n logits belong to the n labels. The model repository also holds float16-weight graphs (*_wfp16.tflite, half the size); this app uses the float32 graphs.

App architecture

Text + labels (+ prompt) → linearized string → Kotlin byte-level BPE tokenizer ([CLS] … [SEP])
  → IDs, attention, one-hot <<LABEL>> routing + memory-mapped float16 table upcast to float32
  → LiteRT graph (GPU FP32 or CPU) → 25 logits
  → Kotlin decoder (softmax argmax, or sigmoid ≥ threshold) → labels and scores

MainActivity observes immutable UiState. MainViewModel owns the classifier and runs every model call on one confined dispatcher (Dispatchers.Default.limitedParallelism(1)), because LiteRT reuses its native input and output buffers; onCleared() closes the classifier. One LiteRT Environment is shared per process. s128 compiles at startup; s256 compiles on first use and stays resident. Before Ready the app repeats the bundled example through the whole pipeline five times; each window/backend also gets one untimed pass. Graph timing covers the first input write through output readback because run() is asynchronous.

Labels go one per line, or on a single line separated by commas; up to 25 labels. Labels are trimmed and empty lines are skipped. The request is linearized as the pipeline does it:

<<LABEL>>label 1<<LABEL>>label 2…<<SEP>>{prompt}{text}

There is no separator between the prompt and the text. Single-label returns the top softmax label; multi-label returns every label whose sigmoid is at least the threshold (0.5 by default), in input order, or none.

The tokenizer follows tokenizer.json the way Hugging Face tokenizers applies it: added tokens are split out first (<<LABEL>>, <<SEP>>, [CLS]; [MASK] takes the whitespace on its left), each remaining segment is NFC-normalized and split on the normalized added tokens (runs of 2–24 spaces, [unused0]–[unused82]), and every remaining piece gets a leading space before the GPT-2 regex, the byte mapping and the BPE merges.

The smallest fitting window of 128/256 tokens is chosen; more than 256 tokens or more than 25 labels is rejected, never truncated (the pipeline would truncate at 1,024 tokens). A text, prompt or label that contains <<LABEL>> is rejected. Few-shot examples and hierarchical labels of the pipeline are not ported. The model is English.

Files

Path	Role
`app/src/main/java/com/gliclass/MainActivity.kt`	Compose host; reads the debug and benchmark launch extras
`app/src/main/java/com/gliclass/MainViewModel.kt`	Owns the classifier on the confined dispatcher: load, warm-up, classify, diagnostics
`app/src/main/java/com/gliclass/UiState.kt`	Immutable screen state and the result rows
`app/src/main/java/com/gliclass/LabelEditor.kt`	The label editor's format
`app/src/main/java/com/gliclass/view/GliclassScreen.kt`	The screen; `view/Theme.kt` and `view/Color.kt` hold the Material 1 theme
`app/src/main/java/com/gliclass/GliclassTokenizer.kt`	Byte-level BPE tokenizer read from `tokenizer.json`
`app/src/main/java/com/gliclass/GliclassInputs.kt`	Linearization, window choice, padding, label routing, float16 table lookup
`app/src/main/java/com/gliclass/GliclassClassifier.kt`	LiteRT `CompiledModel` calls (GPU FP32 or CPU), resident windows, timing
`app/src/main/java/com/gliclass/GliclassDecoder.kt`	Softmax and sigmoid decisions of the pipeline
`app/src/main/java/com/gliclass/GliclassGateFixtures.kt`, `GliclassGateRunner.kt`	Debug fixture gate
`app/src/main/java/com/gliclass/GliclassDiagnostics.kt`	First-request and paced reports (debug and benchmark builds)
`app/src/debug/assets/gate_fixtures.json`	152 captured official requests for the debug gate
`app/src/test/java/com/gliclass/`	JVM parity tests against the official pipeline
`scripts/install_to_device.sh`	Copies the four external files into the app's `files/`
`scripts/TEST_DATA.md`	The gate asset and the optional JVM reference data
`LICENSE`, `NOTICE`, `licenses/`	Apache-2.0 text, attribution and the retained upstream licenses

Measured

Samsung Galaxy S26 SM-S942Q, Android 16, LiteRT 2.2.0, USB powered, thermal status 1, app in the foreground (top-app). Medians are the middle value of the sorted list (the upper one of the two for an even count).

Fixture gate, debug build, GPU FP32 and CPU (four threads), 552 captured official requests (482 at s128, 70 at s256): on each backend 552/552 requests gave the official single-label and multi-label decisions, all logits finite, and the on-device token IDs, <<LABEL>> positions, windows and padding equal the captured Python inputs for all 552. Max |Δlogit| against the fp32 oracle: 9.08e-3 GPU, 9.02e-3 CPU. Both windows run whole on the GPU delegate, in one partition.
Graph (input write → readback), median over the gate requests: GPU s128 5.86 ms, s256 8.23 ms; CPU s128 7.83 ms, s256 15.65 ms.
Cold start, benchmark build, GPU, the prefilled request (61 tokens, five labels): Ready 0.59 s after process start; the first request took 8.05 / 8.38 / 8.20 ms end to end in three cold starts (graph 6.1 ms, tokenize + embed 1.9–2.1 ms).
Paced, benchmark build, 20 requests one every 2 s: median end to end 13.15 ms on GPU (min 9.55, max 18.78; graph 6.92 ms, tokenize + embed 5.9 ms) and 22.36 ms on CPU.
Not measured: Pixel phones, other Android devices, sustained runs.

Tests

JVM tests compare the Kotlin host with the official pipeline and the Python tokenizers library. They skip unless -Pgliclass.fixtures=/path/to/data is set; see scripts/TEST_DATA.md. With the conversion run's data: tokenizer 552/552 oracle strings and 261/261 stress strings, inputs 552/552 requests, decoder 552/552 requests, and the 152 rows of the committed gate asset.

./gradlew :app:testDebugUnitTest -Pgliclass.fixtures=/path/to/data

The debug APK runs the fixture gate over the rows of app/src/debug/assets/gate_fixtures.json. The committed asset holds 152 requests (SemIf, the model card's two examples and six invented requests), all at s128. The 552-request gate above used an asset that the conversion run regenerates with the ag_news and banking77 rows, whose text is not redistributed here.

adb shell am force-stop com.gliclass
adb shell am start -W -n com.gliclass/.MainActivity --ez gate true --es backend gpu --es report app_gate_gpu.json
adb exec-out run-as com.gliclass cat files/app_gate_gpu.json > app_gate_gpu.json

For every request the gate checks that the on-device token IDs, <<LABEL>> positions, window and padding equal the captured Python inputs and that the single-label and multi-label (0.5) decisions equal the official results, and records max |Δlogit| against the fp32 oracle and timing medians. The report is files/<report>.partial while running. Completion lines use GLICLASS_GATE. --ez firsttap true records the first classification after a normal startup, and --ez paced true (optional --ei count and --ei interval_ms) classifies the bundled example repeatedly; both work in the debug and the non-debuggable benchmark build and also print their reports to logcat under GLICLASS_GATE. With these extras the activity shows above a secure lock screen so that the measured process is in the foreground; a normal launch is unchanged.