| --- |
| license: apache-2.0 |
| language: |
| - en |
| pipeline_tag: token-classification |
| base_model: fastino/GLiNER2.5-Decide |
| tags: |
| - gliner2 |
| - gguf |
| - antfly-inference |
| - decision-model |
| --- |
| |
| ## Canonical Antfly layout (2026-10-17) |
|
|
| This repository follows the upstream Fastino checkpoint name. The previous |
| `-gguf` repository URL redirects here. The paired filenames identify the |
| **Q8_0 bundle variant**: the encoder matrices are Q8_0, while the auxiliary |
| head remains **FP32** despite its `Q8_0` filename suffix. The weight bytes, |
| tokenizer, and source configuration have not changed. The bundle descriptor |
| and `antfly_inference_variants.json` name both required components explicitly. |
| Historical validation reports retain the original paths and measurements; |
| `gliner_head.gguf` in those reports is now `gliner2-head.Q8_0.gguf`. |
|
|
| This DeBERTa Q8 conversion supports ordinary extraction/classification |
| compatibility. It does **not** qualify the public typed Decide endpoint: |
| the exporter strips `decide` and `typed_decisions`. The source checkpoint's |
| name and the historical browser smoke test do not confer that qualification. |
| Use a qualified FP32 Decide checkpoint or the separate qualified 1B Q8 bundle |
| for typed decisions. Native/Metal full-task and long-document qualification |
| must be established separately for these converted bytes. |
|
|
|
|
| # GLiNER2.5 Decide — Antfly Q8_0 split GGUF bundle |
| |
| This is an Antfly inference bundle converted from [Fastino's GLiNER2.5-Decide](https://huggingface.co/fastino/GLiNER2.5-Decide). The encoder is Q8_0 GGUF; the GLiNER head is a separate GGUF file. The bundle uses Antfly's `gliner2_split_bundle/v1` layout and is intended for Antfly's GLiNER2 inference runtime, including its browser/WASM playground. It is **not** a single-file llama.cpp model. Keep the files and relative paths together. |
|
|
| The source is Apache-2.0 licensed. The local `model.safetensors` used for conversion has SHA-256 `40a5a23ff860dc3dff426cecd1048cacdd29c648c96db209dad818e9686dc997`, matching the file in the [upstream repository](https://huggingface.co/fastino/GLiNER2.5-Decide/tree/main) at revision `5a7adf72a23b4d311abae6ce050d7f0012bb3416`. Model design, intended uses, limitations, and upstream evaluation are described in [Fastino's model card](https://huggingface.co/fastino/GLiNER2.5-Decide). Fastino did not publish this Antfly conversion. |
|
|
| ## Bundle contents |
|
|
| - `antfly_inference_bundle.json` — split-bundle receipt naming the encoder and head. |
| - `gliner2-encoder.Q8_0.gguf` — quantized encoder, SHA-256 `8d9bea7df21ab4a8a17333f51f246faab5280159e5751ebbadd81a85ec96c871`. |
| - `gliner2-head.Q8_0.gguf` — GLiNER head, SHA-256 `52cb6197d3dcfe5ad6276ceb037a3333d2921bbcf3dfb8289aeff82fbd5693dc`. |
| - `config.json`, `encoder_config/config.json`, `special_tokens_map.json`, `tokenizer.json`, `tokenizer_config.json` — configuration and tokenizer files from the source checkpoint. |
| - `SHA256SUMS` — SHA-256 hashes for all eight bundle files. |
|
|
| Download the repository with `hf download antflydb/GLiNER2.5-Decide --local-dir ./GLiNER2.5-Decide-Q8_0`, then load that directory with an Antfly runtime that supports `gliner2_split_bundle/v1`. For reproducible deployments, pin a repository revision instead of using `main`. |
|
|
| ## Validation |
|
|
| Antfly's focused Decide runtime test passed 3/3 cases with these exact local bundle files and the pinned browser/WASM extraction runtime. This is a compatibility smoke test, not a full quality or cross-browser evaluation. Quantization can change outputs relative to the original checkpoint. |
|
|
