Instructions to use trymirai/Flint-S with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Trellis
How to use trymirai/Flint-S with Trellis:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
|
Download README.md from trymirai/Flint-S: direct link, hf CLI and curl.
- Browser
- Download file 7.97 kB
-
https://huggingface.co/trymirai/Flint-S/resolve/main/README.md
- Command line
-
hf download hf://trymirai/Flint-S/README.md
-
curl -L -o README.md https://huggingface.co/trymirai/Flint-S/resolve/main/README.md
7.97 kB
| license: apache-2.0 | |
| base_model: RecursiveSI/flint | |
| base_model_relation: quantized | |
| pipeline_tag: text-generation | |
| tags: | |
| - mirai-s | |
| - quantized | |
| - trellis | |
| - custom-format | |
| - experimental | |
| # Flint S | |
| Mirai S quantization of [Recursive Flint](https://huggingface.co/RecursiveSI/flint), a mixture-of-experts coding-agent model with 39.8B total parameters and approximately 2.5B active parameters per token. | |
| The root-level package is **Flint S 13.3**: **13.157 GB of weights/shared tensors**, **13.173 GB including tokenizer and metadata**. It uses a custom packed S format and includes a standalone reference decoder. | |
| **Validation status:** the KL and agent results below were measured on this checkpoint's dense dequantized evaluation build. The packed files passed reconstruction comparisons against that build, but **decoded-package RomaKL and direct runtime integration remain unvalidated**. These files are not a drop-in Transformers, GGUF, MLX or uzu checkpoint. The supplied decoder reconstructs tensors; it does not provide a serving engine. | |
| ## KL / teacher fidelity | |
| Lower KL is better. Top-1 is agreement with the teacher's next-token choice, not task accuracy. | |
| | Quantization | Size (GB) | On-policy RomaKL ↓ | Teacher top-1 agreement | Dataset-text RomaKL ↓ | | |
| |---|---:|---:|---:|---:| | |
| | **Mirai Flint S 13.3**, dense evaluation build | **13.17** | **0.09117** | **91.49%** | **0.23781** | | |
| | Unsloth UD-IQ1_M | 11.35 | 0.20225 | 86.72% | 0.41156 | | |
| | Unsloth UD-IQ2_XXS | 11.86 | 0.17714 | 87.51% | 0.37993 | | |
| | bartowski IQ2_XXS | 12.03 | 0.27529 | 84.46% | 0.58081 | | |
| | bartowski IQ2_XS | 12.74 | 0.22879 | 86.14% | 0.52038 | | |
| | bartowski IQ2_S | 13.39 | 0.21386 | 86.62% | 0.48050 | | |
| | Unsloth UD-Q2_K_XL | 13.56 | 0.13507 | 89.23% | 0.29978 | | |
| | Unsloth UD-IQ3_XXS | 14.54 | 0.12238 | 89.97% | 0.33076 | | |
| | bartowski IQ2_M | 14.88 | 0.15442 | 88.64% | 0.38832 | | |
| | bartowski Q2_K | 15.48 | 0.17558 | 87.84% | 0.39941 | | |
| | bartowski IQ3_XXS | 17.49 | 0.08920 | 91.42% | 0.25920 | | |
| | bartowski IQ3_XS | 18.32 | 0.06739 | 92.50% | 0.25420 | | |
| | bartowski Q3_K_S | 18.44 | 0.06659 | 92.57% | 0.23567 | | |
| | Unsloth UD-Q3_K_XL | 18.49 | 0.05566 | 93.30% | 0.19121 | | |
| | bartowski Q3_K_M | 19.53 | 0.06101 | 92.74% | 0.21117 | | |
| | bartowski Q3_K_L | 20.59 | 0.05018 | 93.46% | 0.18979 | | |
| | bartowski IQ3_M | 21.05 | 0.05041 | 93.51% | 0.19371 | | |
| | bartowski IQ4_XS | 22.45 | 0.02505 | 95.52% | 0.13466 | | |
| | Recursive nvfp4 | 22.86 | 0.06065 | 93.09% | 0.24904 | | |
| | Recursive int4-q4_1 | 25.99 | 0.05691 | 93.21% | 0.22758 | | |
| | Q8_0 | 42.30 | 0.00479 | 97.92% | 0.05890 | | |
| The GGUF comparisons were measured through llama.cpp; Mirai S uses a dense evaluation twin. Recursive entries follow their recorded evaluation paths. These are cross-runtime measurements. Packed decoder KL is still pending. Both contracts use the same 100-prompt pool, rendered with Flint's chat template. The headline on-policy contract scores **76,946 tokens** from Flint-generated continuations. The dataset-text contract scores **34,631 tokens** from dataset responses. | |
| Teacher cache and dense quantized evaluation use **FP32 activations** to reduce Flint's numerical instability on off-policy text. The BF16-activation teacher's measured floor against that cache is approximately **0.00359 KL / 98.36% top-1** on-policy and **0.0558 KL / 96.03% top-1** on dataset text. These numbers are specific to these contracts and precisions; they do not establish runtime-independent fidelity. | |
| ## Local agent benchmark | |
| These are **Mirai's in-house agent/task evaluations**, not official Terminal-Bench, SWE-bench or BFCL scores. The S column evaluates the dense dequantized twin of **S3-13.3-qB**, the checkpoint with 51/55 hard problems and 444/656 assistant-week v2 tasks. GGUFs use llama.cpp; the teacher uses SGLang. The packed decoder has not been agent-benchmarked. | |
| | Evaluation | Flint S 13.3 | Unsloth UD-Q2_K_XL | bartowski IQ3_XXS | Q8_0 | BF16 Flint | | |
| |---|---:|---:|---:|---:|---:| | |
| | Function calling | 92% | 97% | 94% | 96% | 91% | | |
| | Function calling, hard | 75% | 76% | 73% | 75% | 77% | | |
| | Hard-problem battery | 51/55 | 44/55 | 38/55 | 52/55 | 49/55 | | |
| | AppWorld, 80 tasks × 3 | 158/240 | 158/240 | Unvalidated¹ | 164/240 | 187/240 | | |
| | Assistant week v2, native tools | 444/656 | 242/656 | 333/656 | 358/656 | 460/656 | | |
| | Assistant week v1, text markers | 41/136 | 18/136 | 36/136 | 57/136 | 60/136 | | |
| | Creative build, fully correct | 4/4 | 4/4 | 4/4 | 4/4 | 4/4 | | |
| | Agentic build, declared done | 4/4 | 2/4 | 3/4 | 4/4 | 4/4 | | |
| ¹ The bartowski AppWorld record completed 240 episodes in only 108 seconds. That result requires harness/protocol review and is excluded as a valid model-quality comparison. It is not presented as 0% task ability. | |
| S and BF16 agent runs used SGLang; GGUF agent runs used llama.cpp. Backend, numerical and tool-protocol differences can affect results. These are recorded point estimates; this card does not claim statistically significant differences. | |
| Function-calling sets contain 175 standard and 150 hard cases, each spanning thinking/non-thinking runs and repeated samples (700/600 scored outputs). The hard battery contains 11 problems with five attempts each. Assistant v1 uses four seeds; v2 uses sixteen. AppWorld repeats each task three times. Creative/build sets contain only four tasks each. A declared-done result is not a render-quality pass; no latest-S agentic render result was graded. Historical text-marker sessions can loop or exhaust context, so v1 and native-tools v2 are reported separately. | |
| Sampling uses temperature **1.0**, top-p **0.95**, top-k **20**, with the original Flint template and task-specific token/step budgets. Aggregate scores, counts, checkpoint labels and hashes of the original result records are provided in [benchmarks/results.json](benchmarks/results.json). Training used Flint-generated coding, reasoning and tool traces; these are local evaluations, not an independent external audit. | |
| ## Files and format | |
| - `model-00.safetensors` … `model-49.safetensors`: packed weight shards. | |
| - `shared.safetensors`: codebooks and shared rotation parameters. | |
| - `flint_s.json` and `quantized-*.json`: format and tensor metadata. | |
| - `flint_s_decode.py`: standalone CPU reference decoder, requiring PyTorch and safetensors. | |
| - Original configuration, tokenizer, chat template and license. | |
| - `PACKAGE_SHA256.json`: file sizes and SHA-256 hashes for the latest payload, model card, decoder and benchmark summaries (excluding the manifest itself and legacy directories). | |
| Trellis formats include V4 restart-64 T6/T7/T8 and connected V2 k3/k4, with Kronecker rotations and row scales; other quantized tensors use the package's int8 format. See the decoder's format documentation for the precise bit order and reconstruction arithmetic. | |
| Packing reuses the exact previously recovered k3/k4 codes, and each of the 50 packed shards is decoded and compared against this checkpoint's dense twin. See `PACKING_RESULT.json` for the completed reconstruction checks. This establishes a weight-format check; decoded-model logits and direct runtime integration still require validation. | |
| To inspect/reconstruct the latest package without downloading legacy dense exports: | |
| ```bash | |
| hf download trymirai/Flint-S --local-dir flint-s \ | |
| --include 'model-*.safetensors' shared.safetensors flint_s.json \ | |
| flint_s_decode.py config.json 'tokenizer*' chat_template.jinja generation_config.json | |
| python -m pip install torch safetensors | |
| python flint-s/flint_s_decode.py flint-s flint-s-decoded | |
| ``` | |
| Reconstruction writes approximately 80 GB of dense tensors. The decoder output needs layout/configuration integration before serving; no one-command serving path for this packed layout is claimed. | |
| ## License | |
| Apache 2.0; see [LICENSE](LICENSE). Base model: [RecursiveSI/flint](https://huggingface.co/RecursiveSI/flint). This repository contains Mirai quantization artifacts and local evaluation results; credit for the base model belongs to Recursive. | |