--- license: apache-2.0 base_model: RecursiveSI/flint base_model_relation: quantized pipeline_tag: text-generation tags: - mirai-s - quantized - trellis - custom-format - experimental --- # Flint S Mirai S quantization of [Recursive Flint](https://huggingface.co/RecursiveSI/flint), a mixture-of-experts coding-agent model with 39.8B total parameters and approximately 2.5B active parameters per token. The root-level package is **Flint S 13.3**: **13.157 GB of weights/shared tensors**, **13.173 GB including tokenizer and metadata**. It uses a custom packed S format and includes a standalone reference decoder. **Validation status:** the KL and agent results below were measured on this checkpoint's dense dequantized evaluation build. The packed files passed reconstruction comparisons against that build, but **decoded-package RomaKL and direct runtime integration remain unvalidated**. These files are not a drop-in Transformers, GGUF, MLX or uzu checkpoint. The supplied decoder reconstructs tensors; it does not provide a serving engine. ## KL / teacher fidelity Lower KL is better. Top-1 is agreement with the teacher's next-token choice, not task accuracy. | Quantization | Size (GB) | On-policy RomaKL ↓ | Teacher top-1 agreement | Dataset-text RomaKL ↓ | |---|---:|---:|---:|---:| | **Mirai Flint S 13.3**, dense evaluation build | **13.17** | **0.09117** | **91.49%** | **0.23781** | | Unsloth UD-IQ1_M | 11.35 | 0.20225 | 86.72% | 0.41156 | | Unsloth UD-IQ2_XXS | 11.86 | 0.17714 | 87.51% | 0.37993 | | bartowski IQ2_XXS | 12.03 | 0.27529 | 84.46% | 0.58081 | | bartowski IQ2_XS | 12.74 | 0.22879 | 86.14% | 0.52038 | | bartowski IQ2_S | 13.39 | 0.21386 | 86.62% | 0.48050 | | Unsloth UD-Q2_K_XL | 13.56 | 0.13507 | 89.23% | 0.29978 | | Unsloth UD-IQ3_XXS | 14.54 | 0.12238 | 89.97% | 0.33076 | | bartowski IQ2_M | 14.88 | 0.15442 | 88.64% | 0.38832 | | bartowski Q2_K | 15.48 | 0.17558 | 87.84% | 0.39941 | | bartowski IQ3_XXS | 17.49 | 0.08920 | 91.42% | 0.25920 | | bartowski IQ3_XS | 18.32 | 0.06739 | 92.50% | 0.25420 | | bartowski Q3_K_S | 18.44 | 0.06659 | 92.57% | 0.23567 | | Unsloth UD-Q3_K_XL | 18.49 | 0.05566 | 93.30% | 0.19121 | | bartowski Q3_K_M | 19.53 | 0.06101 | 92.74% | 0.21117 | | bartowski Q3_K_L | 20.59 | 0.05018 | 93.46% | 0.18979 | | bartowski IQ3_M | 21.05 | 0.05041 | 93.51% | 0.19371 | | bartowski IQ4_XS | 22.45 | 0.02505 | 95.52% | 0.13466 | | Recursive nvfp4 | 22.86 | 0.06065 | 93.09% | 0.24904 | | Recursive int4-q4_1 | 25.99 | 0.05691 | 93.21% | 0.22758 | | Q8_0 | 42.30 | 0.00479 | 97.92% | 0.05890 | The GGUF comparisons were measured through llama.cpp; Mirai S uses a dense evaluation twin. Recursive entries follow their recorded evaluation paths. These are cross-runtime measurements. Packed decoder KL is still pending. Both contracts use the same 100-prompt pool, rendered with Flint's chat template. The headline on-policy contract scores **76,946 tokens** from Flint-generated continuations. The dataset-text contract scores **34,631 tokens** from dataset responses. Teacher cache and dense quantized evaluation use **FP32 activations** to reduce Flint's numerical instability on off-policy text. The BF16-activation teacher's measured floor against that cache is approximately **0.00359 KL / 98.36% top-1** on-policy and **0.0558 KL / 96.03% top-1** on dataset text. These numbers are specific to these contracts and precisions; they do not establish runtime-independent fidelity. ## Local agent benchmark These are **Mirai's in-house agent/task evaluations**, not official Terminal-Bench, SWE-bench or BFCL scores. The S column evaluates the dense dequantized twin of **S3-13.3-qB**, the checkpoint with 51/55 hard problems and 444/656 assistant-week v2 tasks. GGUFs use llama.cpp; the teacher uses SGLang. The packed decoder has not been agent-benchmarked. | Evaluation | Flint S 13.3 | Unsloth UD-Q2_K_XL | bartowski IQ3_XXS | Q8_0 | BF16 Flint | |---|---:|---:|---:|---:|---:| | Function calling | 92% | 97% | 94% | 96% | 91% | | Function calling, hard | 75% | 76% | 73% | 75% | 77% | | Hard-problem battery | 51/55 | 44/55 | 38/55 | 52/55 | 49/55 | | AppWorld, 80 tasks × 3 | 158/240 | 158/240 | Unvalidated¹ | 164/240 | 187/240 | | Assistant week v2, native tools | 444/656 | 242/656 | 333/656 | 358/656 | 460/656 | | Assistant week v1, text markers | 41/136 | 18/136 | 36/136 | 57/136 | 60/136 | | Creative build, fully correct | 4/4 | 4/4 | 4/4 | 4/4 | 4/4 | | Agentic build, declared done | 4/4 | 2/4 | 3/4 | 4/4 | 4/4 | ¹ The bartowski AppWorld record completed 240 episodes in only 108 seconds. That result requires harness/protocol review and is excluded as a valid model-quality comparison. It is not presented as 0% task ability. S and BF16 agent runs used SGLang; GGUF agent runs used llama.cpp. Backend, numerical and tool-protocol differences can affect results. These are recorded point estimates; this card does not claim statistically significant differences. Function-calling sets contain 175 standard and 150 hard cases, each spanning thinking/non-thinking runs and repeated samples (700/600 scored outputs). The hard battery contains 11 problems with five attempts each. Assistant v1 uses four seeds; v2 uses sixteen. AppWorld repeats each task three times. Creative/build sets contain only four tasks each. A declared-done result is not a render-quality pass; no latest-S agentic render result was graded. Historical text-marker sessions can loop or exhaust context, so v1 and native-tools v2 are reported separately. Sampling uses temperature **1.0**, top-p **0.95**, top-k **20**, with the original Flint template and task-specific token/step budgets. Aggregate scores, counts, checkpoint labels and hashes of the original result records are provided in [benchmarks/results.json](benchmarks/results.json). Training used Flint-generated coding, reasoning and tool traces; these are local evaluations, not an independent external audit. ## Files and format - `model-00.safetensors` … `model-49.safetensors`: packed weight shards. - `shared.safetensors`: codebooks and shared rotation parameters. - `flint_s.json` and `quantized-*.json`: format and tensor metadata. - `flint_s_decode.py`: standalone CPU reference decoder, requiring PyTorch and safetensors. - Original configuration, tokenizer, chat template and license. - `PACKAGE_SHA256.json`: file sizes and SHA-256 hashes for the latest payload, model card, decoder and benchmark summaries (excluding the manifest itself and legacy directories). Trellis formats include V4 restart-64 T6/T7/T8 and connected V2 k3/k4, with Kronecker rotations and row scales; other quantized tensors use the package's int8 format. See the decoder's format documentation for the precise bit order and reconstruction arithmetic. Packing reuses the exact previously recovered k3/k4 codes, and each of the 50 packed shards is decoded and compared against this checkpoint's dense twin. See `PACKING_RESULT.json` for the completed reconstruction checks. This establishes a weight-format check; decoded-model logits and direct runtime integration still require validation. To inspect/reconstruct the latest package without downloading legacy dense exports: ```bash hf download trymirai/Flint-S --local-dir flint-s \ --include 'model-*.safetensors' shared.safetensors flint_s.json \ flint_s_decode.py config.json 'tokenizer*' chat_template.jinja generation_config.json python -m pip install torch safetensors python flint-s/flint_s_decode.py flint-s flint-s-decoded ``` Reconstruction writes approximately 80 GB of dense tensors. The decoder output needs layout/configuration integration before serving; no one-command serving path for this packed layout is claimed. ## License Apache 2.0; see [LICENSE](LICENSE). Base model: [RecursiveSI/flint](https://huggingface.co/RecursiveSI/flint). This repository contains Mirai quantization artifacts and local evaluation results; credit for the base model belongs to Recursive.