license: apache-2.0
pipeline_tag: text-generation
tags:
- mlx
- int4
- safetensors
Flint INT4
MLX-format INT4 weights for Recursive Flint, primarily intended for local inference on Apple Silicon. Projections and experts use group-64 affine INT4; embeddings use INT8. The weights occupy approximately 22.6 GB.
See Flint’s model card for model details, benchmarks, chat conventions, and limitations.
Use with Rec
Follow Rec’s requirements and installation guide, then launch pi or OpenCode:
rec launch pi
# Or use OpenCode:
rec launch opencode
HTTP output budgets in Rec MLX v0.1.0
For OpenAI-compatible HTTP requests, set max_tokens or max_completion_tokens to bound total generated output while retaining grammar: "flint":
{
"model": "flint",
"messages": [{"role": "user", "content": "Hello"}],
"grammar": "flint",
"max_tokens": 2048,
"stream": false
}
The cap includes reasoning and answer or tool-call tokens. Reaching it can return finish_reason: "length" before an answer or tool call is complete. Non-null max_completion_tokens takes precedence over max_tokens. Omitting both permits generation through the remaining context; the terminal chat token default is separate.
This version has no separate reasoning-only allowance. Unsupported fields such as reasoning_effort, thinking_token_budget, reasoning_budget_tokens and chat_template_kwargs are removed and logged. Nonempty custom stop strings cannot be combined with Flint grammar. Token limits must use integer JSON values.
See the pinned request allowlist and validation and output-limit handling.
Download the weights separately
With the Hugging Face CLI installed:
hf download RecursiveSI/flint-int4 --local-dir flint-int4
For llama.cpp on Metal, use Flint INT4 GGUF.
Training data
Flint’s training data included publicly available and crawled web documents, text and code, commercially licensed data, and AI-generated synthetic data. Post-training included instruction tuning, reinforcement learning and human-preference data. See the base Flint model card for shared model and training information.