README.md

---
license: apache-2.0
pipeline_tag: text-generation
tags:
- mlx
- int4
- safetensors
---

<p align="center">
  <img src="assets/flint-logo.png" width="160" alt="Flint logo">
</p>

<h1 align="center">Flint INT4</h1>

MLX-format INT4 weights for [Recursive Flint](https://huggingface.co/RecursiveSI/flint), primarily intended for local inference on Apple Silicon. Projections and experts use **group-64 affine INT4**; embeddings use INT8. The weights occupy approximately **22.6 GB**.

See [Flint’s model card](https://huggingface.co/RecursiveSI/flint) for model details, benchmarks, chat conventions, and limitations.

## Use with Rec

Follow [Rec’s requirements and installation guide](https://github.com/recursive-oss/rec), then launch pi or OpenCode:

```bash
rec launch pi
# Or use OpenCode:
rec launch opencode
```

### HTTP output budgets in Rec MLX v0.1.0

For OpenAI-compatible HTTP requests, set `max_tokens` or `max_completion_tokens` to bound total generated output while retaining `grammar: "flint"`:

```json
{
  "model": "flint",
  "messages": [{"role": "user", "content": "Hello"}],
  "grammar": "flint",
  "max_tokens": 2048,
  "stream": false
}
```

The cap includes reasoning and answer or tool-call tokens. Reaching it can return `finish_reason: "length"` before an answer or tool call is complete. Non-null `max_completion_tokens` takes precedence over `max_tokens`. Omitting both permits generation through the remaining context; the terminal chat token default is separate.

This version has no separate reasoning-only allowance. Unsupported fields such as `reasoning_effort`, `thinking_token_budget`, `reasoning_budget_tokens` and `chat_template_kwargs` are removed and logged. Nonempty custom stop strings cannot be combined with Flint grammar. Token limits must use integer JSON values.

See the pinned [request allowlist and validation](https://github.com/recursive-oss/rec/blob/8af99abaf50b33903147d89eed91d1bb3ac642dd/engine/src/rec_mlx/server.py#L134-L224) and [output-limit handling](https://github.com/recursive-oss/rec/blob/8af99abaf50b33903147d89eed91d1bb3ac642dd/engine/src/rec_mlx/server.py#L249-L262).

## Download the weights separately

With the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli) installed:

```bash
hf download RecursiveSI/flint-int4 --local-dir flint-int4
```

For llama.cpp on Metal, use [Flint INT4 GGUF](https://huggingface.co/RecursiveSI/flint-int4-gguf).

## Training data

Flint’s training data included publicly available and crawled web documents, text and code, commercially licensed data, and AI-generated synthetic data. Post-training included instruction tuning, reinforcement learning and human-preference data. See the [base Flint model card](https://huggingface.co/RecursiveSI/flint#training) for shared model and training information.

## License

[Apache License 2.0](https://huggingface.co/RecursiveSI/flint-int4/blob/main/LICENSE).