| --- |
| license: apache-2.0 |
| library_name: splash |
| pipeline_tag: text-generation |
| inference: false |
| base_model: |
| - mlx-community/Qwen3.8-27B-4bit |
| - incoai/Qwen3.8-27B-DFlash2 |
| base_model_relation: quantized |
| tags: |
| - splash |
| - apple-silicon |
| - metal |
| - local-inference |
| - dflash2 |
| - speculative-decoding |
| - qwen3.8 |
| - 4-bit |
| --- |
| |
| # Qwen3.8-27B-Splash |
|
|
| **Qwen3.8-27B, packed for Splash on Apple silicon.** |
|
|
| [Splash](https://github.com/incoai/splash) is Inco AI's open-source inference |
| engine for Apple silicon, built around the model. This package contains |
| everything Splash needs to serve Qwen3.8-27B: the 4-bit target, its DFlash 2 |
| draft, the vision encoder, and the tokenizer. It is not a Transformers or MLX |
| checkpoint and does not load anywhere but Splash. |
|
|
| Qwen3.8-27B is the dense launch model. The other, |
| [Qwen3.6-35B-A3B-Splash](https://huggingface.co/incoai/Qwen3.6-35B-A3B-Splash), |
| is a mixture-of-experts model with about 3B active parameters per token, and |
| the faster of the two. |
|
|
| [Engine](https://github.com/incoai/splash) 路 |
| [Launch post and benchmarks](https://inco.ai/blog/splash/) 路 |
| [DFlash 2](https://inco.ai/blog/dflash2/) |
|
|
| ## Quick start |
|
|
| Apple M3 or newer, macOS 26.4 or later, [Homebrew](https://brew.sh), and 36 GB |
| of unified memory (48 GB or more recommended). |
|
|
| ```bash |
| brew install incoai/tap/splash |
| splash serve --model incoai/Qwen3.8-27B-Splash |
| ``` |
|
|
| The first run downloads this package (17.4 GB), verifies it, checks available |
| memory, and starts serving on `127.0.0.1:8000`. When it prints its `Ready` |
| line, open <http://127.0.0.1:8000> or attach an agent you already have |
| installed from another terminal: |
|
|
| ```bash |
| splash opencode # or: splash claude / splash codex / splash hermes |
| ``` |
|
|
| The server binds `127.0.0.1`, and authentication is off by default. Set |
| `SPLASH_API_KEY` before exposing it beyond your Mac. |
|
|
| The API is OpenAI Chat Completions and Responses, and Anthropic Messages, with |
| streaming, tool calls, JSON Schema output, images, and inline PDFs: |
|
|
| ```bash |
| curl http://127.0.0.1:8000/v1/chat/completions \ |
| -H 'Content-Type: application/json' \ |
| -d '{ |
| "model": "incoai/Qwen3.8-27B-Splash", |
| "messages": [{"role": "user", "content": "Explain speculative decoding in one sentence."}] |
| }' |
| ``` |
|
|
| Reasoning is on by default at `xhigh`. `reasoning_effort` takes `low`, |
| `medium`, `xhigh`, or `none`, and reasoning comes back as `reasoning_content`. |
|
|
| There is no config file. The only settings are ceilings such as `--max-memory` |
| and `--max-context`, which the |
| [README](https://github.com/incoai/splash#settings) lists with their defaults. |
|
|
| ## Performance |
|
|
| Measured on an M5 Pro (16-core GPU, 48 GB) with selected SPEED-Bench coding |
| prompts over HTTP, with a 1,024-token output limit and reasoning at medium. |
| The ratio in each cell is against the next-fastest engine we measured. |
|
|
| | Metric | Qwen3.8-27B | |
| | --- | ---: | |
| | Decode 路 short prompt | 74 tok/s (2.0脳) | |
| | Prefill 路 32K prompt | 363 tok/s (1.2脳) | |
| | Time to first token 路 32K prompt, uncached | 96 s (1.2脳) | |
| | Cached time to first token 路 32K replay | 282 ms (7.3脳) | |
| | Aggregate decode 路 4 concurrent short prompts | 170 tok/s (3.9脳) | |
|
|
| Splash led on every measure at every prompt length we tested. With 16 |
| concurrent 32K requests, Splash completed all 16 where a general-purpose |
| memory policy accepted nine. The cached figure replays the prompt exactly, so |
| a real turn also pays for the tokens it adds. The [launch |
| post](https://inco.ai/blog/splash/) has the method and the full comparison. |
|
|
| ## Package contents |
|
|
| ``` |
| target/ 66 files 14.1 GiB Qwen3.8-27B, 4-bit, one packed file per layer |
| draft/ 6 files 1.2 GiB DFlash 2 draft model |
| vision/ 1 file 0.9 GiB bf16 vision encoder |
| tokenizer/ 5 files tokenizer and chat template |
| manifest.json provenance, geometry, and SHA-256 of every artifact |
| ``` |
|
|
| | Component | Source | Revision | |
| | --- | --- | --- | |
| | Target, tokenizer, vision | [`mlx-community/Qwen3.8-27B-4bit`](https://huggingface.co/mlx-community/Qwen3.8-27B-4bit) | `3e6447f082e89cc7f0bc6e5441afd38dfce760ff` | |
| | Draft | [`incoai/Qwen3.8-27B-DFlash2`](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2) | `dedf8df68adfb1afeaf7b7480c0a0243108177b4` | |
|
|
| The weights are fixed-layout binaries that Splash maps directly from disk. The |
| target and the vision encoder total 15 GiB, and the draft is another 1.2 GiB. |
| The draft is a five-layer DFlash 2 model that reads the target's hidden states |
| at five layers and proposes 7 tokens per step, which the target verifies in |
| one pass. The chat template is upstream's with one change: a system message |
| after the first turn is rendered in place instead of rejected, which coding |
| agents that inject instructions mid-conversation need. |
|
|
| `manifest.json` records the revisions above, the execution geometry, and the |
| size and SHA-256 of every artifact. Splash pins an immutable commit of this |
| repository, checks every artifact's SHA-256 before installing it, and |
| re-checks sizes and alignment on every start. The target is the upstream 4-bit |
| conversion, and the target verifies every drafted token, so speculation |
| changes speed and not the output distribution. |
|
|
| ## License |
|
|
| Apache-2.0. Every component is Apache-2.0 upstream as well: Qwen3.8-27B |
| (Alibaba), its 4-bit conversion (mlx-community), and the DFlash 2 draft (Inco |
| AI). |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{inco2026splash, |
| title = {{Splash: A Local Engine Built Around the Model}}, |
| author = {{Inco AI}}, |
| year = {2026}, |
| month = {September}, |
| url = {https://inco.ai/blog/splash/} |
| } |
| ``` |
|
|
