Naive-N0.5-Flash — EXL3 4 bpw (experts 4 / attention 6)

Original model: NaiveAI/Naive-N0.5-Flash by the NaiveAI Team, MIT License. This artifact is quantized from the bf16 master release (not the FP8 one). It contains modified weights (EXL3 trellis quantization); the original model is © its authors. Tokenizer and chat template are unchanged from the original. Served by the veloGB10 engine it is listed by /v1/models as doth4580/Naive-N0.5-Flash-EXL3-4bpw (the model_name: line of this card); base_model: is the original model it was quantized from, not this pack.

A sibling of doth4580/Naive-N0.5-Flash-EXL3-3.5bpw: an EXL3 (ExLlamaV3 trellis) mixed-bitrate quantization of Naive-N0.5-Flash — 309B parameters, 15.5B active, native 1M-token context — with the routed experts at 4 bpw and attention, the DSA indexer, the dense layer-0 MLP and lm_head at 6 bpw (4.03 bpw overall). The pack is 157 GB (147 GiB) against ~315 GB for the FP8 release and ~618 GB for bf16; it fits across two NVIDIA DGX Spark (GB10) machines with less headroom than the 3.5 bpw sibling, or across four comfortably.

Quantized with stock exllamav3 1.5.3 plus the architecture definition from sf-stav/Naive-N0.5-Flash-EXL3-2x-DGX-Spark, run on 8× RTX PRO 6000 (Blackwell) in about 1.5 hours, with a 498-row × 2,048-token calibration set built from code repositories, agent-style tool-calling conversations rendered through the model's own chat template, and general text (instead of the converter's default 250-row general set).

What this pack is: sources and credits

Accuracy

Forward-only gate in exllamav3 (teacher-forced NLL on a 977-token technical text; next-token probability after two code/JSON prefixes), same prompts for every row:

model source NLL (nats/token) p(8-space indent after if n < 0:) p(age) on a JSON key
Naive-N0.5-Flash bf16, transformers bf16 1.627 1.000 1.000
this pack (4.03 bpw) bf16 1.634 1.000 1.000
sibling 3.5 bpw pack FP8 1.619 1.000 1.000

So this pack is within 0.007 nats of its own unquantized source. It is not better than the 3.5 bpw sibling on that text: the FP8 release and the bf16 master are different checkpoints (their embedding tables differ by ~14 % relative), and on this particular text the FP8-derived pack scores marginally lower. Agentic tool use with tool-eval-bench v2.7.1 (69 scenarios, temperature 0, seed 42, thinking on), served by the veloGB10 engine at TP=4: sibling 3.5 bpw pack 86 / 100 (118 / 138 points); this pack 91 / 100 (126 / 138). One run each; the bench itself warns a single trial is not a calibrated difficulty estimate, so read it as "at least as good", not as a five-point lead. Where this pack may pay is code and tool-call output, which its calibration set targets, and in the headroom that 4-bit experts leave against outliers; measure on your own workload. Note that a pack built by mixing the two sources (attention from bf16, experts copied from the FP8-derived pack) scored 2.48 nats — the two releases do not mix.

Running it with veloGB10

Download a release binary of veloGB10 (one self-contained file; see its README), and the pack with hf download doth4580/Naive-N0.5-Flash-EXL3-4bpw --local-dir /path/to/Naive-N0.5-Flash-EXL3-4bpw. The server is OpenAI-compatible on http://<head-ip>:9000/v1. The flags below are the ones this pack was tested with; --max-batch sets how many requests run at once.

TP=2 and TP=4 only: no single GB10 holds this model (the engine says so at boot).

Two GB10s (TP=2)

On the second machine (no model copy, no configuration):

./gb10_inference --node --port 29500

On the head (the machine with the model folder):

./gb10_inference --server \
  --model-dir /path/to/Naive-N0.5-Flash-EXL3-4bpw \
  --tp 2 --nodes <node-ip>:29500 \
  --host 0.0.0.0 --port 9000 \
  --max-seq-len 262144 \
  --max-batch 2 \
  --max-tokens 65536 \
  --prefix-cache on

Four GB10s (TP=4)

Start ./gb10_inference --node --port 29500 on each of the three other machines, then on the head:

./gb10_inference --server \
  --model-dir /path/to/Naive-N0.5-Flash-EXL3-4bpw \
  --tp 4 --nodes <node1-ip>:29500,<node2-ip>:29500,<node3-ip>:29500 \
  --host 0.0.0.0 --port 9000 \
  --max-seq-len 262144 \
  --max-batch 2 \
  --max-tokens 65536 \
  --prefix-cache on

The nodes need no copy of the model. The head sends each node only its own share of the weights over the network and keeps it in a local cache, so a second start of the same model transfers nothing. On a box with two active RoCE port pairs, name the one to use with --rdma-dev <rail1>,<rail2>; with one pair it is found automatically.

With the Naive S8 FP8 drafter (NaiveAI/Naive-N0.5-Flash-FP8-Draft): add --draft-dir /path/to/Naive-N0.5-Flash-FP8-Draft to the head's command line (any topology above).

Measured with veloGB10

VeloBench runs against the served API on NVIDIA DGX Spark-class GB10 machines (2026-11-09, veloGB10 0.7.7 development builds), at production sampling: each request sends the pack's default sampler settings, thinking on, one request at a time for the single-request columns. Decode is in output tokens per second and includes speculative decoding. One run per cell; sampled runs vary by roughly ±10%.

Topology Draft source Code turn decode Story turn decode All-turn median decode Prefill (tok/s) 2 requests, aggregate Draft acceptance
TP=2 Naive S8 FP8 drafter 37 33 45 747 44 32.7%
TP=4 Naive S8 FP8 drafter 82 46 53 1,072 66 36.0%

Specifications

Base model NaiveAI/Naive-N0.5-Flash (bf16 master as the direct source)
Architecture MoE, 309B total / 15.5B active, 48 layers, hidden 4096, 64 heads / 4 KV heads, head_dim 192
Attention hybrid 39 SWA layers (window 128) + 9 DSA layers (top-2048 selection), GQA4, no full attention
MoE 256 routed experts, 8 per token, expert intermediate 2048
Context / vocab 1,048,576 tokens / 152,576
Quantization EXL3 trellis, mul1 codebook — routed experts 4 bpw; attention, DSA indexer, dense MLP and lm_head 6 bpw (4.03 bpw overall); embeddings unquantized; calibration 498×2048 (code / agent / general)
Format / size native exl3 — 157 GB (147 GiB), 24 shards
License MIT (inherited from the original model)

Files

File Notes
model-00001..00024-of-00024.safetensors the quantized weights
model.safetensors.index.json, quantization_config.json tensor index and the full EXL3 metadata
config.json the native ExLlamaV3 pack config (carries quantization_config)
RECIPE.yaml the per-tensor bit-rate recipe the converter was run with
GATE_exllamav3.json, CONVERT.log the gate numbers above and the converter's full log
SHA256SUMS checksums of every file in the pack
modeling_naive_n05_flash.py, configuration_naive_n05_flash.py model code, loaded via trust_remote_code
tokenizer + chat_template.jinja, generation_config.json unchanged from the original

Notes

  • Beyond 2,048 tokens the DSA layers are approximate in runtimes that run them as dense attention (exact up to 2,048 tokens); the 1M native context needs a sparse attention backend.
  • Larger than the sibling: ~14 % more bytes than the 3.5 bpw pack, so expect proportionally lower decode speed on bandwidth-bound hardware such as the DGX Spark.
  • Sampling defaults from the original generation_config.json: temperature 1.0, top_p 0.95.

Intellectual property

We do not claim any intellectual-property ownership of the model, its weights, its architecture or its original documentation. All rights stay with the original authors and with every contributor credited above, under their own licences, which apply to this repository unchanged. Our contribution is limited to what the "What this pack is" section says we did (quantization, packaging, testing and the veloGB10 instructions). If you are a rights holder and want something here changed or removed, open a discussion on this repository.


Original model card — NaiveAI/Naive-N0.5-Flash

Reproduced verbatim from the original repository (metadata header removed).

Naive-N0.5-Flash

Building Frontier AI with AI

Introduction

Naive-N0.5-Flash is an open-weight 309B MoE model with 15.5B active parameters, built for coding and AI R&D. It supports a native 1M-token context window through a hybrid of Sliding-Window Attention (SWA) and lightweight DeepSeek Sparse Attention (DSA), with no full-attention layers.

Key Features

  • Native 1M context, without full attention. Naive-N0.5-Flash combines Sliding-Window Attention (SWA) and lightweight DeepSeek Sparse Attention (DSA) with GQA4 at a predominantly 5:1 SWA–DSA layout. The entire network remains local or sparse, with no full-attention layers.
  • AI-optimized inference up to 2,000 tokens/s. NaiveRT, our inference system for Naive-N0.5-Flash, was built and optimized through AI-centered R&D. It combines mega-kernel fusion, Programmatic Dependent Launch (PDL), and speculative decoding, delivering 50 tokens/s per user in Standard mode and up to 2,000 tokens/s in Ultrafast mode. See the NaiveRT case study in the technical blog for the implementation and optimization process.
  • Open weights and API. Model weights and inference code are released under the MIT license. API access will also be provided, with pricing set at $0.10 / $0.40 / $0.01 per million tokens for input, output, and cache reads, respectively.

Model Architecture

Property Specification
Architecture Mixture-of-Experts (MoE)
Total parameters 309B
Active parameters 15.5B
Context length Native 1M tokens
Transformer layers 48
Attention-layer composition 39 SWA layers + 9 DSA layers
Attention mechanism Hybrid SWA–DSA
SWA window 128 tokens
DSA token selection Top 2,048 tokens for backbone attention
DSA KV groups 4 (GQA4)
Indexer query heads 16

Hybrid SWA–DSA Attention

Naive-N0.5-Flash builds on the open-weight MiMo-V2.5 base model, which has a simple architecture with strong foundational capabilities in world knowledge and deep research. Most layers use Sliding-Window Attention (SWA), whose per-token decoding cost does not grow with context length, while a small number of global-attention layers preserve long-range information. At million-token context lengths, however, these global-attention layers account for much of the decoding overhead.

Naive-N0.5-Flash replaces the global-attention layers with DeepSeek Sparse Attention (DSA). A lightweight indexer scores the full history, while the backbone computes attention only over a selected subset of tokens. Although the indexer still scans the full history and the full KV cache is retained, sparse attention substantially reduces attention computation and memory access. Adapting the model to this new attention structure was one objective of continued pretraining.

Hybrid SWA–DSA architecture showing the network stack and the DSA attention module, including the 16-head indexer and top-2,048 token selection.

Figure 1. The hybrid attention stack and DSA module.

The network consists of eight six-layer modules. A standard module contains five SWA layers followed by one DSA layer, with the first layer of the first module also replaced by DSA. SWA uses a 128-token window, while DSA selects the top 2,048 tokens for backbone attention. Both attention types incorporate sink bias.

Unlike the original MLA-based DSA implementation, Naive-N0.5-Flash replaces MLA with grouped-query attention (GQA) using four KV groups. For the architecture design process and indexer efficiency comparison, see model architecture in the technical blog.

Training Overview

Following the architectural changes, Naive-N0.5-Flash completed 3.25T tokens of multi-stage training with a native 1M-token context window: 50B tokens of Indexer Warmup, 3T tokens of Sparse Attention Training, and 200B tokens of Learning Rate Decay. This process adapted the model to its new sparse attention architecture while substantially improving its AI R&D and coding capabilities. See the technical blog for training details.

Evaluation Results

Coding benchmarks comparing Naive-N0.5-Flash with other models across seven software engineering and agentic tasks.

Figure 2. Coding and agentic task results. Naive-N0.5-Flash is highlighted in yellow.

AI R&D benchmarks covering PostTrainBench, MLE-bench-30, PaperBench, SOL-ExecBench, NanoChat AutoResearch, and NanoGPT SpeedRun.

Figure 3. AI research and systems optimization results. Metric directions are indicated in the figure.

Evaluation setup and metric notes

Evaluation setup. Unless otherwise noted, our evaluations of Naive-N0.5-Flash use Claude Code 2.1.207 with a 1M-token context window, temperature 1.0, and top-p 0.95. The harness exposes only basic file I/O and Bash tools.

Sources for reported benchmark scores are as follows:

Deployment

Naive-N0.5-Flash supports FP8 mixed-precision inference. For general use, we recommend setting the sampling parameters to temperature=1.0 and top_p=0.95.

Quick Start with Transformers

Naive-N0.5-Flash requires FP8-capable NVIDIA GPUs. The model weights occupy approximately 315 GB; allow additional GPU memory for inference.

pip install "transformers[torch,kernels]>=5.17.0"
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NaiveAI/Naive-N0.5-Flash-FP8"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype="auto",
    device_map="auto",
)

inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Hello!"}],
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

output = model.generate(**inputs, max_new_tokens=2048)
response = tokenizer.decode(
    output[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)
print(response)

License

Naive-N0.5-Flash is released under the MIT License.

Citation

If you find Naive-N0.5-Flash useful in your research or work, please cite:

@misc{naiveai2026naiven05flash,
  title  = {Naive-N0.5-Flash: Building Frontier AI with AI},
  author = {{NaiveAI Team}},
  year   = {2026},
  url    = {https://naive.ai/en/research/}
}

Acknowledgments

Naive-N0.5-Flash builds on the work of the open-source community and gives back to it. We thank the Xiaomi MiMo team for making their MiMo-V2.5 base model publicly available, the DeepSeek team for their work on DeepSeek Sparse Attention (DSA), and the SGLang team and community for their open-source inference infrastructure.

Contact

For questions, feedback, or collaboration, please contact us at contact@naive.ai or follow us on X at @naiveailab. You can also find our open-source projects and model releases on GitHub and Hugging Face.

Downloads last month
966
Safetensors
Model size
79B params
Tensor type
BF16
·
F16
·
I16
·
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for doth4580/Naive-N0.5-Flash-EXL3-4bpw

Quantized
(6)
this model