Lernex/Metis-1.6-Logos-NVFP4

Technical report  ·  Metis-1.6  ·  Lernex  ·  X

Logos in BF16  ·  FP8  ·  NVFP4

Metis-1.6 Logos NVFP4

Metis-1.6 Logos in NVFP4 for Dromos, Lernex's inference engine: 6.57 GB instead of 20.80 GB, so the 10.2B model runs entirely on an 8 GB GPU, and it keeps 99.4% of BF16's benchmark average.

Benchmarks

Across seven zero-shot tasks, NVFP4 keeps 99.4% of BF16's average score, and 99.1% with --fast-math, the faster maths explained after Run it.

Task	Metric	BF16	NVFP4	NVFP4 --fast-math
MMLU	acc	53.43	52.88	51.69
HellaSwag	acc_norm	68.46	68.11	67.59
ARC-Easy	acc_norm	70.62	70.79	69.99
ARC-Challenge	acc_norm	44.11	44.80	43.43
OpenBookQA	acc_norm	43.40	41.40	42.80
PIQA	acc_norm	76.06	76.71	75.95
WinoGrande	acc	61.96	61.01	62.90
Average		59.72	59.39	59.19

Each task is scored by log-likelihood over every item in lm-evaluation-harness 0.4.13. The NVFP4 columns decode this repository's weights exactly and run them in the reference implementation with Dromos's activations: BF16 by default, and for --fast-math each input rounded the way Dromos rounds it on an RTX 50 series GPU, to FP4 for the experts and to FP8 per token for the other projections. The BF16 model's card scores generated answers instead, so compare these columns with each other, not with that card.

Run it

curl -fsSL https://github.com/lernex/dromos/releases/latest/download/install.sh | sh   # Linux, macOS
dromos run logos:nvfp4

On Windows, install with irm https://github.com/lernex/dromos/releases/latest/download/install.ps1 | iex; pip install dromos works everywhere. logos:nvfp4 is Dromos's short name for this repository, which it downloads the first time, and dromos serve logos:nvfp4 serves the OpenAI API on http://127.0.0.1:8080/v1.

The model's thinking comes back in reasoning_content and tool calls are parsed into tool_calls. Thinking is on by default; send "chat_template_kwargs": {"enable_thinking": false} for direct answers, or run dromos run logos:nvfp4 --think off. Dromos applies the recommended sampling from generation_config.json by itself: temperature 0.8, top-p 0.95, top-k 20 and repetition penalty 1.05 when thinking, and temperature 0.3 with no penalty for direct answers.

It runs wherever Dromos does: NVIDIA GPUs through CUDA 12 and 13, Apple silicon through Metal, and CPUs. An 8 GB GPU holds it entirely with room for one sequence of 8,192 tokens, and a 12 GB GPU with room for 15. NVFP4 experts are not offloaded, so on a smaller GPU, or an 8 GB one that also drives a display, use Logos FP8, whose experts Dromos offloads to system memory.

vLLM, SGLang and Transformers do not read this format. Use Logos FP8 with vLLM or SGLang, and Logos with Transformers.

What --fast-math does

Dromos can multiply this model's weights in two ways.

• By default it expands the stored weights to 16 bits as it multiplies and does the maths in BF16, as the original model does. The weights are the only thing rounded.
• With --fast-math it multiplies on the GPU's 4-bit and 8-bit tensor cores instead, which means rounding what goes into each multiplication too. That is faster on the GPUs that have those instructions, and a little less accurate: it keeps 99.1% of BF16's benchmark average, against 99.4% by default.

On an RTX 5070 Ti, Dromos 0.1.0 reads a 4,096-token prompt at 9,257 tokens per second with --fast-math, against 7,301 by default: 27% faster.

It only changes the steps where the GPU handles many tokens at once: reading the prompt, and generating for more than 8 conversations together. A single conversation writes its answer one token at a time with the default maths either way, because that step's speed is set by how fast the GPU reads the weights, not by the multiplying. The answer can still come out slightly different, since it builds on the prompt as --fast-math read it.

Your GPU	What --fast-math changes
RTX 50 series, RTX PRO Blackwell	The 4-bit experts multiply in FP4 and the 8-bit projections in FP8, as measured in Benchmarks
RTX 40 series, H100	The 8-bit projections multiply in FP8; the 4-bit experts stay in BF16
RTX 30 series, A100, Apple silicon, CPUs	Nothing: without FP8 or FP4 tensor cores, Dromos keeps the default

Turn it on when reading the prompt is the slow part: long documents, a large codebase in context, or serving many people at once. For everyday chat, leave it off; the default stays closer to the original model.

dromos serve logos:nvfp4 --fast-math

Once the model is downloaded, dromos doctor logos:nvfp4 shows what it changes on your GPU.

Speed

Decode speed on an RTX 5070 Ti (16 GB) in tokens per second with the CUDA 13 build of Dromos 0.1.0, with 512 prompt tokens and 256 generated per stream, greedy decoding without speculation, counting the tokens generated once every stream is decoding:

	1 stream	32 streams	On disk
FP8, Metis-1.6-Logos-FP8	187.4	1,331	11.17 GB
NVFP4, this repository	199.7	1,425	6.57 GB

How close it stays to BF16

Mean top-20 KL divergence from the native model's FP32 logits, and how often the top token agrees, over the whole prompt, on the release's three reference prompts: a thinking chat, a chat with a tool call and a prose passage. None of them is in the calibration set. For comparison, Logos FP8 measures 0.031 to 0.033 on the tool prompt and 0.0079 to 0.0084 on prose in vLLM and SGLang. Lower is closer; 0 would be the original exactly.

Maths	Thinking chat	Chat with tools	Prose
BF16 (default)	0.010, 87.5%	0.026, 93.2%	0.012, 96.7%
--fast-math	0.023, 100.0%	0.062, 89.8%	0.023, 95.0%

What is in NVFP4

The routed experts, most of the model's parameters, are NVFP4: 4-bit E2M1 values with one FP8 (E4M3) scale per 16 inputs and an FP32 scale per expert. The attention and Mamba-2 input projections, the projection into the experts' latent space, the shared expert, the depth memory and the tied embedding are FP8 with one scale per output row. What this architecture is most sensitive to stays as trained: every Mamba-2 output projection and every projection out of the latent space stay BF16, and the routers, norms and Mamba-2's A, D and dt bias keep their native precision. Every 8-bit weight and all but 35 of the 7,680 expert weight matrices were rounded with GPTQ (Hessian error compensation in activation order); the other 35, experts routed too rarely in calibration to estimate their Hessian, use round-to-nearest. The calibration set is 352 windows of 2,048 tokens, and none of it is evaluation data: WikiText-103, Magicoder OSS-Instruct code, GSM8K and MATH training problems, Aya conversations in 24 languages, UltraChat conversations, long-form essays, and Logos FP8's own tool-call transcripts. Every document was checked against the 46 benchmark datasets on our evaluation machine and the three reference prompts above, and any that shared a 13-token run with one was left out.

quantization_config.json lists the recipe tensor by tensor, calibration_mix.json the calibration set, and quantization_receipt.json the checksums and per-module errors of the conversion.

Citation

If Metis helps your work, please cite the technical report, and the model if you used it directly:

@misc{vollmer2026thinktwice,
  title         = {Think Twice: The Metis-1.6 Technical Report},
  author        = {Vollmer, Giulianno},
  year          = {2026},
  eprint        = {2610.37903},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2610.37903}
}

@misc{vollmer2026logos,
  title     = {{Metis-1.6 Logos}},
  author    = {Vollmer, Giulianno},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/Lernex/Metis-1.6-Logos}
}

License

Apache 2.0, the same as the original model.