Arete-27B NVFP4

NousResearch/Arete-27B quantized with ModelOpt 0.48.0 and NVIDIA's recipe for Qwen3.8-27B (the layout of nvidia/Qwen3.8-27B-NVFP4). Made and measured on a DGX Spark (GB10).

Layers

Part Format
MLP, lm_head NVFP4 (Local-Hessian)
Full and linear attention FP8
Embeddings, vision BF16

Calibration: 512 samples from 7 Nemotron post-training v3 sets. Base revision 108f0afb.

Quality against BF16

Thinking off, temperature 0. Both measured by me with the same harness and vLLM build.

GPQA Diamond (198) IFEval (200)
BF16 81.8% 85.5%
This model 80.3% 83.0%

Speed on one DGX Spark

One user, vLLM 0.33.0 nightly. Size 20 GB (BF16 is 55 GB).

Context Decode
short 13.2 tok/s
32K 11.8 tok/s
64K 10.7 tok/s
128K 9.1 tok/s

128K prompt read: 82 s. Second turn on the same 128K: 1.1 s with prefix caching. Lowest free memory during the 128K run: 65 GiB.

Run

vllm serve atakans/Arete-27B-NVFP4 --max-model-len 262144 --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder --enable-prefix-caching

Needs Blackwell (NVFP4). No MTP weights in Arete, so no draft model.

Downloads last month
133
Safetensors
Model size
18B params
Tensor type
BF16
·
F8_E4M3
·
U8
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for atakans/Arete-27B-NVFP4

Base model

Qwen/Qwen3.8-27B
Quantized
(63)
this model