Victoria

70.0% on Terminal-Bench 2.1 (avg@3) from a model that uses 5.9B parameters per token, at 279.6 tok/s on one stream.

Victoria is a compressed, 4-bit derivative of Qwen/Qwen3.8-Flash-Next. 44% of its experts were removed (512 down to 288 per layer) and the model was then retrained at 4-bit to restore quality, so it runs on a single GPU while keeping most of the original model's coding and agent ability.

See also: Maple, a Canadian-first model built on Victoria.

Highlights

• Fast decode. 5.9B active parameters per token, the same as Qwen3.8-Flash-Next. With the included draft head on, the NVFP4 build decodes 279.6 tok/s on one stream on one NVIDIA B300, 2.08x its 134.7 tok/s with the head off.
• 70.0% on Terminal-Bench 2.1, avg@3. Three full runs of all 89 tasks at 8 hours per task scored 75.3%, 68.5% and 66.3%. The previous NVFP4 build scored 62.5% on the identical protocol.
• Size. 48.0 GiB of weights including the draft head, plus the model's 95.4 GiB n-gram lookup table. The GGUF build holds 49.17 GiB of weights on llama.cpp.
• Fewer wasted tokens. The new build used 35% fewer output tokens than the previous NVFP4 build (69.4M against 107.3M across three runs).
• Two runtimes. NVFP4 for vLLM on NVIDIA Blackwell GPUs, and GGUF Q4_K_M for llama.cpp.

Builds

The two builds are different checkpoints: the NVFP4 build is the latest checkpoint, and the GGUF build is the checkpoint from the first release. Each build's scores were measured on that build only.

Build	Runtime	Download	Resident weights	Terminal-Bench 2.1, 8 h per task	HumanEval
nvfp4/	vLLM, NVIDIA B300 or B200	154.0 GB (143.4 GiB)	48.0 GiB of weights plus the 95.4 GiB lookup table; vLLM holds both on the GPU	70.0% avg@3 (75.3 / 68.5 / 66.3)	97.0% (159/164), one sample per problem
gguf/ Q4_K_M	llama.cpp	107.20 GB (smaller download, same model) or 155.20 GB (full-precision lookup table)	52.79 GB (49.17 GiB)	75.28% (67/89), one run	93.2% avg@5

Quick start

Download:

hf download rmonsurate/Victoria --include "nvfp4/*" --local-dir .

# full GGUF set, 155.20 GB
hf download rmonsurate/Victoria --include "gguf/victoria-s410-bitexact-0*" --local-dir .

# smaller download, 107.20 GB: the model's lookup table is stored at 8-bit, GPU memory use is the same
hf download rmonsurate/Victoria --include "gguf/victoria-s410-bitexact-tbl8-*" --local-dir .

nvfp4/SHA256SUMS lists the sha256 of every file in the NVFP4 build.

NVFP4 on vLLM, one GPU, draft head on. This is the command the decode speeds below were measured with:

vllm serve /path/to/nvfp4 \
  --served-model-name victoria \
  --tensor-parallel-size 1 \
  --max-model-len 20480 \
  --max-num-seqs 16 \
  --max-num-batched-tokens 16384 \
  --gpu-memory-utilization 0.80 \
  --enable-prefix-caching \
  --compilation-config '{"inductor_compile_config":{"triton.autotune_at_compile_time":false}}' \
  --kernel-config '{"enable_flashinfer_autotune":false}' \
  --trust-remote-code \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

• The --compilation-config line is required: compile-time Triton autotuning on this model either hangs or blows out startup time.
• For long agent sessions, the Terminal-Bench runs used --max-model-len 262144 --max-num-seqs 64 with everything else the same, minus --speculative-config.
• We ran vLLM from the Qwen3.8-Flash-Next vLLM image with two fixes backported for a prefix-caching crash on this architecture (vllm#53798 and vllm#54076, open upstream when we built it). On a vLLM without those fixes, leave out --enable-prefix-caching.

Run the GGUF files with speculative decoding off, since public llama.cpp cannot use this model's draft head. We ran llama.cpp build b10667, 8 slots at 16384 context each, across two GPUs. The block below is UNTESTED as written and you will need to work out the offload flags for your own machine:

llama-server \
  -m victoria-s410-bitexact-00001-of-00003.gguf \
  -c 16384 \
  --parallel 8 \
  --host 0.0.0.0 --port 8080

Evaluation

NVFP4 build

Model	Terminal-Bench 2.1, avg@3	Run 1	Run 2	Run 3
Victoria NVFP4 (this release)	70.04%	67/89	61/89	59/89
Victoria NVFP4 (previous build)	62.54%	57/89	54/89	56/89

Counting passes out of three runs per task, this release does better than the previous build on 18 tasks and worse on 6, and ties on the rest (two-sided exact sign test, p = 0.023). Harbor 0.22.0 with the Terminus 2 agent, 8 hours per task, 262144 context, all 89 tasks in every denominator; a trial with no verdict counts as a fail.

HumanEval on the NVFP4 weights is 97.0% (159/164): one sample per problem, thinking on, temperature 0.6, top_p 0.95, top_k 20. That is a single run, not an average over five runs like the GGUF figure, so the two numbers are not directly comparable.

Decode speed and the draft head

Setup	Output tok/s, mean of 3 runs (each run)	Draft acceptance	Tokens per forward pass of the main model
Draft head off	134.7 (133.7, 133.6, 136.9)	n/a	1
Draft head as pruned, never retrained	269.3 (270.4, 269.8, 267.6)	64.1%	2.92
Trained draft head, shipped in nvfp4/	279.6 (284.0, 278.5, 276.2)	67.6%	3.03

One NVIDIA B300, one stream, the vLLM command above, 50 fixed prompts (20 coding, 30 agentic), temperature 0.6, top_p 0.95, thinking off, up to 3 draft tokens per step, a fresh server for each run. Acceptance is accepted draft tokens over proposed draft tokens. The draft head only proposes tokens and the main model verifies every one, so it changes speed, not answers.

GGUF build

Model	Terminal-Bench 2.1	Kept vs original
Victoria GGUF Q4_K_M	75.28% (67/89)	84.8%
Qwen3.8-Flash-Next, the model Victoria came from	88.76% (79/89)	baseline
Qwen3.8-27B dense, avg@3 with Claude Code	73.0%	not applicable
Claude Opus 4.6 (Max)	78.2%	as listed on Qwen's Qwen3.8-27B model card

The last two rows come from Qwen's Qwen3.8-27B model card and were not run on our setup, so treat them as a rough guide.

Setup: Harbor 0.22.0 with the Terminus 2 agent, 8 hours per task, avg@1, all 89 tasks in the denominator, 262144 context. One avg@1 run over 89 tasks carries a standard error of roughly 5 points. HumanEval on the same weights is 93.2% avg@5, sd 1.3, at n=164. We evaluated text only.

Training curves

Drawn from the training logs of the NVFP4 build (steps 510 to 810) and of the draft head.

Training loss.

Held-out divergence from the full model, agentic and coding slices. Lower is closer to the original model.

How much the model leans toward ending its turn while still thinking. Lower means fewer empty agent turns.

Draft head training: divergence from the main model and draft acceptance rate.

License

Victoria is a derivative work of Qwen3.8-Flash-Next by Farpoint Technologies and carries the same license, the Qwen Community License 1.0, whose full text ships in this repo as LICENSE. Two conditions travel with it: the copyright and permission notice must be included in all copies, and a licensee running a Model as a Service or AI Work Assistant business must obtain a separate license from Qwen before any commercial use. Read the license text and talk to Qwen if that describes what you are building. Attribution goes to Qwen for Qwen3.8-Flash-Next and to Farpoint for Victoria.

Acknowledgements

This work was made possible by the support of Dell, who provided lab access to the 8x NVIDIA B300 system it was built and evaluated on, and of Mario at Buzz (NASDAQ: HIVE). Thanks to the Qwen team for Qwen3.8-Flash-Next, and to the maintainers of llama.cpp, vLLM, harbor and Terminal-Bench.