keptn/ornith-1.5-9b-fp8

Ornith 1.5 9B · FP8 · for radiance

ornith-ai/Ornith-1.5-9B in one 10.7 GiB file for the radiance inference engine on AMD RDNA4. A dense 9B reasoning model, tuned for coding and agent work. It runs on one card.

File	ornith-1.5-9b-fp8.rad, 10.67 GiB
Needs	radiance 1.15.1 or newer; 1 card with 32 GB (tested on Radeon AI PRO R9700), or 2 cards with --tp 2
Format	weights FP8 (E4M3), one bf16 scale per 128×128 block; activations FP8; lm_head FP8, 128×128 blocks
Speculation	the model's own MTP head, 3 draft tokens
Inputs	text · image
Context	262,144 tokens
Version	1.0 (2026-11-03)

Speed

Radeon AI PRO R9700 at 215 W, radiance 1.15.1, fp8 KV, 3 draft tokens, thinking off. BetterBench 0.6.1, full run, temperature 0.7. Tokens per second (median):

	1 card	2 cards
decode, one stream	142.5	237.9
prefill at 24k tokens	8,106	12,491
4 streams, total	397	657

Quality

Distance to the bf16 original on the 80 MT-Bench questions, both turns. The text is the GPT-4 answers from lmsys/mt_bench_human_judgments, with the model's chat template and thinking off. The reference is the bf16 original in HF transformers, with fp32 arithmetic and the full vocabulary. radiance scores this file with --kld-ref, on the answer tokens only (42,305 tokens).

	mean KL	99th percentile KL	top token the same	perplexity ratio
this file	0.00498	0.0514	97.8%	×1.001
keptn/ornith-1.5-9b-bf16	0.00086	0.00753	98.9%	×1.001

Quick start

With the compose files from the radiance repository (1.15.1 or newer).

Or directly (one card):

hf download keptn/ornith-1.5-9b-fp8 ornith-1.5-9b-fp8.rad --local-dir ~/models
docker run -d --init --name ornith --network host --group-add video --group-add render \
    --device /dev/kfd --device /dev/dri --security-opt seccomp=unconfined --ulimit memlock=-1:-1 \
    -v ~/models:/models:ro -e ROCR_VISIBLE_DEVICES=0 \
    stilldeadcode/radiance:1.15.1 --model /models/ornith-1.5-9b-fp8.rad --host 0.0.0.0 --port 8000 \
    --kv-cache-dtype fp8 --max-model-len 262144 --max-num-seqs 4 \
    --max-num-batched-tokens 2048 --num-speculative-tokens 3 \
    --temp 1.0 --top-p 0.95 --top-k 20 --presence-penalty 1.5

	change
two cards	ROCR_VISIBLE_DEVICES=0,1, add --tp 2 --p2p on
coding	--temp 0.6 --presence-penalty 0 (Ornith's setting for precise coding)
thinking	on by default; per request "chat_template_kwargs": {"enable_thinking": false} turns it off

Memory and context

KV pool at --max-model-len 262144, fp8 KV, radiance 1.15.1, from radiance's startup report. Tokens of context in total, shared by all sequences:

	2 sequences	4 sequences	8 sequences
1 card	524,304	1,048,624	1,238,000
2 cards	524,368	1,048,624	2,097,264

Images, audio and video

The same inputs go to radiance and to the bf16 original in HF transformers. Greedy, thinking off, 1 card, radiance 1.15.1 (release image). There is one measurement for each vision or audio encoder, made with ornith-1.5-9b-fp8.rad.

input	cases	same answer	mean KL	top token the same
images	20	10/20	0.102	244/264

Each image answer is at most 32 tokens. The HF reference scores the same answer tokens.

How it was made

1. Start from the bf16 release ornith-ai/Ornith-1.5-9B (revision 489cb979).
2. Round every linear layer of the trunk and of the MTP layer to FP8. Use one scale per 128×128 block. Round to nearest.
3. Round the lm_head to FP8 with the same block size.
4. Add a 2-bit copy of the lm_head for the draft tokens. This follows radiance's recipe for Qwen3.6-35B-A3B.

rad-convert Ornith-1.5-9B --tokenizer Ornith-1.5-9B/tokenizer.json \
    --recipe ornith-1.5-9b-fp8.recipe \
    --set quantization_config.quant_method=fp8 --set quantization_config.fmt=e4m3 \
    --set "quantization_config.weight_block_size=128 128" --set quantization_config.activation_scheme=dynamic \
    -o ornith-1.5-9b-fp8.rad

Calibration: no calibration data.

Model card with: radiance-model-cards-framework 0.3 (codeberg.org/keptn42/radiance-model-cards-framework).

Limits

• radiance runs on AMD RDNA4 cards.
• Image answers change more than text answers. Mean KL is 0.102 for images and 0.005 for text.
• With thinking on, hard tasks can need more than 32k output tokens.
• Speed on four cards is not measured.

Credits

• Model: the Ornith team (github.com/ornith-ai/Ornith-1), built on Qwen3.5 (Apache 2.0).
• radiance: StillDeadcode (codeberg.org/StillDeadcode/radiance).
• Test sets: MT-Bench, DocVQA, AI2D.

License

MIT, the license of the base model. See LICENSE (from github.com/ornith-ai/Ornith-1).