1beb/clef-flash-radiance

clef-flash for radiance

Cloudflare's clef-flash decision model as two radiance containers, each with the joint schema head inside, for radiance's clef_bf16 and clef_fp8 plugins. radiance serves POST /v1/systemone from them with no Python process beside it. The weights are Cloudflare's, under Apache-2.0. The containers are a repackaging.

File	Size	Weights	Made with
clef-flash-fp8-head.rad	12.2 GB	the trunk's large linears block fp8 (e4m3, 128x128, bf16 scales). Norms, embeddings, the delta net's small projections and the lm_head bf16	the recipe below
clef-flash-head.rad	19.1 GB	bf16 as shipped	no recipe

Both carry joint_head.safetensors and the head's geometry and learned scalars in the container header (decision.type=clef, decision.clef.*), under architecture id clef.

What serves them

radiance built from the clef-flash-arch-plugin branch: the clef_bf16 and clef_fp8 plugins, plugin ABI 32. Other builds of radiance have no plugin for architecture clef.

radiance --model clef-flash-fp8-head.rad --port 8000 --max-model-len 16384

curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "I was charged twice this month, please refund one of them.",
  "questions": {
    "refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"},
    "team": {"type": "choice", "instructions": "Which team should handle this?",
             "criteria": {"billing": "Charges, refunds, invoices", "technical": "App or site faults"}},
    "urgency": {"type": "score", "instructions": "How urgent is this?",
                "criteria": ["Can wait", "Today", "Blocking the customer now"]}}}'

The request and response are clef's own System One shape. /v1/decisions is the same endpoint. The answer to that request has this shape, every question decided jointly in one pass:

{
  "answers": {
    "refund":  {"type": "noul", "noul": 0.97},
    "team":    {"type": "choice", "choice": "billing", "confidence": 0.91,
                "probabilities": {"billing": 0.91, "technical": 0.09}},
    "urgency": {"type": "score", "score": 1.12, "confidence": 0.72,
                "probabilities": {"0": 0.08, "1": 0.72, "2": 0.20},
                "legend": {"0": "Can wait", "1": "Today", "2": "Blocking the customer now"}}
  },
  "usage": {"input_tokens": 231, "output_tokens": 0}
}

The numbers above only show the shape. noul is P(true). A choice names its most probable option with that option's probability as confidence, and lists every option in the order the request gave them. A score is the expected level over its levels (0, 1, ... as in legend), confidence the probability of the likeliest level. Each question's probabilities sum to 1, and every value is rounded to four places, as clef's own code rounds them. input_tokens is the whole prompt: the state, every question and its options.

How they were made

rad-convert clef-flash -o clef-flash-head.rad --arch clef \
    --add clef-flash/joint_head.safetensors --set decision.type=clef
rad-convert clef-flash -o clef-flash-fp8-head.rad --arch clef --recipe clef-flash-fp8block.recipe \
    --add clef-flash/joint_head.safetensors --set decision.type=clef \
    --set quantization_config.quant_method=fp8 --set quantization_config.fmt=e4m3 \
    --set "quantization_config.weight_block_size=128 128" \
    --set quantization_config.activation_scheme=dynamic

The recipe rounds blk.*.attn_qg, attn_k, attn_v, attn_output, ssm_inz, ssm_out, ffn_gate_up and ffn_down to nearest into fp8 e4m3 codes with a bf16 scale per 128x128 block, the trunk lines of radiance's q38-27b-fp8block-df2.recipe. The lm_head stays bf16: the head averages its rows.

Measured on one AMD gfx1201 card, 2026-10-28

100 survey respondents, one request each, one at a time. The guidance prompts carry a rules text in front of each respondent (about 13,100 tokens). The rules come from the prefix cache after the first respondent of each group. The short prompts are 550 to 800 tokens.

Container	Prompt	Prompt tokens/s	Median per decision	Same verdict as clef's head in PyTorch
fp8	guidance, 13,116 tokens	24,344	495 ms	98/100, largest probability difference 0.10
fp8	short, 676 tokens	9,722	71 ms	98/100, largest 0.10
bf16	guidance	14,255	838 ms	100/100, largest 0.01
bf16	short	5,532	125 ms	not compared at bf16

Tokens/s counts every prompt token a decision read, the ones the prefix cache supplied included. On the guidance prompts two thirds of them come from the cache. On the short prompts none do.

The PyTorch column is clef's own JointSchemaHead over the same radiance trunk's hidden states, at the same precision. The fp8 differences are the fp8 trunk's, as the PyTorch head over fp8 rows differs from bf16 by the same amount. That comparison ran on radiance's first, integrated build of the head. The clef plugins' answers equal that build's to four places on all four runs. bf16 was measured before the plugins skipped the lm_head on clef steps.