code-daemon-summary-v1

Compact bilingual (English / Russian) code-documentation generator — a distilled Qwen3.5-4B in GGUF that writes:

• one-sentence entity descriptions for a source file (functions, classes, fields) as a clean markdown bullet list;
• module overviews — short prose plus ASCII architecture / data-flow diagrams;
• hierarchical codebase summaries — subsystem, product-level and whole-project digests built from smaller summaries.

It is the long-output worker of the UltraCode code-intelligence daemon: a purpose-built component, not a general assistant. Outside this task distribution its behaviour is undefined. The output language follows the request — both languages were distilled first-class.

⚠ The weights under this id changed on 2026-09-16

Until then this repository served a Qwen3-4B distilled from Qwen2.5-7B-Instruct (Q3_K_M, Q4_K_M and Q5_K_M). Everything on this page was measured on the Qwen3.5-4B student; the previous card is in this repository's git history. Do not mix figures across the two.

Files

file	size	use
code-daemon-summary-v1-Q4_K_M.gguf	2.58 GB	recommended
code-daemon-summary-v1-Q3_K_M.gguf	2.16 GB	6 GB-VRAM cards; quantized with a domain importance matrix

Both need a llama.cpp build with the qwen35 architecture (b10809 / v0.4.0 or newer).

Architecture — read this before sizing a deployment

• Base: Qwen/Qwen3.5-4B, ChatML, 248 320-token vocabulary.
• Hybrid attention: 32 blocks, every fourth a full-attention block (8), the rest gated DeltaNet (24). The linear blocks carry a recurrent state per sequence instead of a growing KV cache — see Memory.
• No multi-token-prediction block. The base checkpoint has one; this export does not, so there is no built-in speculative head.

Quick start (llama.cpp)

The bundled chat template runs the model in non-thinking mode unless enable_thinking is set — use it as a normal ChatML model. If you build raw prompts yourself, end them with the assistant tag followed by an empty think block (<|im_start|>assistant\n<think>\n\n</think>\n\n); that is the shape the model was trained on. Greedy decoding (temperature 0) is recommended; stop on <|im_end|>.

llama-cli -m code-daemon-summary-v1-Q4_K_M.gguf -c 8192 --temp 0 \
  -p '<|im_start|>system
You write one-sentence descriptions for code entities of a single file. Output ONLY a markdown bullet list, ONE bullet per entity: - **<EntityName>**: <one-sentence description>.<|im_end|>
<|im_start|>user
Entities: parseArray, encodeValue. File excerpt: <...><|im_end|>
<|im_start|>assistant
<think>

</think>

'

Output formats:

• Entity documentation — - **Name**: one-sentence description., one bullet per entity.
• Module overview — ## Overview prose plus ## Architecture / ## Flow ASCII diagrams.
• Hierarchical summaries — paragraph-length subsystem / product / strategy digests.

Evaluation

200 held-out documentation prompts, never trained on. Reference: the teacher's answer to the same prompt. Metrics: ROUGE-L and token F1 against that reference, greedy decoding, 512 new tokens, every model scored as the GGUF file through llama.cpp's server, with the prompt sent as the same token ids.

model (GGUF)	size	ROUGE-L	token F1	length ratio
this model, Q4_K_M	2.58 GB	0.548	0.628	0.93
this model, Q3_K_M + importance matrix	2.16 GB	0.543	0.623	0.88
the same Q3_K_M without the importance matrix	2.16 GB	0.530	0.612	0.88
stock Qwen3.5-4B, same conversion, Q4_K_M	2.58 GB	0.464	0.555	0.93
the Qwen3-4B this id served before	2.38 GB	0.414	0.479	0.69

Paired over the same 200 prompts (4 000 bootstrap resamples):

comparison	ROUGE-L Δ	95 % CI	wins / losses
stock Qwen3.5-4B → this model	+0.084	[+0.066, +0.101]	156 / 42
this model: Q4_K_M → Q3_K_M + importance matrix	−0.004	[−0.020, +0.011]	92 / 102
Q3_K_M: plain → with importance matrix	+0.013	[+0.002, +0.026]	110 / 83

• The Q3 tier keeps the gain. It is indistinguishable from the Q4_K_M here, and the importance matrix earns its place at that width.
• The previous row is not a verdict on the previous model. The references come from the new teacher; the Qwen3-4B was distilled from a smaller one and writes shorter answers in its style (length ratio 0.69), which this ruler penalises — even the untouched Qwen3.5-4B scores above it.
• The gain shrinks with prompt length and stays positive: +0.117 ROUGE-L for prompts under 512 tokens (57 prompts), +0.104 at 512–1 024 (50), +0.056 at 1 024–2 048 (72), +0.041 above 2 048 (21).

Inside a real index

All three models in the same llama.cpp slot of the UltraCode daemon, each writing a first documentation pass over the same 18 directories of a Go repository, everything else unchanged:

	this model	stock Qwen3.5-4B	the previous Qwen3-4B
sections missing after the output guards	2	0	14
duplicate entity descriptions	0.4 %	0.3 %	4.8 %
bullets that only restate the entity name	1.7 %	2.3 %	3.5 %
description words, median	14	14	8
diagram lines repeating an earlier line	26 %	42 %	39 %
tokens generated	168 427	161 166	124 408
decode, 1 slot / 4 slots	86 / 154 tok/s	85 / 153 tok/s	—

Against its base the student is tidier, not longer: its diagrams loop less and its bullets restate names less at the same description length — but it dropped two sections the stock model kept. One repository, one run per model: a direction, not a verdict. (The previous model's speed is omitted: another workload shared the GPU during its run.)

Speed

Laptop RTX 5060 (8 GB), llama.cpp CUDA, one server slot, n_ctx=4096, median decode over answers of 16+ tokens: 88.6 tok/s for the Q4_K_M, 77.0 tok/s for the Q3_K_M. Inside the daemon, sharing the GPU with its embedding and classification workers: 86 tok/s single-stream and 154 tok/s across four concurrent slots.

Memory

Measured through llama.cpp's server with one logits row per sequence:

file	sequences	dedicated VRAM
Q4_K_M	28	4 710 MiB
Q3_K_M	28	4 284 MiB
Q3_K_M	12	3 408 MiB

• The recurrent state costs 50.25 MiB per sequence (1 407 MiB at 28) whatever the context length. It scales with the number of parallel sequences, not with n_ctx — cut sequences, not context, to fit a small card.
• If you embed llama.cpp yourself, set n_outputs_max. Its default reserves logits for a whole n_batch: with a 248 320-token vocabulary that is ~1.9 GB at n_batch=2048, allocated and never used by a generator that reads one logits row per sequence. llama-server sets it to its slot count; a library caller gets the default. The same model measured 6 341 MiB that way.

How it was made

• Base: Qwen/Qwen3.5-4B (Apache-2.0).
• Teacher: Qwen3.8-27B.
• Method: sequence-level knowledge distillation (SeqKD) on the bilingual documentation tasks above; the adapter was merged into the base in full precision and converted to GGUF.

License & attribution

Apache-2.0, matching the Qwen/Qwen3.5-4B base. Not legal advice — check the base and teacher model cards before redistributing. Base and teacher © the Qwen team; please also honour their cards.