zils-lite-body-13: a hold-anchored fine-tune change carried onto the current Zils LITE soup

A decision body for the Zils d3-lite soup pool (Bittensor subnet 127), Qwen/Qwen3.5-0.8B family.

It is the current d3-lite soup plus one change of our own:

• Start. The d3-lite soup of round d3-lite-2026-11-10 (soup hash 4d23ec5a54c2ab6d0c16fc493819ef113c789ab8edd1ef8756d41b645d2762a3), rebuilt by us with the official Zils soup code as the uniform average of its two members, Charlie320/zils-lite-v9a @93b064048a7aeb7cf0d64236136e12c903e67f40 (Apache-2.0; Zils body digest 44edf3875e4ac0bb2d4aa0ae04d00e81d410bbdbfd9588705f23fe1fac3032ff) and qhoenix/zils-lite-decision-v9 @083d5b9aa3d2db48155cd6e7f92ac6f9aa057c65 (Apache-2.0; Zils body digest 8b8e3db3c8a62cbde4b5fb034bffb2c871faa1db450f6ddb0103ad120a44cfe5). The rebuilt soup's hash equals the published one. Zils's mining guide allows this ("You may fine-tune the current soup").
• Our change. A LoRA fine-tune of decision-v9 on our own data (below). The weight difference "fine-tuned minus decision-v9" was added once (factor 1.0) to the soup.

Credit for the starting weights belongs to the authors of the two soup members and the works they were built on (see NOTICE). Nothing else from any other miner is in this body; it is not a re-upload of any public model.

Method

• Training (run on decision-v9, before the soup changed). LoRA rank 32 / alpha 64 / no dropout on the attention, Gated-DeltaNet and MLP (gate / up / down) projections of all language layers, merged into the weights after training. AdamW, learning rate 5e-5, weight decay 1.0, epsilon 1e-4, gradient-norm clip 1.0; cosine decay to 0 over the full run, multiplied by a 50-step warm-up factor; 2 epochs (438 steps), max 2,048 tokens, no label smoothing, no EMA, seed 5002. Each batch of 8 holds 4 labelled rows and 4 hold rows; the loss is the mean over the 8 rows.
  • Labelled rows: cross-entropy on the row's gold label, plus a Brier term on choice and yes/no rows and a normalised ranked-probability term on score rows (each weight 1), plus a small KL term (weight 0.1) to the starting body's own answer on the same row.
  • Hold rows: one from each of the other four categories (reasoning, retrieval, tools, arts) in every step. Their target is the starting body's own answer distribution on that prompt, computed live with the adapter off, with weight 30. No labels are used for them. This penalizes changes to the body's existing answers while the labelled rows are learned.
• This body. The final save of that run, minus decision-v9, added to the rebuilt soup by our weight-difference tool (see the checks for the exact inputs). No weight averaging of our own runs, no other body merged.

Training data (876 rows, file sha256 73d39a818cc31b33f9a4c8be96dc197ffd598685ec4ef6848374f5180037c77b)

A Language-led set in the exam's 15 "known" families (5 categories x choice / yes-no / score; the Tools families are our dialogue-based versions).

• Labels. Every label is one of: the dataset's own label (human annotators, or the dataset's own answer spans or annotations), derived by code from the dataset record, or computed by our own generator. The one exception is 10 rows written by a local model (see the last table row). No other miner's outputs were used as gold labels.
• Builder. our feed builder (sha256 f0711c68), from a freeze of 3,505 rows.
• Difficulty rule. A row was eligible only if the body the run started from (decision-v9) puts probability 0.30-0.80 on its correct answer. 3,505 of the 6,091 read rows were in that band.
• Unit rule. Only units that passed our label audit, or are proven by code, were used (passed-unit list sha256 3e29ebe2). Human-labelled rows from units that failed the audit enter only by exact, individually checked id lists (186 rows: MultiNLI 86, PIQA 55, Social IQa 27, SNLI 18).

category	choice	yes/no	score	total
reasoning	27	55	35	117
language	81	38	133	252
retrieval	26	56	37	119
tools	114	110	94	318
arts	10	22	38	70

source	rows	label	original licence	original release
Schema-Guided Dialogue (DSTC8, TRAIN split): post-call yes/no 58, required-slot score 68 + 26	152	derived by code from the human annotations of the real dialogue (the annotated next system call; the human-annotated dialogue state)	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
Schema-Guided Dialogue (DSTC8, TRAIN split), program-built: function choice 114, function-list yes/no 30 + 22	166	code: the dialogue is real and unchanged; sibling functions / the function list were edited by code; the label is the annotated real call, schema-checked	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
MultiNLI (GLUE, TRAIN split): choice 63, yes/no 23	86	human gold label (3-way; in yes/no rows entailment = true, neutral / contradiction = false)	OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0	https://cims.nyu.edu/~sbowman/multinli/
PIQA (TRAIN split)	55	human gold label: the shown method is the correct or the wrong solution of the goal	AFL-3.0	https://yonatanbisk.com/piqa/
SQuAD 2.0 (TRAIN split): answerable-part count 28, sentence level 19	47	dataset gold: answerability of each part / derived by code from the human answer spans	CC BY-SA 4.0	https://rajpurkar.github.io/SQuAD-explorer/
QNLI (GLUE, TRAIN split; text from SQuAD v1.1)	37	human gold label (the sentence answers the question)	CC BY-SA 4.0 (SQuAD)	https://rajpurkar.github.io/SQuAD-explorer/
Social IQa (TRAIN split)	27	human gold answer	CC BY 4.0	https://allenai.org/data/socialiqa
20 Newsgroups (SetFit mirror, TRAIN split)	26	the newsgroup the post was sent to	no open licence is named by the original release; used under the original authors' terms	http://qwone.com/~jason/20Newsgroups/
iSarcasmEval task A (TRAIN split)	22	the tweet author's own sarcasm label	MIT	https://github.com/iabufarha/iSarcasmEval
SNLI (TRAIN split): 11 with all 5 annotators agreeing, 7 gold	18	human gold label	CC BY-SA 4.0	https://nlp.stanford.edu/projects/snli/
RTE (GLUE, TRAIN split)	15	human gold label (entailment / not entailment)	no open licence is named by the original release; used under the original authors' terms (PASCAL RTE challenges)	https://gluebenchmark.com/
our code generator: language score items (facet rule)	133	computed by the generator from its item spec (facets built to agree / differ); in 13 rows a local model (Mistral-Nemo-Instruct-2407) only reordered each caption and added function words	our own code (Mistral-Nemo: Apache-2.0)	-
our code generator: arts score items (rulebook)	38	computed by the generator (net score of tracked word effects), re-counted; in 5 rows Mistral-Nemo-Instruct-2407 reworded the post and the score was re-counted on the new text	our own code (Mistral-Nemo: Apache-2.0)	-
our code generator: reasoning score items (ASDiv-style word problems)	35	computed from the item spec and re-counted independently from the rendered text	our own code	-
our code generator: retrieval score items (invented passages)	9	facts placed in an invented passage; the label is re-derived from the rendered text	our own code	-
written by a local model (Phi-4 7, Mistral-Nemo-Instruct-2407 3; the public Zils item-authoring recipe, a public TRAIN row as style example only)	10	the writer's own answer (answer steered for label balance); no human label; kept only after our blind check and sealed audit	Phi-4: MIT; Mistral-Nemo: Apache-2.0	-

Program-built rows.

kind	rows	share of 876
Our code generators (invented content)	215	24.5%
SGD rows whose options or function lists were built by code from real dialogues	166	18.9%
All program-built	381	43.5%
Written by a local model	10	1.1%

Licence note. 41 of the 876 rows come from 2 datasets whose original release names no open licence: 20 Newsgroups (SetFit mirror, TRAIN split), 26 rows; RTE (GLUE, TRAIN split), 15 rows. We used their TRAIN splits for training only, under the original authors' terms. This repository contains model weights only, no dataset text. Check those datasets' terms before you reuse this body for anything they may restrict.

Hold set

The hold rows are prompts only. Their targets are the answers of the body the run started from (decision-v9); no labels were used. Hold file sha256 c7f3d0c07f89e59f0c04f80151d125bbc647cf40e2046d9d7f6f0f3b78b11d96, 1,406 prompts (reasoning 380, language 318, retrieval 211, tools 382, arts 115; 653 of them program-built). The run drew its hold rows from the four categories other than language. No hold row is in the labelled set.

• Reasoning: PIQA 109, Social IQa 42, our reasoning score generator 229.
• Language: MultiNLI 67, RTE 43, SNLI 26, our language score generator 182.
• Retrieval: 20 Newsgroups 73, QNLI 59, SQuAD 2.0 51, our retrieval score generator 28.
• Tools: Schema-Guided Dialogue 382 (145 of them program-built).
• Arts: our arts score generator 69, SST-5 (TRAIN split) 21, iSarcasmEval 13, local-model-written 12.

Filters on every row. No exam split files (the validation / test splits the Zils exam-authoring code reads were not used as training inputs), no public Zils seed items, no Glaive data, no BIG-bench / MMLU; our exam-split guard and test-overlap rule removed every hit; no house or relay labels. No external miner's predictions were used as gold labels; the hold (KL) targets are the starting decision-v9 body's own answer distributions, as described above.

Honest limits of this data. 43.5% of the labelled rows are program-built and 10 rows were written by a local model. The difficulty band was measured on decision-v9, not on the soup this body starts from.

Checks we ran before commit (not an official score)

• Change from Qwen/Qwen3.5-0.8B (Zils relative change): 0.035964 (cap 0.15). Zils body digest 22eaa9c4bd2bf95fea7822c456f1582d416326a00425cb559cf7db5fa5a6aaea; model.safetensors sha256 05668582ddd7aea53172f644584101f0d5aa89687737546c5e9d55218fdd620c.
• Build inputs: soup rebuilt from the two members above (soup hash 4d23ec5a, equal to the published one); our fine-tuned body (Zils body digest c71002ddcf3180951a82da1400858a35e3483cf720a4ef872205d9fcc814d0c6) minus decision-v9, factor 1.0.
• Copy screen with the official soup_copies code (our seeds; the validator's seed is private): cosine to the current soup at most 0.9966 across three public sketch seeds (line 0.999); own share beyond the house bodies and the current soup 0.007, i.e. below the 0.05 house-mix line; the joint share of this body's own residual explained by the 74 earlier bodies in our screened field was at most 0.0041 across three public sketch seeds (line 0.30).
• Our own exam-style reads against the soup (same code, same items; not the Zils exam): +1.06 points on our known-family test (10 splits: +0.69 to +1.55), and +0.35 on its public-seed-style part alone, which is not distinguishable from zero (bootstrap -1.4 to +1.7). The same recipe at two other seeds, carried onto the soup the same way, reads +0.18 and +0.36 on that test, so this seed is the best observed local result among these three runs (mean about +0.5). We chose that test view after looking at several, on one finished round, so it is a local reading and not a forecast. We do not claim this body beats the soup on the real exam.
• No family's accuracy fell by more than 5 percentage points across our five tests (largest drop: reasoning score).

Licence

Apache-2.0 (LICENSE). See NOTICE for the upstream works and their notices. Nothing here is legal advice; check the original datasets' terms for your use.