zils-mini-body-25: Language- and Arts-weighted exam-type LoRA fine-tune of a uniform mix of four d3-mini house ingredients

A decision body for the Zils d3-mini soup pool (Bittensor subnet 127), in the d3-mini chassis layout (Qwen3.5-4B language and vision tensors, bf16). Its start is a uniform weight average, built with the official zils.soup.build, of four of the pool's five public house ingredients: vllm-sr/d3-mini @df99bf4c; perplexity-ai/pplx-decider-flash-v1.2 @9ff7cf98; torchcast-ai/torchcast-decision-4b @492bfb19; Maincode/Matilda-Decision-v1-Mini @39479a8a. Start body digest be40457ed0725abfe6a66d0e3cdb29bff6c206a71440d54382bbdca89c01a2fc (soup hash 20c02cae). Credit for the starting weights belongs to the authors of those models and their upstream works (see NOTICE). No other miner's body was merged; it is not a re-upload of any public model.

Method

• Training: LoRA rank 32 / alpha 64 / dropout 0.05 on the attention, Gated-DeltaNet AND MLP (gate / up / down) projections of all language layers, learning rate 5e-5, AdamW with weight decay 1.0 on the LoRA weights only, batch 8 with no accumulation, warm-up 50 steps then cosine, 2 epochs, max 2,048 tokens, seed 4359; loss per row = cross-entropy on the row's label + 1.0 x Brier + 0.1 x KL(start || this model) on the same row. The KL term holds the model near its own start and is computed live from the start weights with the adapter off; it uses no teacher, no extra pin rows and no other miner's outputs. Merged into bf16. No weight interpolation. This body is the final save (the only one read and shipped, fixed before training).
• Target: Data weighted toward Language and Arts (56.4% of the rows together), using only rows that passed our checks. The start is the fixed house mix above, not a soup.

Training data (985 rows, file sha256 41c69ab2496cf0576b362288a27f854df6f3032a1b1a8fb243a59d50cdb015e8)

A Language- and Arts-weighted exam-type set in the exam's 15 known families: 13.1% of the rows are Reasoning, 44.0% Language, 15.2% Retrieval, 15.2% Tools and 12.5% Arts. Category shares do NOT follow the exam weights. Every label is the dataset's own label (human annotators, or the dataset's own prover / generator), derived by code from the dataset record, or computed by our own generator, except the 22 rows written by local open models, whose answer is the writer model's own (see the table). No other miner's outputs were used. Selected with ft_swap_lscore.py v0 (274d6920) from a 2,990-row set frozen for this purpose (sha256 d0920e13a5e098dd65c5eba7af93c020ec710f479852a1ff1b17169fb355e5eb). This set has fewer than 1,600 rows, so it trains at batch 8 with no accumulation (about 246 optimizer steps).

CC BY-SA 4.0 note. 150 of the rows are built from the Schema-Guided Dialogue dataset (Google Research, CC BY-SA 4.0, https://github.com/google-research-datasets/dstc8-schema-guided-dialogue) and 50 from SNLI (CC BY-SA 4.0). They were used for training only; this repository holds model weights only, no dataset text. Check those datasets' terms before you reuse this body.

Difficulty rule: a row enters only if this start puts probability 0.30-0.80 on its correct answer, measured on the start body itself before training (all 985 rows checked: lowest 0.301, highest 0.800). Rows were taken only from sources that passed our checks, and nothing was padded to reach a row count. No family exceeds 20% of the rows (largest 17.4%); our own item generators supply 25.0% of the rows.

category	choice	yes/no	score	total
reasoning	29	50	50	129
language	163	99	171	433
retrieval	50	50	50	150
tools	50	50	50	150
arts	22	35	66	123

source	rows	label	original licence	original release
SST-5 (Stanford Sentiment Treebank, TRAIN split)	66	human 5-class sentiment label	no open licence is named by the original release; used under the original authors' terms	https://nlp.stanford.edu/sentiment/
MultiNLI (GLUE, TRAIN split; second selection)	63	human gold label (3-way); only rows an AI reviewer read and found correct	OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0	https://cims.nyu.edu/~sbowman/multinli/
RTE (GLUE, TRAIN split)	58	human gold label (entailment / not entailment)	no open licence is named by the original release; used under the original authors' terms (PASCAL RTE challenges)	https://gluebenchmark.com/
MultiNLI (GLUE, TRAIN split)	50	human gold label (3-way); only rows an AI reviewer read and found correct	OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0	https://cims.nyu.edu/~sbowman/multinli/
20 Newsgroups (TRAIN split)	50	the newsgroup the post was sent to	no open licence is named by the original release; used under the original authors' terms	http://qwone.com/~jason/20Newsgroups/
Schema-Guided Dialogue (DSTC8, TRAIN split; real dialogues, sibling function options made by code)	50	code: the arguments are the dialogue's annotated call; a schema check on every option passes only for the gold sibling	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
MultiNLI (GLUE, TRAIN split; yes/no form)	41	human gold label (entailment = yes, neutral / contradiction = no); only rows an AI reviewer read and found correct	OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0	https://cims.nyu.edu/~sbowman/multinli/
Schema-Guided Dialogue (DSTC8, TRAIN split; required arguments filled)	38	code: required slots of the active intent that are in the human-annotated dialogue state and whose value occurs verbatim in the conversation so far	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
PIQA (TRAIN split)	37	human gold label (the shown method is the correct or the wrong solution of the goal); rows an AI reviewer marked wrong or unclear were removed	AFL-3.0	https://yonatanbisk.com/piqa/
iSarcasmEval task A (TRAIN split)	35	the tweet author's own sarcasm label	MIT	https://github.com/iabufarha/iSarcasmEval
SNLI (TRAIN split; rows where all 5 annotators agree)	30	human label given by all 5 annotators; only rows an AI reviewer read and found correct	CC BY-SA 4.0	https://nlp.stanford.edu/projects/snli/
Social IQa (TRAIN split)	29	human gold answer; rows an AI reviewer marked wrong or unclear were removed	CC BY 4.0	https://allenai.org/data/socialiqa
Schema-Guided Dialogue (DSTC8, TRAIN split; does the next turn make a call)	26	code: whether the annotated next system turn of the real dialogue makes a service call	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
SQuAD 2.0 (TRAIN split, passage level)	25	dataset gold: the number of answerable question parts	CC BY-SA 4.0	https://rajpurkar.github.io/SQuAD-explorer/
QNLI (GLUE, TRAIN split; text from SQuAD v1.1)	24	human gold label (the sentence answers the question)	CC BY-SA 4.0 (SQuAD)	https://rajpurkar.github.io/SQuAD-explorer/
SNLI (TRAIN split)	20	human gold label; only rows an AI reviewer read and found correct	CC BY-SA 4.0	https://nlp.stanford.edu/projects/snli/
Items written on our own GPU by the local open model microsoft/phi-4 (MIT); a public TRAIN row served as a style example only	16	the writer model's own answer; kept only after a blind check by local open models and a sealed read by an AI reviewer	text written by us; no dataset text reused	https://huggingface.co/microsoft/phi-4
QNLI (GLUE, TRAIN split; text from SQuAD v1.1; second selection)	14	human gold label (the sentence answers the question)	CC BY-SA 4.0 (SQuAD)	https://rajpurkar.github.io/SQuAD-explorer/
Schema-Guided Dialogue (DSTC8, TRAIN split; real dialogues, function list edited by code)	13	code: true = the annotated next call serves the request and the function list is the schema's; false = one edit of the list so no listed function can honour it	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
PIQA (TRAIN split; second selection)	13	human gold label (the shown method is the correct or the wrong solution of the goal); only rows an AI reviewer read and found correct	AFL-3.0	https://yonatanbisk.com/piqa/
Schema-Guided Dialogue (DSTC8, TRAIN split; required arguments filled, corrected or referred-back values)	12	code: as the row above, only turns where a value was corrected, carried from assistant text or a function response, or referred back to	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
SQuAD 2.0 (TRAIN split, sentence level)	12	derived by code from the human answer spans (the sentence holds the answer or not)	CC BY-SA 4.0	https://rajpurkar.github.io/SQuAD-explorer/
Schema-Guided Dialogue (DSTC8, TRAIN split; real dialogues, requested option removed by code)	11	code: true = the annotated next call honours the slot the user just stated; false = that option was taken out of the listed functions	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
Items written on our own GPU by the local open model mistralai/Mistral-Nemo-Instruct-2407 (Apache-2.0); a public TRAIN row served as a style example only	6	the writer model's own answer; kept only after a blind check by local open models and a sealed read by an AI reviewer	text written by us; no dataset text reused	https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407
our code generator: language score items	147	computed by the generator from its item spec and re-counted independently	our own code	-
our code generator: reasoning score items (ASDiv-style word problems)	50	computed by the generator from its item spec and re-counted independently	our own code	-
our code generator: retrieval score items on invented passages	25	re-derived by code from the rendered passage; one hand audit of a sample	our own code	-
our code generator: language score items, captions reworded by a local open model (Mistral Nemo)	24	set by the generator's facet spec; every kept phrase re-checked by code on the reworded text	our own code	-

Licence note. 174 of the 985 rows come from 3 datasets whose original release names no open licence (SST-5 (Stanford Sentiment Treebank, TRAIN split), RTE (GLUE, TRAIN split), 20 Newsgroups (TRAIN split)). We used their TRAIN splits for training only, under the original authors' terms; this repository contains model weights only, no dataset text. Check those datasets' terms before you reuse this body for anything they may restrict.

Filters on every row: no exam split file was used for training (every row comes from a TRAIN split, from our own generators, or was written locally), no Glaive data, no BIG-bench / MMLU. We ran our exam-split 8-gram guard on these exact 985 rows before commit: the only match was one function-description phrase in the schema text of one dialogue service (6 rows), and no conversation or other text matched. Our test-overlap rule found no row sharing text with our own test items. No two rows share the same text; 22 rows have their text and answer written by local open models (listed in the table above); 24 rows have their text reworded by a local open model from code-made items (label set by code); no other row was written by an LLM item writer, no house or relay labels, no other miner's outputs as targets. Share-alike terms (CC BY-SA) apply to redistributing source text; this repository holds model weights only.

Checks we ran before commit (not an official score)

• Change from Qwen/Qwen3.5-4B (Zils relative change): 0.0182 (cap 0.15). Zils body digest 96dc97b6dc0edc71106c4fab0ef376cfba2aefe530e9be34bce10dbcc44fb14b.
• Copy screen with the official soup_copies code (our seeds; the validator's seed is private): own share beyond the house bodies, the listed public models, the soup and the release 0.0560-0.0618 (house-mix line 0.05); largest cosine to any of them or to our other committed bodies 0.968 (line 0.999); largest share explained by earlier bodies 0.235 (line 0.30).
• Our own exam-style reads vs the start (same code, same items; not the Zils exam): on our own test (795 items, without 8 Language items we excluded; not the Zils exam), gain over the start: total +2.06; Reasoning -1.32, Language known part +11.14, Retrieval +0.70, Tools +0.68, Arts +3.58. Reasoning fell, and this body did not meet all of our own pre-set bars on that test.

Licence

Apache-2.0 (LICENSE). See NOTICE for the upstream works and their notices. Nothing here is legal advice; check the original datasets' terms for your use.