zils-mini-body-27: Retrieval-weighted exam-type LoRA fine-tune of our own earlier fine-tune of a uniform mix of four d3-mini house ingredients

A decision body for the Zils d3-mini soup pool (Bittensor subnet 127), in the d3-mini chassis layout (Qwen3.5-4B language and vision tensors, bf16). Its start is one of our own earlier fine-tunes, not published on its own (Zils body digest 6bb4d4d61edc9d123928ddd830aa64855d87126187be00d926c59165e01051b0; its training is described below). That fine-tune in turn starts from a uniform weight average, built with the official zils.soup.build, of four of the pool's five public house ingredients: vllm-sr/d3-mini @df99bf4c; perplexity-ai/pplx-decider-flash-v1.2 @9ff7cf98; torchcast-ai/torchcast-decision-4b @492bfb19; Maincode/Matilda-Decision-v1-Mini @39479a8a. Start body digest be40457ed0725abfe6a66d0e3cdb29bff6c206a71440d54382bbdca89c01a2fc (soup hash 20c02cae). Credit for the starting weights belongs to the authors of those models and their upstream works (see NOTICE). No other miner's body was merged; it is not a re-upload of any public model.

Method

• Training: LoRA rank 32 / alpha 64 / dropout 0.05 on the attention, Gated-DeltaNet AND MLP (gate / up / down) projections of all language layers, learning rate 5e-5, AdamW with weight decay 1.0 on the LoRA weights only, batch 8 with no accumulation, warm-up 50 steps then cosine, 2 epochs, max 2,048 tokens, seed 4361; loss per row = cross-entropy on the row's label + 1.0 x Brier + 0.1 x KL(start || this model) on the same row. The KL term holds the model near its own start and is computed live from the start weights with the adapter off; it uses no teacher, no extra pin rows and no other miner's outputs. Merged into bf16.
• The start body: it was trained with the same recipe on file 2: LoRA rank 32 / alpha 64 / dropout 0.05 on the attention, Gated-DeltaNet AND MLP (gate / up / down) projections of all language layers, learning rate 5e-5, AdamW with weight decay 1.0 on the LoRA weights only, batch 8 x accumulation 2, warm-up 50 steps then cosine, 2 epochs, max 2,048 tokens, seed 4350; loss per row = cross-entropy on the row's label + 1.0 x Brier + 0.1 x KL(house mix || that model) on the same row, computed live from the house-mix weights with the adapter off; no teacher, no extra pin rows and no other miner's outputs. Merged into bf16. No weight interpolation. This body is the final save (the only one read and shipped, fixed before training).
• Target: Data weighted toward Retrieval (49.0% of the rows), using only rows that passed our checks. The start is our own earlier fine-tune, not a soup and not another miner's body.

Training data (2,800 rows, two files (file 1: sha256 941f78a433b56061471f508362f8c78e9d3ade1666a82bdc42524f2d2eb243d8, 800 rows; file 2: sha256 91ee9ec2c267832c8daa7f3b7fc498321b68acdc5bfa6ea02ad4c6e93a294e0a, 2,000 rows); 92 row ids shared)

Two training files. File 1 is this body's own training set, file 2 the training set of its start body; the tables below add the two. File 1: A Retrieval-weighted exam-type set in the exam's 15 known families: 12.0% of the rows are Reasoning, 14.2% Language, 49.0% Retrieval, 12.9% Tools and 11.9% Arts. Category shares do NOT follow the exam weights. File 2: A Reasoning-weighted exam-type set of 2,000 rows: 33.5% of the rows are Reasoning, 22.4% Language, 17.3% Retrieval, 18.3% Tools and 8.7% Arts, so category shares do NOT follow the exam weights. Every label is the dataset's own label (human annotators, or the dataset's own prover / generator), derived by code from the dataset record, or computed by our own generator, except the 22 rows written by local open models, whose answer is the writer model's own (see the table). No other miner's outputs were used. File 1: Selected with ft_emphasis.py v0 (c84826ad) from a 3,004-row set frozen for this purpose (sha256 fbc9e79d4d5931cb4285ca776e53832b56a92be88b554aa48578216fa6943845). This set has fewer than 1,600 rows, so it trains at batch 8 with no accumulation (about 200 optimizer steps). File 2, the start body's own training set: selected with select_final.py v0.6 (f8c693d7) from a 6,983-row set frozen for its start only (sha256 482e664f314ec7b40e6836147ae268dade2c41e0490245458e3cba2648155926); 2,000 rows after the category, family and generator limits; it trained at batch 8 x accumulation 2 (250 optimizer steps).

CC BY-SA 4.0 note. 468 of the rows are built from the Schema-Guided Dialogue dataset (Google Research, CC BY-SA 4.0, https://github.com/google-research-datasets/dstc8-schema-guided-dialogue) and 9 from SNLI (CC BY-SA 4.0). They were used for training only; this repository holds model weights only, no dataset text. Check those datasets' terms before you reuse this body.

Difficulty rule: a row enters only if our fixed house-mix body (uniform mix of four d3-mini house ingredients, digest be40457e) puts probability 0.30-0.80 on its correct answer (all 800 rows of file 1 checked: lowest 0.301, highest 0.800; all 2,000 rows of file 2 likewise: lowest 0.300, highest 0.800). That was measured on the house mix, NOT on this body's own start. Rows were taken only from sources that passed our checks, and nothing was padded to reach a row count. No family exceeds 20% of the rows (largest 16.6%); our own item generators supply 14.8% of the rows of file 1. In file 2, no family exceeds 20% of the rows (largest 17.0%); our own item generators supply 57.0% of its rows, under a limit set for that one feed (the rows built by code from Schema-Guided Dialogue are not counted in that limit).

category	choice	yes/no	score	total
reasoning	88	474	203	765
language	54	130	377	561
retrieval	294	143	301	738
tools	160	160	148	468
arts	80	93	95	268

source	rows	label	original licence	original release
logical-entailment (DeepMind; tasksource mirror)	286	the dataset's entailment label	Apache-2.0 (repository)	https://github.com/google-deepmind/logical-entailment-dataset
RuleTaker (tasksource mirror)	209	the dataset's prover label under the stated closed-world assumption	Apache-2.0 (allenai/ruletaker repository)	https://github.com/allenai/ruletaker
Schema-Guided Dialogue (DSTC8, TRAIN split; real dialogues, sibling function options made by code)	160	code: the arguments are the dialogue's annotated call; a schema check on every option passes only for the gold sibling	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
20 Newsgroups (TRAIN split)	127	the newsgroup the post was sent to	no open licence is named by the original release; used under the original authors' terms	http://qwone.com/~jason/20Newsgroups/
Schema-Guided Dialogue (DSTC8, TRAIN split; required arguments filled)	107	code: required slots of the active intent that are in the human-annotated dialogue state and whose value occurs verbatim in the conversation so far	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
SQuAD 2.0 (TRAIN split, passage level)	81	dataset gold: the number of answerable question parts	CC BY-SA 4.0	https://rajpurkar.github.io/SQuAD-explorer/
QNLI (GLUE, TRAIN split; text from SQuAD v1.1)	71	human gold label (the sentence answers the question)	CC BY-SA 4.0 (SQuAD)	https://rajpurkar.github.io/SQuAD-explorer/
Schema-Guided Dialogue (DSTC8, TRAIN split; does the next turn make a call)	60	code: whether the annotated next system turn of the real dialogue makes a service call	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
Schema-Guided Dialogue (DSTC8, TRAIN split; real dialogues, function list edited by code)	57	code: true = the annotated next call serves the request and the function list is the schema's; false = one edit of the list so no listed function can honour it	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
Schema-Guided Dialogue (DSTC8, TRAIN split; real dialogues, requested option removed by code)	43	code: true = the annotated next call honours the slot the user just stated; false = that option was taken out of the listed functions	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
Schema-Guided Dialogue (DSTC8, TRAIN split; required arguments filled, corrected or referred-back values)	41	code: as the row above, only turns where a value was corrected, carried from assistant text or a function response, or referred back to	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
SST-5 (Stanford Sentiment Treebank, TRAIN split)	38	human 5-class sentiment label	no open licence is named by the original release; used under the original authors' terms	https://nlp.stanford.edu/sentiment/
iSarcasmEval task A (TRAIN split)	35	the tweet author's own sarcasm label	MIT	https://github.com/iabufarha/iSarcasmEval
SQuAD 2.0 (TRAIN split, sentence level)	33	derived by code from the human answer spans (the sentence holds the answer or not)	CC BY-SA 4.0	https://rajpurkar.github.io/SQuAD-explorer/
Social IQa (TRAIN split)	29	human gold answer; rows an AI reviewer marked wrong or unclear were removed	CC BY 4.0	https://allenai.org/data/socialiqa
QNLI (GLUE, TRAIN split; text from SQuAD v1.1; second selection)	29	human gold label (the sentence answers the question)	CC BY-SA 4.0 (SQuAD)	https://rajpurkar.github.io/SQuAD-explorer/
PIQA (TRAIN split)	21	human gold label (the shown method is the correct or the wrong solution of the goal); rows an AI reviewer marked wrong or unclear were removed	AFL-3.0	https://yonatanbisk.com/piqa/
RTE (GLUE, TRAIN split)	20	human gold label (entailment / not entailment)	no open licence is named by the original release; used under the original authors' terms (PASCAL RTE challenges)	https://gluebenchmark.com/
MultiNLI (GLUE, TRAIN split; yes/no form)	18	human gold label (entailment = yes, neutral / contradiction = no); only rows an AI reviewer read and found correct	OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0	https://cims.nyu.edu/~sbowman/multinli/
PIQA (TRAIN split; second selection)	17	human gold label (the shown method is the correct or the wrong solution of the goal); only rows an AI reviewer read and found correct	AFL-3.0	https://yonatanbisk.com/piqa/
Items written on our own GPU by the local open model microsoft/phi-4 (MIT); a public TRAIN row served as a style example only	16	the writer model's own answer; kept only after a blind check by local open models and a sealed read by an AI reviewer	text written by us; no dataset text reused	https://huggingface.co/microsoft/phi-4
MultiNLI (GLUE, TRAIN split; second selection)	14	human gold label (3-way); only rows an AI reviewer read and found correct	OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0	https://cims.nyu.edu/~sbowman/multinli/
MultiNLI (GLUE, TRAIN split)	11	human gold label (3-way); only rows an AI reviewer read and found correct	OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0	https://cims.nyu.edu/~sbowman/multinli/
Items written on our own GPU by the local open model mistralai/Mistral-Nemo-Instruct-2407 (Apache-2.0); a public TRAIN row served as a style example only	6	the writer model's own answer; kept only after a blind check by local open models and a sealed read by an AI reviewer	text written by us; no dataset text reused	https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407
SNLI (TRAIN split)	5	human gold label; only rows an AI reviewer read and found correct	CC BY-SA 4.0	https://nlp.stanford.edu/projects/snli/
MultiNLI (GLUE, TRAIN split; third selection)	4	human gold label (3-way); only rows an AI reviewer read and found correct	OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0	https://cims.nyu.edu/~sbowman/multinli/
SNLI (TRAIN split; rows where all 5 annotators agree)	4	human label given by all 5 annotators; only rows an AI reviewer read and found correct	CC BY-SA 4.0	https://nlp.stanford.edu/projects/snli/
our code generator: language score items	335	computed by the generator from its item spec and re-counted independently	our own code	-
our code generator: reasoning score items (ASDiv-style word problems)	203	computed by the generator from its item spec and re-counted independently	our own code	-
our code generator: retrieval choice items on invented passages (computed over all current records after later corrections)	167	re-derived from the rendered text by an independent parser	our own code	-
our code generator: retrieval score items on invented passages (with later corrections)	155	re-derived from the rendered text by an independent parser	our own code	-
our code generator: language yes/no items (does one caption facet agree; pairs built to the facet spec)	92	set by the generator's facet spec (the pair is built to agree or differ); one hand audit of a sample	our own code	-
our code generator: retrieval score items on invented passages	65	re-derived by code from the rendered passage; one hand audit of a sample	our own code	-
our code generator: arts yes/no items (a post score rule stated in the item; label recounted by the generator)	58	computed by the generator's recount on the final post; one hand audit of a sample	our own code	-
our code generator: arts choice items (which listed word a flip rule moves; label re-checked on the final post)	58	set by the generator and re-checked on the final post; one hand audit of a sample	our own code	-
our code generator: arts score items	45	computed by the generator from its item spec and re-counted independently	our own code	-
our code generator: language score items, captions reworded by a local open model (Mistral Nemo)	42	set by the generator's facet spec; every kept phrase re-checked by code on the reworded text	our own code	-
our code generator: language choice items (the one agreeing caption facet; pairs built to the facet spec)	16	set by the generator's facet spec (the pair is built to agree on one facet); one hand audit of a sample	our own code	-
our code generator: arts score items, post text reworded by a local open model (Mistral Nemo)	12	computed by the generator from its item spec and re-counted independently	our own code	-
our code generator: retrieval yes/no items on invented passages (the place is named through a second fact)	9	re-derived by code from the rendered passage; one hand audit of a sample	our own code	-
our code generator: retrieval yes/no items on invented passages	1	re-derived by code from the rendered passage; one hand audit of a sample	our own code	-

Licence note. 185 of the 2,800 rows come from 3 datasets whose original release names no open licence (20 Newsgroups (TRAIN split), SST-5 (Stanford Sentiment Treebank, TRAIN split), RTE (GLUE, TRAIN split)). We used their TRAIN splits for training only, under the original authors' terms; this repository contains model weights only, no dataset text. Check those datasets' terms before you reuse this body for anything they may restrict.

Filters on every row: no exam split file was used for training (every row comes from a TRAIN split, from our own generators, or was written locally), no Glaive data, no BIG-bench / MMLU. We ran our exam-split 8-gram guard on these exact 800 rows and on the 2,000 rows of the earlier file before commit: the only match was one function-description phrase in the schema text of one dialogue service (6 rows here, 23 rows in the earlier file), and no conversation or other text matched. Our test-overlap rule found no row sharing text with our own test items. No two rows share the same text inside either file; 92 rows of this file also appear in the earlier 2,000-row file; 22 rows have their text and answer written by local open models (listed in the table above); 54 rows have their text reworded by a local open model from code-made items (label set by code); no other row was written by an LLM item writer, no house or relay labels, no other miner's outputs as targets. Share-alike terms (CC BY-SA) apply to redistributing source text; this repository holds model weights only.

Checks we ran before commit (not an official score)

• Change from Qwen/Qwen3.5-4B (Zils relative change): 0.0186 (cap 0.15). Zils body digest f17c4e1c788e14a036eb66f79def2c84ea6649da79280557b6e93928d36531a5.
• Copy screen with the official soup_copies code (our seeds; the validator's seed is private): own share beyond the house bodies, the listed public models, the soup and the release 0.0947-0.0983 (house-mix line 0.05); largest cosine to any of them or to our other committed bodies 0.952 (line 0.999); largest share explained by earlier bodies 0.262 (line 0.30).
• Our own exam-style reads vs the start (same code, same items; not the Zils exam): on our own test (795 items, without 8 Language items we excluded; not the Zils exam), gain over the start: total +0.44; Reasoning -1.18, Language known part +1.15, Retrieval +1.35, Tools +0.30, Arts +1.13. Reasoning fell, and this body did not meet all of our own pre-set bars on that test.

Licence

Apache-2.0 (LICENSE). See NOTICE for the upstream works and their notices. Nothing here is legal advice; check the original datasets' terms for your use.