zils-mini-body-13: class-balanced human-label LoRA fine-tune of the current Zils MINI soup
A decision body for the Zils d3-mini soup pool (Bittensor subnet 127), in the d3-mini chassis layout (Qwen3.5-4B language and vision
tensors, bf16). It is a fine-tune of the current MINI soup: the uniform weight average (official zils.soup.build: bf16-canonical, fp32
mean, chassis vision tower) of qhoenix/zils-mini-decision-v2 @ecb9cadb39a2f84cefd2f27a755ec12095b62933 and Charlie320/zils-mini-s1
@53eb3ffa768c1ad1eace7acbfdb3ebb9674cefe6 (soup body digest df87aaa9…), as Zils's mining guide allows ("You may fine-tune the current
soup"). Credit for the starting weights belongs to the authors of those two bodies and their upstream works (see NOTICE). No other miner's
body was merged; it is not a re-upload of any public model.
Method
- Training: LoRA rank 32 / alpha 64 / dropout 0.05 on the attention and Gated-DeltaNet projections of all language layers (152 matrices), learning rate 5e-5, AdamW with weight decay 1.0 on the LoRA weights only, batch 8 x accumulation 2, warm-up 50 steps then cosine, 2 epochs, max 2,048 tokens, seed 4343; loss per row = cross-entropy on the row's label + 1.0 x Brier. No KL / distillation term, no reference or teacher probabilities, no weight interpolation. Merged into bf16. This body is the final save (no picking among saves).
Training data (2,006 rows, file sha256 510d4b0a58d60128fd2775bf6fced31d263286dedddf586899fa53cc1c8d49a1)
A class-balanced set in the exam's 15 "known" families (5 categories x choice / yes-no / score; category shares follow the published exam weights). Every label is the dataset's own label (human annotators, or the dataset's own prover / generator), derived by code from the dataset record, or computed by our own generator; no label was predicted by a model we ran, and no other miner's outputs were used. Half A of a 4,007-row set frozen with MINI-soup difficulty and the band rule below (two disjoint, label-balanced halves; freeze sha256 af43a4f16f5079c9...; tool balM_freeze.py v0.2 3458b9d9); the other half trains a separate body. Difficulty rule per cell (to avoid the label-noise tail of a pure hardest-first pick): rows with a human label were taken in a fixed hash order from those on which the MINI soup puts probability 0.30-0.90 on the correct answer (rows at or above 0.90 only to fill a short cell, the hardest of them first); code-labelled and proven rows hardest-first; 50% cap per source except where noted in the freeze receipt.
| category | choice | yes/no | score | total |
|---|---|---|---|---|
| reasoning | 216 | 215 | 216 | 647 |
| language | 215 | 214 | 237 | 666 |
| retrieval | 167 | 166 | 0 | 333 |
| tools | 88 | 0 | 0 | 88 |
| arts | 72 | 75 | 125 | 272 |
| source | rows | label | original licence | original release |
|---|---|---|---|---|
| MultiNLI (GLUE, TRAIN split) | 216 | human gold label (3-way; in yes/no rows entailment = yes, neutral / contradiction = no) | OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0 | https://cims.nyu.edu/~sbowman/multinli/ |
| PIQA (TRAIN split) | 215 | human gold label: the shown method is the correct or the wrong solution of the goal | AFL-3.0 | https://yonatanbisk.com/piqa/ |
| 20 Newsgroups (SetFit mirror, TRAIN split) | 167 | the newsgroup the post was sent to | no open licence is named by the original release; used under the original authors' terms | http://qwone.com/~jason/20Newsgroups/ |
| STS Benchmark (TRAIN split, captions genre) | 162 | human 0-5 similarity mean mapped to the nearest of 5 levels; rows near a level edge dropped | no open licence is named by the original release; used under the original authors' terms (mixed sources; the mirror card says 'unknown') | https://huggingface.co/datasets/mteb/stsbenchmark-sts |
| Social IQa (TRAIN split) | 136 | human gold answer | CC BY 4.0 | https://allenai.org/data/socialiqa |
| SNLI (TRAIN split) | 112 | human gold label | CC BY-SA 4.0 | https://nlp.stanford.edu/projects/snli/ |
| RTE (GLUE, TRAIN split) | 101 | human gold label (entailment / not entailment) | no open licence is named by the original release; used under the original authors' terms (PASCAL RTE challenges) | https://gluebenchmark.com/ |
| QNLI (GLUE, TRAIN split; text from SQuAD v1.1) | 92 | human gold label (the sentence answers the question) | CC BY-SA 4.0 (SQuAD) | https://rajpurkar.github.io/SQuAD-explorer/ |
| SST-5 (Stanford Sentiment Treebank, TRAIN split) | 85 | human 5-class sentiment label | no open licence is named by the original release; used under the original authors' terms | https://nlp.stanford.edu/sentiment/ |
| iSarcasmEval task A (TRAIN split) | 75 | the tweet author's own sarcasm label | MIT | https://github.com/iabufarha/iSarcasmEval |
| SQuAD 2.0 (TRAIN split, sentence level) | 74 | derived by code from the human answer spans (the sentence holds the answer or not) | CC BY-SA 4.0 | https://rajpurkar.github.io/SQuAD-explorer/ |
| RuleTaker (tasksource mirror) | 64 | the dataset's prover label under the stated closed-world assumption | Apache-2.0 (allenai/ruletaker repository) | https://github.com/allenai/ruletaker |
| ToolACE @6bda777c | 48 | read off the record: the real call is the answer; distractors break a required argument (code-checked) | Apache-2.0 | https://huggingface.co/datasets/Team-ACE/ToolACE |
| GoEmotions (simplified, TRAIN split) | 43 | human label (rows with exactly one of the 6 emotion names) | Apache-2.0 (google-research repository) | https://github.com/google-research/google-research/tree/master/goemotions |
| Hermes function-calling v1 @dae3e1d2 (func-calling file; the authors' Glaive-derived glaive-function-calling-5k file excluded) | 40 | read off the record: the real call is the answer; distractors break a required argument (code-checked) | Apache-2.0 | https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1 |
| TweetEval emotion (SemEval-2018 Task 1, TRAIN split) | 29 | human emotion label | no open licence is named by the original release; used under the original authors' terms (TweetEval: 'restrictions may apply to individual tasks'; Twitter terms) | https://github.com/cardiffnlp/tweeteval |
| Corr2Cause (tasksource mirror) | 16 | the dataset's generator label | MIT (repository) | https://github.com/causalNLP/corr2cause |
| our code generator: reasoning score items (ASDiv-style word problems) | 216 | computed by the generator from its item spec and re-counted independently | our own code | - |
| our code generator: language score items | 75 | computed by the generator from its item spec and re-counted independently | our own code | - |
| our code generator: arts score items | 40 | computed by the generator from its item spec and re-counted independently | our own code | - |
Licence note. 544 of the 2,006 rows come from 5 datasets whose original release names no open licence (20 Newsgroups (SetFit mirror, TRAIN split), STS Benchmark (TRAIN split, captions genre), RTE (GLUE, TRAIN split), SST-5 (Stanford Sentiment Treebank, TRAIN split), TweetEval emotion (SemEval-2018 Task 1, TRAIN split)). We used their TRAIN splits for training only, under the original authors' terms; this repository contains model weights only, no dataset text. Check those datasets' terms before you reuse this body for anything they may restrict.
Filters on every row: no exam split files (the validation / test splits the Zils exam-authoring code reads were never opened), no Glaive data, no BIG-bench / MMLU, no real-event rows; our exam-split 8-gram guard and test-overlap rule (both directions) removed every hit; near-duplicates removed; no rows written by an LLM item writer, no house or relay labels, no other miner's outputs as targets. Share-alike terms (CC BY-SA) apply to redistributing source text; this repository holds model weights only.
Checks we ran before commit (not an official score)
- Change from Qwen/Qwen3.5-4B (Zils relative change): 0.01242 (cap 0.15). Zils body digest 16298d1bdf6eeb660837916ccedb2fdd369f994ee24c41d5ffd8ae2ff16ce3de.
- Copy screen with the official
soup_copiescode (our seeds; the validator's seed is private): own share beyond the house bodies, the listed public models, the soup and the release 0.096-0.101 (house-mix line 0.05); largest cosine to any of them or to our other committed bodies 0.956 (line 0.999); largest share explained by earlier bodies 0.268 (line 0.30). - Our own exam-style reads vs the start (same code, same items; not the Zils exam): predicted score +1.24 points overall; by category reasoning +1.44, language +1.13, retrieval -2.17, tools -0.48, arts -0.29. Retrieval, tools and arts fell; the retrieval drop comes from worse calibration (its accuracy was unchanged).
Licence
Apache-2.0 (LICENSE). See NOTICE for the upstream works and their notices. Nothing here is legal advice; check the original datasets' terms for your use.
- Downloads last month
- 28
Model tree for tensor-tailor/zils-mini-body-13
Base model
Qwen/Qwen3.5-4B-Base