zils-lite-body-14: a seed-mean LoRA change carried onto the current Zils LITE soup

A decision body for the Zils d3-lite soup pool (Bittensor subnet 127), Qwen/Qwen3.5-0.8B family.

It is the current d3-lite soup plus one change of our own:

• Start. The d3-lite soup of round d3-lite-2026-11-10 (soup hash 4d23ec5a54c2ab6d0c16fc493819ef113c789ab8edd1ef8756d41b645d2762a3), which is still the soup with the same two members today, rebuilt by us with the official Zils soup code (soup.build) as the uniform average of its two members, Charlie320/zils-lite-v9a @93b064048a7aeb7cf0d64236136e12c903e67f40 (Apache-2.0; Zils body digest 44edf3875e4ac0bb2d4aa0ae04d00e81d410bbdbfd9588705f23fe1fac3032ff) and qhoenix/zils-lite-decision-v9 @083d5b9aa3d2db48155cd6e7f92ac6f9aa057c65 (Apache-2.0; Zils body digest 8b8e3db3c8a62cbde4b5fb034bffb2c871faa1db450f6ddb0103ad120a44cfe5). The rebuilt soup's hash equals the published one. Our rebuilt start has Zils body digest 8bebfcbebee0c6fef293680fc44d0f05c2f05a8c014f37eb055602c157cb58dd (weights file sha256 4d456d8c77bb63d04bed623f7f15d75678f9006de17e926e3f82e7c94c9af123). Zils's mining guide allows this ("You may fine-tune the current soup").
• Our change. Six LoRA fine-tunes of that start on our own data (below), one per seed; the body is the start plus 1.0 x (the uniform mean of the six fine-tuned bodies minus the start), that is, the uniform mean of the six fine-tuned bodies.

Credit for the starting weights belongs to the authors of the two soup members and the works they were built on (see NOTICE). Nothing else from any other miner is in this body; it is not a re-upload of any public model. No other miner's outputs were used as labels.

Method

• Training (six runs, seeds 5002, 5012, 5022, 5032, 5042, 5052, all from the start above). LoRA rank 32 / alpha 64 / no dropout on the attention, Gated-DeltaNet and MLP projections of all language layers, merged into the weights after training. Learning rate 5e-5, weight decay 1.0, 1 epoch, batch 8, 349 optimiser steps per run (2,500 labelled rows + 290 anchor prompts). Trainer train_soft_c_pin_v388.py (sha256 75d17f362d67f8036e86aa893e50e2872a62c9c2b756aa69dfeb09ad9243bb0f).
  • Labelled rows (2,500): cross-entropy on the row's gold label with label smoothing 0.05, plus a Brier term on choice and yes/no rows or a ranked-probability (RPS) term on score rows (weight 1 each), plus a KL term with weight 0.1 to the start's own answer distribution, computed live from the start weights.
  • Anchor prompts (290): prompts only; their only loss is a KL term with weight 1.0 to the start's own answer distribution, computed live from the start weights (temperature 1). No labels are used for them.
• This body. The six final saves averaged with equal weights: factor 1.0, coefficient 1/6 per run (body = start + sum of 1/6 x (run - start), every tensor; 186 tensors differ from the start), built with our weight-difference tool (delta_combo.py v1.4, sha256 0a9b49b1...). No other body merged.

Training data (2,500 rows, file sha256 7e159a477438c85c85cd3197a8712f4e48cba1e9fe8d454803247595abe1e317)

Rows in the exam's 15 "known" families (5 categories x choice / yes-no / score; the Tools families are our dialogue-based versions).

category	choice	yes/no	score	total
reasoning	216	216	215	647
language	216	216	215	647
retrieval	167	167	166	500
tools	153	152	152	457
arts	83	83	83	249
total	835	834	831	2,500

• Most rows were written by AI writers of our team. 2,426 rows (97.0%) were written by two Claude sessions of our team (1,458 and 968 rows) with the public Zils item-authoring recipe (scripts/author_exam.py @6809491), and 5 rows (0.2%) by a local model (Phi-4) following that recipe's method (their records name the style example in seed_source / seed_row_id and carry no recipe field). Their label is the writer's own answer; there is no human label. The Claude-written rows were used only after a second model answered them blind and matched the writer's answer (rows it flagged as unclear were dropped) and after our row guard and audit; the 5 Phi-4 rows (stock re-lay rows) were checked by our own reads instead.
• Style examples. Each Claude-written row was written with one style or topic example: a public TRAIN row (2,399 rows) or an item from our own code generators (27 rows). The rows are new writing, not copies, but we did not check every written row against its own example; what we did check is listed under "Copy checks" below.
• Rows from datasets (69, 2.8%). Their labels are the dataset's own labels or are derived by code from the dataset record (table below).
• Builder. our feed builder (sha256 e7f56b45), from ledger cb388464; difficulty band and hard-row exceptions as recorded in the receipt (exception file 1256f0f8).

source	rows	label	original licence	original release
written by our Claude sessions (public recipe; one public TRAIN row or one of our own code-generator items as style example)	2,426	the writer's own answer; blind-answer match + audit	our own writing	-
written by a local model (Phi-4; the recipe's method, answer steered for label balance; style example: GoEmotions TRAIN 3, TweetEval emotion TRAIN 2)	5	the writer's own answer; checked by our own reads	our own writing	-
SQuAD 2.0 (TRAIN split): answerable-part count	30	dataset gold: answerability of each part, counted by code	CC BY-SA 4.0	https://rajpurkar.github.io/SQuAD-explorer/
Schema-Guided Dialogue (DSTC8, TRAIN split), re-formed to the exam's layout by code	24	code: required slots of the active intent in the human-annotated dialogue state	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
PIQA (TRAIN split)	13	human gold label: the shown method is the correct or the wrong solution	AFL-3.0	https://yonatanbisk.com/piqa/
SNLI (TRAIN split)	2	human gold label	CC BY-SA 4.0	https://nlp.stanford.edu/projects/snli/

Style examples of the 2,426 Claude-written rows (shown to the writer as a style or topic example; never a label): Schema-Guided Dialogue 433, QNLI 274, RTE 216, Social IQa 216, GSM8K 215, MultiNLI 214, PIQA 203, 20 Newsgroups 167, COCO captions 116, STS Benchmark 99, TweetEval 88 (emotion 78, from SemEval-2018 Task 1; sentiment 10, from SemEval-2017 Task 4), News Headlines for Sarcasm Detection 83, SST-5 73, our own code generators 27, SQuAD 2.0 2. For 66 written rows the example was one specific TRAIN row (20 Newsgroups 23, PIQA 22, Social IQa 21), used as topic and style example only.

Program-built and team-written shares.

kind	rows	share of 2,500
Written by our AI writers (Claude 2,426, Phi-4 5)	2,431	97.2%
Labels computed by code from dataset records (SQuAD 2.0 30, SGD 24)	54	2.2%
Human dataset labels (PIQA 13, SNLI 2)	15	0.6%

Licence notes. 56 rows carry dataset text under CC BY-SA 4.0 (SQuAD 2.0 30, SGD 24, SNLI 2); the rows are not distributed with this body. For the style-example datasets below (shown to the writer; none of them has a row of its own in the training data):

• GSM8K: MIT. COCO captions: the caption annotations are CC BY 4.0 (the images have separate terms; no image was used). News Headlines for Sarcasm Detection: CC BY 4.0 (as its author's Kaggle listing states).
• 20 Newsgroups, RTE: no open licence is named by the original release; used under the original authors' terms.
• SST-5, STS Benchmark: we could not read the terms of the original release (its pages were not reachable); its original terms remain unverified.
• TweetEval emotion (80 rows: 78 Claude, 2 Phi-4): written by an AI writer with a SemEval-2018 Task 1 (via TweetEval) tweet shown as a style example only; no row shares a run of 5 words with its own example tweet, and no training row or anchor prompt shares a run of 8 words with any TweetEval emotion TRAIN tweet (see Copy checks); that dataset's terms limit it to research use.
• TweetEval sentiment (10 Claude rows): written by an AI writer with a SemEval-2017 Task 4 (via TweetEval) tweet shown as a style example only; no row shares a run of 5 words with its own example tweet; that task's terms ask users to follow the Twitter terms of service.
• GoEmotions (3 Phi-4 rows): CC BY 4.0.

Copy checks

• Tweet-seeded rows (90: TweetEval emotion 80, TweetEval sentiment 10). Each was compared with its own example tweet (the 3,257-tweet TweetEval emotion TRAIN file, sha256 448fb3d7..., or, for the 10 sentiment rows, the seed text shown to the writer): no shared run of 8 words, nor of 5 words; the longest shared character run is 15 characters (" bottled water ", the spaces included). Against all 3,257 TweetEval emotion TRAIN tweets: no shared run of 8 words in any of the 2,500 training rows or the 290 anchor prompts.
• Test content. Our exam-split guard and test-overlap rule removed rows that share text with test content. This is a filter against test overlap, not a proof that no source text was copied.
• The other written rows were not checked against their own style example.

Anchor set

290 prompts (file sha256 27de0417dcc9b6f09e4a0a2bfc8deab81ce6a4530b8c7de7e6104ec6aba0c5d3), prompts only. Their targets are the start's own answers, computed live; no labels were used. None of them is in the training rows.

Filters on every row. No exam split files, no public Zils seed items, no Glaive data, no BIG-bench / MMLU; our exam-split guard and test-overlap rule removed rows that share text with test content. Our 149 hold-out rows share no id, no identical state text and no identical state + question with the training rows, and none of the 294 rows of our sealed hold-out list is a training row (by id).

Honest limits of this data. 97.2% of the rows were written by AI writers and carry the writer's own answer, not a human label: the 2,426 Claude-written rows were checked by a second model's blind answer and a sampled audit, the 5 Phi-4 rows by our own reads; none by people. The difficulty band was measured on our rebuilt start.

Build record and our own read (not an official score)

• This body. model.safetensors sha256 6e086eea534371dd8757dfb781bb880991175282e16e5014a9e09c79201329f2; Zils body digest 91a3308487e4184bd128b1e11393d9db815188f86dd0179c96c2f23c3047de67. Zils relative change from Qwen/Qwen3.5-0.8B, measured on the committed form: 0.035919 (cap 0.15; the start itself is 0.03584).

• Build inputs. Start file sha256 4d456d8c...; the six runs' model.safetensors sha256: seed 5002 11be6a12..., 5012 31dc95e9..., 5022 d8ee3c50..., 5032 ef93414f..., 5042 93bb15ff..., 5052 c99428cf... (full values in the build receipt).

• Copy screen (official soup_copies code, 11-11, with the current soup kept as the soup): no copy detected in this screen. Copy lines: cosine >= 0.999, or a share explained by earlier bodies >= 0.3, on any of three screen seeds. This body: share explained by the 74 d3-lite bodies on record on 11-10 jointly at most 0.00215, by our earlier LITE body at most 0.0014, by our other body of tonight at most 0.0055; highest cosine 0.9962 (to our other body of tonight; to other miners' bodies at most 0.9861). 27 later d3-lite commits were not in the screen. This holds only while the current soup stays the soup (with the soup counted as an earlier body, the explained share is 0.987); no claim about a later field.

• Our own exam-style read (C3; not the Zils exam). Gain of this body over the soup (our rebuilt S10) on the same items, in points, on 10 registered splits (5 dev + 5 robustness), registered before the read:

  split	dev-0	dev-1	dev-2	dev-3	dev-4	rob-0	rob-1	rob-2	rob-3	rob-4	mean
  gain	+1.42	+1.90	+2.85	+0.96	+2.20	+0.81	-0.62	-2.46	+0.91	+2.51	+1.05

  By category (10-split mean): reasoning -1.11, language +3.63, retrieval +0.64, tools +0.81, arts +1.22.

• Our commit bar, and this body. Our pre-set bar is a 10-split mean gain of at least +2.00 points AND at least 8 of the 10 splits at +2.00 or more. This body is below our own pre-set bar of +2.00: mean +1.05, 3 of 10 splits at +2.00 or more.

Licence

Apache-2.0 (LICENSE). See NOTICE for the upstream works and their notices. Nothing here is legal advice; check the original datasets' terms for your use.