zils-mini-body-32: six LoRA runs on our lh3500 data, averaged and scaled by 0.5 on the MS11 start

This is a full d3-mini body for the Zils d3-mini pool. It uses the model d3-mini-4.5b-v1 from the Zils repository at revision e9e87b3e.

Body

The body's model.safetensors sha256 is feca81c3a13e0ad69a19687ddf5a93334c8a8a427892617f438044d4dc2f2139 (9,078,616,200 bytes); its Zils body digest is 896da8f87ff0e45df095f65d0ec747ed1888bdff6ea1619ac0a87677380fe538. It was built by our combination tool delta_combo.py v1.4 (sha256 prefix 0a9b49b1; build receipt sha256 prefix 09b6e384) as the start plus 0.5 times the average change of the six runs, on every tensor (152 tensors differ from the start). At 0.5 this is halfway between the start and the uniform mean of the six run bodies:

body = start + 0.5 x (1/6) x sum(run - start)

Start (MS11)

MS11 is the official soup of the d3-mini round of 2026-11-11 (soup hash 9a21f188dc04284d8ef526870e8a37b19533556eabaccc31aecf1457b0b3ce1a). It was rebuilt with the Zils repository at revision e9e87b3e from the soup's two public member bodies:

  • qhoenix/zils-mini-decision-l13_avg678 at revision 091c6ebe (body sha256 prefix 11bfca0b)
  • joamCriff123/zils-mini-sx3 at revision c9432ada (body sha256 prefix e0ccf728)

The start body's sha256 is 8c26d1ce07e3d26e6e9b04c3cdff13dafc37fbcab70d7c18e40d14cb78fdd57c. The build receipt's sha256 prefix is 02af29da.

Training

Each of the six runs starts from MS11, using the flags in run_mini_seed_v6.sh (sha256 prefix 16672cc6) and the trainer train_soft_f.py (sha256 prefix b78912c4). These values are read back from each run's train_meta.json:

  • adapters: LoRA rank 32, alpha 64, dropout 0.05; no adapters on the MLP layers
  • learning rate 5e-5, for 2 epochs; batch 8 with 2 accumulation steps
  • LoRA weight decay 1.0
  • loss: cross-entropy (weight 1) plus Brier score (weight 1); KL weight 0
  • pin term: pin weight 1.0, --pin-train 0.1 with the live pin; the pin file is empty (0 pin rows)

The runs differ only in their seed. These are the six run files, as the build receipt names them:

Seed Run model.safetensors sha256
4602 f6637c2265628ebfc6b1decfacb4cc4a59f2dae0d55ca467c24e42b24077a5a9
4612 01049277c753b5c6d4d1fe1b7f39845f81ec9c868f38ee6f339fe9f16e607321
4622 c7ed3516fe923acff57676639f20a621c55bff0961b249c115bcf8dbc05f09c3
4632 d5d20bb4dd785d5790dcb3335755c95ee242ab8863ace67074fc865d5cefb7b2
4642 96d2e7780c3d757ea11da912e46090e4a193d7171d75bdc8dd40ae04e9a27631
4652 103e70af31f8e1c05f36152c47723bab4c4292214a1df4288ad7f752d5bac2ac

Data

The training file has 2,929 rows (sha256 prefix 4c53fddd). It is the same file that trains our LITE bodies. This is its own description, as each run records it:

lh3500 = LITE v24 Y-HARD feed 4c53fddd (2,929 rows; gate v3f PASS, feed_final/LH3500_1112i; S10 start 8bebfcbe)

Where the labels come from, counted from each row's label_method:

  • 2,882 rows: the writer's own answer (1,486 by one Claude session, 1,395 by another, 1 by a local Phi-4 model)
  • 41 rows: the human gold label of a public TRAIN row (MNLI 26, QNLI 15)
  • 4 rows: derived by code from the human answer spans of SQuAD v2
  • 2 rows: computed by code from the item's own specification

Most rows also name the public training set whose row served as their style example, or as their topic and style example (48 rows record none). NOTICE lists these sets with their counts.

Evaluation

These are our own reads, not an official score.

  • C3 read (read_body v2): this body against the MS11 start, on the same items, in points, on 10 registered splits.

    • Mean change: +1.42. The splits range from +0.17 to +5.10.
    • 10 of 10 splits are above 0, and 2 of 10 are at +2.00 or more.
    category mean change
    reasoning +1.30
    language +3.14
    retrieval -0.64
    tools +2.06
    arts +0.18
  • C3 with category weights (reasoning 0.26, language 0.26, retrieval 0.20, tools 0.18, arts 0.10): +0.86, with 9 of 10 splits above 0. Our own line for this read is +2.8, so the read is below it.

  • Hold-out read (mean change in the probability of the gold answer, our sealed hold-out rows): +0.0359 on the 140 exam-length rows (+0.0356 on all 149). This passes our hold-out line.

  • Per-family check: passes: no family is below the floor of -3.0 (the lowest is retrieval-score at -2.01).

  • Copy screen: (official sketch method, one public seed) cosine to the soup 0.99399 (line 0.999); share of its change explained by our earlier MINI body 0.185 (line 0.30).

  • Our commit bar. Our pre-set bar is a 10-split mean gain of at least +2.00 points over the start, with at least 8 of the 10 splits at +2.00 or more. This body is below our own pre-set bar: mean +1.42, 2 of 10 splits at +2.00 or more.

This card reports no other results.

Licence

The licence is the start's LICENSE, unchanged. NOTICE lists the sources this body depends on.

Downloads last month
8
Safetensors
Model size
5B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for tensor-tailor/zils-mini-body-32

Finetuned
Qwen/Qwen3.5-4B
Finetuned
(5)
this model