org card: per-experiment hyperparameters and VQA/caption results

README.md

@@ -512,11 +512,17 @@ The cost argument is secondary but real: each adapter is ~821 MB, so the five
 unpublished runs would add roughly 5 GB of weights that no one should load in
 preference to what is already here.
 
-**Aspaklarya-111 is still training.** It will be published if it clears the
-`train,other` arm it is built on - meaning a MEAN4 above roughly 76.8, which
-is the arm mean plus two sigma. If it lands inside the arm it will be
-reported here as another null and left unpublished, on the same reasoning as
-108-110.
+**Aspaklarya-111 has finished, and it is a null.** Its best checkpoint
+(step 4496) scores **76.62**, against the pre-registered bar of roughly 76.8 -
+the `train,other` arm mean plus two sigma. It lands *inside* the arm it was
+built on (76.37 +/- 0.23, so +0.25 is about 1.1 sigma), which is exactly the
+case the bar was written to catch. It is therefore **not published**, on the
+same reasoning as 108-110.
+
+That is worth stating plainly because 76.62 is the second-highest single
+number on this page, and the temptation to ship it is precisely the error the
+sigma measurement exists to prevent. Raising QA depth from 2 to 4 per image
+did not extend the one confirmed lever.
 
 ### Caveat
 
@@ -547,6 +553,251 @@ That is why the legacy numbers are reported but never used to gate a decision.
   *What is published here, and what is not* above - they are either strictly
   dominated, refuted, or seed replications whose output is a sigma rather
   than a model.
-- Open: whether QA depth (111, running) extends the one confirmed lever, and
-  whether the `other` split helped by adding **images** or merely **items** -
-  111 isolates items, since it adds no new images.
+- **QA depth is answered: it does not help.** 111 raised QA per image from 2
+  to 4 and landed inside its own arm. Since 111 adds items but no new images,
+  the `train,other` gain is better explained by the **images** it brings than
+  by raw item count.
+
+## Round 11X - six axes against a replicated control
+
+Round 1XX ended with one confirmed lever (data scaling) and a warning that
+single-run deltas below about 1.0 MEAN4 are unreadable. Round 11X was built
+to test six *new* axes cheaply, under a short-budget probe (SBP) design: one
+epoch, checkpoints at steps 562 and 1124, every checkpoint judged on all four
+slices. Ten jobs ran unattended over three days.
+
+**No axis produced a win.** That is the headline, and it is not a failed
+round - six levers were eliminated, and the round also corrected two errors
+in how this project measures things.
+
+### The control had to be rebuilt first
+
+The SBP arms were initially compared against a single control checkpoint that
+ran a **different step grid** (590/1180 rather than 562/1124) and a different
+`max-seq-length`. That mismatch produced a reported **+2.32** VQA_DM gain for
+the paraphrase arm which **does not exist**; re-measured against the matched
+three-seed control the same arm is **+1.00**, against a VQA_DM sigma of 0.88.
+The claim is retracted here in full, and so is the recommendation to
+paraphrase RSICD questions that was built on it.
+
+The lesson generalises past this round: *a control must match the treatment on
+grid, split and budget, or the delta measures the mismatch rather than the
+treatment.* Four-slice reporting did not catch it, because the slicing was
+right and the **baseline** was wrong.
+
+Three matched controls were then trained from scratch - identical recipe,
+seeds 3407 / 1234 / 2024, nothing else varied:
+
+| control | MEAN4 @1124 |
+|---|---|
+| seed 3407 | 75.71 |
+| seed 1234 | 75.34 |
+| seed 2024 | 75.50 |
+| **mean** | **75.52** (sd 0.19) |
+
+Those three runs also give, for the first time, a **per-slice** noise floor -
+which turns out to matter more than the aggregate one:
+
+| slice | control mean | sd |
+|---|---|---|
+| VQA_DM | 72.82 | **0.88** |
+| VQA_RSICD | 70.15 | **0.74** |
+| CAP_DM | 81.06 | 0.25 |
+| CAP_RSICD | 78.06 | 0.52 |
+
+**The two VQA slices are three to four times noisier than the caption
+slices.** MEAN4 averages that down to 0.19, which is why aggregate deltas
+look more trustworthy than the slice deltas underneath them. Any single-slice
+VQA claim below about 1.8 (two sigma) is noise, and several claims made
+earlier in this project sit under that line.
+
+### Results
+
+All arms at the matched 1124 grid point, against control mean 75.52. Bands
+were pre-registered as WIN >= +0.76, SIGNAL +0.51..+0.76, NULL +/-0.51,
+HARM <= -0.51.
+
+| arm | axis tested | MEAN4 | delta | verdict |
+|---|---|---|---|---|
+| **control** | no treatment | **75.52** | - | baseline |
+| 115-para | question paraphrase augmentation | 75.25 | -0.27 | null |
+| 111-qa4 | 4 QA per image | 74.96 | -0.56 | harm |
+| 116-neg | hard negative examples | 74.86 | -0.66 | harm |
+| 113-frz | frozen vision tower | 74.78 | -0.74 | harm |
+| 119-teach | teacher-generated captions | 68.58 | -6.94 | severe harm |
+| 120-cur | caption -> VQA curriculum | 62.60 | -12.92 | catastrophic |
+
+One caveat on the 111 row, in the spirit of the retraction above: 111 is a
+**3-epoch** run, so its step-1124 checkpoint sits at a different point on the
+LR schedule than the 1-epoch control's does. That comparison is *not* clean,
+and the -0.56 should be read as indicative only. 111's clean result is its
+full-budget one, reported in the Round 1XX section: 76.62, a null against its
+own arm. Both readings agree that QA depth did not help, which is why the row
+is kept rather than dropped.
+
+**Nothing beat doing nothing.** Two arms are worth reading past the headline
+number, because their MEAN4 hides the mechanism. All slice deltas below are
+against the three-seed control mean, with that slice's sigma in brackets.
+
+**119-teach** replaced reference captions with captions generated by a strong
+general-purpose model. The damage is **concentrated in the caption slices**
+(CAP_RSICD **-16.11** [31 sigma], CAP_DM **-9.18** [37 sigma]) while the VQA
+slices lose only -1.85 and -0.63, both around two sigma or less. The teacher
+writes fluent generic prose and overwrites domain-specific remote-sensing
+vocabulary; a judge scoring against domain references punishes exactly that.
+Distillation fails when the teacher lacks the dataset's domain knowledge - it
+is not a statement about distillation in general, and it does not condemn
+115, which only rephrases *questions* and leaves reference captions intact.
+
+**120-cur** trained captions first, then VQA only. It produced **the best
+VQA_DM in the entire round - 75.48, or +2.66 (3.0 sigma)** - and
+simultaneously destroyed captioning (CAP_RSICD 78.06 -> **35.55**, a 42-point
+collapse). This is the round's strongest evidence that **the joint objective
+is load-bearing**: captioning is not a free passenger on a VQA model, it is
+actively maintained by the mixed loss, and it unlearns within one phase of
+being dropped. It is also the only slice-level *gain* in the round that
+survives its own noise floor - at a price nobody would pay.
+
+**116-neg** was predicted to buy VQA assertiveness with hard negatives. It did
+the opposite: **VQA_DM -1.97**, VQA_RSICD -0.51, CAP_DM -0.56, with only
+CAP_RSICD nominally positive at +0.38 (well inside its 0.52 sigma, so: no
+effect). The predicted mechanism is refuted rather than merely unproven.
+
+**115-para**, the retracted arm, is worth one more line. Re-measured properly
+it shows **VQA_DM +1.00** - but VQA_DM's sigma is 0.88, so that is 1.1 sigma,
+and the other three slices are negative. There is no paraphrase effect here
+in either direction. The original **+2.32** was baseline mismatch almost
+end to end.
+
+### Two axes that could not be tested
+
+**Decoding (112).** Greedy, beam-4, temperature 0.7 and repetition-penalty
+1.1 were run against the same checkpoint: **76.72 / 76.42 / 75.58 / 75.52**.
+Greedy wins. The axis is closed, not deferred.
+
+**Resolution (114).** This one is a negative result about the *corpus*, not
+the model. The run was configured with `--max-side 1536` and trained
+**faster** than baseline, which is impossible if images were being upscaled.
+Measuring 600 images directly: **none exceed 1024 px** - they are 224x224
+(71%), 896x896 (27%) and 768x768 (2%) - and `--max-side` only ever
+downscales. Raising it above the 1024 default is a **guaranteed no-op**. A
+real resolution test on this data has to lower the cap, not raise it.
+
+The run was allowed to finish anyway, because its configuration made it the
+missing SBP-native control - and that accident paid for itself twice over.
+
+### Hyperparameters
+
+Every arm in this round shares one SBP recipe and changes **exactly one
+thing**. The shared recipe:
+
+| field | value |
+|---|---|
+| base | `Qwen/Qwen3-VL-8B-Instruct` (bf16, not 4-bit) |
+| LoRA r / alpha / dropout | 16 / 16 / 0.0 |
+| vision tower | trained |
+| epochs | 1.0 |
+| lr / scheduler / warmup | 2e-4 / linear / 0.03 |
+| batch x grad-accum | 4 x 8 (effective 32) |
+| weight decay / grad clip | 0.001 / 0.3 |
+| max seq length | 2048 |
+| max side | 1024 |
+| split | `train,other` (11,994 images -> 35,979 items) |
+| per image | 2 QA + 1 caption |
+| loss weighting | token, caption and VQA both 1.0 |
+| checkpoints | step 562 and step 1124 |
+
+And the single change per arm:
+
+| arm | the one change |
+|---|---|
+| control | none (seeds 3407 / 1234 / 2024) |
+| 113-frz | `finetune_vision = 0` |
+| 115-para | source CSV -> paraphrased questions (35,981 items) |
+| 116-neg | source CSV -> hard negatives added (37,178 items) |
+| 119-teach | source CSV -> teacher captions (35,980 items) |
+| 120-cur | `captions_per_image = 0`, initialised from the caption-phase adapter |
+| 111-qa4 | `qa_per_image = 4`, `captions_per_image = 2`, 3 epochs (full budget, not SBP) |
+| 112-dec | no training - decoding sweep on an existing checkpoint |
+| 114-res | `max_seq_length = 3072`, `max_side = 1536` (both no-ops; see below) |
+
+**Only the best checkpoint of each run is reported**, here and in the table
+above. Intermediate checkpoints exist but are not quoted, because picking the
+best of several grid points and then quoting it as a result is the same
+selection bias the sigma bands exist to control - the bar was raised
+accordingly rather than the practice hidden.
+
+### Sigma, corrected again
+
+The resolution no-op generated the same checkpoint twice under two different
+flags, giving two independent generate-and-judge passes over effectively
+identical weights: **75.66 vs 75.67**, every slice within 0.02.
+
+**Judge reproducibility is therefore ~0.01 MEAN4 - effectively
+deterministic.** That *refutes* an earlier hypothesis on this page that
+sigma 0.31 might be partly judge drift across invocations. It is not. Cross
+-invocation comparisons in the results table are safe.
+
+A second accident sharpened it further. One control run was killed by an
+out-of-memory event at 87% - a neighbouring job on the shared A100 took
+20 GB - and was re-run from scratch at the **same seed**. The two step-562
+checkpoints, same seed and same data order, score **74.36 and 74.66**: a
+**0.30** spread with the seed held fixed.
+
+So sigma is not mostly *seed* variance, as assumed after the previous
+correction. A large share is **irreducible run-to-run nondeterminism** from
+non-deterministic CUDA kernels. Two practical consequences:
+
+- The 0.31 band stands, and is if anything well calibrated.
+- **Exact replication is unattainable even holding the seed.** No delta under
+  roughly 0.5 should be believed from a single pair of runs, however carefully
+  the configuration is matched.
+
+The `max-seq-length` confound inherited from the old control was also settled
+empirically: 2048 vs 3072 at the same seed gives **75.71 vs 75.66**, a 0.05
+difference. Inert, as the token arithmetic predicted - an 896 px image is
+about 784 vision tokens, so a 2048 budget is rarely the binding constraint.
+
+### What is published from this round, and why there are no new adapters
+
+**Nothing is published, and no adapter repo was created for any run in this
+round** - not for 112-120, and not for the three controls.
+
+The reasoning is the same one applied to 106b-110, stated again because it
+keeps being asked: an adapter repo on this org is a **claim that a
+configuration is worth using**. Every arm here is either a *refuted*
+configuration (113, 115, 116, 119, 120, 111), a *calibration* run whose entire
+purpose is to measure noise (the three controls), or a *no-op* that trained
+the baseline recipe under a differently-named flag (114). Publishing weights
+for a refuted recipe would make the org a dump of artefacts rather than a
+record of what is known, and a user downloading 119 would get a model
+measurably worse at captioning than the base it came from.
+
+There is also a standing rule that no result goes to the hub off a one-epoch
+probe. A win in the SBP grid earns a **full-budget confirmation run** first.
+No arm produced a win, so the question never arose.
+
+The round's deliverables are therefore a replicated control (75.52 +/- 0.19),
+a per-slice noise floor, six closed axes, a corrected sigma, one retracted
+result - and roughly 30 GPU-hours spent to establish that none of it should
+be shipped. The findings are the product; the weights would have been the
+liability.
+
+### Status after Round 11X
+
+- **Nothing on this org changed.** Aspaklarya-101-step4500 (76.37 +/- 0.23
+  across seeds) remains the in-domain reference; Aspaklarya-020-step4500
+  remains the better general model; Aspaklarya-106-soup remains the cheap
+  default. Round 11X added no model and removed none.
+- **One confirmed lever, total, across both rounds: data scale.** Nine other
+  axes have now been tested and closed - rank, loss weighting, decoding,
+  vision-tower freezing, resolution, QA depth, paraphrase augmentation, hard
+  negatives, teacher distillation and curriculum ordering.
+- **The measurement apparatus is the part that improved.** Round 11X ends
+  with a three-seed control, a per-slice noise floor, judge reproducibility
+  pinned at ~0.01, and a demonstration that same-seed reruns still differ by
+  0.30. Future deltas on this page are readable in a way earlier ones were
+  not.
+- **Two results elsewhere on this page are retracted or corrected by this
+  round** - the paraphrase gain, and the judge-drift explanation of sigma.
+  Both are left visible above rather than quietly edited out.

index.html

@@ -430,7 +430,8 @@ a { color: #0969da; }
 <p>108, 109 and 110 are not new recipes. <strong>They are the recipes of 102 and 101 re-run with a different random seed and nothing else changed.</strong> Their adapters are therefore near-duplicates of adapters already published: same corpus, same rank, same schedule, same everything the config file records. What they produced is not a model, it is <strong>a number</strong> - sigma 0.31 - and a number belongs in the text, where it now is.</p>
 <p>Publishing them would also do active harm. Six more checkpoints spanning 75.22-76.56 would sit in the listing looking like six more options to choose between, and the obvious way to choose would be to take the highest one. That is exactly the mistake the sigma measurement exists to prevent: within an arm, the spread between seeds <strong>is</strong> the noise, so picking the best seed is selecting on noise and will not reproduce. The honest artifact is the published 101 and 102 adapters plus a stated uncertainty on what they score, not a shelf of lottery tickets.</p>
 <p>The cost argument is secondary but real: each adapter is ~821 MB, so the five unpublished runs would add roughly 5 GB of weights that no one should load in preference to what is already here.</p>
-<p><strong>Aspaklarya-111 is still training.</strong> It will be published if it clears the <code>train,other</code> arm it is built on - meaning a MEAN4 above roughly 76.8, which is the arm mean plus two sigma. If it lands inside the arm it will be reported here as another null and left unpublished, on the same reasoning as 108-110.</p>
+<p><strong>Aspaklarya-111 has finished, and it is a null.</strong> Its best checkpoint (step 4496) scores <strong>76.62</strong>, against the pre-registered bar of roughly 76.8 - the <code>train,other</code> arm mean plus two sigma. It lands <em>inside</em> the arm it was built on (76.37 +/- 0.23, so +0.25 is about 1.1 sigma), which is exactly the case the bar was written to catch. It is therefore <strong>not published</strong>, on the same reasoning as 108-110.</p>
+<p>That is worth stating plainly because 76.62 is the second-highest single number on this page, and the temptation to ship it is precisely the error the sigma measurement exists to prevent. Raising QA depth from 2 to 4 per image did not extend the one confirmed lever.</p>
 <h3>Caveat</h3>
 <p>The judge is <strong>text-only</strong>. It compares an answer against a reference and never sees the image, so it measures agreement with the reference, not verified truth. For a round whose whole premise is longer answers this matters more than it did in round 0XX: the judge can confirm that a model supplies reference-matching content, but not that the content is true of the pixels. Human spot-checks are the only way to close that gap.</p>
 <p>The legacy slices also reward a terse style the product no longer wants - the legacy VQA rubric caps a padded-but-correct answer at 89/100 by construction, so a 290-character model cannot reach the top band however accurate it is. That is why the legacy numbers are reported but never used to gate a decision.</p>
@@ -452,9 +453,133 @@ a { color: #0969da; }
 </ul>
 <p>  <em>What is published here, and what is not</em> above - they are either strictly   dominated, refuted, or seed replications whose output is a sigma rather   than a model.</p>
 <ul>
-<li>Open: whether QA depth (111, running) extends the one confirmed lever, and</li>
+<li><strong>QA depth is answered: it does not help.</strong> 111 raised QA per image from 2</li>
 </ul>
-<p>  whether the <code>other</code> split helped by adding <strong>images</strong> or merely <strong>items</strong> -   111 isolates items, since it adds no new images.</p>
+<p>  to 4 and landed inside its own arm. Since 111 adds items but no new images,   the <code>train,other</code> gain is better explained by the <strong>images</strong> it brings than   by raw item count.</p>
+<h2>Round 11X - six axes against a replicated control</h2>
+<p>Round 1XX ended with one confirmed lever (data scaling) and a warning that single-run deltas below about 1.0 MEAN4 are unreadable. Round 11X was built to test six <em>new</em> axes cheaply, under a short-budget probe (SBP) design: one epoch, checkpoints at steps 562 and 1124, every checkpoint judged on all four slices. Ten jobs ran unattended over three days.</p>
+<p><strong>No axis produced a win.</strong> That is the headline, and it is not a failed round - six levers were eliminated, and the round also corrected two errors in how this project measures things.</p>
+<h3>The control had to be rebuilt first</h3>
+<p>The SBP arms were initially compared against a single control checkpoint that ran a <strong>different step grid</strong> (590/1180 rather than 562/1124) and a different <code>max-seq-length</code>. That mismatch produced a reported <strong>+2.32</strong> VQA_DM gain for the paraphrase arm which <strong>does not exist</strong>; re-measured against the matched three-seed control the same arm is <strong>+1.00</strong>, against a VQA_DM sigma of 0.88. The claim is retracted here in full, and so is the recommendation to paraphrase RSICD questions that was built on it.</p>
+<p>The lesson generalises past this round: <em>a control must match the treatment on grid, split and budget, or the delta measures the mismatch rather than the treatment.</em> Four-slice reporting did not catch it, because the slicing was right and the <strong>baseline</strong> was wrong.</p>
+<p>Three matched controls were then trained from scratch - identical recipe, seeds 3407 / 1234 / 2024, nothing else varied:</p>
+<table style="border-collapse:collapse;margin:1rem 0;font-size:.85rem;border:1px solid #d0d7de"><thead><tr>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">control</th>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">MEAN4 @1124</th>
+</tr></thead><tbody>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">seed 3407</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.71</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">seed 1234</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.34</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">seed 2024</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.50</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>mean</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>75.52</strong> (sd 0.19)</td></tr>
+</tbody></table>
+<p>Those three runs also give, for the first time, a <strong>per-slice</strong> noise floor - which turns out to matter more than the aggregate one:</p>
+<table style="border-collapse:collapse;margin:1rem 0;font-size:.85rem;border:1px solid #d0d7de"><thead><tr>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">slice</th>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">control mean</th>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">sd</th>
+</tr></thead><tbody>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">VQA_DM</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">72.82</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>0.88</strong></td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">VQA_RSICD</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">70.15</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>0.74</strong></td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">CAP_DM</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.06</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.25</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">CAP_RSICD</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.06</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.52</td></tr>
+</tbody></table>
+<p><strong>The two VQA slices are three to four times noisier than the caption slices.</strong> MEAN4 averages that down to 0.19, which is why aggregate deltas look more trustworthy than the slice deltas underneath them. Any single-slice VQA claim below about 1.8 (two sigma) is noise, and several claims made earlier in this project sit under that line.</p>
+<h3>Results</h3>
+<p>All arms at the matched 1124 grid point, against control mean 75.52. Bands were pre-registered as WIN &gt;= +0.76, SIGNAL +0.51..+0.76, NULL +/-0.51, HARM &lt;= -0.51.</p>
+<table style="border-collapse:collapse;margin:1rem 0;font-size:.85rem;border:1px solid #d0d7de"><thead><tr>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">arm</th>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">axis tested</th>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">MEAN4</th>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">delta</th>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">verdict</th>
+</tr></thead><tbody>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>control</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">no treatment</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>75.52</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">-</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">baseline</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">115-para</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">question paraphrase augmentation</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.25</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">-0.27</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">null</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">111-qa4</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">4 QA per image</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">74.96</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">-0.56</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">harm</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">116-neg</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">hard negative examples</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">74.86</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">-0.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">harm</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">113-frz</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">frozen vision tower</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">74.78</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">-0.74</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">harm</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">119-teach</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">teacher-generated captions</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">68.58</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">-6.94</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">severe harm</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">120-cur</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">caption -&gt; VQA curriculum</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">62.60</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">-12.92</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">catastrophic</td></tr>
+</tbody></table>
+<p>One caveat on the 111 row, in the spirit of the retraction above: 111 is a <strong>3-epoch</strong> run, so its step-1124 checkpoint sits at a different point on the LR schedule than the 1-epoch control&#x27;s does. That comparison is <em>not</em> clean, and the -0.56 should be read as indicative only. 111&#x27;s clean result is its full-budget one, reported in the Round 1XX section: 76.62, a null against its own arm. Both readings agree that QA depth did not help, which is why the row is kept rather than dropped.</p>
+<p><strong>Nothing beat doing nothing.</strong> Two arms are worth reading past the headline number, because their MEAN4 hides the mechanism. All slice deltas below are against the three-seed control mean, with that slice&#x27;s sigma in brackets.</p>
+<p><strong>119-teach</strong> replaced reference captions with captions generated by a strong general-purpose model. The damage is <strong>concentrated in the caption slices</strong> (CAP_RSICD <strong>-16.11</strong> [31 sigma], CAP_DM <strong>-9.18</strong> [37 sigma]) while the VQA slices lose only -1.85 and -0.63, both around two sigma or less. The teacher writes fluent generic prose and overwrites domain-specific remote-sensing vocabulary; a judge scoring against domain references punishes exactly that. Distillation fails when the teacher lacks the dataset&#x27;s domain knowledge - it is not a statement about distillation in general, and it does not condemn 115, which only rephrases <em>questions</em> and leaves reference captions intact.</p>
+<p><strong>120-cur</strong> trained captions first, then VQA only. It produced <strong>the best VQA_DM in the entire round - 75.48, or +2.66 (3.0 sigma)</strong> - and simultaneously destroyed captioning (CAP_RSICD 78.06 -&gt; <strong>35.55</strong>, a 42-point collapse). This is the round&#x27;s strongest evidence that <strong>the joint objective is load-bearing</strong>: captioning is not a free passenger on a VQA model, it is actively maintained by the mixed loss, and it unlearns within one phase of being dropped. It is also the only slice-level <em>gain</em> in the round that survives its own noise floor - at a price nobody would pay.</p>
+<p><strong>116-neg</strong> was predicted to buy VQA assertiveness with hard negatives. It did the opposite: <strong>VQA_DM -1.97</strong>, VQA_RSICD -0.51, CAP_DM -0.56, with only CAP_RSICD nominally positive at +0.38 (well inside its 0.52 sigma, so: no effect). The predicted mechanism is refuted rather than merely unproven.</p>
+<p><strong>115-para</strong>, the retracted arm, is worth one more line. Re-measured properly it shows <strong>VQA_DM +1.00</strong> - but VQA_DM&#x27;s sigma is 0.88, so that is 1.1 sigma, and the other three slices are negative. There is no paraphrase effect here in either direction. The original <strong>+2.32</strong> was baseline mismatch almost end to end.</p>
+<h3>Two axes that could not be tested</h3>
+<p><strong>Decoding (112).</strong> Greedy, beam-4, temperature 0.7 and repetition-penalty 1.1 were run against the same checkpoint: <strong>76.72 / 76.42 / 75.58 / 75.52</strong>. Greedy wins. The axis is closed, not deferred.</p>
+<p><strong>Resolution (114).</strong> This one is a negative result about the <em>corpus</em>, not the model. The run was configured with <code>--max-side 1536</code> and trained <strong>faster</strong> than baseline, which is impossible if images were being upscaled. Measuring 600 images directly: <strong>none exceed 1024 px</strong> - they are 224x224 (71%), 896x896 (27%) and 768x768 (2%) - and <code>--max-side</code> only ever downscales. Raising it above the 1024 default is a <strong>guaranteed no-op</strong>. A real resolution test on this data has to lower the cap, not raise it.</p>
+<p>The run was allowed to finish anyway, because its configuration made it the missing SBP-native control - and that accident paid for itself twice over.</p>
+<h3>Hyperparameters</h3>
+<p>Every arm in this round shares one SBP recipe and changes <strong>exactly one thing</strong>. The shared recipe:</p>
+<table style="border-collapse:collapse;margin:1rem 0;font-size:.85rem;border:1px solid #d0d7de"><thead><tr>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">field</th>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">value</th>
+</tr></thead><tbody>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">base</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><code>Qwen/Qwen3-VL-8B-Instruct</code> (bf16, not 4-bit)</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">LoRA r / alpha / dropout</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16 / 0.0</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">vision tower</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">epochs</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1.0</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">lr / scheduler / warmup</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4 / linear / 0.03</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">batch x grad-accum</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">4 x 8 (effective 32)</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">weight decay / grad clip</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.001 / 0.3</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">max seq length</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2048</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">max side</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1024</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">split</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><code>train,other</code> (11,994 images -&gt; 35,979 items)</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">per image</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2 QA + 1 caption</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">loss weighting</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">token, caption and VQA both 1.0</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">checkpoints</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">step 562 and step 1124</td></tr>
+</tbody></table>
+<p>And the single change per arm:</p>
+<table style="border-collapse:collapse;margin:1rem 0;font-size:.85rem;border:1px solid #d0d7de"><thead><tr>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">arm</th>
+<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">the one change</th>
+</tr></thead><tbody>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">control</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">none (seeds 3407 / 1234 / 2024)</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">113-frz</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><code>finetune_vision = 0</code></td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">115-para</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">source CSV -&gt; paraphrased questions (35,981 items)</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">116-neg</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">source CSV -&gt; hard negatives added (37,178 items)</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">119-teach</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">source CSV -&gt; teacher captions (35,980 items)</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">120-cur</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><code>captions_per_image = 0</code>, initialised from the caption-phase adapter</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">111-qa4</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><code>qa_per_image = 4</code>, <code>captions_per_image = 2</code>, 3 epochs (full budget, not SBP)</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">112-dec</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">no training - decoding sweep on an existing checkpoint</td></tr>
+<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">114-res</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><code>max_seq_length = 3072</code>, <code>max_side = 1536</code> (both no-ops; see below)</td></tr>
+</tbody></table>
+<p><strong>Only the best checkpoint of each run is reported</strong>, here and in the table above. Intermediate checkpoints exist but are not quoted, because picking the best of several grid points and then quoting it as a result is the same selection bias the sigma bands exist to control - the bar was raised accordingly rather than the practice hidden.</p>
+<h3>Sigma, corrected again</h3>
+<p>The resolution no-op generated the same checkpoint twice under two different flags, giving two independent generate-and-judge passes over effectively identical weights: <strong>75.66 vs 75.67</strong>, every slice within 0.02.</p>
+<p><strong>Judge reproducibility is therefore ~0.01 MEAN4 - effectively deterministic.</strong> That <em>refutes</em> an earlier hypothesis on this page that sigma 0.31 might be partly judge drift across invocations. It is not. Cross -invocation comparisons in the results table are safe.</p>
+<p>A second accident sharpened it further. One control run was killed by an out-of-memory event at 87% - a neighbouring job on the shared A100 took 20 GB - and was re-run from scratch at the <strong>same seed</strong>. The two step-562 checkpoints, same seed and same data order, score <strong>74.36 and 74.66</strong>: a <strong>0.30</strong> spread with the seed held fixed.</p>
+<p>So sigma is not mostly <em>seed</em> variance, as assumed after the previous correction. A large share is <strong>irreducible run-to-run nondeterminism</strong> from non-deterministic CUDA kernels. Two practical consequences:</p>
+<ul>
+<li>The 0.31 band stands, and is if anything well calibrated.</li>
+<li><strong>Exact replication is unattainable even holding the seed.</strong> No delta under</li>
+</ul>
+<p>  roughly 0.5 should be believed from a single pair of runs, however carefully   the configuration is matched.</p>
+<p>The <code>max-seq-length</code> confound inherited from the old control was also settled empirically: 2048 vs 3072 at the same seed gives <strong>75.71 vs 75.66</strong>, a 0.05 difference. Inert, as the token arithmetic predicted - an 896 px image is about 784 vision tokens, so a 2048 budget is rarely the binding constraint.</p>
+<h3>What is published from this round, and why there are no new adapters</h3>
+<p><strong>Nothing is published, and no adapter repo was created for any run in this round</strong> - not for 112-120, and not for the three controls.</p>
+<p>The reasoning is the same one applied to 106b-110, stated again because it keeps being asked: an adapter repo on this org is a <strong>claim that a configuration is worth using</strong>. Every arm here is either a <em>refuted</em> configuration (113, 115, 116, 119, 120, 111), a <em>calibration</em> run whose entire purpose is to measure noise (the three controls), or a <em>no-op</em> that trained the baseline recipe under a differently-named flag (114). Publishing weights for a refuted recipe would make the org a dump of artefacts rather than a record of what is known, and a user downloading 119 would get a model measurably worse at captioning than the base it came from.</p>
+<p>There is also a standing rule that no result goes to the hub off a one-epoch probe. A win in the SBP grid earns a <strong>full-budget confirmation run</strong> first. No arm produced a win, so the question never arose.</p>
+<p>The round&#x27;s deliverables are therefore a replicated control (75.52 +/- 0.19), a per-slice noise floor, six closed axes, a corrected sigma, one retracted result - and roughly 30 GPU-hours spent to establish that none of it should be shipped. The findings are the product; the weights would have been the liability.</p>
+<h3>Status after Round 11X</h3>
+<ul>
+<li><strong>Nothing on this org changed.</strong> Aspaklarya-101-step4500 (76.37 +/- 0.23</li>
+</ul>
+<p>  across seeds) remains the in-domain reference; Aspaklarya-020-step4500   remains the better general model; Aspaklarya-106-soup remains the cheap   default. Round 11X added no model and removed none.</p>
+<ul>
+<li><strong>One confirmed lever, total, across both rounds: data scale.</strong> Nine other</li>
+</ul>
+<p>  axes have now been tested and closed - rank, loss weighting, decoding,   vision-tower freezing, resolution, QA depth, paraphrase augmentation, hard   negatives, teacher distillation and curriculum ordering.</p>
+<ul>
+<li><strong>The measurement apparatus is the part that improved.</strong> Round 11X ends</li>
+</ul>
+<p>  with a three-seed control, a per-slice noise floor, judge reproducibility   pinned at ~0.01, and a demonstration that same-seed reruns still differ by   0.30. Future deltas on this page are readable in a way earlier ones were   not.</p>
+<ul>
+<li>**Two results elsewhere on this page are retracted or corrected by this</li>
+</ul>
+<p>  round** - the paraphrase gain, and the judge-drift explanation of sigma.   Both are left visible above rather than quietly edited out.</p>
 </main>
 </body>
 </html>