Aspaklarya - Qwen3-VL-8B for remote-sensing VQA + captioning

LoRA finetunes of Qwen3-VL-8B-Instruct, trained jointly on visual question answering and image captioning over overhead imagery. All model repos are private.

Start here

Nothing has beaten 101-step4500 since it was trained. Three rounds and roughly 30 experiments later it is still the reference, and this page is mostly a record of what does not work. Two published results have been retracted after control errors were found; both retractions are kept visible below rather than edited away.

The programme has run in three rounds. Rounds 0XX and 1XX trained against two different answer styles and are not scored on the same VQA benchmark; round 11X is a set of probes against a replicated control.

Round 11X is at the bottom of this page and is the most recent work. If you only read one section, read that one: it contains the sigma measurement that determines which differences on this page are real.

Both rounds are now scored on the same four-slice benchmark, so the MEAN4 column below is comparable across rounds. The legacy and new VQA columns individually are not: The two benchmarks reward opposite answer styles; see Why the legacy score drops below.

The four tables

Everything measured in this programme is in the four tables below. The sections after them are the reasoning, the controls and the caveats; nothing in those sections introduces a number that is not here.

Two rounds, because they answer different questions and are not comparable to each other: round 0XX trained on the one-word VQA column and is scored on terse exact answers, round 1XX trained on the verbose grounded VQA_1126 column. Aspaklarya-020 and Aspaklarya-101 are identical in every hyperparameter - the whole gap between the rounds is answer style, not recipe.

Table 1 - exp0XX settings

Constant everywhere: effective batch 32 (except 014 at 16, 015-017 at 64), warmup 0.03, grad clip 0.3 (1.0 for 011/012), seed 3407. 4-bit base for runs 001-017, bf16 base from 018 on.

Table 2 - exp0XX evaluation results

Best checkpoint per run on a fixed 1000-image draw (2000 graded items), with bootstrap standard error. Overall = mean of caption and VQA. In-dom. = the target overhead-imagery subset (491 images); RSICD = the public RSICD remote-sensing benchmark subset (509 images). Sorted by in-domain overall; bold marks the best value in each column.

Every finetune beats both references overall. The selected model, Aspaklarya-020, is +17.9 over the base model and +9.4 over the in-house reference on the full set.

Table 3 - exp1XX settings

One row per experiment, stating only what that experiment changed. Rounds 1XX and 11X share a scoring harness and are directly comparable; the 11X runs are 1-epoch screens against a replicated 1-epoch control, the 1XX runs are full 4-epoch budgets.

Constant across every row above: dropout 0, bf16 base (no 4-bit), effective batch 32 (4 x 8), warmup 0.03, weight decay 0.001, grad clip 0.3, max side 1024, seq len 2048, seed 3407 unless the row says otherwise, LoRA on all linear layers of both towers, VQA_1126 answer column.

Table 4 - exp1XX evaluation results

Best checkpoint per experiment only. Four slices, 500 graded items each (491 for caption in-domain), one fixed draw, judged by gemma-4-31b-it-imiai. MEAN4 is the unweighted mean of the four. The last column is the gap to the 3-seed 1-epoch control mean of 75.52 (replicates 75.71 / 75.34 / 75.50), which is the same reference the individual model cards quote.

The noise floor is sigma = 0.31 MEAN4, measured from three same-config replicates, so a difference under +-0.51 is not a result. Per slice, sigma is VQA in-dom. 0.88, VQA RSICD 0.74, cap in-dom. 0.25, cap RSICD 0.52 - the VQA slices are 3-4x noisier than the caption slices, which is why MEAN4 rather than any single slice is the gate.

Read the bottom of this table as carefully as the top. Aspaklarya-120 B is the worst MEAN4 in the programme and simultaneously holds the best VQA in-domain score in the programme (75.48, +2.66 over control, 3.0 sigma): it dropped captioning entirely and its caption RSICD collapsed 42 points. That is a direct measurement of the two objectives competing for LoRA capacity, and it is the only effect in round 11X that clears its own noise floor. It is what Aspaklarya-121 is being run to exploit.

Round 0XX - terse answers (Aspaklarya-001 ... 021)

The original 21-experiment programme, trained on the one-word VQA column and scored on the 1,000-image union draw. Everything below this heading refers to that round only.

Setup

Winners are merged into the bf16 base and re-validated. For runs 001-017 that merge also removes a train/serve mismatch, since those adapters were fitted on a 4-bit base but are served from bf16 weights; runs 018 onward have no such mismatch.

Configurations

Moved to Table 1 at the top of this page.

Constant everywhere: effective batch 32 (except 014 at 16, 015-017 at 64), warmup 0.03, grad clip 0.3 (1.0 for 011/012), seed 3407.

Results

Best checkpoint per run on a fixed 1000-image draw (2000 graded items), with bootstrap standard error. Overall = mean of caption and VQA. In-dom. = the target overhead-imagery subset (491 images); RSICD = the public RSICD remote-sensing benchmark subset (509 images). Sorted by in-domain overall; bold marks the best value in each column.

Moved to Table 2 at the top of this page.

Every finetune beats both references overall. The selected model, Aspaklarya-020, is +17.9 over the base model and +9.4 over the in-house reference on the full set.

Learning rate at large batch

Aspaklarya-015 gave the best captioning in the table at effective batch 64, but it was trained at 2e-4 — the rate tuned for batch 32. Two follow-ups tested whether the batch-64 recipe was simply under-stepped. Scores below come from a separate 500-image draw used for live checkpoint selection, so they compare only with each other.

Linear LR scaling is wrong for LoRA here: 4e-4 peaked at epoch 1 and then lost ground at every later checkpoint, ending below where 2e-4 started. The adapter's effective step size does not follow the full-finetune scaling rule, so raising LR with batch size overshoots rather than compensating. The intermediate rate 3e-4 traced almost exactly the same curve as 2e-4 and peaked at the same epoch, one tenth of a point lower.

On the shared 1000-image set, neither batch-64 rate beat the reference: 2e-4 is -0.56 overall and -0.09 in-domain, 3e-4 is -0.34 and -0.87. Effective batch 64 is therefore refuted at every learning rate tried, and the loss is consistently in VQA while captioning holds or improves. Together with the batch-16 result this closes the batch-size and learning-rate axes: no hyperparameter lever remains untested, and further gains have to come from the training data.

Error bars

Bootstrap over images, 3000 resamples. Absolute standard error:

Paired model-vs-model error is smaller because image difficulty cancels: 0.42 overall on the full set, 0.56 in-domain. Use the paired figure for any "A beats B" claim.

Applied exhaustively to the five leading checkpoints (020, 018, 019, 001, 011), the paired bootstrap separates 0 of 10 pairs overall and 0 of 10 in-domain at 95% — in-domain every pair sits at p ≥ 0.67. The only significant pairs anywhere are three on the RSICD slice, all of them 020 against another model. The ordering of the results table below the top line is not a measurement.

Caveat

The hyperparameter runs are statistically tied with each other: among runs 001-019, every confidence interval against the leader straddles zero. VQA error is ~2.3x caption error, so no finetune-vs-finetune VQA difference in any column is resolvable; only caption gaps, the gaps to the two references, and the data-scaling result of run 020 are real. Doubling the eval set from 500 to 1000 images did not separate the tied field, and it reshuffled its order.

Findings

• Hyperparameters are saturated; data is not. LR, rank, alpha, dropout, weight decay, gradient clipping, batch geometry, effective batch size, LR scaling at large batch and base-weight precision were all swept without beating the noise floor. Adding 91% more training images cleared it on the first attempt.
• The slices disagree — but data breaks the trade. Ranking by in-domain score almost inverts the RSICD ranking, and every hyperparameter lever that moved one slice moved the other the opposite way. Extra data is the exception: it lifted RSICD by 1.57 while leaving in-domain unchanged.
• Captioning is the real gain. It improves on every slice, by +14 over the base model and +10 to +19 over the in-house reference.
• The in-domain VQA win is not established. The in-house reference posts the single highest in-domain VQA score in the table; it collapses outside that domain, where it falls below the untuned base model.
• The vision tower matters. Freezing it is the one ablation that consistently hurt.
• Extra QA per image does not help. 4-QA and 6-QA mixtures tie with 2-QA.
• Larger batch trades generality for domain fit. Effective batch 64 gives the best in-domain captioning in the table and the worst RSICD VQA.
• Training in bf16 instead of 4-bit is a small, consistent positive — the only change in the whole sweep that never loses on any slice — but it is still inside the error bar.
• Data is the remaining lever, not hyperparameters — and it is not exhausted: the run that used it had not converged when its schedule ended.

Base-weight precision

Runs 001-017 all trained their adapters against 4-bit quantized base weights, while evaluation and the final merge both use the bf16 base. That is a train/serve mismatch: the adapter was fitted to compensate for quantization error that is absent at inference time.

Runs 018 and 019 repeat the two strongest recipes with the base loaded in bf16 and nothing else changed. Against the reference, paired on 1000 images:

These are the first two runs in the sweep to post a positive overall delta, and 018 is the only run that is not worse than the reference on any slice. Both are still well inside the paired error bar (p = 0.34 that each is in fact worse), so neither is promoted.

The mechanism is not the expected one. Removing quantization error should mostly help the harder task, VQA; instead the gain is entirely in captioning (+0.48 and +0.35) while VQA is flat to slightly negative. That is the same caption-up / VQA-flat signature produced by every other lever in the sweep, which suggests precision is acting through the same saturated channel rather than unlocking anything new.

Removing quantization also costs ~9 GB of GPU memory during training (19 GB vs 10 GB), which is a real constraint on a shared 40 GB card.

Training data

Every run up to 017 trained on 6,288 images / 18,864 samples. The source table also carries 5,706 further images that the dataset author left unassigned (Notes == other, RSICD only). They are fully populated, statistically indistinguishable from the assigned training rows, and have zero image-name overlap with either the train or the validation split, so they are safe to train on.

Aspaklarya-020 adds them, taking training data to 11,994 images / 35,981 samples (+91%). Everything else is identical to 018. Paired against the previous selection on the same 1000 images:

This is the first change in the whole programme to clear the error bar. The RSICD interval excludes zero, and the overall caption gain is 3.2 standard errors. It is also the first result that does not trade one slice against the other: in-domain is a clean tie while the general slice moves.

Two further points matter for anyone repeating this:

• The in-domain penalty is transient. At matched optimizer steps the extra data looked actively harmful early — step 2,360 was 1.65 behind on in-domain — and that deficit had vanished by step 4,500. Stopping early would have refuted the best run in the programme.
• The climb to the final checkpoint was schedule, not headroom. Overall score rose at all seven checkpoints including the last, which suggested an unexhausted budget. Aspaklarya-021 tested that directly and refuted it — see below.

More data is therefore the live axis, and the only one still producing movement.

Schedule length

Aspaklarya-021 repeats 020 on the same enlarged corpus with 8 epochs instead of 4 (9,000 steps, checkpoints step-aligned with 020). It peaked at step 4,720 and then declined; the stop rule ended it at step 5,900 with 3,100 steps unspent.

Its best checkpoint, paired against 020 on the same 1000 images, is negative on every slice and significant on none: overall -0.53 ±0.43, in-domain -0.39 ±0.60, RSICD -0.66 ±0.64.

Data volume helped; data duration did not. Four epochs is already the right length at this corpus size.

The failure mode is asymmetric and worth recording: at step 5,900 captioning was intact (80.75) while VQA collapsed (76.52 → 74.45). Overtraining this recipe degrades VQA first, so a stop rule watching captioning alone would not catch it.

Status

The hyperparameter search is closed. Rank, alpha, dropout, learning rate, schedule, weight decay, gradient clipping, vision-tower freezing, QA-per-image, caption prompt pool, batch geometry, effective batch size, learning rate at large batch and base-weight precision were all swept across 19 recipes under identical conditions. None displaced the reference recipe by more than the paired error bar.

Batch size and learning rate illustrate that ceiling. Effective batch 16 came in at -0.59 overall, batch 64 at 2e-4 at -0.56, and batch 64 at 3e-4 at -0.34 overall and -0.87 in-domain; batch 64 at 4e-4 was stopped at epoch 4 while declining. In each case captioning held steady or improved while VQA fell.

Enlarging the training set did what none of those could, and training it longer did not extend the gain. That narrows the remaining question to data composition: the corpus added was RSICD-only, and the gain it produced is correspondingly RSICD-only while in-domain stays flat. Additional in-domain imagery is the targeted next step.

Round 1XX - verbose grounded answers (Aspaklarya-101 ... 111)

Why a second round

The 0XX round closed with hyperparameters saturated and data identified as the only live lever. It also left a product problem: every 0XX model answers in one word, because its supervision was one word. yes is a correct answer to "are there any anomalies in the scene?" and a useless one.

The dataset author then supplied a new column, VQA_1126, replacing the terse answers for the in-domain (DM) images with ~300-character grounded answers that name objects, locations and context. The RSICD rows are byte-identical to the old column; only the 5,148 DM rows changed, from a mean of 5.9 characters to ~304.

Configurations

All runs share one corpus recipe - VQA_1126, 1 caption + 2 QA per image - and one optimiser: bf16 base, seed 3407, lr 2e-4, linear schedule, warmup 0.03, weight decay 0.001, grad clip 0.3, effective batch 32 (4 x 8), 4 epochs, max seq 2048, vision tower trained. One variable each.

Hyperparameters in full

Every value actually passed to the trainer, read back from each run's run_config.json. Cells that differ from the Aspaklarya-102 reference are bold - that is the single variable the run was built to test.

Constant across the whole round and therefore omitted above: base model Qwen/Qwen3-VL-8B-Instruct, no 4-bit quantisation, VQA column VQA_1126, caption prompts drawn from the 50-query bank, eval-loss probe of 400 held-out items (200 caption / 200 VQA).

Aspaklarya-106 and 106b are not training runs. They are exact LoRA soups: four rank-16 adapters stacked into a single rank-64 adapter, which reproduces the arithmetic mean of the parent weight deltas to within 2.3e-9. 106 averages the epoch-3 parents, 106b the epoch-4 parents. Neither consumed GPU training time; both are listed here so their rank and alpha are not mistaken for a tuned choice.

Aspaklarya-108, 109 and 110 are replications, not new configurations. 108 re-runs 102 at two further seeds; 109 and 110 re-run 101 at one further seed each. They exist to measure the run-to-run spread that every comparison on this card silently assumed was small, and to test whether the one positive result survives a change of seed. They are measurements, not deliverables.

How these are scored

Round-1XX checkpoints were first ranked on the DM VQA slice alone - the slice the round was trained for. Every model in the programme has since been re-judged on one fixed sample drawn with seed 1105, drawn once outside the model loop so all 24 checkpoints see byte-identical items:

MEAN4 is the unweighted mean, so a model cannot win by specialising on one slice. Judge: gemma-4-31b-it-imiai, greedy, temperature 0, scored 0-100.

Two caveats on the shape of the benchmark. CAP DM carries 491 items, not 500, because the underlying 500-image sweep drew 491 DM / 509 RSICD. And RSICD VQA has no verbose rewrite - it is still the legacy question set generated at a 48-token budget, which clips verbose answers and penalises the 1XX models there.

Results

One row per experiment: its best checkpoint. Every model is scored on the same 1,991 judged items under the same rubric, so the column is comparable all the way down. The ckpts scored column says how many checkpoints that experiment contributed; the best of n is an order statistic, so it is biased upward by roughly 0.3-0.5 for n=4 and should not be read as that recipe's expected quality. Use it to rank recipes, not to size effects.

For that, and for every individual checkpoint, see Run-to-run sigma below and the full 36-row table in rescore_slices.csv. Seed replicates of one recipe (108, 109, 110) are pooled into a single row here.

Run-to-run sigma, measured

The previous version of this card said sigma had never been measured and that no one-point difference should be trusted. That has now been fixed, at the cost of four replication runs.

Aspaklarya-108 re-ran the 102 recipe at seeds 1234 and 2024, changing nothing else. Aspaklarya-109 and 110 did the same for the 101 recipe, at seeds 1234 and 2024. With the two original seed-3407 runs that gives two arms of six scored checkpoints each.

sigma = 0.31 MEAN4 for a single checkpoint (pooled, the wider arm). Two consequences, and they are not optional:

• A single run targeting less than about 1.0 point cannot be interpreted.
• The difference between two single checkpoints carries SE `0.31 x sqrt(2)

= 0.44`, so a gap needs roughly 0.9 points before it means anything, and 1.3 before it is comfortable.

Re-reading the whole round against that floor:

Only one row of that table survives. Everything else the round tested is noise at the resolution the budget bought - which is the honest reading, not a disappointing one: five levers were eliminated for 25 GPU-hours.

The two soup rows are worth reading together. 106 beat its parents' average and 106b lost to it, by similar amounts in opposite directions. The defensible claim is therefore that souping returns roughly the average of its parents for zero GPU time, not that it adds anything.

What the round actually bought

Data scaling is the only lever that survived replication. It is also the only one that was replicated - and it was replicated specifically because it was the only candidate worth the GPU time. Three independent seeds of the large-corpus recipe produced six checkpoints spanning 76.09-76.72; six small-corpus checkpoints span 75.22-76.04. The two distributions do not overlap. +0.83 at SE 0.16 is 5.2 SE.

Note what this costs in humility about the original claim: 101's headline was +1.07, measured from one seed against one seed. The replicated estimate is +0.83. The effect is real and the first number was 29% too generous - a clean illustration of why single-seed deltas are quoted here only with a sigma attached.

The hyperparameter contrasts are all unresolved or null. Rank, LoRA scale, objective balance and schedule shape each moved MEAN4 by less than 1.4 SE. None of them is worth a further run at this resolution; resolving a 0.5-point effect would need roughly 6 seeds per arm, or 60 GPU-hours per lever.

Aspaklarya-020, a full round older, still wins three slices of four. It takes VQA RSICD, CAP DM and CAP RSICD, and loses only DM VQA, by 8.0 points. Round 1XX bought specialisation onto the rewritten DM question set, not a better model overall. Part of that 8.0 is real; part is the 48-token legacy harness clipping verbose answers.

The epoch-3 vs epoch-4 question is closed, and the answer is "neither". Across the six replicated large-corpus checkpoints the epoch-3/epoch-4 gap is +0.49, -0.18 and -0.16 depending on seed - it changes sign. Both earlier claims about a best epoch were reading noise.

The corpus is still only 27% spent. ImageTable.csv holds 89,119 QA pairs over 11,994 train+other images, a mean of 7.43 per image, and every run above used 2. Aspaklarya-111 is the first run to spend more of it, raising QA depth to 4 while doubling captions to 2 so the 2:1 VQA:caption ratio - and therefore the objective balance - is held exactly where every previous run had it.

Aspaklarya-106: an exact LoRA soup

Averaging LoRA adapters naively is wrong, because mean(B) @ mean(A) is not mean(B @ A). The parents are instead stacked, so the product reproduces the mean of the weight deltas exactly:

` A_cat = [A_1; ...; A_N] (Nr x k) B_cat = [c_1B_1, ..., c_NB_N] (d x Nr) c_i = (1/N) * (alpha_i/r_i) / (alpha_new/r_new) `

With r_new = alpha_new = 64 the reconstruction is exact - verified against the explicit mean of the parent deltas to 1.9e-9 max absolute error. The trap is Aspaklarya-104, which carries alpha/r = 2.0 while its three siblings carry 1.0: its coefficient must be doubled, and a soup that ignored this would have silently halved that parent's contribution.

It cost zero GPU-hours and lands 2nd of 24.

What is published here, and what is not

This org holds 22 adapter repos: Aspaklarya-101 through 105 at four epoch checkpoints each, Aspaklarya-106-soup, and Aspaklarya-020-step4500 from the previous round. Five of the runs described above have no repo at all - 106b, 107, 108, 109 and 110. That is deliberate, and the reasons differ.

A checkpoint is published when it is a distinct recipe and not strictly dominated by something already here. Both tests have to pass.

The last three deserve a fuller answer, because they are the ones that produced the most useful result on this page.

108, 109 and 110 are not new recipes. They are the recipes of 102 and 101 re-run with a different random seed and nothing else changed. Their adapters are therefore near-duplicates of adapters already published: same corpus, same rank, same schedule, same everything the config file records. What they produced is not a model, it is a number - sigma 0.31 - and a number belongs in the text, where it now is.

Publishing them would also do active harm. Six more checkpoints spanning 75.22-76.56 would sit in the listing looking like six more options to choose between, and the obvious way to choose would be to take the highest one. That is exactly the mistake the sigma measurement exists to prevent: within an arm, the spread between seeds is the noise, so picking the best seed is selecting on noise and will not reproduce. The honest artifact is the published 101 and 102 adapters plus a stated uncertainty on what they score, not a shelf of lottery tickets.

The cost argument is secondary but real: each adapter is ~821 MB, so the five unpublished runs would add roughly 5 GB of weights that no one should load in preference to what is already here.

Aspaklarya-111 has finished, and it is a null. Its best checkpoint (step 4496) scores 76.62, against the pre-registered bar of roughly 76.8 - the train,other arm mean plus two sigma. It lands inside the arm it was built on (76.37 +/- 0.23, so +0.25 is about 1.1 sigma), which is exactly the case the bar was written to catch. It is therefore not published, on the same reasoning as 108-110.

That is worth stating plainly because 76.62 is the second-highest single number on this page, and the temptation to ship it is precisely the error the sigma measurement exists to prevent. Raising QA depth from 2 to 4 per image did not extend the one confirmed lever.

Caveat

The judge is text-only. It compares an answer against a reference and never sees the image, so it measures agreement with the reference, not verified truth. For a round whose whole premise is longer answers this matters more than it did in round 0XX: the judge can confirm that a model supplies reference-matching content, but not that the content is true of the pixels. Human spot-checks are the only way to close that gap.

The legacy slices also reward a terse style the product no longer wants - the legacy VQA rubric caps a padded-but-correct answer at 89/100 by construction, so a 290-character model cannot reach the top band however accurate it is. That is why the legacy numbers are reported but never used to gate a decision.

Status

• Aspaklarya-101-step4500 is the round reference at MEAN4 76.72, but the

fair description of its recipe is 76.37 +/- 0.23 across three seeds. Any of the six large-corpus checkpoints is a defensible choice; picking the 76.72 one is selecting on noise.

• Aspaklarya-106-soup is 2nd at 76.36 for no training cost, and remains

the better default when GPU time is the constraint - with the caveat above that souping buys the parents' average, not a gain over it.

• Aspaklarya-020-step4500 remains the better general model (3 slices of 4).

Choose a 1XX model only when verbose in-domain VQA is what matters.

• Aspaklarya-106b, 107, 108, 109 and 110 have no adapter repos. See

What is published here, and what is not above - they are either strictly dominated, refuted, or seed replications whose output is a sigma rather than a model.

• QA depth is answered: it does not help. 111 raised QA per image from 2

to 4 and landed inside its own arm. Since 111 adds items but no new images, the train,other gain is better explained by the images it brings than by raw item count.

Round 11X - six axes against a replicated control

Round 1XX ended with one confirmed lever (data scaling) and a warning that single-run deltas below about 1.0 MEAN4 are unreadable. Round 11X was built to test six new axes cheaply, under a short-budget probe (SBP) design: one epoch, checkpoints at steps 562 and 1124, every checkpoint judged on all four slices. Ten jobs ran unattended over three days.

No axis produced a win. That is the headline, and it is not a failed round - six levers were eliminated, and the round also corrected two errors in how this project measures things.

The control had to be rebuilt first

The SBP arms were initially compared against a single control checkpoint that ran a different step grid (590/1180 rather than 562/1124) and a different max-seq-length. That mismatch produced a reported +2.32 VQA_DM gain for the paraphrase arm which does not exist; re-measured against the matched three-seed control the same arm is +1.00, against a VQA_DM sigma of 0.88. The claim is retracted here in full, and so is the recommendation to paraphrase RSICD questions that was built on it.

The lesson generalises past this round: a control must match the treatment on grid, split and budget, or the delta measures the mismatch rather than the treatment. Four-slice reporting did not catch it, because the slicing was right and the baseline was wrong.

Three matched controls were then trained from scratch - identical recipe, seeds 3407 / 1234 / 2024, nothing else varied:

Those three runs also give, for the first time, a per-slice noise floor - which turns out to matter more than the aggregate one:

The two VQA slices are three to four times noisier than the caption slices. MEAN4 averages that down to 0.19, which is why aggregate deltas look more trustworthy than the slice deltas underneath them. Any single-slice VQA claim below about 1.8 (two sigma) is noise, and several claims made earlier in this project sit under that line.

Results

All arms at the matched 1124 grid point, against control mean 75.52. Bands were pre-registered as WIN >= +0.76, SIGNAL +0.51..+0.76, NULL +/-0.51, HARM <= -0.51.

One caveat on the 111 row, in the spirit of the retraction above: 111 is a 3-epoch run, so its step-1124 checkpoint sits at a different point on the LR schedule than the 1-epoch control's does. That comparison is not clean, and the -0.56 should be read as indicative only. 111's clean result is its full-budget one, reported in the Round 1XX section: 76.62, a null against its own arm. Both readings agree that QA depth did not help, which is why the row is kept rather than dropped.

Nothing beat doing nothing. Two arms are worth reading past the headline number, because their MEAN4 hides the mechanism. All slice deltas below are against the three-seed control mean, with that slice's sigma in brackets.

119-teach replaced reference captions with captions generated by a strong general-purpose model. The damage is concentrated in the caption slices (CAP_RSICD -16.11 [31 sigma], CAP_DM -9.18 [37 sigma]) while the VQA slices lose only -1.85 and -0.63, both around two sigma or less. The teacher writes fluent generic prose and overwrites domain-specific remote-sensing vocabulary; a judge scoring against domain references punishes exactly that. Distillation fails when the teacher lacks the dataset's domain knowledge - it is not a statement about distillation in general, and it does not condemn 115, which only rephrases questions and leaves reference captions intact.

120-cur trained captions first, then VQA only. It produced the best VQA_DM in the entire round - 75.48, or +2.66 (3.0 sigma) - and simultaneously destroyed captioning (CAP_RSICD 78.06 -> 35.55, a 42-point collapse). This is the round's strongest evidence that the joint objective is load-bearing: captioning is not a free passenger on a VQA model, it is actively maintained by the mixed loss, and it unlearns within one phase of being dropped. It is also the only slice-level gain in the round that survives its own noise floor - at a price nobody would pay.

116-neg was predicted to buy VQA assertiveness with hard negatives. It did the opposite: VQA_DM -1.97, VQA_RSICD -0.51, CAP_DM -0.56, with only CAP_RSICD nominally positive at +0.38 (well inside its 0.52 sigma, so: no effect). The predicted mechanism is refuted rather than merely unproven.

115-para, the retracted arm, is worth one more line. Re-measured properly it shows VQA_DM +1.00 - but VQA_DM's sigma is 0.88, so that is 1.1 sigma, and the other three slices are negative. There is no paraphrase effect here in either direction. The original +2.32 was baseline mismatch almost end to end.

Two axes that could not be tested

Decoding (112). Greedy, beam-4, temperature 0.7 and repetition-penalty 1.1 were run against the same checkpoint: 76.72 / 76.42 / 75.58 / 75.52. Greedy wins. The axis is closed, not deferred.

Resolution (114). This one is a negative result about the corpus, not the model. The run was configured with --max-side 1536 and trained faster than baseline, which is impossible if images were being upscaled. Measuring 600 images directly: none exceed 1024 px - they are 224x224 (71%), 896x896 (27%) and 768x768 (2%) - and --max-side only ever downscales. Raising it above the 1024 default is a guaranteed no-op. A real resolution test on this data has to lower the cap, not raise it.

The run was allowed to finish anyway, because its configuration made it the missing SBP-native control - and that accident paid for itself twice over.

Hyperparameters

Every arm in this round shares one SBP recipe and changes exactly one thing. The shared recipe:

And the single change per arm:

Only the best checkpoint of each run is reported, here and in the table above. Intermediate checkpoints exist but are not quoted, because picking the best of several grid points and then quoting it as a result is the same selection bias the sigma bands exist to control - the bar was raised accordingly rather than the practice hidden.

Sigma, corrected again

The resolution no-op generated the same checkpoint twice under two different flags, giving two independent generate-and-judge passes over effectively identical weights: 75.66 vs 75.67, every slice within 0.02.

Judge reproducibility is therefore ~0.01 MEAN4 - effectively deterministic. That refutes an earlier hypothesis on this page that sigma 0.31 might be partly judge drift across invocations. It is not. Cross -invocation comparisons in the results table are safe.

A second accident sharpened it further. One control run was killed by an out-of-memory event at 87% - a neighbouring job on the shared A100 took 20 GB - and was re-run from scratch at the same seed. The two step-562 checkpoints, same seed and same data order, score 74.36 and 74.66: a 0.30 spread with the seed held fixed.

So sigma is not mostly seed variance, as assumed after the previous correction. A large share is irreducible run-to-run nondeterminism from non-deterministic CUDA kernels. Two practical consequences:

• The 0.31 band stands, and is if anything well calibrated.
• Exact replication is unattainable even holding the seed. No delta under

roughly 0.5 should be believed from a single pair of runs, however carefully the configuration is matched.

The max-seq-length confound inherited from the old control was also settled empirically: 2048 vs 3072 at the same seed gives 75.71 vs 75.66, a 0.05 difference. Inert, as the token arithmetic predicted - an 896 px image is about 784 vision tokens, so a 2048 budget is rarely the binding constraint.

What is published from this round

Every trained arm of this round now has an adapter repo, and every one of them is labelled on its own card as a rejected configuration. The three controls are not published, and neither are 112 or the 1536 variant of 114 - those changed decoding or input resolution only and reuse an existing adapter, so there are no distinct weights to publish.

This reverses the earlier policy on this page, so the earlier argument is restated rather than deleted: an adapter repo was treated as a claim that a configuration is worth using, and on that reading publishing a refuted recipe makes the org a dump of artefacts. The counter-argument won. A negative result that cannot be re-examined is worth less than one that can, and someone repeating this programme should be able to load 119 and see the caption collapse for themselves rather than take this page's word for it. The claim now lives in the card text, not in the existence of the repo.

The same reasoning applied to the round-1XX tail, which had also never been pushed: 107, 108, 109, 110 and 111 now have repos too.

What has not changed is the standing rule that no result is recommended off a one-epoch probe. A win in the SBP grid earns a full-budget confirmation run before anything is called a model to use. No arm produced a win, so nothing here is recommended - the repos are evidence, not recommendations.

The round's deliverables are a replicated control (75.52 +/- 0.19), a per-slice noise floor, six closed axes, a corrected sigma, one retracted result, and roughly 30 GPU-hours spent establishing that none of it improves the model.

Status after Round 11X

• Nothing on this org changed. Aspaklarya-101-step4500 (76.37 +/- 0.23

across seeds) remains the in-domain reference; Aspaklarya-020-step4500 remains the better general model; Aspaklarya-106-soup remains the cheap default. Round 11X added no model and removed none.

• One confirmed lever, total, across both rounds: data scale. Nine other

axes have now been tested and closed - rank, loss weighting, decoding, vision-tower freezing, resolution, QA depth, paraphrase augmentation, hard negatives, teacher distillation and curriculum ordering.

• The measurement apparatus is the part that improved. Round 11X ends

with a three-seed control, a per-slice noise floor, judge reproducibility pinned at ~0.01, and a demonstration that same-seed reruns still differ by 0.30. Future deltas on this page are readable in a way earlier ones were not.

• **Two results elsewhere on this page are retracted or corrected by this

round** - the paraphrase gain, and the judge-drift explanation of sigma. Both are left visible above rather than quietly edited out.