Spaces:
Running
Running
org card: per-experiment hyperparameters and VQA/caption results
Browse files- README.md +22 -2
- index.html +22 -2
README.md
CHANGED
|
@@ -50,6 +50,7 @@ Winners are merged into the bf16 base and re-validated: adapters are evaluated o
|
|
| 50 |
| Aspaklarya-018 | 16 / 16 | 0.0 | 2e-4 | linear | trained | 1c + 2q | 18,864 | **bf16 base** (no 4-bit) |
|
| 51 |
| Aspaklarya-019 | 16 / 16 | 0.05 | 2e-4 | linear | trained | 1c + 2q | 18,864 | **bf16 base** + 011 recipe |
|
| 52 |
| Aspaklarya-020 | 16 / 16 | 0.0 | 2e-4 | linear | trained | 1c + 2q | **35,981** | bf16 base + **+91% data** |
|
|
|
|
| 53 |
|
| 54 |
Constant everywhere: effective batch 32 (except 014 at 16, 015-017 at 64), warmup 0.03, grad clip 0.3 (1.0 for 011/012), seed 3407.
|
| 55 |
|
|
@@ -66,6 +67,7 @@ Best checkpoint per run on a fixed 1000-image draw (2000 graded items), with boo
|
|
| 66 |
| Aspaklarya-018 | 82.28 ±0.69 | 82.70 ±1.51 | 82.49 ±0.86 | 80.46 ±0.49 | 75.80 ±1.11 | 78.13 ±0.63 | 78.70 ±0.67 | 69.15 ±1.61 | 73.92 ±0.90 |
|
| 67 |
| Aspaklarya-015 | 82.14 ±0.64 | 82.76 ±1.46 | 82.45 ±0.82 | 80.07 ±0.49 | 74.73 ±1.13 | 77.40 ±0.63 | 78.07 ±0.72 | 66.98 ±1.65 | 72.52 ±0.92 |
|
| 68 |
| Aspaklarya-006 | 81.40 ±0.71 | 82.95 ±1.50 | 82.18 ±0.90 | 79.81 ±0.50 | 75.79 ±1.12 | 77.80 ±0.65 | 78.26 ±0.68 | 68.88 ±1.63 | 73.57 ±0.91 |
|
|
|
|
| 69 |
| Aspaklarya-005 | 81.37 ±0.67 | 82.37 ±1.50 | 81.87 ±0.85 | 79.55 ±0.49 | 75.74 ±1.11 | 77.64 ±0.62 | 77.79 ±0.71 | 69.35 ±1.65 | 73.57 ±0.93 |
|
| 70 |
| Aspaklarya-012 | 81.68 ±0.66 | 82.05 ±1.53 | 81.86 ±0.86 | 79.73 ±0.48 | 75.02 ±1.13 | 77.37 ±0.64 | 77.85 ±0.71 | 68.23 ±1.63 | 73.04 ±0.90 |
|
| 71 |
| Aspaklarya-014 | 81.51 ±0.67 | 81.93 ±1.48 | 81.72 ±0.83 | 79.95 ±0.49 | 74.77 ±1.13 | 77.36 ±0.63 | 78.45 ±0.71 | 67.87 ±1.66 | 73.16 ±0.92 |
|
|
@@ -160,14 +162,32 @@ Every run up to 017 trained on 6,288 images / 18,864 samples. The source table a
|
|
| 160 |
Two further points matter for anyone repeating this:
|
| 161 |
|
| 162 |
* **The in-domain penalty is transient.** At matched optimizer steps the extra data looked actively harmful early — step 2,360 was 1.65 behind on in-domain — and that deficit had vanished by step 4,500. Stopping early would have refuted the best run in the programme.
|
| 163 |
-
* **The
|
| 164 |
|
| 165 |
More data is therefore the live axis, and the only one still producing movement.
|
| 166 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 167 |
## Status
|
| 168 |
|
| 169 |
The **hyperparameter** search is closed. Rank, alpha, dropout, learning rate, schedule, weight decay, gradient clipping, vision-tower freezing, QA-per-image, caption prompt pool, batch geometry, effective batch size, learning rate at large batch and base-weight precision were all swept across 19 recipes under identical conditions. None displaced the reference recipe by more than the paired error bar.
|
| 170 |
|
| 171 |
Batch size and learning rate illustrate that ceiling. Effective batch 16 came in at **-0.59** overall, batch 64 at 2e-4 at **-0.56**, and batch 64 at 3e-4 at **-0.34** overall and **-0.87** in-domain; batch 64 at 4e-4 was stopped at epoch 4 while declining. In each case captioning held steady or improved while VQA fell.
|
| 172 |
|
| 173 |
-
Enlarging the training set did what none of those could
|
|
|
|
| 50 |
| Aspaklarya-018 | 16 / 16 | 0.0 | 2e-4 | linear | trained | 1c + 2q | 18,864 | **bf16 base** (no 4-bit) |
|
| 51 |
| Aspaklarya-019 | 16 / 16 | 0.05 | 2e-4 | linear | trained | 1c + 2q | 18,864 | **bf16 base** + 011 recipe |
|
| 52 |
| Aspaklarya-020 | 16 / 16 | 0.0 | 2e-4 | linear | trained | 1c + 2q | **35,981** | bf16 base + **+91% data** |
|
| 53 |
+
| Aspaklarya-021 | 16 / 16 | 0.0 | 2e-4 | linear | trained | 1c + 2q | **35,981** | as 020, **8 epochs** not 4 |
|
| 54 |
|
| 55 |
Constant everywhere: effective batch 32 (except 014 at 16, 015-017 at 64), warmup 0.03, grad clip 0.3 (1.0 for 011/012), seed 3407.
|
| 56 |
|
|
|
|
| 67 |
| Aspaklarya-018 | 82.28 ±0.69 | 82.70 ±1.51 | 82.49 ±0.86 | 80.46 ±0.49 | 75.80 ±1.11 | 78.13 ±0.63 | 78.70 ±0.67 | 69.15 ±1.61 | 73.92 ±0.90 |
|
| 68 |
| Aspaklarya-015 | 82.14 ±0.64 | 82.76 ±1.46 | 82.45 ±0.82 | 80.07 ±0.49 | 74.73 ±1.13 | 77.40 ±0.63 | 78.07 ±0.72 | 66.98 ±1.65 | 72.52 ±0.92 |
|
| 69 |
| Aspaklarya-006 | 81.40 ±0.71 | 82.95 ±1.50 | 82.18 ±0.90 | 79.81 ±0.50 | 75.79 ±1.12 | 77.80 ±0.65 | 78.26 ±0.68 | 68.88 ±1.63 | 73.57 ±0.91 |
|
| 70 |
+
| Aspaklarya-021 | 82.14 ±0.70 | 81.93 ±1.53 | 82.03 ±0.88 | 80.93 ±0.48 | 75.41 ±1.13 | 78.17 ±0.65 | 79.76 ±0.64 | 69.12 ±1.65 | 74.44 ±0.92 |
|
| 71 |
| Aspaklarya-005 | 81.37 ±0.67 | 82.37 ±1.50 | 81.87 ±0.85 | 79.55 ±0.49 | 75.74 ±1.11 | 77.64 ±0.62 | 77.79 ±0.71 | 69.35 ±1.65 | 73.57 ±0.93 |
|
| 72 |
| Aspaklarya-012 | 81.68 ±0.66 | 82.05 ±1.53 | 81.86 ±0.86 | 79.73 ±0.48 | 75.02 ±1.13 | 77.37 ±0.64 | 77.85 ±0.71 | 68.23 ±1.63 | 73.04 ±0.90 |
|
| 73 |
| Aspaklarya-014 | 81.51 ±0.67 | 81.93 ±1.48 | 81.72 ±0.83 | 79.95 ±0.49 | 74.77 ±1.13 | 77.36 ±0.63 | 78.45 ±0.71 | 67.87 ±1.66 | 73.16 ±0.92 |
|
|
|
|
| 162 |
Two further points matter for anyone repeating this:
|
| 163 |
|
| 164 |
* **The in-domain penalty is transient.** At matched optimizer steps the extra data looked actively harmful early — step 2,360 was 1.65 behind on in-domain — and that deficit had vanished by step 4,500. Stopping early would have refuted the best run in the programme.
|
| 165 |
+
* **The climb to the final checkpoint was schedule, not headroom.** Overall score rose at all seven checkpoints including the last, which suggested an unexhausted budget. **Aspaklarya-021 tested that directly and refuted it** — see below.
|
| 166 |
|
| 167 |
More data is therefore the live axis, and the only one still producing movement.
|
| 168 |
|
| 169 |
+
## Schedule length
|
| 170 |
+
|
| 171 |
+
**Aspaklarya-021** repeats 020 on the same enlarged corpus with **8 epochs instead of 4** (9,000 steps, checkpoints step-aligned with 020). It peaked at step 4,720 and then declined; the stop rule ended it at step 5,900 with 3,100 steps unspent.
|
| 172 |
+
|
| 173 |
+
| step | overall | caption | vqa | in-domain |
|
| 174 |
+
|---|---|---|---|---|
|
| 175 |
+
| 1,180 | 77.66 | 79.19 | 76.12 | 81.56 |
|
| 176 |
+
| 2,360 | 78.23 | 80.55 | 75.92 | 81.25 |
|
| 177 |
+
| 3,540 | 78.37 | 80.33 | 76.41 | 82.18 |
|
| 178 |
+
| **4,720** | **78.69** | 80.86 | **76.52** | 82.39 |
|
| 179 |
+
| 5,900 | 77.60 | 80.75 | 74.45 | 81.15 |
|
| 180 |
+
|
| 181 |
+
Its best checkpoint, paired against 020 on the same 1000 images, is negative on every slice and significant on none: overall **-0.53 ±0.43**, in-domain **-0.39 ±0.60**, RSICD **-0.66 ±0.64**.
|
| 182 |
+
|
| 183 |
+
**Data volume helped; data duration did not.** Four epochs is already the right length at this corpus size.
|
| 184 |
+
|
| 185 |
+
The failure mode is asymmetric and worth recording: at step 5,900 **captioning was intact (80.75) while VQA collapsed (76.52 → 74.45)**. Overtraining this recipe degrades VQA first, so a stop rule watching captioning alone would not catch it.
|
| 186 |
+
|
| 187 |
## Status
|
| 188 |
|
| 189 |
The **hyperparameter** search is closed. Rank, alpha, dropout, learning rate, schedule, weight decay, gradient clipping, vision-tower freezing, QA-per-image, caption prompt pool, batch geometry, effective batch size, learning rate at large batch and base-weight precision were all swept across 19 recipes under identical conditions. None displaced the reference recipe by more than the paired error bar.
|
| 190 |
|
| 191 |
Batch size and learning rate illustrate that ceiling. Effective batch 16 came in at **-0.59** overall, batch 64 at 2e-4 at **-0.56**, and batch 64 at 3e-4 at **-0.34** overall and **-0.87** in-domain; batch 64 at 4e-4 was stopped at epoch 4 while declining. In each case captioning held steady or improved while VQA fell.
|
| 192 |
|
| 193 |
+
Enlarging the training set did what none of those could, and training it longer did not extend the gain. That narrows the remaining question to **data composition**: the corpus added was RSICD-only, and the gain it produced is correspondingly RSICD-only while in-domain stays flat. Additional in-domain imagery is the targeted next step.
|
index.html
CHANGED
|
@@ -86,6 +86,7 @@ a { color: #0969da; }
|
|
| 86 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-018</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.0</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">18,864</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>bf16 base</strong> (no 4-bit)</td></tr>
|
| 87 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-019</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.05</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">18,864</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>bf16 base</strong> + 011 recipe</td></tr>
|
| 88 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-020</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.0</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>35,981</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">bf16 base + <strong>+91% data</strong></td></tr>
|
|
|
|
| 89 |
</tbody></table>
|
| 90 |
<p>Constant everywhere: effective batch 32 (except 014 at 16, 015-017 at 64), warmup 0.03, grad clip 0.3 (1.0 for 011/012), seed 3407.</p>
|
| 91 |
<h2>Results</h2>
|
|
@@ -109,6 +110,7 @@ a { color: #0969da; }
|
|
| 109 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-018</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.28 ±0.69</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.70 ±1.51</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.49 ±0.86</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.46 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.80 ±1.11</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.13 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.70 ±0.67</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">69.15 ±1.61</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.92 ±0.90</td></tr>
|
| 110 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-015</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.14 ±0.64</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.76 ±1.46</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.45 ±0.82</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.07 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">74.73 ±1.13</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.40 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.07 ±0.72</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">66.98 ±1.65</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">72.52 ±0.92</td></tr>
|
| 111 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-006</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.40 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.95 ±1.50</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.18 ±0.90</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.81 ±0.50</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.79 ±1.12</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.80 ±0.65</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.26 ±0.68</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">68.88 ±1.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.57 ±0.91</td></tr>
|
|
|
|
| 112 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-005</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.37 ±0.67</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.37 ±1.50</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.87 ±0.85</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.55 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.74 ±1.11</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.64 ±0.62</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.79 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">69.35 ±1.65</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.57 ±0.93</td></tr>
|
| 113 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-012</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.68 ±0.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.05 ±1.53</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.86 ±0.86</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.73 ±0.48</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.02 ±1.13</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.37 ±0.64</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.85 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">68.23 ±1.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.04 ±0.90</td></tr>
|
| 114 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-014</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.51 ±0.67</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.93 ±1.48</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.72 ±0.83</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.95 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">74.77 ±1.13</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.36 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.45 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">67.87 ±1.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.16 ±0.92</td></tr>
|
|
@@ -203,13 +205,31 @@ a { color: #0969da; }
|
|
| 203 |
<p>Two further points matter for anyone repeating this:</p>
|
| 204 |
<ul>
|
| 205 |
<li><strong>The in-domain penalty is transient.</strong> At matched optimizer steps the extra data looked actively harmful early — step 2,360 was 1.65 behind on in-domain — and that deficit had vanished by step 4,500. Stopping early would have refuted the best run in the programme.</li>
|
| 206 |
-
<li><strong>The
|
| 207 |
</ul>
|
| 208 |
<p>More data is therefore the live axis, and the only one still producing movement.</p>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 209 |
<h2>Status</h2>
|
| 210 |
<p>The <strong>hyperparameter</strong> search is closed. Rank, alpha, dropout, learning rate, schedule, weight decay, gradient clipping, vision-tower freezing, QA-per-image, caption prompt pool, batch geometry, effective batch size, learning rate at large batch and base-weight precision were all swept across 19 recipes under identical conditions. None displaced the reference recipe by more than the paired error bar.</p>
|
| 211 |
<p>Batch size and learning rate illustrate that ceiling. Effective batch 16 came in at <strong>-0.59</strong> overall, batch 64 at 2e-4 at <strong>-0.56</strong>, and batch 64 at 3e-4 at <strong>-0.34</strong> overall and <strong>-0.87</strong> in-domain; batch 64 at 4e-4 was stopped at epoch 4 while declining. In each case captioning held steady or improved while VQA fell.</p>
|
| 212 |
-
<p>Enlarging the training set did what none of those could
|
| 213 |
</main>
|
| 214 |
</body>
|
| 215 |
</html>
|
|
|
|
| 86 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-018</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.0</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">18,864</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>bf16 base</strong> (no 4-bit)</td></tr>
|
| 87 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-019</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.05</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">18,864</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>bf16 base</strong> + 011 recipe</td></tr>
|
| 88 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-020</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.0</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>35,981</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">bf16 base + <strong>+91% data</strong></td></tr>
|
| 89 |
+
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-021</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.0</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>35,981</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">as 020, <strong>8 epochs</strong> not 4</td></tr>
|
| 90 |
</tbody></table>
|
| 91 |
<p>Constant everywhere: effective batch 32 (except 014 at 16, 015-017 at 64), warmup 0.03, grad clip 0.3 (1.0 for 011/012), seed 3407.</p>
|
| 92 |
<h2>Results</h2>
|
|
|
|
| 110 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-018</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.28 ±0.69</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.70 ±1.51</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.49 ±0.86</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.46 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.80 ±1.11</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.13 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.70 ±0.67</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">69.15 ±1.61</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.92 ±0.90</td></tr>
|
| 111 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-015</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.14 ±0.64</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.76 ±1.46</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.45 ±0.82</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.07 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">74.73 ±1.13</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.40 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.07 ±0.72</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">66.98 ±1.65</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">72.52 ±0.92</td></tr>
|
| 112 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-006</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.40 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.95 ±1.50</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.18 ±0.90</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.81 ±0.50</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.79 ±1.12</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.80 ±0.65</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.26 ±0.68</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">68.88 ±1.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.57 ±0.91</td></tr>
|
| 113 |
+
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-021</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.14 ±0.70</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.93 ±1.53</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.03 ±0.88</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.93 ±0.48</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.41 ±1.13</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.17 ±0.65</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.76 ±0.64</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">69.12 ±1.65</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">74.44 ±0.92</td></tr>
|
| 114 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-005</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.37 ±0.67</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.37 ±1.50</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.87 ±0.85</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.55 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.74 ±1.11</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.64 ±0.62</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.79 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">69.35 ±1.65</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.57 ±0.93</td></tr>
|
| 115 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-012</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.68 ±0.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.05 ±1.53</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.86 ±0.86</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.73 ±0.48</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.02 ±1.13</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.37 ±0.64</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.85 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">68.23 ±1.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.04 ±0.90</td></tr>
|
| 116 |
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-014</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.51 ±0.67</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.93 ±1.48</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.72 ±0.83</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.95 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">74.77 ±1.13</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.36 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.45 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">67.87 ±1.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.16 ±0.92</td></tr>
|
|
|
|
| 205 |
<p>Two further points matter for anyone repeating this:</p>
|
| 206 |
<ul>
|
| 207 |
<li><strong>The in-domain penalty is transient.</strong> At matched optimizer steps the extra data looked actively harmful early — step 2,360 was 1.65 behind on in-domain — and that deficit had vanished by step 4,500. Stopping early would have refuted the best run in the programme.</li>
|
| 208 |
+
<li><strong>The climb to the final checkpoint was schedule, not headroom.</strong> Overall score rose at all seven checkpoints including the last, which suggested an unexhausted budget. <strong>Aspaklarya-021 tested that directly and refuted it</strong> — see below.</li>
|
| 209 |
</ul>
|
| 210 |
<p>More data is therefore the live axis, and the only one still producing movement.</p>
|
| 211 |
+
<h2>Schedule length</h2>
|
| 212 |
+
<p><strong>Aspaklarya-021</strong> repeats 020 on the same enlarged corpus with <strong>8 epochs instead of 4</strong> (9,000 steps, checkpoints step-aligned with 020). It peaked at step 4,720 and then declined; the stop rule ended it at step 5,900 with 3,100 steps unspent.</p>
|
| 213 |
+
<table style="border-collapse:collapse;margin:1rem 0;font-size:.85rem;border:1px solid #d0d7de"><thead><tr>
|
| 214 |
+
<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">step</th>
|
| 215 |
+
<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">overall</th>
|
| 216 |
+
<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">caption</th>
|
| 217 |
+
<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">vqa</th>
|
| 218 |
+
<th style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;background:#f6f8fa;font-weight:600;white-space:nowrap">in-domain</th>
|
| 219 |
+
</tr></thead><tbody>
|
| 220 |
+
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1,180</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.19</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">76.12</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.56</td></tr>
|
| 221 |
+
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2,360</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.23</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.55</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.92</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.25</td></tr>
|
| 222 |
+
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">3,540</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.37</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.33</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">76.41</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.18</td></tr>
|
| 223 |
+
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>4,720</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>78.69</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.86</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>76.52</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.39</td></tr>
|
| 224 |
+
<tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">5,900</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.60</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.75</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">74.45</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.15</td></tr>
|
| 225 |
+
</tbody></table>
|
| 226 |
+
<p>Its best checkpoint, paired against 020 on the same 1000 images, is negative on every slice and significant on none: overall <strong>-0.53 ±0.43</strong>, in-domain <strong>-0.39 ±0.60</strong>, RSICD <strong>-0.66 ±0.64</strong>.</p>
|
| 227 |
+
<p><strong>Data volume helped; data duration did not.</strong> Four epochs is already the right length at this corpus size.</p>
|
| 228 |
+
<p>The failure mode is asymmetric and worth recording: at step 5,900 <strong>captioning was intact (80.75) while VQA collapsed (76.52 → 74.45)</strong>. Overtraining this recipe degrades VQA first, so a stop rule watching captioning alone would not catch it.</p>
|
| 229 |
<h2>Status</h2>
|
| 230 |
<p>The <strong>hyperparameter</strong> search is closed. Rank, alpha, dropout, learning rate, schedule, weight decay, gradient clipping, vision-tower freezing, QA-per-image, caption prompt pool, batch geometry, effective batch size, learning rate at large batch and base-weight precision were all swept across 19 recipes under identical conditions. None displaced the reference recipe by more than the paired error bar.</p>
|
| 231 |
<p>Batch size and learning rate illustrate that ceiling. Effective batch 16 came in at <strong>-0.59</strong> overall, batch 64 at 2e-4 at <strong>-0.56</strong>, and batch 64 at 3e-4 at <strong>-0.34</strong> overall and <strong>-0.87</strong> in-domain; batch 64 at 4e-4 was stopped at epoch 4 while declining. In each case captioning held steady or improved while VQA fell.</p>
|
| 232 |
+
<p>Enlarging the training set did what none of those could, and training it longer did not extend the gain. That narrows the remaining question to <strong>data composition</strong>: the corpus added was RSICD-only, and the gain it produced is correspondingly RSICD-only while in-domain stays flat. Additional in-domain imagery is the targeted next step.</p>
|
| 233 |
</main>
|
| 234 |
</body>
|
| 235 |
</html>
|