etzion commited on
Commit
b61cff5
·
verified ·
1 Parent(s): 3dcf5f7

org card: per-experiment hyperparameters and VQA/caption results

Browse files
Files changed (2) hide show
  1. README.md +6 -3
  2. index.html +5 -3
README.md CHANGED
@@ -46,7 +46,7 @@ Winners are merged into the bf16 base and re-validated: adapters are evaluated o
46
  | Aspaklarya-014 | 16 / 16 | 0.0 | 2e-4 | linear | trained | 1c + 2q | 18,864 | effective batch 16 |
47
  | Aspaklarya-015 | 16 / 16 | 0.0 | 2e-4 | linear | trained | 1c + 2q | 18,864 | effective batch 64 |
48
  | Aspaklarya-016 | 16 / 16 | 0.0 | **4e-4** | linear | trained | 1c + 2q | 18,864 | batch 64 + linear LR scaling |
49
- | Aspaklarya-017 | 16 / 16 | 0.0 | **3e-4** | linear | trained | 1c + 2q | 18,864 | batch 64 + sqrt LR scaling (running) |
50
 
51
  Constant everywhere: effective batch 32 (except 014 at 16, 015-017 at 64), warmup 0.03, grad clip 0.3 (1.0 for 011/012), seed 3407.
52
 
@@ -64,6 +64,7 @@ Best checkpoint per run on a fixed 1000-image draw (2000 graded items), with boo
64
  | Aspaklarya-012 | 81.68 ±0.66 | 82.05 ±1.53 | 81.86 ±0.86 | 79.73 ±0.48 | 75.02 ±1.13 | 77.37 ±0.64 | 77.85 ±0.71 | 68.23 ±1.63 | 73.04 ±0.90 |
65
  | Aspaklarya-014 | 81.51 ±0.67 | 81.93 ±1.48 | 81.72 ±0.83 | 79.95 ±0.49 | 74.77 ±1.13 | 77.36 ±0.63 | 78.45 ±0.71 | 67.87 ±1.66 | 73.16 ±0.92 |
66
  | Aspaklarya-003 | 81.23 ±0.71 | 82.18 ±1.47 | 81.71 ±0.85 | 79.55 ±0.49 | 75.52 ±1.12 | 77.53 ±0.63 | 77.92 ±0.66 | 69.10 ±1.66 | 73.51 ±0.91 |
 
67
  | Aspaklarya-001 (ep5) | 80.60 ±0.72 | 82.25 ±1.49 | 81.42 ±0.84 | 79.30 ±0.50 | **76.17 ±1.12** | 77.73 ±0.63 | 78.05 ±0.69 | **70.31 ±1.64** | 74.18 ±0.91 |
68
  | Aspaklarya-013 | 80.58 ±0.69 | 82.20 ±1.48 | 81.39 ±0.84 | 79.11 ±0.49 | 75.35 ±1.10 | 77.23 ±0.61 | 77.70 ±0.69 | 68.74 ±1.60 | 73.22 ±0.90 |
69
  | Aspaklarya-009 | 80.01 ±0.71 | 82.34 ±1.46 | 81.18 ±0.84 | 78.53 ±0.52 | 75.85 ±1.11 | 77.19 ±0.63 | 77.09 ±0.73 | 69.59 ±1.63 | 73.34 ±0.91 |
@@ -84,9 +85,11 @@ Aspaklarya-015 gave the best captioning in the table at effective batch 64, but
84
  |---|---|---|---|---|---|---|
85
  | 2e-4 (015) | 76.92 | 77.03 | 77.25 | 77.64 | **77.92** | peak at epoch 5 |
86
  | 4e-4 (016) | **77.67** | 76.93 | 76.42 | 76.89 | — | stopped at epoch 4, declining |
87
- | 3e-4 (017) | running | | | | | |
88
 
89
- Linear LR scaling is **wrong** for LoRA here: 4e-4 peaked at epoch 1 and then lost ground at every later checkpoint, ending below where 2e-4 started. The adapter's effective step size does not follow the full-finetune scaling rule, so raising LR with batch size overshoots rather than compensating.
 
 
90
 
91
  ## Error bars
92
 
 
46
  | Aspaklarya-014 | 16 / 16 | 0.0 | 2e-4 | linear | trained | 1c + 2q | 18,864 | effective batch 16 |
47
  | Aspaklarya-015 | 16 / 16 | 0.0 | 2e-4 | linear | trained | 1c + 2q | 18,864 | effective batch 64 |
48
  | Aspaklarya-016 | 16 / 16 | 0.0 | **4e-4** | linear | trained | 1c + 2q | 18,864 | batch 64 + linear LR scaling |
49
+ | Aspaklarya-017 | 16 / 16 | 0.0 | **3e-4** | linear | trained | 1c + 2q | 18,864 | batch 64 + sqrt LR scaling |
50
 
51
  Constant everywhere: effective batch 32 (except 014 at 16, 015-017 at 64), warmup 0.03, grad clip 0.3 (1.0 for 011/012), seed 3407.
52
 
 
64
  | Aspaklarya-012 | 81.68 ±0.66 | 82.05 ±1.53 | 81.86 ±0.86 | 79.73 ±0.48 | 75.02 ±1.13 | 77.37 ±0.64 | 77.85 ±0.71 | 68.23 ±1.63 | 73.04 ±0.90 |
65
  | Aspaklarya-014 | 81.51 ±0.67 | 81.93 ±1.48 | 81.72 ±0.83 | 79.95 ±0.49 | 74.77 ±1.13 | 77.36 ±0.63 | 78.45 ±0.71 | 67.87 ±1.66 | 73.16 ±0.92 |
66
  | Aspaklarya-003 | 81.23 ±0.71 | 82.18 ±1.47 | 81.71 ±0.85 | 79.55 ±0.49 | 75.52 ±1.12 | 77.53 ±0.63 | 77.92 ±0.66 | 69.10 ±1.66 | 73.51 ±0.91 |
67
+ | Aspaklarya-017 | 81.49 ±0.67 | 81.86 ±1.48 | 81.67 ±0.85 | 79.50 ±0.50 | 75.73 ±1.10 | 77.62 ±0.63 | 77.59 ±0.70 | 69.83 ±1.60 | 73.71 ±0.89 |
68
  | Aspaklarya-001 (ep5) | 80.60 ±0.72 | 82.25 ±1.49 | 81.42 ±0.84 | 79.30 ±0.50 | **76.17 ±1.12** | 77.73 ±0.63 | 78.05 ±0.69 | **70.31 ±1.64** | 74.18 ±0.91 |
69
  | Aspaklarya-013 | 80.58 ±0.69 | 82.20 ±1.48 | 81.39 ±0.84 | 79.11 ±0.49 | 75.35 ±1.10 | 77.23 ±0.61 | 77.70 ±0.69 | 68.74 ±1.60 | 73.22 ±0.90 |
70
  | Aspaklarya-009 | 80.01 ±0.71 | 82.34 ±1.46 | 81.18 ±0.84 | 78.53 ±0.52 | 75.85 ±1.11 | 77.19 ±0.63 | 77.09 ±0.73 | 69.59 ±1.63 | 73.34 ±0.91 |
 
85
  |---|---|---|---|---|---|---|
86
  | 2e-4 (015) | 76.92 | 77.03 | 77.25 | 77.64 | **77.92** | peak at epoch 5 |
87
  | 4e-4 (016) | **77.67** | 76.93 | 76.42 | 76.89 | — | stopped at epoch 4, declining |
88
+ | 3e-4 (017) | 77.00 | 76.56 | 77.16 | 77.68 | 77.86 | peak at epoch 5, stopped at epoch 6 |
89
 
90
+ Linear LR scaling is **wrong** for LoRA here: 4e-4 peaked at epoch 1 and then lost ground at every later checkpoint, ending below where 2e-4 started. The adapter's effective step size does not follow the full-finetune scaling rule, so raising LR with batch size overshoots rather than compensating. The intermediate rate 3e-4 traced almost exactly the same curve as 2e-4 and peaked at the same epoch, one tenth of a point lower.
91
+
92
+ On the shared 1000-image set, neither batch-64 rate beat the reference: 2e-4 is **-0.56** overall and **-0.09** in-domain, 3e-4 is **-0.34** and **-0.87**. Effective batch 64 is therefore refuted at every learning rate tried, and the loss is consistently in VQA while captioning holds or improves. Together with the batch-16 result this closes the batch-size and learning-rate axes: **no hyperparameter lever remains untested**, and further gains have to come from the training data.
93
 
94
  ## Error bars
95
 
index.html CHANGED
@@ -82,7 +82,7 @@ a { color: #0969da; }
82
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-014</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.0</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">18,864</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">effective batch 16</td></tr>
83
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-015</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.0</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">18,864</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">effective batch 64</td></tr>
84
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-016</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.0</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>4e-4</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">18,864</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">batch 64 + linear LR scaling</td></tr>
85
- <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-017</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.0</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>3e-4</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">18,864</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">batch 64 + sqrt LR scaling (running)</td></tr>
86
  </tbody></table>
87
  <p>Constant everywhere: effective batch 32 (except 014 at 16, 015-017 at 64), warmup 0.03, grad clip 0.3 (1.0 for 011/012), seed 3407.</p>
88
  <h2>Results</h2>
@@ -107,6 +107,7 @@ a { color: #0969da; }
107
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-012</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.68 ±0.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.05 ±1.53</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.86 ±0.86</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.73 ±0.48</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.02 ±1.13</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.37 ±0.64</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.85 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">68.23 ±1.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.04 ±0.90</td></tr>
108
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-014</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.51 ±0.67</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.93 ±1.48</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.72 ±0.83</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.95 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">74.77 ±1.13</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.36 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.45 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">67.87 ±1.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.16 ±0.92</td></tr>
109
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-003</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.23 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.18 ±1.47</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.71 ±0.85</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.55 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.52 ±1.12</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.53 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.92 ±0.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">69.10 ±1.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.51 ±0.91</td></tr>
 
110
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-001 (ep5)</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.60 ±0.72</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.25 ±1.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.42 ±0.84</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.30 ±0.50</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>76.17 ±1.12</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.73 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.05 ±0.69</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>70.31 ±1.64</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">74.18 ±0.91</td></tr>
111
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-013</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.58 ±0.69</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.20 ±1.48</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.39 ±0.84</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.11 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.35 ±1.10</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.23 ±0.61</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.70 ±0.69</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">68.74 ±1.60</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.22 ±0.90</td></tr>
112
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-009</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.01 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.34 ±1.46</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.18 ±0.84</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.53 ±0.52</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.85 ±1.11</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.19 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.09 ±0.73</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">69.59 ±1.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.34 ±0.91</td></tr>
@@ -132,9 +133,10 @@ a { color: #0969da; }
132
  </tr></thead><tbody>
133
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4 (015)</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">76.92</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.03</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.25</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.64</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>77.92</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">peak at epoch 5</td></tr>
134
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">4e-4 (016)</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>77.67</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">76.93</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">76.42</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">76.89</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">—</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">stopped at epoch 4, declining</td></tr>
135
- <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">3e-4 (017)</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">running</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"></td></tr>
136
  </tbody></table>
137
- <p>Linear LR scaling is <strong>wrong</strong> for LoRA here: 4e-4 peaked at epoch 1 and then lost ground at every later checkpoint, ending below where 2e-4 started. The adapter&#x27;s effective step size does not follow the full-finetune scaling rule, so raising LR with batch size overshoots rather than compensating.</p>
 
138
  <h2>Error bars</h2>
139
  <p>Bootstrap over images, 3000 resamples. Absolute standard error:</p>
140
  <table style="border-collapse:collapse;margin:1rem 0;font-size:.85rem;border:1px solid #d0d7de"><thead><tr>
 
82
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-014</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.0</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">18,864</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">effective batch 16</td></tr>
83
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-015</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.0</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">18,864</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">effective batch 64</td></tr>
84
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-016</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.0</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>4e-4</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">18,864</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">batch 64 + linear LR scaling</td></tr>
85
+ <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-017</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">16 / 16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">0.0</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>3e-4</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">linear</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">trained</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">1c + 2q</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">18,864</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">batch 64 + sqrt LR scaling</td></tr>
86
  </tbody></table>
87
  <p>Constant everywhere: effective batch 32 (except 014 at 16, 015-017 at 64), warmup 0.03, grad clip 0.3 (1.0 for 011/012), seed 3407.</p>
88
  <h2>Results</h2>
 
107
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-012</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.68 ±0.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.05 ±1.53</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.86 ±0.86</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.73 ±0.48</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.02 ±1.13</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.37 ±0.64</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.85 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">68.23 ±1.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.04 ±0.90</td></tr>
108
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-014</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.51 ±0.67</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.93 ±1.48</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.72 ±0.83</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.95 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">74.77 ±1.13</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.36 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.45 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">67.87 ±1.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.16 ±0.92</td></tr>
109
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-003</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.23 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.18 ±1.47</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.71 ±0.85</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.55 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.52 ±1.12</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.53 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.92 ±0.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">69.10 ±1.66</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.51 ±0.91</td></tr>
110
+ <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-017</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.49 ±0.67</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.86 ±1.48</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.67 ±0.85</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.50 ±0.50</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.73 ±1.10</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.62 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.59 ±0.70</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">69.83 ±1.60</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.71 ±0.89</td></tr>
111
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-001 (ep5)</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.60 ±0.72</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.25 ±1.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.42 ±0.84</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.30 ±0.50</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>76.17 ±1.12</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.73 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.05 ±0.69</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>70.31 ±1.64</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">74.18 ±0.91</td></tr>
112
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-013</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.58 ±0.69</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.20 ±1.48</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.39 ±0.84</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">79.11 ±0.49</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.35 ±1.10</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.23 ±0.61</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.70 ±0.69</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">68.74 ±1.60</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.22 ±0.90</td></tr>
113
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">Aspaklarya-009</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">80.01 ±0.71</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">82.34 ±1.46</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">81.18 ±0.84</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">78.53 ±0.52</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">75.85 ±1.11</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.19 ±0.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.09 ±0.73</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">69.59 ±1.63</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">73.34 ±0.91</td></tr>
 
133
  </tr></thead><tbody>
134
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">2e-4 (015)</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">76.92</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.03</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.25</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.64</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>77.92</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">peak at epoch 5</td></tr>
135
  <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">4e-4 (016)</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap"><strong>77.67</strong></td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">76.93</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">76.42</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">76.89</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">—</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">stopped at epoch 4, declining</td></tr>
136
+ <tr><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">3e-4 (017)</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.00</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">76.56</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.16</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.68</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">77.86</td><td style="border:1px solid #d0d7de;padding:5px 10px;text-align:left;white-space:nowrap">peak at epoch 5, stopped at epoch 6</td></tr>
137
  </tbody></table>
138
+ <p>Linear LR scaling is <strong>wrong</strong> for LoRA here: 4e-4 peaked at epoch 1 and then lost ground at every later checkpoint, ending below where 2e-4 started. The adapter&#x27;s effective step size does not follow the full-finetune scaling rule, so raising LR with batch size overshoots rather than compensating. The intermediate rate 3e-4 traced almost exactly the same curve as 2e-4 and peaked at the same epoch, one tenth of a point lower.</p>
139
+ <p>On the shared 1000-image set, neither batch-64 rate beat the reference: 2e-4 is <strong>-0.56</strong> overall and <strong>-0.09</strong> in-domain, 3e-4 is <strong>-0.34</strong> and <strong>-0.87</strong>. Effective batch 64 is therefore refuted at every learning rate tried, and the loss is consistently in VQA while captioning holds or improves. Together with the batch-16 result this closes the batch-size and learning-rate axes: <strong>no hyperparameter lever remains untested</strong>, and further gains have to come from the training data.</p>
140
  <h2>Error bars</h2>
141
  <p>Bootstrap over images, 3000 resamples. Absolute standard error:</p>
142
  <table style="border-collapse:collapse;margin:1rem 0;font-size:.85rem;border:1px solid #d0d7de"><thead><tr>